Tensors (aka multiway arrays) can be instrumental in revealing latent correlations residing in high dimensional spaces. Despite their applicability to a broad range of applications in machine learning, speech recognition, and imaging, inconsistencies between tensor and matrix algebra have been complicating their broader utility.  Researchers seeking to overcome those discrepancies have introduced several different candidate extensions, each introducing unique advantages and challenges. In this talk, we review some of the common tensor definitions, discuss their limitations, and introduce our tensor product framework which permits the elegant extension of linear algebraic concepts and algorithms to tensors.  Following introduction of fundamental tensor operations, we discuss tensor decompositions in further depth, focusing on tensor SVDs based on the tensor product framework which can be computed efficiently in parallel and for which randomized variants are possible.  We present theoretical results regarding compressibility and provide numerical results that show the promise of our approach for compression/approximation of operators and datasets, highlighting examples such as image compression and model reduction.