# Identities and relationships of tensors

There are many mathematical relationships, identities, and connections among tensors. These identities are presented here and show the versatility of the Tensor Toolbox. The propositions indicated below are references to the following report:

T.G. Kolda, "Multilinear operators for higher-order decompositions", Tech. Rep. SAND2006-2081, Sandia National Laboratories, April 2006, http://csmr.ca.sandia.gov/~tgkolda/pubs/index.html#SAND2006-2081.

## Contents

- N-mode product properties
- N-mode product and matricization
- Norm of difference between two tensors
- Tucker tensor properties
- Tucker operator and matricized tensors
- Orthogonalization of Tucker factors
- Kruskal operator properties
- Norm of Kruskal operator
- Inner product of Kruskal operator with a tensor

## N-mode product properties

Create some data.

    Y = tenrand([4 3 2]);
    A = rand(3,4);
    B = rand(3,3);

Prop 3.4(a): The order of the multiplication in different modes is irrelevant.

    X1 = ttm( ttm(Y,A,1), B, 2); %<-- Y x_1 A x_2 B
    X2 = ttm( ttm(Y,B,2), A, 1); %<-- Y x_2 B x_1 A
    norm(X1 - X2) %<-- difference is zero

    ans =

       4.6278e-16

## N-mode product and matricization

Generate some data to work with.

    Y = tenrand([5 4 3]);
    A = rand(4,5); B = rand(3,4); C = rand(2,3); U = {A,B,C};

Prop. 3.7a: N-mode multiplication can be expressed in terms of matricized tensors.

    for n = 1:ndims(Y)
      X = ttm(Y,U,n); %<-- X = Y x_n U{n}
      Xn = U{n} * tenmat(Y,n); %<-- Xn = U{n} * Yn
      norm(tenmat(X,n) - Xn)  % <-- should be zero
    end

    ans =

         0

    ans =

         0

    ans =

         0

Prop. 3.7b: We can do matricizations in various ways and still be equivalent.

    X = ttm(Y,U); %<-- X = Y x_1 A x_2 B x_3 C
    Xm1 = kron(B,A)*tenmat(Y,[1 2])*C';  %<-- Kronecker product version
    Xm2 = tenmat(X,[1 2]); %<-- Matriczed version
    norm(Xm1 - Xm2)  % <-- should be zero
    Xm1 = B * tenmat(Y,2,[3 1]) * kron(A,C)'; %<-- Kronecker product version
    Xm2 = tenmat(X,2,[3 1]); %<-- Matricized version
    norm(Xm1 - Xm2) % <-- should be zero
    Xm1 = tenmat(Y,[],[1 2 3]) * kron(kron(C,B),A)'; %<-- Vectorized via Kronecker
    Xm2 = tenmat(X,[],[1 2 3]); %<-- Vectorized via matricize
    norm(Xm1 - Xm2)

    ans =

       1.6616e-15

    ans =

       2.1756e-15

    ans =

       2.8087e-15

## Norm of difference between two tensors

Prop. 3.9: For tensors X and Y, we have:

    X = tenrand([5 4 3]); Y = tenrand([5 4 3]);
    % The following 2 results should be equal
    norm(X-Y)
    sqrt(norm(X)^2 - 2*innerprod(X,Y) + norm(Y)^2)

    ans =

        2.9316

    ans =

        2.9316

This relationship makes it more convenient to compare the norm of the difference between two different tensor objects. Imagine if we have a `sptensor` and a `ktensor` and we want the norm of the difference, which may be needed to check for convergence, for example, but which is very expensive to convert to a full (dense) tensor. Because `innerprod` and `norm` are defined for all types of tensor objects, this is a handy formula.

    X = sptensor(X);
    Y = ktensor({[1:5]',[1:4]',[1:3]'});
    % The following 2 results should be equal
    norm(full(X)-full(Y))
    sqrt(norm(X)^2 - 2*innerprod(X,Y) + norm(Y)^2)

    ans =

      148.5941

    ans =

      148.5941

## Tucker tensor properties

The properties of the Tucker operator follow directly from the properties of n-mode multiplication.

    % Initialize data
    Y = tensor(1:24,[4 3 2]);
    A1 = reshape(1:20,[5 4]);
    A2 = reshape(1:12,[4 3]);
    A3 = reshape(1:6,[3 2]);
    A = {A1,A2,A3};
    B1 = reshape(1:20,[4 5]);
    B2 = reshape(1:12,[3 4]);
    B3 = reshape(1:6,[2 3]);
    B = {B1,B2,B3};

Proposition 4.2a

    X = ttensor(ttensor(Y,A),B)

    X is a ttensor of size 4 x 3 x 2
    	X.core is a ttensor of size 5 x 4 x 3
    		X.core.core is a tensor of size 4 x 3 x 2
    			X.core.core(:,:,1) =
    	     1     5     9
    	     2     6    10
    	     3     7    11
    	     4     8    12
    			X.core.core(:,:,2) =
    	    13    17    21
    	    14    18    22
    	    15    19    23
    	    16    20    24
    		X.core.U{1} =
    		     1     6    11    16
    		     2     7    12    17
    		     3     8    13    18
    		     4     9    14    19
    		     5    10    15    20
    		X.core.U{2} =
    		     1     5     9
    		     2     6    10
    		     3     7    11
    		     4     8    12
    		X.core.U{3} =
    		     1     4
    		     2     5
    		     3     6
    	X.U{1} =
    		     1     5     9    13    17
    		     2     6    10    14    18
    		     3     7    11    15    19
    		     4     8    12    16    20
    	X.U{2} =
    		     1     4     7    10
    		     2     5     8    11
    		     3     6     9    12
    	X.U{3} =
    		     1     3     5
    		     2     4     6

    AB = {B1*A1, B2*A2, B3*A3};
    Y = ttensor(Y,AB)

    Y is a ttensor of size 4 x 3 x 2
    	Y.core is a tensor of size 4 x 3 x 2
    		Y.core(:,:,1) =
    	     1     5     9
    	     2     6    10
    	     3     7    11
    	     4     8    12
    		Y.core(:,:,2) =
    	    13    17    21
    	    14    18    22
    	    15    19    23
    	    16    20    24
    	Y.U{1} =
    		         175         400         625         850
    		         190         440         690         940
    		         205         480         755        1030
    		         220         520         820        1120
    	Y.U{2} =
    		    70   158   246
    		    80   184   288
    		    90   210   330
    	Y.U{3} =
    		    22    49
    		    28    64

    norm(full(X)-full(Y))  %<-- should be zero

    ans =

         0

Proposition 4.2b

    Y = tensor(1:24,[4 3 2]);
    X = ttensor(Y,A);
    Apinv = {pinv(A1),pinv(A2),pinv(A3)};
    Y2 = ttensor(full(X),Apinv);
    norm(full(Y)-full(Y2))  %<-- should be zero

    ans =

       3.8576e-13

Proposition 4.2c

    Y = tensor(1:24,[4 3 2]);
    rand('state',0);
    Q1 = orth(rand(5,4));
    Q2 = orth(rand(4,3));
    Q3 = orth(rand(3,2));
    Q = {Q1,Q2,Q3};
    X = ttensor(Y,Q)

    X is a ttensor of size 5 x 4 x 3
    	X.core is a tensor of size 4 x 3 x 2
    		X.core(:,:,1) =
    	     1     5     9
    	     2     6    10
    	     3     7    11
    	     4     8    12
    		X.core(:,:,2) =
    	    13    17    21
    	    14    18    22
    	    15    19    23
    	    16    20    24
    	X.U{1} =
    		   -0.4727    0.5608    0.0275   -0.3954
    		   -0.4394   -0.4243    0.3178    0.5707
    		   -0.4659   -0.6116   -0.1037   -0.5982
    		   -0.4209    0.3259    0.5458    0.1138
    		   -0.4350    0.1587   -0.7679    0.3837
    	X.U{2} =
    		   -0.2570    0.1257    0.8908
    		   -0.3751   -0.2111    0.2636
    		   -0.4640   -0.7988   -0.1591
    		   -0.7602    0.5492   -0.3341
    	X.U{3} =
    		   -0.3907   -0.0625
    		   -0.8045   -0.4616
    		   -0.4473    0.8849

    Qt = {Q1',Q2',Q3'};
    Y2 = ttensor(full(X),Qt)
    norm(full(Y)-full(Y2))  %<-- should be zero

    Y2 is a ttensor of size 4 x 3 x 2
    	Y2.core is a tensor of size 5 x 4 x 3
    		Y2.core(:,:,1) =
    	    1.4195   -0.0317   -1.4127   -0.3848
    	   -0.7708    0.2767    1.3323    0.4969
    	    8.6788   -0.0536   -8.3316   -2.1970
    	   -3.0735    0.1529    3.2424    0.9267
    	    2.9652    0.0889   -2.6130   -0.6316
    		Y2.core(:,:,2) =
    	    4.6979   -0.3995   -5.3170   -1.6004
    	   -2.2186    0.8006    3.8440    1.4349
    	   28.9023   -2.1266  -31.9901   -9.4788
    	  -10.0638    1.0545   11.8230    3.6490
    	   10.0122   -0.4854  -10.5344   -3.0047
    		Y2.core(:,:,3) =
    	   -3.4733    0.9238    5.3001    1.8807
    	    0.9310   -0.3464   -1.6360   -0.6138
    	  -21.7522    5.7319   33.0762   11.7190
    	    7.2102   -1.9498  -11.0723   -3.9398
    	   -7.8266    2.0225   11.8142    4.1724
    	Y2.U{1} =
    		   -0.4727   -0.4394   -0.4659   -0.4209   -0.4350
    		    0.5608   -0.4243   -0.6116    0.3259    0.1587
    		    0.0275    0.3178   -0.1037    0.5458   -0.7679
    		   -0.3954    0.5707   -0.5982    0.1138    0.3837
    	Y2.U{2} =
    		   -0.2570   -0.3751   -0.4640   -0.7602
    		    0.1257   -0.2111   -0.7988    0.5492
    		    0.8908    0.2636   -0.1591   -0.3341
    	Y2.U{3} =
    		   -0.3907   -0.8045   -0.4473
    		   -0.0625   -0.4616    0.8849

    ans =

       3.8190e-14

## Tucker operator and matricized tensors

The Tucker operator also has various epressions in terms of matricized tensors and the Kronecker product. Proposition 4.3a

    Y = tensor(1:24,[4 3 2]);
    A1 = reshape(1:20,[5 4]);
    A2 = reshape(1:12,[4 3]);
    A3 = reshape(1:6,[3 2]);
    A = {A1,A2,A3};
    X = ttensor(Y,A)
    for n = 1:ndims(Y)
      rdims = n;
      cdims = setdiff(1:ndims(Y),rdims);
      Xn = A{n} * tenmat(Y,rdims,cdims) * kron(A{cdims(2)}, A{cdims(1)})';
      norm(tenmat(full(X),rdims,cdims) - Xn)  % <-- should be zero
    end

    X is a ttensor of size 5 x 4 x 3
    	X.core is a tensor of size 4 x 3 x 2
    		X.core(:,:,1) =
    	     1     5     9
    	     2     6    10
    	     3     7    11
    	     4     8    12
    		X.core(:,:,2) =
    	    13    17    21
    	    14    18    22
    	    15    19    23
    	    16    20    24
    	X.U{1} =
    		     1     6    11    16
    		     2     7    12    17
    		     3     8    13    18
    		     4     9    14    19
    		     5    10    15    20
    	X.U{2} =
    		     1     5     9
    		     2     6    10
    		     3     7    11
    		     4     8    12
    	X.U{3} =
    		     1     4
    		     2     5
    		     3     6

    ans =

         0

    ans =

         0

    ans =

         0

## Orthogonalization of Tucker factors

Proposition 4.4

    Y = tensor(1:24,[4 3 2]);
    A1 = rand(5,4);
    A2 = rand(4,3);
    A3 = rand(3,2);
    A = {A1,A2,A3};
    X = ttensor(Y,A)

    X is a ttensor of size 5 x 4 x 3
    	X.core is a tensor of size 4 x 3 x 2
    		X.core(:,:,1) =
    	     1     5     9
    	     2     6    10
    	     3     7    11
    	     4     8    12
    		X.core(:,:,2) =
    	    13    17    21
    	    14    18    22
    	    15    19    23
    	    16    20    24
    	X.U{1} =
    		    0.2026    0.3795    0.3046    0.5417
    		    0.6721    0.8318    0.1897    0.1509
    		    0.8381    0.5028    0.1934    0.6979
    		    0.0196    0.7095    0.6822    0.3784
    		    0.6813    0.4289    0.3028    0.8600
    	X.U{2} =
    		    0.8537    0.8216    0.3420
    		    0.5936    0.6449    0.2897
    		    0.4966    0.8180    0.3412
    		    0.8998    0.6602    0.5341
    	X.U{3} =
    		    0.7271    0.5681
    		    0.3093    0.3704
    		    0.8385    0.7027

    [Q1,R1] = qr(A1);
    [Q2,R2] = qr(A2);
    [Q3,R3] = qr(A3);
    R = {R1,R2,R3};
    Z = ttensor(Y,R);
    norm(X) - norm(Z)  %<-- should be zero

    ans =

         0

## Kruskal operator properties

Proposition 5.2

    A1 = reshape(1:10,[5 2]);
    A2 = reshape(1:8,[4 2]);
    A3 = reshape(1:6,[3 2]);
    K = ktensor({A1,A2,A3});
    B1 = reshape(1:20,[4 5]);
    B2 = reshape(1:12,[3 4]);
    B3 = reshape(1:6,[2 3]);
    X = ttensor(K,{B1,B2,B3})

    Y = ktensor({B1*A1, B2*A2, B3*A3});
    norm(full(X) - full(Y))  %<-- should be zero

    X is a ttensor of size 4 x 3 x 2
    	X.core is a ktensor of size 5 x 4 x 3
    		X.core.lambda = [ 1  1 ]
    		X.core.U{1} =
    		     1     6
    		     2     7
    		     3     8
    		     4     9
    		     5    10
    		X.core.U{2} =
    		     1     5
    		     2     6
    		     3     7
    		     4     8
    		X.core.U{3} =
    		     1     4
    		     2     5
    		     3     6
    	X.U{1} =
    		     1     5     9    13    17
    		     2     6    10    14    18
    		     3     7    11    15    19
    		     4     8    12    16    20
    	X.U{2} =
    		     1     4     7    10
    		     2     5     8    11
    		     3     6     9    12
    	X.U{3} =
    		     1     3     5
    		     2     4     6

    ans =

         0

Proposition 5.3a (second part)

    A1 = reshape(1:10,[5 2]);
    A2 = reshape(1:8,[4 2]);
    A3 = reshape(1:6,[3 2]);
    A = {A1,A2,A3};
    X = ktensor(A);
    rdims = 1:ndims(X);
    Z = double(tenmat(full(X), rdims, []));
    Xn = khatrirao(A{rdims},'r') * ones(length(X.lambda),1);
    norm(Z - Xn)  % <-- should be zero

    ans =

         0

    cdims = 1:ndims(X);
    Z = double(tenmat(full(X), [], cdims));
    Xn = ones(length(X.lambda),1)' * khatrirao(A{cdims},'r')';
    norm(Z - Xn)  % <-- should be zero

    ans =

         0

Proposition 5.3b

    A1 = reshape(1:10,[5 2]);
    A2 = reshape(1:8,[4 2]);
    A3 = reshape(1:6,[3 2]);
    A = {A1,A2,A3};
    X = ktensor(A);
    for n = 1:ndims(X)
      rdims = n;
      cdims = setdiff(1:ndims(X),rdims);
      Xn = khatrirao(A{rdims}) * khatrirao(A{cdims},'r')';
      Z = double(tenmat(full(X),rdims,cdims));
      norm(Z - Xn)  % <-- should be zero
    end

    ans =

         0

    ans =

         0

    ans =

         0

Proposition 5.3a (first part)

    X = ktensor(A);
    for n = 1:ndims(X)
      cdims = n;
      rdims = setdiff(1:ndims(X),cdims);
      Xn = khatrirao(A{rdims},'r') * khatrirao(A{cdims})';
      Z = double(tenmat(full(X),rdims,cdims));
      norm(Z - Xn)  % <-- should be zero
    end

    ans =

         0

    ans =

         0

    ans =

         0

## Norm of Kruskal operator

The norm of a ktensor has a special form because it can be reduced to summing the entries of the Hadamard product of N matrices of size R x R. Proposition 5.4

    A1 = reshape(1:10,[5 2]);
    A2 = reshape(1:8,[4 2]);
    A3 = reshape(1:6,[3 2]);
    A = {A1,A2,A3};
    X = ktensor(A);
    M = ones(size(A{1},2), size(A{1},2));
    for i = 1:numel(A)
      M = M .* (A{i}'*A{i});
    end
    norm(X) - sqrt(sum(M(:)))  %<-- should be zero

    ans =

         0

## Inner product of Kruskal operator with a tensor

The inner product of a ktensor with a tensor yields Proposition 5.5

    X = tensor(1:60,[5 4 3]);
    A1 = reshape(1:10,[5 2]);
    A2 = reshape(2:9,[4 2]);
    A3 = reshape(3:8,[3 2]);
    A = {A1,A2,A3};
    K = ktensor(A);
    v = khatrirao(A,'r') * ones(size(A{1},2),1);
    % The following 2 results should be equal
    double(tenmat(X,1:ndims(X),[]))' * v
    innerprod(X,K)

    ans =

          935340

    ans =

          935340