Semi-orthogonal matrix

In linear algebra, a semi-orthogonal matrix is a non-square matrix with real entries where: if the number of columns exceeds the number of rows, then the rows are orthonormal vectors; but if the number of rows exceeds the number of columns, then the columns are orthonormal vectors.

Properties

Let $A$ be an $m\times n$ semi-orthogonal matrix.

• Either $A^{\operatorname {T} }A=I{\text{ or }}AA^{\operatorname {T} }=I.\,$
• A semi-orthogonal matrix is an isometry. This means that it preserves the norm either in row space, or column space.
• A semi-orthogonal matrix always has full rank.
• A square matrix is semi-orthogonal if and only if it is an orthogonal matrix.
• A real matrix is semi-orthogonal if and only if its non-zero singular values are all equal to 1.
• A semi-orthogonal matrix A is semi-unitary (either A^†A = I or AA^† = I) and either left-invertible or right-invertible (left-invertible if it has more rows than columns, otherwise right invertible).

Examples

Tall matrix (sub-isometry)

Consider the $3\times 2$ matrix whose columns are orthonormal: $A={\begin{pmatrix}1&0\\0&1\\0&0\end{pmatrix}}$ Here, its columns are orthonormal. Therefore, it is semi-orthogonal, which is confirmed by: $A^{T}A={\begin{pmatrix}1&0&0\\0&1&0\end{pmatrix}}{\begin{pmatrix}1&0\\0&1\\0&0\end{pmatrix}}={\begin{pmatrix}1&0\\0&1\end{pmatrix}}=I_{2}$

Short matrix

Consider the $2\times 3$ matrix whose rows are orthonormal: $B={\begin{pmatrix}1&0&0\\0&1&0\end{pmatrix}}$ Here, its rows are orthonormal. Therefore, it is semi-orthogonal, which is confirmed by: $BB^{T}={\begin{pmatrix}1&0&0\\0&1&0\end{pmatrix}}{\begin{pmatrix}1&0\\0&1\\0&0\end{pmatrix}}={\begin{pmatrix}1&0\\0&1\end{pmatrix}}=I_{2}$

Non-example

The following $3\times 2$ matrix has orthogonal, but not orthonormal, columns and is therefore not semi-orthogonal: $C={\begin{pmatrix}2&0\\0&1\\0&0\end{pmatrix}}$ The calculation confirms this: $C^{T}C={\begin{pmatrix}2&0&0\\0&1&0\end{pmatrix}}{\begin{pmatrix}2&0\\0&1\\0&0\end{pmatrix}}={\begin{pmatrix}4&0\\0&1\end{pmatrix}}\neq I_{2}$

Proofs

Preservation of Norm

If a matrix $A$ is tall or square ($m\geq n$), its semi-orthogonality implies $A^{T}A=I_{n}$. For any vector $x\in \mathbb {R} ^{n}$, $A$ preserves its norm: $\|Ax\|_{2}^{2}=(Ax)^{T}(Ax)=x^{T}A^{T}Ax=x^{T}I_{n}x=\|x\|_{2}^{2}$ If a matrix $A$ is short ($m<n$), it preserves the norm of vectors in its row space.

Justification for Full Rank

If $A^{T}A=I_{n}$, then the columns of $A$ are linearly independent, so the rank of $A$ must be $n$. If $AA^{T}=I_{m}$, then the rows of $A$ are linearly independent, so the rank of $A$ must be $m$. In both cases, the matrix has full rank.

Singular Value Property

The statement is that a real matrix $A$ is semi-orthogonal if and only if all of its non-zero singular values are 1.

This follows directly from the SVD, $A=U\Sigma V^{T}$.

($\implies$) Assume $A$ is semi-orthogonal. Then either $A^{T}A=I$ or $AA^{T}=I$. The non-zero singular values of $A$ are the square roots of the non-zero eigenvalues of both $A^{T}A$ and $AA^{T}$. Since one of these "Gramian" matrices is an identity matrix, its eigenvalues are all 1. Thus, the non-zero singular values of $A$ must be 1.

($\Leftarrow$) Assume all non-zero singular values of $A$ are 1. This forces the block of $\Sigma$ containing the non-zero values to be an identity matrix. This structure ensures that either $\Sigma ^{T}\Sigma =I_{n}$ (if $A$ has full column rank) or $\Sigma \Sigma ^{T}=I_{m}$ (if $A$ has full row rank). Substituting this into the expressions for $A^{T}A=V(\Sigma ^{T}\Sigma )V^{T}$ or $AA^{T}=U(\Sigma \Sigma ^{T})U^{T}$ respectively shows that one of them must simplify to an identity matrix, satisfying the definition of a semi-orthogonal matrix.