Classical Vision Transformers (ViT) achieve state-of-the-art image recognition, but their linear projection layers require $O(d^2)$ parameters per layer (where $d$ is the embedding dimension). The Quantum Vision Transformer (QViT) proposes replacing these costly linear layers with orthogonal quantum layers built from Reconfigurable Beam Splitter (RBS) gates, reducing the parameter count to $O(n)$ for an $n$-qubit circuit while implementing a $2^n \times 2^n$ orthogonal transformation.
The architecture is hybrid: image patching, positional encoding, and the classification head remain classical, while the projection matrices inside each Transformer block are quantum.
The foundational quantum primitive is the RBS gate, a two-qubit gate that performs a parameterised beam-splitter operation in the $\{|01\rangle, |10\rangle\}$ subspace:
The RBS gate acts as a controlled rotation: it only mixes $|01\rangle$ and $|10\rangle$ states, leaving $|00\rangle$ and $|11\rangle$ unchanged. This corresponds to a Givens rotation in the single-excitation subspace:
Key properties that make RBS ideal for this purpose:
q₀ ──[RBS(θ₀₁)]──[RBS(θ₀₂)]────────── ...
│ │
q₁ ──[RBS(θ₀₁)]──[RBS(θ₁₂)]──[RBS(θ₁₃)]── ...
│ │
q₂ ──────────────[RBS(θ₁₂)]──[RBS(θ₂₃)]── ...
│
q₃ ──────────────────────────[RBS(θ₂₃)]── ...
Pyramid depth: ⌊n/2⌋ layers → implements O(n) ∈ SO(2ⁿ)
Parameters: n(n-1)/2 for n-dimensional orthogonal matrix
Consider $n$ qubits prepared in a computational basis state encoding a classical vector $\mathbf{x} \in \mathbb{R}^n$ via amplitude encoding in the single-excitation subspace:
A pyramid of RBS gates with parameters $\boldsymbol{\theta}$ implements the unitary $U(\boldsymbol{\theta})$, which in the single-excitation subspace corresponds to an orthogonal matrix $O(\boldsymbol{\theta}) \in O(n)$:
The full orthogonal layer (including bias) is:
| Property | Classical Linear ($W \in \mathbb{R}^{d\times d}$) | Quantum Orthogonal (RBS pyramid) |
|---|---|---|
| Parameters | $O(d^2)$ | $O(n) = O(\log_2 d)$ |
| Expressivity | Full $\mathbb{R}^{d\times d}$ | Orthogonal subgroup $O(d)$ |
| Inference cost | $O(d^2)$ | $O(n^2) = O(\log^2 d)$ circuit depth |
| Gradient | Standard backprop | Parameter-shift rule |
The QViT replaces linear projection matrices inside each Transformer block with quantum orthogonal layers:
INPUT IMAGE (H×W×C)
│
▼
┌─────────────────────────────────┐
│ Patch Embedding (Classical) │
│ Split into N patches → d-dim │
└──────────────┬──────────────────┘
│
▼ [N+1 tokens]
┌─────────────────────────────────┐
│ + CLS token + Pos. Embedding │ (Classical)
└──────────────┬──────────────────┘
│
┌───────────┴──────────────┐
│ Transformer Block × L │
│ │
│ ┌─────────────────────┐ │
│ │ Layer Norm │ │ (Classical)
│ └──────────┬──────────┘ │
│ │ │
│ ┌──────────▼──────────┐ │
│ │ Quantum Multi-Head │ │
│ │ Self-Attention │ │
│ │ │ │
│ │ Q = U_Q(θ_Q)·x │ │ ← Quantum Orthogonal Layer
│ │ K = U_K(θ_K)·x │ │ ← Quantum Orthogonal Layer
│ │ V = U_V(θ_V)·x │ │ ← Quantum Orthogonal Layer
│ │ │ │
│ │ Attn = softmax(QKᵀ/√d)·V │ (Classical computation)
│ └──────────┬──────────┘ │
│ │ │
│ ┌──────────▼──────────┐ │
│ │ + Residual │ │ (Classical)
│ └──────────┬──────────┘ │
│ │ │
│ ┌──────────▼──────────┐ │
│ │ Layer Norm │ │ (Classical)
│ └──────────┬──────────┘ │
│ │ │
│ ┌──────────▼──────────┐ │
│ │ Quantum MLP │ │
│ │ U_1(θ₁)·x + b₁ │ │ ← Quantum Orthogonal Layer
│ │ ReLU │ │
│ │ U_2(θ₂)·x + b₂ │ │ ← Quantum Orthogonal Layer
│ └──────────┬──────────┘ │
│ │ │
│ ┌──────────▼──────────┐ │
│ │ + Residual │ │
│ └─────────────────────┘ │
└───────────┬──────────────┘
│
▼ [CLS token]
┌─────────────────────────────────┐
│ Classification Head (Classical) │
│ Linear → Softmax │
└─────────────────────────────────┘
│
▼
Class Probabilities
Classical input vector x ∈ ℝⁿ
│
▼ (Amplitude encoding into single-excitation basis)
|ψ_x⟩ = Σᵢ xᵢ|eᵢ⟩
│
▼ (RBS pyramid with parameters θ)
q₀: ──●──────●────────── ...
│ │
q₁: ─[θ₀₁]──●──────●── ...
│ │
q₂: ────────[θ₁₂]──●── ...
│
q₃: ───────────────[θ₂₃]── ...
▼ (Measure all qubits → probability amplitudes)
│
O(θ)·x (orthogonal projection of x)
The standard multi-head self-attention computes:
In QViT, the projection matrices are replaced by quantum orthogonal layers:
The attention computation itself remains classical — only the projections are quantum. This is a hybrid approach where quantum hardware handles the high-dimensional linear algebra while classical hardware performs the non-linear operations.
For $h$ heads with dimension $d_k = d/h$ each:
Classical data must be loaded into quantum states. QViT uses amplitude encoding in the single-excitation subspace:
| Strategy | Qubits for $d$-dim | Circuit Depth | Use case |
|---|---|---|---|
| Basis encoding | $d$ qubits | $O(1)$ | Binary/integer data |
| Single-excitation (QViT) | $d$ qubits | $O(d)$ | Real-valued unit vectors |
| Amplitude encoding | $\log_2 d$ qubits | $O(d)$ | Compact representation |
| FRQI (images) | $1 + \log_2(HW)$ qubits | $O(HW)$ | Pixel-level image encoding |
The circuit depth for computing the orthogonal projection is:
Training uses the quantum parameter-shift rule to compute exact gradients:
Key finding: the orthogonality constraint (inherent to quantum layers) acts as a regulariser, providing some accuracy improvement in low-data regimes. The primary benefit is parameter efficiency rather than outright accuracy gain.