Making Analog Training Scale

Analog in-memory computing

Matrix multiplication is a central operation in neural networks. In conventional digital hardware, computing it requires moving weights from memory to a processor. Analog in-memory computing (AIMC) stores weights as conductances in non-volatile memory crossbars and performs the multiplication where those weights reside, reducing data movement and offering an energy-efficient way to execute matrix operations.

A crossbar computes through electrical currents: each conductance multiplies an applied input voltage, and currents add along the output lines. Digital-to-analog converters (DACs) encode the inputs, while analog-to-digital converters (ADCs) return the result to digital circuitry. The array supports both the forward matrix pass and the transposed matrix pass used in backpropagation.

Why analog training is hard

Training requires more than fast matrix multiplication. Each iteration must propagate activations forward, propagate error signals backward, and use weight gradients to update the stored conductances. Small updates must remain effective over thousands of iterations as the weights and signal distributions evolve.

Two hardware constraints make this difficult. DACs and ADCs have finite precision and range, so quantization and clipping can distort the forward and backward signals. Weight programming introduces a second constraint: updates arrive as discrete pulses whose effects depend on the current conductance and can differ between increasing and decreasing directions. Together with read and write noise, these non-idealities perturb the learning signal and the update trajectory.

Hardware errors propagate and accumulate across layers during both the forward and backward passes. As models grow in depth and width, these errors can increasingly distort gradients and weight updates, with effects that compound over the training trajectory. This scaling challenge has kept most existing demonstrations of analog training at relatively small model and dataset scales. Scaling therefore requires preserving gradient fidelity while keeping logical weights, physical conductances, and converter ranges compatible. The training-scale landscape below compares these prior demonstrations with this work.

The mixed-signal architecture

The framework assigns each operation to the domain that best fits its numerical requirements. Analog arrays perform the forward matrix pass and the transpose operation in backpropagation. The precision-sensitive update path stays digital.

This partition follows the established mixed-precision architecture of Nandakumar et al. (2020): analog arrays execute forward and backward matrix passes, while digital circuitry computes and accumulates weight updates. Retaining digital weight-gradient (W-grad) computation, however, incurs dense arithmetic and storage overhead.

In principle, W-grad could instead use the parallel analog rank updates of Gokmen and Vlasov (2016). Activation and error signals drive outer-product updates directly on the crossbar, reducing the update complexity of a \(D\times D\) layer from \(O(D^2)\) to \(O(D)\). Nandakumar et al. point to device-endurance concerns with this approach. We examine the tradeoff from the perspective of scaling: can this computational saving preserve the gradient information needed for stable training as models grow?

Why weight-gradient accumulation should remain digital

We accept the arithmetic and storage cost of digital W-grad for two compounding reasons: gradient reduction is highly sensitive to precision loss, and fragmenting a batch update into frequent quantized transfers repeatedly introduces write-path error. The experiments below trace these mechanisms from signal precision to the realized array update.

W-grad is sensitive to precision loss

First, consider a purely digital Transformer training experiment (panel a). Quantizing weights or activations to FP8 barely changes final validation loss relative to FP16, while the same precision reduction in the gradient path severely impairs convergence. This control quantizes backward error signals and parameter gradients while arithmetic and reductions remain FP32. Gradient fidelity is therefore already critical before analog accumulation or device programming enters the picture.

Cancellation amplifies accumulation error

A weight gradient sums many signed contributions. For one coordinate, \(+10.0\) and \(-9.9\) leave a residual of only \(0.1\): small rounding errors in the large terms can overwhelm the useful update. The cancellation factor \(\kappa=\sum_b|g_b|/|\sum_b g_b|\) measures this imbalance. Its mean is far above one across all six measured Transformer projection types (panel b). Replaying the same contributions with rounding after every addition shows why this matters: relative gradient error grows with cancellation strength, with substantial error even in FP16 and much larger error in FP8 (panel c). Accurate accumulation must preserve the small signed residual throughout the reduction.

Consolidation avoids repeated quantized-transfer error

Digital accumulation also controls how often an update reaches the array. Panel d holds the total batch at 128 on a symmetric finite-state device. Each micro-batch gradient enters a digital residual and immediately requests a quantized full-rank transfer. Smaller micro-batches create more transfer events; each reintroduces pulse-count truncation and capping, update-management effects, and finite-state programming error. These discrepancies accumulate over the fixed batch. Larger micro-batches consolidate more contributions digitally and request fewer transfers, reducing the one-step realized-update error. The direct rank-update result is independent of the micro-batch partition in this test and is shown once.

Why the gradient path needs higher fidelity

Four experiments, one argument

Explore precision sensitivity, cancellation, accumulation error, and fragmented transfers. Select a label to view each experiment; hover, tap, or focus a bar or point for its values.

1. Precision sensitivity: FP8 gradient-path representations impair convergence while weights and activations remain tolerant.
2. Gradient cancellation: large opposing contributions leave a small signed residual.
3. Accumulation error: rounding after each addition increasingly distorts that residual as cancellation grows.
4. Transfer fragmentation: consolidating updates digitally reduces repeated quantized-transfer error.

Four-seed means; whiskers and bands show the observed seed min–max range. View all four panels (SVG).

Three parts of the co-design

Digital accumulation is the starting point. Scaling also requires the logical model, the physical array, and the converters to operate in compatible numerical ranges.

Technical detail: ACIQ formulation and why we use max alignment

Analytical clipping for integer quantization

ACIQ selects a scalar rail under an assumed, centered population density \(f\). Its approximate mean squared error combines clipping beyond \([-C,C]\) with rounding error \(\Delta^2/12\), where \(\Delta=2C/K\):

\[\begin{aligned} C_{\mathrm{ACIQ}}&\in\operatorname*{arg\,min}_{C>0}\widetilde E_{f,K}(C),\\ \widetilde E_{f,K}(C)&=\int_{|z|>C}(|z|-C)^2f(z)\,dz+\frac{C^2}{3K^2}. \end{aligned}\]

For Gaussian inputs \(z\sim\mathcal N(0,\sigma^2)\), and Laplace inputs \(z\sim\operatorname{Laplace}(0,b_L)\), the clipping integrals give:

\[\begin{aligned} \widetilde E_{G,K}(C)&=(\sigma^2+C^2)\operatorname{erfc}\!\left(\frac{C}{\sqrt2\sigma}\right)\\ &\quad-\sqrt{\frac2\pi}\,C\sigma\,e^{-C^2/(2\sigma^2)}+\frac{C^2}{3K^2}. \end{aligned}\]

\[\widetilde E_{L,K}(C)=2b_L^2e^{-C/b_L}+\frac{C^2}{3K^2}.\]

Here \(\sigma\) is the Gaussian standard deviation, \(b_L=\mathbb E|z|\) is the centered Laplace scale, and \(\operatorname{erfc}\) is the complementary error function. Setting the derivative to zero gives \(C=t_G(K)\sigma\) or \(C=t_L(K)b_L\); these resolution-dependent multipliers can be precomputed.

Resolution convention. Original ACIQ uses \(2^b\) bins at \(b\) bits. The playground adapts its rounding term to the displayed endpoint quantizer, with \(K=2^b-1\) intervals, while keeping the same clipping integrals. These formulas are analytical approximations to population error, not the displayed vector’s exact MSE. Authors’ ACIQ formulation.

Why our training architecture uses max alignment

Adapt to changing distributions without choosing a prior. During training, forward activations and backward error signals evolve. A distribution-specific threshold can become suboptimal as its assumed law or scale changes, requiring repeated estimation and recalibration. \(C=\lVert x\rVert_\infty\) adapts directly to each vector with one maximum-absolute-value reduction and avoids input clipping. The concern is maintaining a suitable distribution model during training, rather than recomputing ACIQ’s scalar multiplier.

Approximate the norm–rail optimum with a simple rule. The manuscript optimizes \(C=A\lVert x\rVert_p\) through \(r_K(\mathcal V)\). For fixed dimension \(D\ge2\) and an absolutely continuous input law with finite second moment, it proves that max alignment approaches the best distortion in this family as precision increases:

\[\lim_{K\to\infty}\frac{R_K(\infty,1;\mathcal V)}{r_K(\mathcal V)}=1.\]

Check precision and width, not one sample. The manuscript compares calibrated norm–rail policies and fixed Gaussian/Laplace ACIQ thresholds across both precision and dimension. At \(D=48\), max alignment is within 10% of the calibrated distortion from four bits onward for both synthetic product laws; the curves nearly coincide at higher precision. The fixed ACIQ comparators show a remaining gap and sensitivity to the assumed prior.

These results support a low-overhead, distribution-agnostic training policy, without claiming universal finite-bit optimality. At six bits, the 10% criterion holds through \(D=1{,}024\), and the gap opens around \(D=2{,}048\), especially for Laplace inputs. ACIQ can be useful when its prior matches, particularly at low precision. This experiment isolates input quantization; it does not measure end-to-end training accuracy or energy.

Future Work

We believe analog training chips deserve a broader research effort connecting machine learning, optimization, computer architecture, circuits, and device physics. This work explores one way to make analog training scale, but many fundamental questions remain open. Several directions seem especially promising:

Resources

BibTeX

@misc{wu2026makinganalogtrainingscale,
  title         = {Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters},
  author        = {Wu, Zhaoxian and Gokmen, Tayfun and Fagbohungbe, Omobayode and Xiao, T. Patrick and Chen, Tianyi},
  year          = {2026},
  eprint        = {2609.36584},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2609.36584}
}