MatX

About

MatX designs optimized chips and systems for training and inference of large language models, with a focus on maximizing low-precision compute performance for frontier AI labs.

Featured work

Future Leakage in Block-Quantized Attention

Block quantization is essential for high-throughput LLM inference — grouping matrix elements into blocks that share a scale factor enables efficient low-precision compute on modern accelerators. But when those blocks span multiple token positions in the attention mechanism, a subtle problem emerges: later tokens can leak information back to earlier ones during training, creating a mismatch between how the model learns and how it runs at inference.

MatX researchers identified this future leakage problem in MXFP4-quantized attention, showed it causes measurable divergence between training and inference loss, and proposed a targeted fix: keep block-diagonal tiles unquantized where leakage occurs while quantizing everywhere else. Validating on 1B-parameter models trained on C4, the corrected approach maintained consistent performance — a key step toward quantized attention that matches float baselines while delivering real end-to-end speedups.

Message from Grant

A message from Grant will be included here.