Yimin Gao

Yimin Gao

FlexPosit

Running an LLM at the edge, under a real compute and energy budget, asks for two things at the same time. You want precision tunability, so you can land on the right accuracy-efficiency point for whatever budget you’ve been handed. And you want high quantization accuracy on the heavy-tailed, sensitivity-prone weight distributions that modern LLMs actually have.

Those two rarely show up together. Mixed-precision quantization exposes genuinely rich tradeoffs — in software. Push that fine-grained precision variation down into hardware and it collides head-on with the regular dataflow that makes an accelerator efficient in the first place. You can have the flexibility or you can have the systolic array.

Our insight is to stop treating precision as a width fixed at design time and start treating it as a tunable fractional hardware resource — a knob, set at runtime, that trades accuracy, throughput, and energy on demand. FlexPosit realizes that with algorithm-hardware co-design built on posit arithmetic.

Both halves

Algorithm side: a distribution-aware, hardware-aligned mixed-precision quantization scheme that jointly decides per-channel scaling and sensitivity-guided precision allocation. Jointly matters — these two decisions constrain each other, and picking them separately leaves accuracy behind.

Architecture side: a unified bit-serial systolic array with lightweight per-column decoders and a global precision controller. It tunes precision continuously from 4 to 8 bits while keeping the systolic dataflow fully regular — precision scales temporally, per GEMM kernel, so it never introduces spatial irregularity into the array. That’s what makes the software flexibility survive contact with hardware instead of being flattened out by it.

Precision scales in time, not in spaceA fixed systolic array feeds a timeline. Each layer occupies a span proportional to its bit width — four, six, five, eight, four bits — so cheaper layers finish sooner on the same unchanged array. One array, unchanged bit-serial systolic array fully regular dataflow Time per layer = precision 4 bit 6 bit 5 bit 8 bit 4 bit layer 1 layer n Sensitive layers get more bits; the rest finish sooner. Average lands below 5.
Precision is spent in cycles rather than in silicon, so a layer that needs fewer bits simply takes less time — and the array never changes shape.

I led the design and verification of the array in Verilog, and built a custom cycle-accurate simulator with integrated memory and energy modeling (CACTI, Ramulator) to check architectural decisions against end-to-end inference behavior rather than against peak numbers.

What it gets you

FlexPosit reaches near-FP16 accuracy at sub-5-bit average weight precision. At matched accuracy, it delivers up to 1.8× higher throughput and 2× lower energy than state-of-the-art LLM inference accelerators, evaluated across LLaMA, OPT, Phi, and Yi.

The design is currently being taped out in GlobalFoundries 12nm.

To appear at the IEEE/ACM International Symposium on Microarchitecture (MICRO) 2026. Work with L. Dai, J. Yin, X. Guo, and M. R. Stan.