Yimin Gao

Yimin Gao

EdgeScope

Design space exploration tools — Timeloop, MAESTRO, roofline analysis — model the accelerator in isolation: compute against memory, with the host CPU outside the design space entirely.

On an edge NPU that’s a bad assumption. The host isn’t a spectator. It sequences the accelerator through a compiled operator stream and does real per-operator work along the way, and on many workloads that host work is what actually sets the wall. It isn’t that these tools weigh the host wrongly — they structurally cannot represent it. The practical consequence is that they’ll happily recommend spending your whole area budget on a bigger array for a workload that was never compute-bound.

EdgeScope treats the host as a first-class resource alongside compute, memory, and dispatch, and allocates a fixed area budget across two coupled axes: sizing, making the operators you already offload run faster, and coverage, moving whole operator classes onto the accelerator to relieve the host.

Grow, then cover

Those axes don’t decompose, which is the part worth knowing.

The bottleneck moves when the array growsTwo bar charts. With a small array, compute is the tallest bar and the binding resource. After growing the array, compute shrinks below host, and the host becomes the tallest bar and the new bottleneck. Small array After growing the array bottleneck compute memory dispatch host grow bottleneck compute memory dispatch host The host work was always there — compute was just hiding it.
Growing the array doesn't only shrink compute. It exposes the host, which is where the next spend has to go.

Start small on a transformer workload and it’s compute-bound, so growing the array is the right move and a coverage unit would buy nothing. Grow it far enough and the bottleneck flips to the host — the CPU-side work was there all along, just hidden behind compute. Now the coverage unit, worthless a few steps earlier, becomes the best remaining spend.

Explore either axis alone and you stop short. Array-alone runs into the host wall; coverage-alone does nothing while compute is still binding. Only a loop that watches the bottleneck shift finds “grow, then cover” — and the allocator finds it on its own rather than being told. It also declines to spend when nothing helps, from the same code path, with no special case.

Built on FlexNPU

Any of this is only trustworthy if the measurements are. EdgeScope is built on FlexNPU, an end-to-end AI accelerator platform I developed on a Xilinx Kria KV260 (Zynq SoC) — custom compiler backend, custom C/C++ runtime, and a double-buffered RTL shell with AXI DMA.

Custom MLIR/IREE passes lower ML operators (GEMM, softmax, depthwise convolution) onto the accelerator slots, and an ARM-host runtime with a register and DMA interface can swap RTL tiles without recompiling the model. The compiled binary binds to the slot contract rather than to a particular tile size, so every configuration runs one unchanged binary — which means any single design change can be measured end-to-end in isolation, with nothing else moving.

The platform reaches up to 31× end-to-end speedup on MobileBERT INT8 over the on-chip ARM CPU.

Platform running in silicon. DSE tool in submission.