How DPS proves simple transfers about 3x faster
Until recently, alcove generated its zero-knowledge proofs through a third-party delegated proving service. Over the past two months Aleo Labs built its own delegated proving service (DPS) and moved the app onto it. We measured DPS against the prior solution side by side to understand where the time goes and where DPS still has work to do. This post is what we found.
Summary
On Aleo mainnet, the Labs delegated proving service proves private credits and USDCx transfers 2.6–3.4× faster than the prior solution: a median of 0.9–1.8s against 3.2–4.7s, over about 213 identical requests per case from 6 to 11 November. The speed comes from more than 50 small, measured changes rather than one trick. The H100 does the heavy math and never silently falls back to the CPU. We cut the data copied to the GPU, which matters because confidential computing encrypts every copy on the CPU: one contiguous upload took a proof from 5.8s to 1.1s, and keeping the SRS and Varuna's fourth round resident on the GPU took off more. We fused whole sequences of operations onto the device, deleted CPU work that profiling exposed, sped up witness generation, and moved key setup and program parsing out of the request path.
We are not faster everywhere. On executions with many transitions, such as a 7-transition onchain swap or a 16-record USDCx join, the prior solution is faster, because about half of our time there is witness generation that runs on one core. Building those witnesses in parallel cut that time by 34–40% on a test box, and is the next change we are measuring on production hardware.
The result
On Aleo mainnet, the Labs delegated proving service (DPS) returns a proved private transfer in 0.9–1.8s, where the prior solution takes 3.2–4.7s. From 6 to 11 November we sent the same authorization to both services every two minutes, about 213 times per case, and timed each request from key fetch to decoded transaction. On every request where both services answered, they returned the same transaction ID.
| Case | DPS median | Prior solution median | Speedup |
|---|---|---|---|
credits transfer_public, public fee | 0.45s | 0.71s | 1.6× |
credits transfer_public, private fee | 0.93s | 3.16s | 3.4× |
credits transfer_private, public fee | 0.99s | 3.22s | 3.3× |
credits transfer_private, private fee | 1.46s | 4.31s | 2.9× |
USDCx transfer_private, public fee | 1.35s | 3.84s | 2.8× |
USDCx transfer_private, private fee | 1.83s | 4.67s | 2.6× |
USDCx transfer_private_to_public, public fee | 1.29s | 3.84s | 3.0× |
USDCx transfer_private_to_public, private fee | 1.77s | 4.67s | 2.6× |
The lead is on executions with one or two transitions. On larger executions, such as a 7-transition onchain swap or a 16-record USDCx join, the prior solution is faster. The last section explains why.
What made it fast
The gap follows the records a transfer spends. transfer_public with a public fee spends none, and there we are only 1.6× faster. Each record spent adds about 2.5s to the prior solution's time. For us, a spent record costs about 140ms of inclusion witness plus its share of a two-instance Varuna proof of about 400ms. The prior solution's swap also spends records but does not pay that 2.5s: a private fee adds only 0.14s to it. That pattern would be consistent with transfers being proved on CPUs and other programs on GPUs, but we infer this from timings alone and have not confirmed it.
Our side of the comparison comes from more than 50 merged changes across DPS and our snarkVM fork, each removing a measured cost. Most were A/B tested in interleaved blocks, since late September on the production machine type: an H100 in confidential-computing mode inside an Intel TDX enclave. The percentages below were measured at different times against different baselines, so they do not multiply.
1. The GPU does the heavy math, and never quietly stops.
Varuna's multi-scalar multiplications (MSMs) and FFTs run on the H100 through sppark kernels. Early on we found that the device memory pool was sized too small for a private transfer. Every polynomial multiplication in a private proof failed the size check and silently fell back to the CPU. We resized the pool, and made any GPU failure an error rather than a CPU fallback.
2. Move fewer bytes to the GPU.
In confidential-computing mode, the driver encrypts every host-to-device copy on the CPU, so a copy costs far more than the bandwidth suggests. That cost comes from proving inside an enclave, so these changes keep DPS fast despite the enclave rather than put it ahead of the prior solution. Four changes did most of the work:
- Uploading MSM bases in one contiguous copy instead of one per row cut a proof from 5.8s to 1.09s.
- Keeping the SRS bases resident on the GPU cut a proof from 1,023 to 920ms and raised saturated throughput 27%.
- Keeping Varuna's fourth round on the GPU cut a proof by 18%.
- Shipping OpenSSL 3.5, whose AES-GCM the driver uses for that encryption, raised peak throughput 35%.
3. Put whole sequences on the GPU, not single operations.
Offloading one operation at a time lost every time, because each call paid its own round trip. What paid was fusing adjacent passes over the same data into one device pass. For example, fusing four passes in the fourth round saved 7.85% of a proof's CPU time, and accumulating the opening on the device saved another 7.8%.
4. Delete CPU work from the prover.
Several of these came from profiling the code no timer covered, which was 28% of a proof when we first looked. Dividing by the vanishing polynomial in closed form cut 10.5%. Tuning the MSM window, which sppark defaulted to a value meant for unknown sizes, cut 9.6%. Smaller wins came from Ruffini division, Horner's rule, computing Lagrange coefficients once per circuit, and checking whether a polynomial exists instead of evaluating it and discarding the result.
5. Speed up witness generation.
A faster field inversion (4.1×) and exponentiation that skips the leading zeros of a constant exponent each made executing a transfer 15–32% faster.
6. Pay nothing per request that can be paid once.
Proving keys for common programs are derived or loaded at startup, before the service reports ready. Programs a request uses are compared by hash instead of re-parsed, which turned 610ms of CPU into 3.3ms. A request's whole program closure arrives in one round trip, and state paths come from an attested service that holds the trees in memory.
7. Run proofs concurrently.
The service proves eight requests at once. It plays no part in the comparison above, which sends one request at a time. A single proof leaves most of the CPU idle, so running several at once raises how many proofs the machine finishes per second.
Where the prior solution is still faster
On executions with many transitions, about half of our time goes to witness generation that runs on one core. On the production machine, three phases do not speed up with more cores: executing each transition, building each record's inclusion assignment, and building each translation assignment. They take 2.05s of a 3.68s onchain swap and 6.56s of a 12.95s 16-way join. The prior solution answers the swap in less time than that serial part alone, so it must build these witnesses in parallel or avoid them.
The fix is under way. A build that runs those loops in parallel cut the swap by 34% and the join by 40% on an 8-core test box. It is not yet measured on the production machine. Past that, Varuna's own per-instance CPU work remains, about 95–100ms per instance for us, and a whole request on the prior solution comes in under that.
One limit runs the other way. The prior solution accepts requests of up to 16 records, so a 16-record join paid with a private fee record is out of its range. DPS proves that request in about 13s.