Research · Execution
Fill at Touch Is a Backtest Lie
The maker fill gap on public order-book data: fill at touch grants three fills for every two a queue-aware rule allows, and invents the good ones.
Research · Execution
The maker fill gap on public order-book data: fill at touch grants three fills for every two a queue-aware rule allows, and invents the good ones.
In brief
A market-making or mean-reversion backtest has to decide when a resting limit order fills. The common rule, in most open-source engines and nearly every home-grown one, is fill at touch: when the traded price reaches the order’s price, the order is done. Some variants require the price to trade through. Either way the queue does not exist. Every order is first in line.
A real exchange runs price-time priority. When you join the best bid, everyone already resting there trades before you do. Your order fills only when enough volume executes at that level to clear the queue ahead of you, or when the level is swept entirely, and before the best price moves away. The public feed cannot show your queue position, but it shows enough to estimate it.
This piece is about one thing: resting limit orders. Market and other marketable orders have a different lie, price impact and slippage, which top-of-book data cannot measure, and the rest of what goes wrong in a backtest is for another piece.
Data: a major exchange’s public historical archive for USDT-M perpetual futures (the README has the file paths). Two files per day, bookTicker (every change to the best bid and ask, with displayed size) and aggTrades (every trade, with the side that was passive; consecutive fills at one price from one aggressing order are collapsed into a single row, which keeps the volume the queue arithmetic needs). Ten consecutive days, 2024-03-21 to 2024-03-30, the last ten complete days in the bookTicker archive. Pair BTCUSDT; ETHUSDT and SOLUSDT as checks.
The simulation: whenever the best bid or ask moves to a new price level, join it with a small order: 0.01 BTC, 0.01 ETH or 1 SOL, worth about $680, $35 and $190. Queue ahead is the displayed size at that moment. Then two fill rules on the same join, over the window from the join until the best price moves away:
Both rules share that window, so no order outlives the touch: every time the best price returns to a level, it is a fresh join at the back of the queue. That is the impatient end of maker behaviour, and a resting order that keeps its priority through a one-tick flicker would do better than either number below. It is also the same window for both rules, which is what makes the comparison between them clean.
The proxy is an estimate, and its errors run both ways. It ignores cancels ahead of you: a cancel shrinks the queue, so fewer prints are needed to reach you than the proxy demands, which makes it too strict. It ignores your own latency and impact, which make it too generous: a real order joins some milliseconds after the best price moves, behind faster participants, and cannot take a fill that lands before it arrives. Latency is large, and the section below sizes it at assumed delays; unlike impact it does not shrink with order size. Hidden size would push the same way, but it is absent here: the exchange’s API change log, which runs from March 2022, introduces the order type that rests outside the public depth on 18 November 2025, twenty months after the sample, and the futures order endpoint has no way to place an iceberg. The net depends on venue, size and regime, and nothing in the public feed settles it. What the feed does settle is the naive rule: it cannot be right, only high.
Ten days, 2024-03-21 to 2024-03-30, every join on both sides. Fill rates are shares of joins; the proxy’s fills split into the level being swept and the queue clearing.
| Pair | Joins | Naive fill | Queue-proxy fill | Level swept | Queue cleared |
|---|---|---|---|---|---|
| BTCUSDT | 7.8M | 40.2% | 27.2% | 17.5% | 9.7% |
| ETHUSDT | 7.4M | 41.5% | 30.2% | 20.6% | 9.7% |
| SOLUSDT | 15.8M | 23.3% | 16.0% | 10.1% | 5.9% |
On BTC the naive rule grants three fills for every two the proxy grants: one touch fill in three is a phantom. The ratio is 0.68 on BTC, 0.73 on ETH and 0.69 on SOL, around 0.7 on every pair we ran, though a third pair is not a law.
What the pairs agree on is the mix: the level being swept is about two thirds of the proxy’s fills on all three (64.5% on BTC, 68.0% on ETH, 63.3% on SOL). The patient fill, the queue clearing at the price, is never more than a tenth of all joins, but how much less varies: 9.7% on BTC and ETH, 5.9% on SOL.
The orders are 0.01 BTC, 0.01 ETH and 1 SOL, worth about $680, $35 and $190, and SOL’s touch is by far the thinnest of the three. Its rates are lower; the ratio and the mix are not. Re-running SOL at a fifth of the size gives the same rates to four decimals: the queue, not the order, decides the fill.
Median time to fill is 25 ms under the proxy and 21 ms under the naive rule, though the two medians are taken over different sets of fills — 27% of joins against 40% — and neither distribution is tight: the proxy’s quartiles are 4 ms and 199 ms. The queue-aware fill is not slow. It is late in a different sense: most often it arrives in the millisecond the level breaks.
Two places a modeller might hope to hide:
The proxy joins at the millisecond the best price moves. A real order arrives later, behind faster participants, and cannot take a fill that lands before it arrives. The released backtest2live CLI re-runs the same proxy on the same joins with every order delayed (BTCUSDT, same ten days); at zero delay its touch fills equal this script’s naive fills, and its proxy grants about 3% fewer fills (2.06M against 2.12M), from reading the book after all updates in the join’s millisecond.
| Arrival delay | Orders that rest | Touch rule | Queue proxy | Phantom share |
|---|---|---|---|---|
| 0 ms | 7.6M | 41.4% | 27.1% | 34.5% |
| 5 ms | 4.1M | 56.0% | 32.0% | 42.8% |
| 20 ms | 2.5M | 69.8% | 35.8% | 48.7% |
The rates are over the orders that actually rest: about half of all joins have nothing left to rest at after 5 ms, because the level they would have joined is already gone, and two thirds after 20 ms. What survives is the long-lived levels, and those are where the touch rule is most wrong: a lot of trading at the price and queues that still do not clear.
The phantom share is the touch fills the proxy does not grant, over all touch fills: a ratio within the same set of orders, so it does not depend on how you count the population; the point gaps do.
Latency is the one error source in this piece that can be sized, and it pushes the phantom share up, not down. One in three is what the proxy measures at zero latency. A real order’s share is higher.
To show the count gap in money, a toy: quote both sides at touch, take a 2 basis-point target after a fill, a 5 basis-point stop, 60-second maximum hold, 1 basis point maker fee. The exit path is the mid in 100 ms bars; if a bar touches both the target and the stop, the stop wins. The fee is charged once, on the fill: the exit is taken at the mid and costs nothing, and positions are counted independently, with no netting and no inventory cap. Every one of those is generous, and generous by the same amount under both rules, which is the only reason the two columns can be compared. It is a toy and it loses under both rules. That is the point.
| Ten days, BTCUSDT | Fills | Per fill, before fees | P&L at 1 bp fee |
|---|---|---|---|
| Naive | 3.14M | −0.63 bp | −$347k |
| Queue proxy | 2.12M | −0.86 bp | −$268k |
| of which the level was swept | 1.37M | −1.04 bp | |
| of which the queue cleared | 0.75M | −0.54 bp | |
| Fills only the naive rule grants | 1.02M | −0.15 bp |
The count is the first problem. A fill-at-touch backtest multiplies whatever your per-fill edge is by one and a half, and the fee bill in live trading arrives on a volume the backtest never promised.
The mix is the second, and the worse one. The fills the naive rule invents are the touches that bounced: price reached the level, did not clear the queue ahead, did not sweep through, and left. For a mean-reversion quote that is the ideal fill, and it is exactly the one you do not get. Two thirds of the fills you do get are sweeps, where the market is moving through your price as it fills you, and they lose about twice what a fill earned by clearing the queue loses, and the pattern repeats on SOL (−1.09 bp swept, −0.64 bp cleared). Adverse selection is not a footnote to the fill model. On public data it is most of the fill model.
The proxy is an estimate with errors in both directions. It does not know how much of the displayed queue cancels before you, which would fill you sooner. It does not know how far behind the feed your own order arrives, which would fill you later, and that one can now be sized at assumed delays, in the Latency section: a real order’s phantom share is higher than the headline. Hidden size is absent in this window. So the two directions are not equally known: latency has a size, cancels do not, and the feed cannot say which is larger. The sample is also March 2024. A window after 2025 would contain the hidden-from-depth orders above, which is one reason this piece does not simply swap in a newer one; running the same test on a recent window is the obvious follow-up. The only calibration that closes all of it is your own fills on the venue you trade, compared against the same simulation, and kept current.
That is what we do for our own book: simulated maker fills calibrated against our live fills on the same venues, re-fit as the venue changes.
The script is fillgap.py, about 200 lines of Python with pandas, and the README lists every assumption and the archive URLs. The full output of the run behind this piece ships with it — BTCUSDT, ETHUSDT and SOLUSDT, every table above and several that did not fit — so you can check the numbers before deciding whether to download the data. The README also has the two commands that reproduce the latency table. Then change the pair, the days and the order size, and post what you find.
python3 fillgap.py # ten days of BTCUSDT
python3 fillgap.py --sym SOLUSDT # another pair (tick and size are built in)
python3 fillgap.py --export-joins # join log for the latency table
Data download and setup are in the README.
If you rest limit orders and want the gap measured on your own backtest: for the first ten people, we re-run your order log with calibrated fills and send back the same curve with the fills you would have got. No strategy code, nothing published. How it works.