# Fill at touch is a backtest lie: measuring the maker gap on public order-book data

Reproducible, public-data-only analysis of how often a passive order that joins the
best bid/ask plausibly fills, versus the naive backtest assumption "price traded at my
level, therefore I filled". Everything is in `fillgap.py` (single deterministic script,
no randomness); outputs are `results.md`, `fill_by_hour.png`, `cum_pnl.png`, `summary.txt`.
Headline on BTCUSDT, 10 days, 7.8M joins: naive 40.2%, queue proxy 27.2%.
The full output of the re-run is committed next to this file as `fillgap-results-btcusdt.md`,
`fillgap-summary-btcusdt.txt` and the `-ethusdt` and `-solusdt` equivalents, so every number quoted in the
article can be checked without downloading 3 GB of data. Every table and figure in them is the script's `results.md` /
`summary.txt` output unaltered; the only edit is the two image links at the end, repointed to
`article-4-fill-by-hour.png` / `article-4-cum-pnl.png` (the same two plots under the names the site
uses) in the BTCUSDT file and dropped from the ETHUSDT and SOLUSDT ones, whose plots are not committed.

## Data (public, free)

Source: the exchange's public historical data archive (file paths below).

**Caveat on availability.** The exchange does not publish `bookTicker` for spot at all. For USDT-M
futures the daily files stop after **2024-03-30**; the final monthly file (`...-2024-04.zip`, 38 MB
against 6.7 GB for March) carries roughly the first seven hours of 2024-04-01 and the dataset ends
there. A 2026 window therefore does not exist for any bookTicker dataset.
We use **USDT-M futures (`um`) BTCUSDT**
and the **most recent 10 full days available: 2024-03-21 .. 2024-03-30** (a high-activity post-ATH week; realized 5-min range
terciles in `fillgap-results-btcusdt.md`). `aggTrades` is taken from the same market so book and prints agree.

Files (daily zips, ~90-220 MB each for bookTicker, 7-30 MB for aggTrades):

```
https://data.binance.vision/data/futures/um/daily/bookTicker/BTCUSDT/BTCUSDT-bookTicker-YYYY-MM-DD.zip
https://data.binance.vision/data/futures/um/daily/aggTrades/BTCUSDT/BTCUSDT-aggTrades-YYYY-MM-DD.zip
```

bookTicker columns used: `best_bid_price, best_bid_qty, best_ask_price, best_ask_qty, transaction_time`.
aggTrades columns used: `price, quantity, transact_time, is_buyer_maker`.
Both timestamps are exchange matching-engine time in ms.

## Method

1. **Join events.** For each side independently (bid, then ask), every time the best price
   changes to a new level we simulate a passive order of `SIZE = 0.01 BTC` joining that level
   at that instant. Queue ahead = the displayed best qty on the first bookTicker row of the new
   level. Every level run (contiguous rows with the same best price) is exactly one join, so
   joins are non-overlapping per side.
2. **Fill rule (public-data proxy).** Trades hitting our side are aggTrades with
   `is_buyer_maker = true` for a bid join (aggressive seller) and `false` for an ask join.
   Over the window `(t_join, t_end]`, where `t_end` is the timestamp of the first bookTicker
   row whose best price differs from ours, the order fills on the earlier of:
   - **queue cleared at P**: the cumulative hit volume at exactly our price reaches
     `queue_ahead + SIZE`;
   - **level swept (print through P)**: a hit trade prints *through* our price (a sell below
     a bid at P, a buy above an ask at P). On a price-time book that print can only execute
     after every order resting at P, ours included, has filled, so it is a fill by
     construction. Most of these land in the millisecond the touch moves away.

   If the touch moves first with neither condition met, the order is unfilled (cancelled).
   - Window is open at `t_join` (trades printed in the same ms as the level change happened
     before we could join) and closed at `t_end` (trades in the ms of the level move are the
     ones that consumed the level; excluding them would be unfairly pessimistic).
   - **Cancels ahead of us are ignored**: every unit of displayed queue must be traded through
     before we fill at P. This is *strict* for us: real queues also shrink by cancellation,
     so fewer prints are needed to reach us than the proxy demands.
   - **Hidden/iceberg liquidity, our own latency and our own impact are ignored.** These are
     *generous* for us: they add queue we cannot see or arrive later than the feed implies.
   - The proxy's errors therefore run in both directions and its net bias is unknown; it is
     an estimate, not a bound. The naive rule *is* a bound: a resting order cannot fill unless
     something trades at or through its price, so naive ≥ real.
   - Level runs that begin and end in the same ms have an empty window and count as unfilled.
3. **Naive rule.** Fill probability 1.0 whenever any trade prints at or through our level on
   our side inside the same window. Its fill time is the first such trade. This is the
   "fill at touch" assumption; the "must trade through" variant is a subset of it.
4. **Reporting.** Fill rate (proxy vs naive), overall, by UTC hour, and by tercile of the
   trailing 5-minute realized range of the mid (max-min over the previous five 1-minute bins,
   in bps of the min; terciles taken over all joins). Also median time-to-fill (fill ts minus
   join ts) and the share of joins whose level trades at all (identical to the naive rate by
   construction).
5. **Toy mean-reversion maker.** Quote both sides at every join. After a fill at price `P`
   (long if bid fill, short if ask fill), exit at `+2 bps` target, `-5 bps` stop, or after 60 s.
   The price path is the mid, bucketed into 100 ms bars (high/low/last of mid); if target and
   stop are hit in the same bar the stop wins (conservative). Timeout exits at the bar's last
   mid. P&L is reported per day in USD for 0.01 BTC per fill, with 0 fee and with a 1 bps
   maker fee charged once per fill (exit is treated as fee-free, and at mid). Positions are
   evaluated independently (no netting, no inventory limits). **This is a toy**: it exists only
   to show how the fill rule changes a backtest, not as a strategy.

Constants at the top of `fillgap.py`: `SIZES, TICKS, TARGET_BPS, STOP_BPS, MAX_HOLD_MS, BAR_MS`.
`SIZES` is the own order size per symbol in base-asset units (BTCUSDT 0.01, ETHUSDT 0.01, SOLUSDT 1, which is
about $680, $35 and $190 on the sample's prices); `--size` overrides it. Fill rates did not depend on the size
on SOLUSDT: 1 SOL and 0.2 SOL gave identical rates to four decimals (only the P&L scales), because the queue
dominates the fill test.

**`TICK` must be the venue's price tick for the symbol** (`TICKS`: BTCUSDT 0.1, ETHUSDT 0.01, SOLUSDT 0.001;
`--tick` overrides). It is load-bearing, not cosmetic: prices are bucketed with `rint(price / TICK)`,
so a tick coarser than the venue's merges neighbouring levels and a sweep one or two ticks deep is
then counted as volume *at our own price* — the proxy fills more often, and by the wrong mechanism.
On ETHUSDT, where the touch is one 0.01 tick wide, `TICK = 0.1` puts the bid and the ask in the same
bucket. The script refuses to run when more than 1% of a day's best bid/ask prices are off the `TICK` grid.

## How to run

Put `fillgap.py` in an empty folder and `cd` into it, then:

```
python3 -m venv venv && ./venv/bin/pip install pandas numpy matplotlib tabulate
mkdir -p data && cd data
for i in $(seq 21 30); do d=2024-03-$i; for t in bookTicker aggTrades; do
  curl -sSO https://data.binance.vision/data/futures/um/daily/$t/BTCUSDT/BTCUSDT-$t-$d.zip; done; done
cd .. && ./venv/bin/python fillgap.py            # ~3-5 min for 10 days, ~4 GB RAM peak
./venv/bin/python fillgap.py --days 2024-03-30   # single day
./venv/bin/python fillgap.py --sym ETHUSDT       # if ETHUSDT files are in data/ (tick 0.01)
./venv/bin/python fillgap.py --sym SOLUSDT       # if SOLUSDT files are in data/ (tick 0.001, 1 SOL)
```

Outputs are written next to the script. Re-running produces byte-identical tables.

## ETHUSDT (secondary run)

Same 10 days, same method, `--sym ETHUSDT`; outputs are `results-ethusdt.md`, `summary-ethusdt.txt`,
`fill_by_hour-ethusdt.png`, `cum_pnl-ethusdt.png`. Note `SIZE` is 0.01 units of the base asset,
so the ETH order is ~$35 vs ~$680 for BTC. ETH gives naive 41.5%, proxy 30.2% (proxy/naive = 0.73)
against BTC's 40.2% / 27.2% (0.68): the same effect, a narrower gap. The mix is where they agree —
9.7% of joins fill by the queue clearing at P on BTC and 9.7% on ETH, with everything above that
(17.5% BTC, 20.6% ETH) coming from the level being swept. A fill is classed "swept" when the through
print came before the queue at P would have cleared.

An earlier run of this table reported ETH as 28.5% proxy with a 12.3% / 16.2% split, i.e. the queue
clearing more often than the level broke, and read that as a thin-touch effect. It was `TICK = 0.1`
on a 0.01-tick symbol: sweeps inside the 0.1 bucket were being counted as volume at our own price.
With the right tick the split points the same way as BTC's.

## SOLUSDT (third pair)

Same 10 days, same method, `--sym SOLUSDT` (tick 0.001, one SOL per order); outputs are
`fillgap-results-solusdt.md` and `fillgap-summary-solusdt.txt`. Naive 23.3%, proxy 16.0%
(proxy/naive = 0.69), of which 10.1% swept and 5.9% queue-cleared, on 15.8M joins. SOL's touch is the
thinnest of the three (median displayed depth about $3.4k, against about $48k on ETH and $105k on BTC; single-day spot checks, BTCUSDT and ETHUSDT 2024-03-26 and SOLUSDT 2024-03-21, not computed by the script), so
its absolute rates are lower, but the ratio is not: 0.68, 0.73, 0.69 across BTC, ETH, SOL. Sweeps are
64.5%, 68.0% and 63.3% of the proxy's fills, and the per-fill pattern holds (swept −1.09 bp, queue-cleared
−0.64 bp on SOL; −1.04 and −0.54 on BTC). The queue-cleared *share* of joins does not agree across pairs
(9.7%, 9.7%, 5.9%): the BTC and ETH match to a hundredth of a point was a coincidence, and the article
no longer says otherwise.

## Latency sensitivity

The proxy joins at the millisecond the best price moves. A real order arrives later. `--export-joins`
writes the join log (`joins.csv`: one row per simulated join, about 50 bytes each) in the order-log format
of the released [`backtest2live`](https://pypi.org/project/backtest2live/) CLI, whose `fill-gap` is an
independent implementation of the same proxy with a latency option:

```
./venv/bin/python fillgap.py --export-joins        # also writes joins.csv ("-sym" suffix for other pairs)
pip install backtest2live
bt2live fill-gap joins.csv --venue binance-um --pair BTCUSDT --latency-ms 5
```

At 0 ms the tool's touch-rule fills equal this script's naive fills (checked on a one-day export: 113,112 of
255,392 joins on ETHUSDT 2024-03-30), and its proxy grants about 3% fewer fills on the full BTCUSDT log (2.06M
against 2.12M), from reading the book after all updates in the join's millisecond, as under Cross-check. The export
has no `filled` column on purpose: with one, the tool measures every delay against that 0 ms flag (39.3% instead of
37.6% at 5 ms on that day); without it, against its own touch rule at the delayed arrival, which is how the table
below was made. Run on the full BTCUSDT log (2026-10-06), rates over the orders that actually rest as makers
(orders that arrive to find their level gone, and orders that would arrive marketable, drop out):

| arrival delay | orders that rest | touch rule | queue proxy | share of touch fills the proxy does not grant |
|---|---:|---:|---:|---:|
| 0 ms | 7,597,831 | 41.4% | 27.1% | 34.5% |
| 5 ms | 4,096,000 | 56.0% | 32.0% | 42.8% |
| 20 ms | 2,537,737 | 69.8% | 35.8% | 48.7% |

Roughly half of all joins have nothing left to rest at after 5 ms, and two thirds after 20 ms; the
survivors are the long-lived levels, where the touch rule is most wrong. The last column is a ratio within
the same set of orders, so it does not depend on how the population is counted; the point gaps do.
(Until `backtest2live` PR #22 lands, its off-grid price guard rejects a log containing the archive's one
malformed price; the export writes tick-bucketed prices, which avoids it.)

## Cross-check

The 2024-03-30 BTCUSDT day (192,290 joins) gives naive 42.4%, proxy 28.8%, of which 18.3% swept and
10.6% queue-cleared. It is a one-day cross-check on the lightest and least typical day in the sample
— a Saturday, 2.5% of the joins and the highest fill rates of the ten (`## By day` in `fillgap-results-btcusdt.md`);
it checks that the script reproduces, not that the day is representative.

An earlier version of this script counted only volume at exactly P and reported
12.6% for the proxy on that day; the difference is entirely the through-print rule. An independent
implementation of the same proxy on the same join log returns 28.5%; the 0.3 pt comes from reading
the book after all updates in the join's millisecond rather than at the first row.

## Known limitations

- Futures, not spot (spot bookTicker is not published). Futures BTCUSDT is the deeper book;
  spot queues are thinner, so the spot gap is likely smaller in absolute queue terms but the
  method is identical.
- bookTicker is a best-level snapshot stream, not a full L2/L3 feed: we cannot see cancels
  or queue position. Ignoring cancels makes the proxy too strict; ignoring latency and
  impact makes it too generous (hidden size would too, but see below). Only live fills on the same
  venue settle the net.
- Hidden size: absent in the sample window. The venue's own API change log (https://developers.binance.com/docs/derivatives/change-log), which runs from 2022-03-01
  and so covers the window, introduces the order type that rests outside the public depth (RPI) on
  2025-11-18, about twenty months after the sample, and the futures order endpoint has no iceberg
  quantity parameter (the spot endpoint does; the futures order *response* schema carries an
  `icebergQuantity` field that is always null, because nothing can set it).
- The `results.md` per-fill table splits proxy fills into "level swept" and "queue cleared at
  P". The split is diagnostic: a swept-level fill is one where the market moved through the
  order's price right after filling it.
- ms timestamps: multiple events share a ms; ordering within a ms is by file order only.
- The archive carries the occasional malformed price: 2024-03-21 has exactly one best bid off the
  0.1 grid (65285.56, in 18.9M rows), which `rint(price / TICK)` folds onto the nearest level. The
  tick guard therefore trips on the *share* of off-grid rows, not on any single one — a wrong TICK
  puts ~88% of rows off the grid, a bad row puts one.
- The 10 days are March 2024, not the present day; hour-of-day and volatility patterns reflect
  that regime, and this archive offers no way to re-check them on a recent one. At least one other
  exchange publishes daily order-book archives into 2025 (existence checked, not used here). A window
  after 2025-11-18 on this venue would also contain the hidden-from-depth orders above.