September 2026 — What a conversion gate is for

The litter detector behind Trash Trail is a 120 MB RF-DETR-S ONNX graph that runs server-side. It is the largest thing in the request path, and shrinking it is the obvious win: fewer bytes to pull from S3 on a cold start, less memory to hold, and — on the right hardware — less time per image.

There are two standard ways to do that. Our exporter offers one of them by default and refuses the other.

int8: everything a deployment asks for

The refusal first, because it turns out to be the interesting part. export_rfdetr rejects any -int8 version string with “INT8 export requires a calibration pass; not wired for V2. Use fp16 instead.”

That is true of static post-training quantization, which needs representative data to place activation scales. It is not true of dynamic quantization, which quantizes weights ahead of time and computes activation scales at run time. No calibration set, no data pipeline. The path the exporter refuses was reachable the whole time, in about ninety seconds.

It loads. ONNX Runtime builds a session on the first try, both outputs are finite, the output shapes are unchanged, and the fp32 input/output boundary is untouched, so nothing downstream needs to know. Three and a third times smaller, and it halves inference time on the same CPU.

By every number a deployment review normally asks for, this artifact is strictly better than the one in production.

Then we ran the gate

Decode mirrors the backend exactly: argmax over the 43 named columns, sigmoid of the winner, threshold at the sidecar’s recommended 0.4, top-k at 100, no NMS. Across 200 real photographs:

fp32 produced 202 detections. int8 produced 202 detections.

Not close. Equal. A release check that compared detection counts — which is the check most people write, because it is the one that is easy to write — would have reported exact parity and waved this through.

Because both artifacts are the same graph, query slot q means the same thing in each, so we can compare them query by query instead of in aggregate. Doing that:

At the operating point, 112 of those detections disagree — 56 present only in fp32, 56 present only in int8. The totals match because the losses and the gains happen to be the same size, not because the artifacts agree. Nineteen further queries survive in both and change predicted class; those are mislabels, and no count-based check can see them at all. Fifty of 200 images — a quarter of the corpus — come back different.

The largest confidence shift on a single query was 0.78. That is not rounding. That is a detection appearing or vanishing.

Six recipes, one wall

The obvious next move is to assume the recipe is wrong rather than the idea. So we swept the knobs, screening each variant against the same fp32 reference on the first 40 fixtures (35 detections in fp32):

Six rows, one answer. Per-channel scales, unsigned weights, quantizing only MatMuls, protecting the prediction heads, and finally holding all 47 transformer and head nodes in fp32 while quantizing only the 105 backbone convolutions — every one of them lands between 18 and 22 flips. The spread is smaller than the difference between recipes is meaningful.

The last row is the informative one. Keeping the whole decoder in fp32 costs 14 MB and changes nothing, which places the sensitivity in the backbone — the 105 convolutions you cannot leave alone if you want the speedup at all. There is no knob here. Quantizing this model post-hoc at all is what moves the decisions.

The other conversion: fp16 never even loaded

The path the exporter does offer, and defaults to — --version v2.0.0-fp16, with precision read straight off that string — had never been run end to end. Nothing fp16 has ever served a request; production has always served fp32.

The first attempt to open the converted file failed, on two independent defects in onnxconverter-common 1.16.0:

Colliding, degenerate casts. The converter names inserted casts <node>_input_cast<i>. RF-DETR’s own torch export had already emitted casts around its TopK using that exact convention, so the converter regenerated two names that were taken — and emitted them as self-loops, output name equal to input name, giving those tensors two producers. The source graph has 1,573 uniquely named nodes; the converted graph has 1,578 with 2 duplicates.

Orphaned casts. Of 1,076 FLOAT tensor annotations the converter flipped all but 2 to FLOAT16, while leaving 35 explicit Cast(to=FLOAT) nodes producing float into positions the graph now declares float16.

onnx.checker accepts this graph. ONNX Runtime refuses it. The cheapest imaginable gate — construct one inference session — would have caught this the day the fp16 path was written, and the schema validator never would. That check now runs inside both conversion scripts.

Repaired conservatively (2 casts removed, 35 retyped, the fp32 IO boundary left where the exporter put it), it loads. Measured on the same 200 images, it is 1.89× slower — ONNX Runtime’s CPU provider has thin fp16 kernel coverage, so the graph pays conversion overhead at every boundary and never gets fp16 arithmetic back — with 19 threshold flips and 6 class flips at 0.4, against 202 detections in fp32 and 203 in fp16.

Which is the same trap in a quieter register: 202 against 203 looks like parity too. It is 19 disagreements that nearly cancel. int8 is the sharper illustration only because its disagreements cancel exactly.

We also tried the accelerator route, since fp16 is normally a speedup on hardware with fp16 units. CoreML declined to compile the fp16 graph and fell back to CPU, so that number is unchanged.

What this licenses

On this fixture set, this runtime and this execution provider: both conversions change the output the product would serve, and no post-training recipe we tried avoids it.

It does not license “quantization is unsafe for this model.” Specifically —

• 200 images from three subreddits is not the production distribution.
• Flips are not accuracy — and that is now measured, not asserted. A query moving from 0.41 to 0.39 counts as a disagreement here, and it might be the better answer. Deciding that needed labelled ground truth, so we built some: the same 200 fixtures, labelled blind from the pixels with no model output in front of the reviewer. Image-level litter recall is 52.6% for fp32 and 53.2% for int8, and the false-positive rate on genuinely clean scenes is identical — the same four images of 44. The decisions moved; the accuracy did not. The full result, its limits, and what it corrects are in the update at the end of this post.
• Nothing here separates “int8 arithmetic is lossy for this architecture” from “this converter quantizes this graph badly.” Those have different fixes.
• Latency describes ONNX Runtime CPU on Apple silicon, not the arm64 host that actually serves, and not a GPU or an embedded target. The fp32 reference’s own p50 wandered between 284 and 330 ms across runs — about 16% — so only within-run ratios are quoted above.

What the gate is for

Two conversions were built. One was 1.89× slower and would have been rejected on latency alone. The other was 3.3× smaller and 1.85× faster and would have been accepted by any check anyone had actually written down — including, precisely, a comparison of detection counts.

That is the whole argument for the harness. It is not that conversions are dangerous. It is that the cheap check and the real check disagree, and the cheap check disagrees in the direction of shipping.

Still missing, honestly:

• A fixture set drawn from production. 200 cleanup photos from three subreddits describes Reddit, not the uploads the model sees.
• A threshold placed by a human, on this evidence. The harness deliberately refuses to pick one. Whether 112 flips and 19 mislabels per 200 images is acceptable turns on the fact that a person confirms every suggestion in this flow — a product judgement about a review queue, not a numerical fact.
• Somewhere to put the answer. The serving contract has no field for precision at all: it is encoded only in a version string and an S3 key, and the loader never looks. An int8 artifact dropped at the same key would serve, and nothing in the system would record that precision had changed or that this evidence existed. Today the parity report is a file in a directory, tied to its artifact by filename. It belongs on the release record.

And the finding that actually redirects the work: six post-training recipes hit the same wall, so the next lever is not a converter flag. It is quantization-aware training or a distilled student — which is already what the Pi 5 edge path is waiting on.

Update · 2026-09-08 · the flips, measured

Added the day after publication. Everything above is as published, apart from the bullet that named this measurement as missing. This section is the measurement, and it corrects what the flip count was taken to mean.

The open question was whether 112 changed decisions made the model worse. Answering it needed labels, so the 200 fixtures were split into four shards and read by four reviewers working independently — pixels only, no model output, no boxes, neither artifact’s predictions available. Each image got a yes/no on whether litter is present and a set of the repo’s coarse-10 category rollup for what is in it.

This is presence-level ground truth, not mAP. Nobody drew a box. Nothing here is a localization score, and none of it is comparable to the 0.215 mAP on /benchmarks.

Of the 200 images, 156 contain litter and 44 are genuinely clean. Reviewers recorded their own confidence: 122 high, 71 medium, 7 low. Where a reviewer was unsure whether a category was present, that image/category pair was dropped from scoring entirely — neither credited nor penalised — rather than resolved in either direction.

Then the one question a flip count cannot answer: on an image a human says contains litter, does the artifact return at least one detection at 0.4?

On genuinely clean scenes the false-positive rate is identical on every subset — 9.1% across all 200, 4 images of 44, and the same four images in both artifacts.

The gaps are one to three images out of 81 to 156. They point in both directions depending on subset, and on a set this size none of them is distinguishable from noise. 46.9% against 50.6% is three images.

Per category, across all 200 images, counting an image as recovered when the artifact reports that category somewhere in it:

Two of those rows are not results. glass_broken never appeared confidently in any of the 200 images and smoking appeared once, so both are unmeasured, not good — a zero denominator and a denominator of one say nothing about either artifact. The paper row is a result, and it is the same finding the August post reached from the other direction: loose paper is the category this lineage does not see.

Six of the fixtures are not photographs

Reading 200 images by hand turned up something no harness was looking for. Roughly six of the fixtures are not photographs of scenes at all: two video-game level-select screenshots, a cartoon event flyer, a text-only promotional slide, a marketing graphic, and a phone screenshot of a web search result showing a juice bottle. They are artifacts of a Reddit scrape, and they had been sitting in the reference fixture set under both conversions.

A parity harness cannot find this, structurally. Both artifacts agree happily on a screenshot — it is exactly the kind of input two builds of the same graph have no reason to disagree about, so it produces no flip, no delta, and no signal of any kind. Parity measures whether two artifacts say the same thing. It has no opinion about whether the question was worth asking. That took human eyes on the pixels, and it is the second thing this exercise bought.

What this corrects

The gate was right that 112 decisions changed. It was wrong to imply that mattered.

Those flips were churn around a threshold, not degradation. The queries that cross 0.4 are the queries already sitting near 0.4, and which side of the line they land on is close to arbitrary — which is why 56 losses and 56 gains produce the same recall against a human label and the same false positives on the same clean images.

A parity gate detects instability. Instability is not degradation. That distinction is the whole practical consequence: a flip count belongs in front of a person, not wired to an automatic reject. “A quarter of your corpus moved” is worth someone’s attention and is not by itself evidence of harm. Auto-rejecting on it here would have rejected an artifact that is 30% of the size, 1.85× faster, and — as far as 200 blind-labelled images can tell — exactly as accurate.

What it does not license

It does not show int8 is better. Every difference above is inside the noise of this fixture set. The finding is “no measurable difference,” which is not the same sentence.

It does not retract the 202-against-202 result, which stands exactly as written. A count-based check would still have reported perfect parity on a model whose decisions genuinely moved — and a check that cannot see 112 changed decisions cannot see a real regression either, on the day there is one. What has changed is the conclusion drawn from the flip count, not the flip count.

And the ground truth has its own limits, which are worth stating before anyone leans on it. No image was labelled twice, so there is no inter-rater agreement number: this is one careful human per image, not a consensus. Presence is not localization, so an artifact could find the right category in the wrong place and score here. 156 positive images is small, and the per-category denominators — 8, 12, 16, 19 — are smaller still. And it is the same 200 Reddit photographs as before, which is still not the distribution the model serves.

It does close the specific gap this post left open. The next honest measurement is not a bigger flip count. It is more labels.