rfdetr-s-litter

Precision is encoded in the version string and in the S3 prefix, and nowhere else. The serving contract has no precision field — see the pipeline page for why that is a gap rather than a detail. The two target rows above are also the two senses of the word in this codebase: what was built, and where it runs. The evidence on this record was gathered under Python onnxruntime 1.29.0, not under the runtime named on the serving row.

The two exporters do not mean the same thing by "fp16"

This repository has two exporters that both accept a -fp16 version string, and they produce artifacts with different IO boundaries by different mechanisms:

Both write fp16 into the version string. A consumer reading only the version string cannot tell whether it must feed float16 or float32, and the loader does not read the graph to find out. That is a real defect in the naming scheme, not a stylistic difference between two scripts — it is stated here because the card is where it would have to be caught.

The 44th column

classCount is 43 and the logits tensor has 44 columns. RF-DETR builds its class head with one extra background column, so columns 0–42 are the canonical classes and column 43 is background. Decode iterates 0 <= c < 43 and ignores the trailing column; it raises if the channel count is neither classCount nor classCount + 1. That check catches an arity mismatch. It cannot catch a permutation, which is why the ordering is enforced upstream instead.

Column i of the logits tensor must be canonical class i. Nothing at serving time can verify that — a permuted head produces confident, well-formed, silently mislabelled output — so it is enforced where the artifact is produced.

Fetched alongside the graph on cold start and parsed into the backend's ModelMeta. These are the fields the artifact actually carries:

There is no precision field. There is no field for a parity result, a fixture set, or a runtime the artifact was measured on. The right-hand column is deliberately blunt about which fields do nothing: four of twelve are provenance a human reads, one is inert, and that is worth seeing on the record rather than discovering during an incident.

0.4 is a deliberate move below the upstream default, and the reason is a property of the product rather than of the model: every suggestion this detector produces is surfaced to a contributor who confirms or rejects it before it becomes data. A missed detection costs more than an extra suggestion — the extra suggestion is one tap to dismiss, the miss is a piece of litter that never enters the record. So the threshold is biased toward recall.

DETR confidences are not comparable to YOLO's. There is no NMS score-shaping here, so a number that means "loose" on the YOLO path does not mean "loose" on this one. 0.4 is not transferable to a model with a different decode.

The numbers below are the ones published on /benchmarks, measured on a shared internal validation split (826 images / 1,004 boxes). They are reproduced rather than restated so the release record and the benchmark page cannot drift apart:

Internal validation only. The 3× bundles an architecture change and a data change together, so it is a relative signal and not a headline accuracy claim.

Status: this artifact is the reference. fp32 is what serves, and no converted artifact has been promoted past it. Two conversions have been built and measured against it; both were held. The measurement and the reasoning are in the build log: What a conversion gate is for.

All rows: 200 real photographs, ONNX Runtime 1.29.0, CPU execution provider, decode mirroring the backend exactly, at the sidecar's 0.4.

The int8 row is the one to read twice. It is 30% of the size, 1.85× faster, and returns exactly the same number of detections as the reference — and a quarter of the corpus comes back different. The counts match because 56 losses and 56 gains happen to be the same size, not because the artifacts agree. A release check that compared detection counts would have called that perfect parity.

What those flips cost, measured against human labels

Parity says the decisions changed. It does not say the model got worse, and that gap has since been closed from outside the harness: the same 200 fixtures labelled blind from the pixels — four reviewers, no model output, no boxes, per-image presence of a coarse 10-category rollup. Presence-level ground truth, not mAP, and not comparable to § 07.

One to three images out of 81 to 156, pointing both ways depending on subset: on a set this size none of it is distinguishable from noise. So the 112 flips are instability, not degradation — an argument for putting a flip count in front of a person rather than wiring it to an automatic reject, and not an argument that int8 is better. The 202-against-202 result stands unchanged: a count-based check would still have reported perfect parity on a model whose decisions moved.

Reading those fixtures by hand also found that roughly six of the 200 are not photographs of scenes at all — video-game screenshots, a cartoon flyer, a marketing graphic, a phone screenshot of a web search. No parity harness can surface that: both artifacts agree happily on a screenshot. Status is unchanged by any of this. fp32 is still the reference, and nothing has been promoted.

• Hazard classes are at zero recall. Needles and batteries are in the 43-class space and the model does not find them. Nothing on this card should be read as hazard coverage.
• Labels are weaker than boxes. An earlier blind grading of this detector's lineage found ~95% of boxes on real litter and ~36% carrying the right label, with the material composition inverted relative to ground truth. See what a litter detector actually sees.
• Counting is not usable. Presence is. Quantity is not.
• Presence recall is about half. Blind human labels over the 200 conversion fixtures put image-level litter recall at 52.6% — this artifact returns nothing at all on roughly half the images a person says contain litter. Per category it recovers bottles at 49%, bags_film_wrappers at 30% and paper at 2%. glass_broken appeared in none of the 200 images and smoking in one, so both are unmeasured, not good. Presence only — no boxes were drawn, so this is not a localization number and not comparable to the mAP above.
• No Pi 5 number is published for this model. The edge benchmarks on /benchmarks are a stock COCO YOLOv8n proving the board can sustain inference; nothing on that page describes this artifact.
• The class-order gate can be bypassed by --skip-class-check, and it degrades to a warning when the checkpoint ships without training_config.json.

What these numbers do not license

• The eval numbers are internal validation only. They do not license a comparison against any published litter benchmark, and they are a relative signal between v1 and v2, not an accuracy claim.
• The parity numbers describe these two artifacts, this runtime, this execution provider, this fixture set. They say nothing about a GPU, an accelerator, the arm64 Lambda host that actually serves, or a different input distribution.
• Flips are still not accuracy — and the accuracy has now been measured. A query moving from 0.41 to 0.39 counts as a disagreement and might be the better answer. Blind presence-level labels on the fixture set say the 112 flips cost nothing measurable: 52.6% against 53.2% image-level recall, with an identical false-positive rate on clean scenes. That licenses "no measurable difference", not "int8 is better" — every gap is inside the noise of this fixture set.
• The ground truth behind those recall numbers is one reviewer per image — 200 fixtures split into four independently-labelled shards — so there is no inter-rater agreement figure, and where a reviewer was unsure of a category that image was excluded from scoring rather than resolved either way.
• The fixture set is 200 cleanup photographs from three subreddits. It describes Reddit, not the uploads this model sees in service.
• Nothing here separates "int8 arithmetic is lossy for this architecture" from "this converter quantizes this graph badly". Those have different fixes.
• Latency was measured on desktop CPU. The fp32 reference's own p50 wandered between 284 and 330 ms across runs — about 16% — so only within-run ratios are quoted.

This page is the registry record for one artifact. The edges it travels — what each stage is allowed to change, and who owns the contract when it does — are drawn once on how a model ships here.