In September, Dany, the head coach of my running club, organized a college cross-country race. I went to help. My job was to gather the results and publish them on the club’s website, from a tent at the finish.
The timing was done by hand. One volunteer sat at the finish line and pressed the space bar each time a runner crossed, which saved a time. Another volunteer and I stood at the end of the finish chute and wrote down bib numbers in the order runners reached us. Afterwards, the first time went with the first bib, the second time with the second bib, and so on.
It got hectic when groups of runners arrived together. If two runners swapped places in the chute, their times swapped too.
Two iPads had also filmed the finish line from each side. The day after the race, I gave Claude those videos and the registration lists, and together we rebuilt the results for all 437 finishers: 227 men and 210 women. It was a one-off tool, built for this one race.
Finding every finish in the video
One iPad stood still and had the race clock in its frame. The other followed the runners, which gave a better view of the bibs. The two recordings started at different times, so we lined them up using their sound.
To get race times, we read the finish clock in the video about once a minute and fit a straight line through the readings. Every reading landed within five hundredths of a second of that line, so any frame of the video could be turned into a race time.
The first pass was ready by early afternoon. It used Apple’s Vision framework to find people and a language model to read bib numbers. The results looked plausible, but the detection kept losing runners from one frame to the next, and I didn’t trust it.
I asked for tools made for video instead. The second version ran an object detection model, RF-DETR, and a tracker, ByteTrack, on my MacBook and my Mac mini at the same time. It followed each runner through the finish and recorded the frame where they crossed the line.
Gemini read the bib numbers from small crops of each runner. It never saw the full videos. Every bib had to be on the registration list for that race and appear only once. About twenty cases were still unclear, like a folded bib or a runner hidden in a group. I looked at those frames myself and decided. One runner had pinned his bib upside down, so 86 read as 98.
Replaying the finish
In the end, I shipped a small web app to view the results. It plays both videos in sync, with the race clock on top. The ranking fills in as each runner crosses the line, and scrubbing back takes them off again. Each runner also has their own page that starts a few seconds before their finish.
Five finishes from the fixed camera. Faces, bibs and names are blurred. The places and times come from the rebuilt results.
The site runs on Cloudflare. A Worker serves the pages and streams the videos from R2 storage. By the evening, the replay was online.
Comparing with the official results
Both lists had the same 437 finishers.
The official times were about half a second later than the video, and that gap barely changed from one runner to the next. Once I took it out, most times agreed within about a tenth of a second, and 430 of the 437 agreed within half a second.
The order was different in a few spots. In four of them, the video clearly showed another order: three groups in the men’s race, where both cameras agreed, and one pair in the women’s race that I called from the frames. One more group of three finished within about a third of a second, too close to call. In two other close finishes, my pipeline had it wrong and the official order was right.
Keeping it private
The first version was an unlisted link with no password. That evening, I realized the site showed runners’ faces next to their names and bib numbers, and some of them are minors.
I removed the share button, deleted the full-length recordings I had uploaded, and put the site behind a private access code. That’s why there’s no link to it here.
What happened next
I sent Dany the corrected results as a PDF and spreadsheets, with a finish-line image for each change.