Boresight by Raycast
An operator opens a camera and the peak labels float a finger's width above the mountains. The camera's motors report pan 144°, tilt −0.3°. The mountains have not moved. If the labels are wrong by the same amount all the way across the frame, the camera is not pointing where it says it is, and the difference is a property of the bracket it sits on. That property is the , and this page is how it gets measured, in plain English, from nothing but the camera's own photographs and a map of the land.
Is the calibration done in 3D?
Yes. The world it reasons about is fully three-dimensional; only the answer is small. The camera has a real position: latitude, longitude, a height above sea level, a mast. The land around it is a real 3D surface taken from an elevation map of the earth. Every pixel of the photo is treated as a straight line leaving the lens.
What the solver does not do is build or draw a 3D model. There is no mesh and no rendering. It needs one number per direction — how high the ridge appears from the lens — so it walks along each line, reads heights off the map, and keeps the highest angle it meets. The scene below draws that same calculation so you can see it. Drag it to look around.
Stage 1. One ray per compass direction leaves the camera, tilted up just enough to graze the highest ground on its path. Where it touches the land is the skyline. Where it passes through the glass is where that piece of skyline should appear in the photo: the grey dots. Hollow dots came from the near ridge on the left and will be ignored.
What you are looking at. The grid is the land as the elevation map describes it, with heights stretched seven times so the hills read. The dot on the near hill is the camera on its mast. The upright rectangle is the photo: a sheet of glass a few kilometres in front of the lens, and a pixel in the photo and a point on that glass are the same thing. Each orange line is one ray, tilted up just enough to graze the highest ground on its path; where it touches the land is the skyline in that direction, and where it crosses the glass is where that piece of skyline should appear in the photo. Grey dots are those predictions. Red dots are where the ridge actually is in the photograph.
In stage 2 the red dots sit below the grey ones by the same amount everywhere. The land has not sunk; the glass is not where we assumed. In stage 3 we tip our assumed camera until the rays pass through the red dots, and the amount we had to tip it is the calibration.
The same thing, one flat slice at a time
The film cuts the scene into flat views because each one makes a single idea easier to read. Step 1 is a vertical slice along one ray, seen from the side. Step 2 is the scene from straight above. Steps 3 to 5 are what the camera itself sees: the sheet of glass, face on.
Pick one compass bearing. Step outward from the camera through the elevation tiles, and at every step ask how high that ground appears from the lens, after subtracting how far the curved earth has fallen away. The highest apparent angle along the ray is the skyline in that direction.
- ray being marched
- skyline found in the photo
- terrain ridge at the encoder pose
- terrain ridge with the correction
Predict the ridge: the ray cast
Three things go in. Where the camera is. A handful of ordinary photos it already took, each stamped with the pan, tilt and zoom its motors reported. And a public elevation map — a — that gives the ground height at any point.
Pick one compass direction, say 144°. Start at the camera and walk outward in steps, 300 m, 600 m, a kilometre, on out to tens of kilometres. At every step look up the ground height and ask one question: from the lens, how many degrees above level would I have to look to see this spot? Close, low ground gives a negative angle; a tall ridge far away gives a positive one. Keep the biggest angle you met along the whole walk. That is the skyline in that direction, because everything behind it is hidden and everything above it is sky. Do it for every direction the photo covers and you have a curve: ridge height against compass bearing. It comes from geography alone.
Over 28 km the surface drops about 54 m beneath a level line, so a ridge that far off looks a tenth of a degree lower than flat-map arithmetic says — three pixels. Air bends light slightly back the other way, so the solver applies the standard surveying allowance (k = 0.13) too. Skip this and every distant camera gets a small false correction.
Find the ridge in the photo, then compare
Now the photograph. In each of 128 vertical strips, scan down from the top for the place where bright sky turns into darker land and stays land below it. A cloud fails that test, because there is sky again under it. The solver already knows roughly where the ridge should be from the ray cast, so it only searches a band around that, which stops it grabbing a rooftop or a foreground tree. It also measures how sharp the edge is: a crisp ridge scores 70 to 110, haze about 20, and anything under 28 is thrown away, because the top of a haze layer looks like a ridge and sits about a degree too high.
A camera is a pinhole: every pixel is one direction, fixed by the lens's field of view and by which way the camera is pointed. Using the pan and tilt the motors reported, each red ridge pixel becomes a direction, and its height angle is compared with the predicted ridge at the same bearing. One small number per strip: the gap, in degrees.
If the camera were pointed exactly where it reports, the gaps would scatter around zero. If the mount is tipped, every gap has the same offset. If the camera is rotated about its lens, the gaps grow steadily from one side of the frame to the other. So the middle value of the gaps is the pitch error, and the left-to-right slope is the roll. The compass error works the same way but sideways: a ridge has a shape, and if the predicted shape only matches after sliding it left or right, that slide is the pan offset — trusted only when views at least 15° apart ask for the same slide.
The map is bare earth. On a ridge 3 km off, 20 m of forest raises the skyline by 0.4°, as large as the error being hunted; at 30 km the same forest is 0.04°. Strips whose skyline is nearer than 15 km are left out (8 km if nothing farther exists).
Why it is allowed to say nothing
Some strips will always be wrong — a tower on the ridge, a wisp of cloud, a smoke column. An average is dragged by those; the median stays put until about half the strips are bad. And one frame proves little: for each view the camera regularly holds, the pipeline takes up to twelve photos spread over seven days, drops the hazy ones, and takes the per-strip median across the rest. A view also needs at least two photos that agree within half a degree, which rejects frames caught while the camera was still moving.
A wrong correction is worse than none, because it moves every peak label for every user. So a result is kept only when it clears all of these, and otherwise the camera is left as it was:
- at least 15 distant strips, spread over at least half the frame width;
- at least 80 % of them within 0.35° of the answer, with a typical scatter () of 0.2° or less;
- a correction no larger than 3° in pitch or pan and 2° in roll — bigger than that is bad data, not a bent bracket;
- removing any single view must not change the answer much.
Even then the solver never writes to production. It sends a proposal to the Admin app with a picture — the photo, red dots for the ridge it found, grey for the prediction before the correction, green for after — and a reviewer approves or rejects. What comes out per camera is a pan offset and a short table of pitch and roll against pan, because a leaning mast looks tipped by different amounts in different directions. The overlay adds the row for the nearest calibrated view before it asks where the peaks are.
One strip, with numbers
| Quantity | Value | Where it comes from |
|---|---|---|
| Lens height | 1,700 m | site elevation plus the mast |
| Highest ground along bearing 144° | 2,326 m at 28 km | walking the ray across the elevation map |
| Angle on a flat earth | +1.28° | 626 m up over 28,000 m out |
| Drop from earth's curve, less refraction | 54 m | distance² × 0.87 ÷ (2 × earth radius) |
| Predicted ridge angle | +1.17° | 572 m up over 28,000 m out |
| Ridge angle read from the photo | +0.55° | red dot, converted using the reported tilt |
| Gap for this strip | 0.62° | prediction minus photo |
| What 0.62° means on screen | 18 px | a 1920 px frame covering 65° is 29.5 px per degree |
| What 0.62° means on the ground | 303 m at 28 km | a peak label floating that far above its summit |
One strip proves nothing. Thirty distant strips that all say about 0.62°, on nine photos from different days, is a bent bracket.
Wrapping up
The mountains are fixed, the map is public, and a camera that is pointed where it says it is would put the ridge exactly where the ray cast predicts. When it does not, by one constant across the frame, that constant is the mount — measured, not guessed, and only from ridges far enough away that trees cannot fake it. Read next, for what happens when the frames are hazy, lying about their pose, or one frame short of trustworthy: The Overnight Boresight.