Where Gestures Become Degrees

There's an invisible layer between the operator and every camera in California. It's a transparent div stretched over the image — XYMouseTrackerOverlay, mounted by IImage/SignalRIImage — and it's the one licensed driver that turns everything the app wants into actual PTZ commands.

Open the file and you'll meet ~1,600 lines that read like a pile of handlers. Don't read it top to bottom. Instead, hold one distinction and the pile organizes itself: input arrives two ways — pointer gestures, which the overlay owns end to end, and store signals, wishes that other components post to the request board for the overlay to execute. This lesson takes each in turn, with the two live demos that make the math tangible.

Gestures

Three, and which one you get depends on cameraMovementMode from cameraMovementStore:

Gesture	Mode	Handler	Result
Double-click	either	handleDoubleClick	Camera centres on that point. Pano also forces zoom: 1
Drag	panTool	handleMouseUpOnSwipe	Pan/tilt by the drag delta — doubled on the pan axis
Drag	zoomTool	handleMouseUpOnZoom	Camera moves to the box's centre and zooms to the box's angular width

Everything is computed on mouseup; there is no incremental movement while dragging. During a drag the only live feedback is onDragChange, which writes a "+ 12.4" style string into the centre tick's label, and — in zoom-tool mode — a black rectangle drawn from transformMousePositionToRectangle.

Feel the math before reading about it — hover the frame below to watch pixels become degrees, drag to pan, and slide the zoom to see the same pixel change meaning:

Hover + drag-to-pan

// getFieldOfView (calculateZoom.ts) + transformMousePositionToImage
// (useMousePosition.tsx) + handleMouseUpOnSwipe (XYMouseTrackerOverlay.tsx).
const { useState, useRef } = React;
const SPEC = { max_fov: 65.1, min_fov: 2.0, sensorWidth: (1 / 2.8) * 25.4, focalMin: 4.25, focalMax: 170 };
const MAX_TILT = 20, MIN_TILT = -180, W = 640, H = 360, AR = 16 / 9;
const fovDeg = fl => 2 * Math.atan(SPEC.sensorWidth / (2 * fl)) * (180 / Math.PI);

function getFieldOfView(zoom) {
const ratio = (Math.max(1, Math.min(zoom, 9999)) - 1) / 9998;
const raw = fovDeg(SPEC.focalMin + (SPEC.focalMax - SPEC.focalMin) * ratio);
const rawMax = fovDeg(SPEC.focalMin), rawMin = fovDeg(SPEC.focalMax);
return SPEC.min_fov + ((raw - rawMin) / (rawMax - rawMin)) * (SPEC.max_fov - SPEC.min_fov);
}

// NOTE the horizontal divisor: H * AR is the IMAGE width, never the overlay width.
function toDegrees(mx, my, center, fov) {
const x = (mx / (H * AR)) * fov - fov / 2 + center.x;
const down = fov / AR;
const y = (1 - my / H) * down - down / 2 + center.y;
return { x: Math.round(x * 10) / 10, y: Math.round(y * 10) / 10 };
}

function Overlay() {
const ref = useRef(null);
const [zoom, setZoom] = useState(2500);
const [center, setCenter] = useState({ x: 137, y: -3 });
const [pix, setPix] = useState(null);
const [drag, setDrag] = useState(null);
const fov = getFieldOfView(zoom);
const toPix = e => { const r = ref.current.getBoundingClientRect(); return { x: e.clientX - r.left, y: e.clientY - r.top }; };
const deg = p => toDegrees(p.x, p.y, center, fov);
const hover = pix ? deg(pix) : null;
const live = drag && pix ? (drag.startDeg.x - deg(pix).x) * 2 : null;

const onUp = e => {
  if (!drag) return;
  const end = deg(toPix(e));
  // The !== 0 guard: an out-of-bounds resize returns { x: 0, y: 0 }.
  if (drag.startDeg.x !== end.x && drag.startDeg.x !== 0) {
    const pan = (drag.startDeg.x - end.x) * 2;  // doubled, ADO Backlog Item 683
    const tilt = drag.startDeg.y - end.y;
    setCenter({ x: center.x + pan,              // no clamp - Axis owns pan bounds
      y: Math.max(Math.min(center.y + tilt, MAX_TILT), MIN_TILT) });
  }
  setDrag(null);
};

const box = { padding: '8px 10px', borderRadius: 6, background: 'var(--ifm-color-emphasis-100)', fontSize: 13, fontFamily: 'var(--ifm-font-family-monospace)' };
const rule = (s, c) => <div style={{ position: 'absolute', background: c, ...s }} />;
const P = 'var(--ifm-color-primary)', E = 'var(--ifm-color-emphasis-500)';

return (
  <div style={{ display: 'grid', gap: 12 }}>
    <label style={{ fontSize: 13 }}>
      Zoom {zoom} → {fov.toFixed(2)}° across, {(fov / AR).toFixed(2)}° down
      <input type="range" min="1" max="9999" value={zoom} style={{ width: '100%' }} onChange={e => setZoom(Number(e.target.value))} />
    </label>
    <div ref={ref}
      onMouseMove={e => setPix(toPix(e))}
      onMouseLeave={() => { setPix(null); setDrag(null); }}
      onMouseDown={e => setDrag({ startDeg: deg(toPix(e)) })}
      onMouseUp={onUp}
      style={{ width: W, height: H, maxWidth: '100%', position: 'relative', overflow: 'hidden',
        border: '1px solid var(--ifm-color-emphasis-300)', borderRadius: 8,
        background: 'var(--ifm-color-emphasis-200)', cursor: drag ? 'grabbing' : 'crosshair', userSelect: 'none' }}>
      {rule({ left: '50%', top: 0, width: 1, height: '100%' }, E)}
      {rule({ top: '50%', left: 0, height: 1, width: '100%' }, E)}
      {pix && rule({ left: pix.x, top: 0, width: 1, height: '100%' }, P)}
      {pix && rule({ top: pix.y, left: 0, height: 1, width: '100%' }, P)}
      <div style={{ position: 'absolute', left: 8, top: 8, ...box }}>
        center {center.x.toFixed(1)}°, {center.y.toFixed(1)}°
        {live !== null && <div>drag Δpan (×2): {live >= 0 ? '+' : ''}{live.toFixed(1)}°</div>}
      </div>
    </div>
    <div style={box}>{hover ? 'pixel (' + pix.x.toFixed(0) + ', ' + pix.y.toFixed(0) + ') → pan ' + hover.x.toFixed(1) + '° tilt ' + hover.y.toFixed(1) + '°' : 'hover the frame'}</div>
  </div>
);
}
render(<Overlay />);

Three things to try:

• Drag right, then left. center.x moves twice as far as the pointer did. That × 2 is the single most surprising line in the handler, and it is deliberate — ADO Backlog Item 683.
• Drag down repeatedly until tilt pins at -180. Nothing warns you; the clamp is silent, and the camera simply stops responding to further downward drags.
• Slide zoom to 9999 (FOV 2.0°) and hover the same pixel. It is now a completely different angle — which is why every consumer of this transform has to be handed the camera's current zoom rather than a cached one.

Three things to try:

• Drag right, then left. center.x moves twice as far as the pointer did. That × 2 is the single most surprising line in the handler, and it is deliberate — ADO Backlog Item 683.
• Drag down repeatedly until tilt pins at -180. Nothing warns you; the clamp is silent.
• Slide zoom to 9999 (FOV 2.0°) and hover the same pixel. It is now a completely different angle — which is why every consumer of this transform must be handed the camera's current zoom, never a cached one.

Two guards sit in front of every drag path:

dragStartLocation.x != mouseLocation.x && dragStartLocation.x !== 0;

The second one is load-bearing. transformMousePositionToImage returns the sentinel { x: 0, y: 0 } when the pointer is outside the overlay, and a container resize can produce exactly that — so without the !== 0 check a window resize mid-drag fires a real camera movement. Do not "clean up" that comparison.

Tilt is clamped to MAX_TILT = 20 / MIN_TILT = -180 so the camera cannot be aimed into the black space above its mount. Pan is deliberately unclamped — Axis enforces pan limits camera-side, and adding a client-side clamp would fight it.

The drag-to-zoom clamp

Zoom-tool mode is not a rectangle crop. handleMouseUpOnZoom takes the horizontal drag distance in degrees and treats it directly as the target field of view:

const panMovement = dragStartLocation.x - mouseLocation.x;
const newZoom = getZoomFromFov(
  cameraSelected ? cameraSelected?.cMo : 'Q6075-E',
  -1 * panMovement,
);

Dragging right makes panMovement negative, so the FOV handed over is positive and the camera zooms to that span. Dragging left hands it a negative FOV — and getZoomFromFov opens with Math.max(min_fov, Math.min(fov, max_fov)), which clamps any negative input straight to min_fov. A leftward drag therefore always commands maximum zoom, whatever the distance.

Run both directions and compare newZoom:

handleMouseUpOnZoom, both drag directions

// getFieldOfView / getZoomFromFov (util/cameras/calculateZoom.ts)
// and handleMouseUpOnZoom (XYMouseTrackerOverlay.tsx).
const SW = (1 / 2.8) * 25.4, FMIN = 4.25, FMAX = 170, MAXF = 65.1, MINF = 2.0;
const fovDeg = fl => 2 * Math.atan(SW / (2 * fl)) * (180 / Math.PI);
const rawMax = fovDeg(FMIN), rawMin = fovDeg(FMAX);
const r2 = n => Math.round(n * 100) / 100;

const getFieldOfView = zoom => {
const raw = fovDeg(FMIN + (FMAX - FMIN) * ((Math.max(1, Math.min(zoom, 9999)) - 1) / 9998));
return MINF + ((raw - rawMin) / (rawMax - rawMin)) * (MAXF - MINF);
};

const getZoomFromFov = fov => {
const clamped = Math.max(MINF, Math.min(fov, MAXF)); // negative fov -> min_fov
const raw = rawMin + ((clamped - MINF) / (MAXF - MINF)) * (rawMax - rawMin);
const focal = SW / (2 * Math.tan((raw * Math.PI) / 360));
return Math.round(1 + ((focal - FMIN) / (FMAX - FMIN)) * 9998);
};

const drag = (start, end) => {
const panMovement = start.x - end.x;
const newZoom = getZoomFromFov(-1 * panMovement);
return { panMovement: r2(panMovement), fovHandedIn: r2(-1 * panMovement), newZoom,
  resultingFov: r2(getFieldOfView(newZoom)),
  newCenter: [r2((start.x + end.x) / 2), r2((start.y + end.y) / 2)] };
};

// ---- edit these ----
const START = { x: 137.0, y: -3.0 };
const RIGHT = { x: 140.5, y: -3.4 };
const LEFT  = { x: 133.5, y: -3.4 };
// --------------------

render(<pre>{JSON.stringify({
draggedRight: drag(START, RIGHT),
draggedLeft: { ...drag(START, LEFT), note: 'clamped to min_fov, i.e. maximum zoom' },
}, null, 2)}</pre>);

Output

Digital zoom is unrelated to all of this: addDigitalZoom derives a CSS scale factor (1 + zoomChange / 2000) from the slider and never calls the PTZ API. Every PTZ call, successful or not, resets it to 1.

Store signals: the request board in action

Four things move the camera without anyone touching the overlay. Each is a store value the overlay watches in a useEffect:

Store value	Raised by	Effect
shouldMoveCurrCamToHome	Home button	moveToHome() — reason code Camera Moved Home, no modal
aiAlertMovementTarget	AI smoke alert	Moves to the alert's pan/tilt/zoom
activityLogMovementPayload	Activity log replay	Re-sends a historical payload verbatim
newZoom	Zoom slider	handleZoomChange — keeps current pan/tilt, changes zoom

Two of these are guarded with usedAsOverlayFor === 'camera', and the other two with addImgDragEvents. Both guards exist for the same reason: when the 60° image and the 360° pano are on screen together, two overlays are mounted, and an unguarded effect fires two commands and opens two modals. Any new store-driven movement needs the same guard.

The lease decision

Every movement funnels through moveCamera(payload, isLeaseConfirmed), which asks the backend who holds the camera before deciding whether the reason-code modal is required:

const allowCamMove =
  isLeaseConfirmed || // user already confirmed
  leasedBySelf || // we hold it
  (freshRecentMovementPayload != null && !leasedByAnother); // we have a template, nobody blocks

freshRecentMovementPayload is read with useCameraMovementStore.getState() after the await, not from the closure — the lease check is async, and the closed-over value is stale by the time it resolves. When the modal is needed, the payload is stashed in pendingModalPayloadRef before the modal opens, so a second interaction cannot overwrite what the user is about to confirm.

If the move is allowed, moveCamera still branches one more time: shouldApplyBrightness routes to adjustBrightness, shouldApplyFocus to adjustFocus, and only otherwise to callMoveCamera. The focus and brightness panels never call an API themselves.

callMoveCamera rounds zoom to an integer, POSTs, and on any outcome — success, a false body meaning the camera did not move, or a rejection — calls removeDigitalZoom(). A false body is not an error path from the transport's point of view, so it is easy to miss.

Touch

handleTouchEnd re-implements both drag handlers rather than delegating to them, and the duplication is intentional: it converts the final touch point to degrees itself (endDeg) and uses that value directly, instead of reading mouseLocation, which the throttled window listener may not have updated yet. If you change the pan-doubling or the tilt clamp, change it in both places — the touch copy will not follow.

handleTouchMove calls preventDefault() only when signed in, so page scrolling still works over the image for anonymous users.

Wrapping up

One invisible layer, two kinds of input. Gestures compute on mouseup through pure math you can now literally feel; store signals arrive from the request board, each guarded against the two-overlays problem; and every path funnels through moveCamera, where the lease question is asked fresh every time. When a movement misbehaves, first ask: gesture or signal?

Last stop in this chapter: the formula reference behind every number you just dragged.