Reference implementation · FH-Ego v1
One clip. Every layer. Every number checkable.
This is the complete annotation output of a single real egocentric recording, produced by our pipeline, not a mock-up and not a best case. Every figure below is computed from the JSON at build time, and the file itself is downloadable so you can verify each one yourself.
Five containers ship. MCAP and WebDataset do not. What arrives, and in what container.
Every figure above is computed from reference-run.json at build time. Two are derived: the mean is an aggregate of the hand-tracking model's own confidence output, and the segment count follows from fixed four-second windows rather than from boundaries found in the footage.
Download the full annotation (JSON)
Verify it yourself, in about thirty seconds
The page above says the file is downloadable so you can check our figures. That is only worth saying if we tell you how, so: here is the digest of the exact file that button serves, and the three commands that pull our own headline numbers back out of it.
shasum -a 256 reference-run.json
ce6591895a1d3b2071bcf4b8c11b3575b3385d03aefc99c531d7be72a1edee7a reference-run.jsonjq '.actions | length' reference-run.json # 16
jq '.tracks.kinematics_3d | length' reference-run.json # 294
jq -r '.provenance.caption_source' reference-run.json # heuristic_stub
The third returns heuristic_stub: the file's own record that its action
track and captions came from a template.
If a number on this page disagrees with the file, the file is right.
How to read this clip
Scope: one clip. It shows the pipeline running end to end on a single recording.
Confidence is the model's own score. The values below come
from the hand-tracking model, and they are one score per detected hand, not per
joint: keypoints_21 holds x, y and z and nothing else.
The action track follows fixed windows. On this clip
the segment boundaries are fixed four-second windows, the verbs cycle a closed
five-item list, and scene_semantic and camera_motion are
single constants. The artifact records this about itself:
provenance.caption_source is heuristic_stub, and
provenance.derived_fields names all nine generated fields.
Dense captions are derived. The schema labels every generated language layer as derived.
Confidence on this run
Computed from the same file you can download. We publish the floor, not just the mean, because worst case is what determines whether a dataset is usable.
Confidence distribution, computed from reference-run.json
| Measure | Value |
|---|---|
| Sampled frames carrying a hand below 0.70 confidence | 44 of 185 (23.8%) |
| Entries below 0.70 confidence (one entry per hand) | 45 of 294 (15.3%) |
| Lowest per-hand confidence observed | 0.508 |
| Median per-hand confidence | 0.9415 |
| Highest per-hand confidence observed | 0.995 |
| Kinematic coverage of total frames | 10.0% (the pipeline emits keyframes, not every frame) |
| Hand distribution | left: 183 · right: 111 |
How to read this. Roughly one sampled frame in four on this run carries a hand below 0.70 confidence, concentrated where the hand is partially occluded. For a buyer that is the number that matters: it sets the filtering threshold you would apply before training, and it is why we ship confidence per frame rather than a single quality score.
Four frames from this run
The renders below are pipeline output, not illustrations. Each overlay is drawn by the engine from the same JSON you can download: frame index, action code, hand, detection confidence and the camera-relative position vector. The kinematics track is sampled every ten frames, so each caption names both the frame that was rendered and the nearest sampled entry behind it (two different numbers, with two different confidences), which is why we never print one under the other's label. The callouts on each still, and the readout a mouse pointer brings up, say exactly what the caption says; nothing is behind the hover.
Frame 1197 shows a low-confidence hand. The right hand is tracked at 0.60 and the sampled entry behind it at 0.548, under the 0.70 floor in the table above, because the hand is partially behind the machine head. The overlay still draws it, and the confidence value is what tells you not to trust it. That is the whole argument for shipping confidence per frame rather than one quality score.
The three layers, from this run
Tier 1: session
{
"task_id": "20260722_Egocentric_Sample_2_task_1",
"task_label": "T0_Other",
"scene": "Industrial_Floor",
"usable": true,
"start_frame": 0,
"end_frame": 1843,
"fps": 30
}Tier 2: actions
16 action segments in this run; the first 8 shown
Read this table as schema shape. On this clip the boundaries are fixed four-second windows and the verbs cycle a closed five-item list; see how to read this clip.
| Frames | Hand | Action | Description |
|---|---|---|---|
| 0–119 | left | grasp_object | The left hand operates sewing machinery and reaches forward to grasp fabric. |
| 120–239 | left | organize_items | The left hand aligns and organizes fabric items on the workstation surface. |
| 240–359 | both | wipe_surface | Both hands hold material cloth and wipe the surface in smooth motions. |
| 360–479 | both | inspect_component | Both hands pick up the component to inspect alignment and seam orientation. |
| 480–599 | both | place_item | Both hands place the processed item back into the designated workstation tray. |
| 600–719 | left | grasp_object | The left hand operates sewing machinery and reaches forward to grasp fabric. |
| 720–839 | left | organize_items | The left hand aligns and organizes fabric items on the workstation surface. |
| 840–959 | both | wipe_surface | Both hands hold material cloth and wipe the surface in smooth motions. |
Verbs observed across all 16 segments: grasp_object,
inspect_component, organize_items, place_item,
wipe_surface.
Tier 3: timeline tracks
Entry counts per track in this run
| Track | Entries |
|---|---|
| hand_control | 5 |
| object_interactions | 5 |
| action_verbs | 5 |
| dense_captions | 16 |
| kinematics_3d | 294 |
One kinematic entry, verbatim
This is the shape the coordinate data on this page is read from: 294 of these, one per
detected hand, on 185 frames sampled every tenth, in tracks.kinematics_3d.
{
"frame_idx": 0,
"timestamp_sec": 0.0,
"hand": "left",
"position_3d_m": {
"x": -0.384,
"y": 0.282,
"z": 0.356
},
"velocity_3d_mps": {
"vx": 0.0,
"vy": 0.0,
"vz": 0.0
},
"confidence": 0.889,
"keypoints_21": [
{
"x": 0.1801,
"y": 0.8129,
"z": 0.0
},
{
"x": 0.2565,
"y": 0.8872,
"z": -0.0038
},
{
"x": 0.3251,
"y": 0.8668,
"z": 0.0054
},
{
"x": 0.3645,
"y": 0.7889,
"z": 0.0156
},
{
"x": 0.3798,
"y": 0.7276,
"z": 0.0259
},
{
"x": 0.35,
"y": 0.7193,
"z": 0.0082
},
{
"x": 0.3903,
"y": 0.6318,
"z": 0.0165
},
{
"x": 0.411,
"y": 0.5881,
"z": 0.0228
},
{
"x": 0.4252,
"y": 0.5607,
"z": 0.0275
},
{
"x": 0.3191,
"y": 0.6446,
"z": 0.0119
},
{
"x": 0.3648,
"y": 0.5688,
"z": 0.0269
},
{
"x": 0.3939,
"y": 0.5386,
"z": 0.0363
},
{
"x": 0.409,
"y": 0.516,
"z": 0.0426
},
{
"x": 0.2807,
"y": 0.6066,
"z": 0.0145
},
{
"x": 0.3157,
"y": 0.5379,
"z": 0.0167
},
{
"x": 0.3174,
"y": 0.5457,
"z": 0.0163
},
{
"x": 0.3084,
"y": 0.5569,
"z": 0.0191
},
{
"x": 0.2441,
"y": 0.5904,
"z": 0.0171
},
{
"x": 0.2671,
"y": 0.542,
"z": 0.0113
},
{
"x": 0.2677,
"y": 0.5459,
"z": 0.0084
},
{
"x": 0.2591,
"y": 0.5593,
"z": 0.0104
}
]
}Provenance: the part you can audit
A delivered clip must carry an active consent receipt in a SHA-256 ledger with revocation semantics: package assembly refuses one that does not. A receipt records who consented, when, on which device, under which DPDP policy, and whether consent was later withdrawn; and the hash lets you verify the record has not been altered.
Consent receipt: none — recorded before the consent flow