HiFiEgo: High Fidelity Egocentric Video
Affordance Grounding Dataset

Embodied AI demands more than recognizing objects: it requires knowing precisely where to make contact, down to the fingertip, to enable dexterous manipulation. Despite growing interest in affordance grounding, the field lacks evaluation infrastructure capable of measuring this physical precision, with existing benchmarks relying on coarse heatmaps that conflate object-level localization with contact-level accuracy. We introduce HiFiEgo, a large-scale egocentric benchmark with over 20K manually annotated interaction events, each with temporally anchored contact frames, pixel-precise contact point labels, bounding boxes, and semantic action labels, sourced from unconstrained real-world egocentric videos. Using HiFiEgo, we expose a systematic physicality gap: state-of-the-art affordance methods that achieve up to 38% accuracy under coarse spatial thresholds collapse to roughly 1% at contact-level precision. We further provide the first standardized evaluation of egocentric data mining pipelines across three tasks, affordance grounding, contact frame localization, and backward label propagation, and show that compounding errors across pipeline stages prevent the pixel-level precision needed for reliable automatic supervision.