# Hierarchy-T5 20k: full versus missing-level inputs

The same trained20k model is evaluated with all levels present, each single
level masked, and each pair masked. Twelve fixed validation cases, exact matched
initial observations/proprio/seeds,1/3/8 chunks, GT or generated feedback.
The training function replaces L3 with `<extra_id_0>`, L2 with `<extra_id_1>`,
and L1 with `<extra_id_2>`. Robot metadata and other levels remain unchanged.
Missing levels are not supplied to the encoder. The auxiliary decoder does not
generate replacement text during this evaluation. There is no new training.

Download both `load_inputs.py` and `load_masked_inputs.py`, install numpy, run:

```sh
python load_masked_inputs.py --case case15 --pattern mask_l1 --chunk 0
python load_masked_inputs.py --case case15 --pattern mask_l2_l3 --chunk 7
```

`--base` accepts a local site folder or a deployed base URL. The helper provides
GT-conditioned inputs and scoring targets. For generated feedback, only the
first call uses these real inputs. Later image history and proprio must come
from your own predictions; never feed downloaded future GT as generated history.
The helper's `generated_feedback_history` reproduces previous-sample-hold image
selection. Proprio becomes the previous prediction's final normalized EE16,
masked by the initial validity mask. Prompts follow the recorded segment schedule.

Times use25Hz episode-relative ticks, independent of native video FPS/offsets.
Condition images are nine uint8 RGB composites: fixed frame0 plus eight frames
from max(0,t-175) through t in25-tick intervals. Use exact lossless NPZ pixels;
MP4s are display-only. The normalized tensor is RGB/127.5-1, shape[3,9,240,416].
Every chunk predicts eight RGB frames at t+4,...,t+32 and32 observed EE16 states
at t+1,...,t+32. Advance32ticks=1.28s;8chunks cover10.24s.

EE16 layout is left xyz(m),qwqxqyqz,gripper followed by right counterparts.
Use supplied valid masks and per-tree mean/std. Current proprio is
where(valid,(state-mean)/std,0); scoring uses physical xyz over valid dimensions.
These are future observed end-effector states, not raw motor commands. No common
world coordinate frame or universally valid quaternion channels are assumed.

Each case JSON contains source dataset/episode IDs, packed-camera offsets,
sampling times, SHA256s, lossless image packs and state arrays. Actual seven-way
prompts/seeds are in `masked-prompts.json`; full prompt fields in case.json are
the unmasked reference. The webpage also offers each condition's action NPZ.

Model: T5LoRA128 + VideoLoRA128/fullAction, hierarchy stage20k after Shared-FM40k.
Inference: video10 -> zero-timestep clean latent cache -> action10, shift3.
Seed=42+selected_case_ordinal*100+chunk_index, shared across conditions.
The complete-input rerun matches prior20k evaluation actions exactly, and all
72 full-input decoded videos are checked against that earlier evaluation.

Interpretation: errors against recorded GT measure recovery of the recorded
future with less text. Missing instructions can permit multiple reasonable
behaviors. Prediction differences and lower GT error do not establish semantic
task success. Twelve selected cases are not a representative benchmark.