Models

Once you have labelled some trials by hand, a model can label the rest — and you then curate what it predicted. Which model depends on the shape of the label and on what data exists when the model runs.

        flowchart TD
    start([Hand-labelled trials]) --> shape{What kind of label?}

    shape -->|"State event<br/>(an interval: onset + offset)"| segment[Action segmentation<br/><code>eto.segment</code><br/><i>DLC2Action models + added architectures</i>]
    shape -->|"Point event<br/>(one moment per trial)"| inference{What is available<br/>when the model runs?}

    inference -->|"Pose / time-series features,<br/>and you want to stay in the GUI"| lgbm[LightGBM model<br/><i>Model ▸ LightGBM: Train… / Predict…</i>]
    inference -->|Video only| spot[Event spotting: E2E-Spot<br/><code>eto.spot</code>]
    inference -->|Video + pose features| spotfeat[E2E-Spot + features]
    inference -->|"Video now,<br/>pose only for training"| distil[Pose teacher,<br/>distilled into E2E-Spot]

    segment --> curate
    lgbm --> curate
    spot --> curate
    spotfeat --> curate
    distil --> curate
    curate([Curate the predictions in the GUI]) --> conf[Read and threshold confidence]

    click segment "segment/index.html"
    click lgbm "onset_model.html"
    click spot "spot/index.html"
    click spotfeat "spot/multimodal.html"
    click distil "spot/multimodal.html"
    click curate "curation.html"
    click conf "confidence.html"
    

Where the models come from

• LightGBM [Ke et al., 2017].

• Action segmentation — eto.segment vendors the models and loss of DLC2Action [Kozlova et al., 2025], adapted there to pose/kinematic input:

  • DLC2Action’s own variants: mstcn (MS-TCN3, which feeds the last two layers of the first stage into the second; from MS-TCN++ [Li et al., 2020]) and c2f_transformer (C2F-TCN [Singhania et al., 2021] with attention in place of convolution).

  • Original architectures, as adapted in DLC2Action: asformer [Yi et al., 2021], c2f_tcn [Singhania et al., 2021], edtcn [Lea et al., 2017], motionbert [Zhu et al., 2023], and mlp, a per-frame baseline.

  • Added in EthoGraph: rnn (a bidirectional GRU/LSTM baseline), specscalpel [Ji et al., 2026] and lady [Ji et al., 2026].

• Precise event spotting — E2E-Spot [Hong et al., 2022].