Models

Once you have labelled some trials by hand, a model can label the rest — and you then curate what it predicted. Which model depends on whether you have a GPU, on the shape of the label, and on what the model sees.

        flowchart TD
    gpu{Do you have<br/>a GPU?}
    gpu -->|No| lgbm[<b>LightGBM</b><br/>point events, CPU<br/><i>Model ▸ LightGBM</i>]
    gpu -->|Yes| shape{What kind<br/>of label?}

    shape -->|"Point event<br/>(one moment)"| input{What does the<br/>model see?}
    shape -->|"State event<br/>(onset + offset)"| big{A large GPU<br/>or a cluster?}

    input -->|Video only| spot[<b>E2E-Spot</b><br/><code>eto.spot</code>]
    input -->|"Video + pose, sensors,<br/>custom features…"| spotfeat[<b>E2E-Spot</b><br/>+ features]

    big -->|Yes| sees{What does<br/>FERAL see?}
    sees -->|Video only| feral[<b>FERAL alone</b><br/><code>export_feral</code>]
    sees -->|"Video + pose, sensors,<br/>custom features…"| feralseg[<b>FERAL embeddings</b><br/>as a video feature]
    big -->|No| segment[<b>Action segmentation</b><br/><code>eto.segment</code>]
    feralseg --> segment

    click segment "segment/index.html"
    click lgbm "onset_model.html"
    click spot "spot/index.html"
    click spotfeat "spot/multimodal.html"
    click feral "segment/feral.html"
    click feralseg "segment/feral.html"
    

Every box is a link. Whichever model you pick, its predictions come back to the GUI: curate them, then read and threshold their confidence.

Model	Label	Description
LightGBM	point event	Simple to use, in the GUI, on a CPU. Fed a few hand-crafted time-series features.
E2E-Spot (eto.spot)	point event	Simple to use: it learns end to end from the video, on a GPU. For when you want a few alignment points per trial — the frame a beak touches the tool, a lever press, the first step of a reach.
Action segmentation (eto.segment)	state event	For action intervals — a bout of grooming, sniffing or tool use movement, each with an onset and an offset — and have pose data. Good in a lab environment, where a calibrated camera setup gives robust features.
FERAL alone	state event	Fine-tunes the V-JEPA 2 video foundation model. Can learn individual and collective behaviours directly from video, even in naturalistic setups. Requires a heavy GPU.
FERAL embeddings	state event	The best of two above: FERAL’s embedding of the video, read by eto.segment beside your pose features.

Where the models come from

• LightGBM [Ke et al., 2017] — runs on a CPU and lives entirely in the GUI: no scripting, and no separate Python environment. The GUI install (uv tool install "ethograph[gui,audio]", see Installation) is all it needs.

• Action segmentation — eto.segment vendors the models and loss of DLC2Action [Kozlova et al., 2025], adapted there to pose/kinematic input:

  • DLC2Action’s own variants: mstcn (MS-TCN3, which feeds the last two layers of the first stage into the second; from MS-TCN++ [Li et al., 2020]) and c2f_transformer (C2F-TCN [Singhania et al., 2021] with attention in place of convolution).

  • Original architectures, as adapted in DLC2Action: asformer [Yi et al., 2021], c2f_tcn [Singhania et al., 2021], edtcn [Lea et al., 2017], motionbert [Zhu et al., 2023], and mlp, a per-frame baseline.

  • Added in Ethograph: rnn (a bidirectional GRU/LSTM baseline), specscalpel [Ji et al., 2026] and lady [Ji et al., 2026].

• Precise event spotting — E2E-Spot [Hong et al., 2022].

• FERAL [Skovorodnikov et al., 2025] — a video foundation model (V-JEPA 2) fine-tuned on the pixels alone, in its own environment on a large GPU. Either on its own, with the GUI’s labels exported to its labels.json, or as a video feature whose per-frame embedding eto.segment reads next to changepoint and pose columns (FERAL).