Trial windows#

Every model in Ethograph learns from and predicts over trials. A trial is one window of the session: the stretch of time you navigate to, curate, and that a model reads as one input. How to store and align trials is covered in Trials and Media alignment (w. neuroconv); this page is about where to draw the windows so they suit a model.

Natural trials#

If your behaviour already comes in trials — a pellet is dispensed, a stimulus plays, an animal enters an arena — or your recording system is triggered, writing one file per trial, the trial is already there. Carve your trials out this way in alignment.nwb (Media alignment (w. neuroconv)).

A free-running camera, as one file or split into several

Free-running camera. Figure from neuroconv’s how-to (BSD-3-Clause).#

A triggered camera, one file per trial

Triggered camera. Figure from neuroconv’s how-to (BSD-3-Clause).#

Passing these trials to a model as its samples is a natural fit, both for action segmentation, where a sample is one (trial, individual) pair, and for event spotting, which shares the same sessions, trial filter and train/val/test split drawn by whole trial. (E2E-Spot then draws its clips of context_s inside each trial; see Every temporal setting is a duration.) For event spotting the trial is also the unit of the prediction rule — one event per class per trial, so cut the trials that way (From probabilities to events).

Both read a trial as its window of the video, not the video file: a trial carved out of a longer video gives the model only the frames between its start and stop, and a video file that already matches its trial is read whole.

A free-running system that writes several files at fixed intervals (the third row of the free-running figure, part_01.mp4, part_02.mp4, …) but has no defined trials may also already give you a natural split: one file, one trial.

One long recording#

A single recording lasting an hour, or a whole day, is not a good sample. It may not fit in memory for the model, and with one sample per recording the gradient updates come rarely. Divide the recording into trial windows instead, and each window is passed to the model on its own.

The right length depends on the model, but we recommend windows of around 1024, 2048 or 4096 frames. In seconds this differs a lot between datasets: 4096 frames is over two minutes of a slow behaviour filmed at 30 fps, but only about 20 s of a crow’s tool use filmed at 200 fps. Build the windows with from_continuous() or as the trials table of the alignment.

Why non-overlapping windows#

Tools such as DLC2Action split a recording into much shorter segments (e.g. 256 frames; Kozlova et al., 2025, Figure S5) that overlap. The clear advantage is that a behaviour cut at the edge of one segment is seen whole in the next.

We take a simpler approach: larger windows with no overlap, following the convention of the temporal action segmentation literature [Ding et al., 2024] (see also awesome-temporal-action-segmentation). This

  • keeps new architectures easy to plug in, since they expect exactly this input;

  • avoids majority voting or averaging over overlapping frames;

  • makes the context a model sees the same as the trial a user curates.

The cost is that an action can be bisected by a window boundary. Don’t make trial windows too small (e.g. 256 frames): the shorter the window, the more often an action is cut in two.