Roadmap#

What Ethograph already does, what is in progress, and what is still open. Grouped by theme; within each theme items run roughly in the order they were (or will be) tackled.

Legend: ✅ done · 🚧 in progress · ⬜ not started

Milestones#

When

Milestone

April 2026

Movement community call demo (slides)

2026

Segmentation pipeline, lightgbm models, pixel event spotting, curation workflows

September 2026

Ethograph demo at the Hahnloser lab

Next

Shared feature schema with movement, NWB video alignment upstream

1. Interop with segmentation models#

  • Import predictions from action segmentation models (DLC2Action, ASFormer, MS-TCN): File → Import predictions… turns per-trial (T, n_classes) or (T,) arrays into labels with a confidence overlay (1 − normalised entropy)

  • Scripted segmentation pipeline (ethograph.segment): materialise → search → cross-validate, predictions written as GUI label files

  • lightgbm models for point events, with confidence read off the curve

  • Pixel event spotting from video (ethograph.spot)

  • Curation of model output: the Curation section, grids and saved workflows

  • 🚧 A shared schema for segmentation feature data with movement (movement#978)

2. Aligning video and data streams via .nwb#

  • Read and edit alignment directly in .nwb sources; .ethograph/alignment.nwb sidecars for everything else

  • 🚧 Video alignment in NWB tooling upstream (nwb-video-widgets#34, nwb-schema#677)

3. Changepoints#

  • Fast changepoint detection (gradient-, RMS-based, …) and changepoint correction of label boundaries

  • Changepoint features (more_changepoint_features()), which massively improved fine-grained accuracy for ASFormer

  • Changepoint features available to every segmentation model (features.changepoint_features in the segment pipeline)

  • Changepoint features for pynapple / NWB sessions (currently .nc only), e.g. to segment extracellular recordings (sharp-wave ripple detection) or other neural time series

  • ML-based changepoint detection. It must stay reproducible so it is a reliable feature. Note: post-model changepoint correction sometimes makes things worse, since the transformer learns a better representation than simple gradient-based methods.

  • 🚧 Audio changepoints (ethograph.features.audio_changepoints)

4. Neural data#

  • Interactive PSTH (ethograph.gui.widgets_psth)

  • Burst detection module to find behavioural periods of interest from neural data (pynapple#668)

  • Single-trial neural dimensionality reduction; visualise label segments in latent space

5. Make ethograph fast#