Guide/A test suite for working with miniscope data
Format: Tutorial
Audience: Intermediate
| Review interval | 6 months |
| Status | No reviews recorded yet — needs first review |
This guide is a map of an open, end-to-end toolkit for working with one-photon (1p) Miniscope data: going from a raw movie to single-cell footprints and calcium traces, and, just as importantly, knowing how much to trust them. It ties together two packages maintained alongside the Miniscope project: Minisim, a physical forward model that generates recordings with exact ground truth, and Minian, the analysis pipeline that recovers cells from a recording. Used together they form a small test suite: you can simulate data whose answer you already know, run the analysis on it, and measure exactly what the pipeline recovered before you ever point it at your own experiment.
If you are new to what a 1p recording actually contains, start with Anatomy of a Miniscope recording, and for the choices that shape a good dataset see Data processing considerations. This guide assumes you have (or will soon have) recordings of your own and want a dependable path from those files to results.
The idea: a forward model and its inverse
An analysis pipeline runs backward: it takes a movie and tries to reconstruct the neurons that produced it. That inverse problem is hard for 1p data, because the optics blur and dim cells, neighbors overlap, the brain moves, and the background is neither flat nor static (all of this is the subject of Anatomy of a Miniscope recording). minisim runs the forward direction instead: it starts from cells, optics, tissue, motion, and a camera, and builds the movie, so the true footprints, traces, spike times, and positions are known at every step. MiniAn is the inverse. Putting them back to back closes the loop, which is what lets you grade an analysis rather than just eyeball it.
Step 1: Understand what you are analyzing
Before processing anything, it pays to know which features of a 1p movie come from which physical step, because that tells you what to fix at the rig and what the analysis has to undo. The Anatomy of a Miniscope recording guide walks through each one (out-of-focus glow, depth blur and dimming, neuropil and vasculature, cross-talk, brain motion, uneven illumination, sensor noise), every figure simulated with minisim. To explore it hands-on, the 01_anatomy interactive notebook builds a recording one stage at a time with live sliders for depth, light, motion, and noise:
pip install "minisim[notebook]" minisim-notebooks copy 01_anatomy jupyter lab minisim-notebooks/01_anatomy/01_anatomy.ipynb
Step 2: Generate test data with known ground truth (minisim)
The build_recording studio notebook is minisim's data-generation tool. Where 01_anatomy isolates one effect at a time to teach, the studio exposes every knob at once so you can dial in a recording that matches your preparation and write it to disk. It has three interactive panels that share one configuration: an anatomy and scope panel (optics, sensor, tissue, cell placement, vasculature, illumination, vignette, glow, and brain motion) with a live example frame and a depth side-view; a neural activity panel (the calcium model) with live ground-truth traces; and a generate panel that writes the full recording. One-click presets seed a realistic starting point, including Miniscope V4 imaging CA1 and Miniscope V4 imaging cortex layer 2/3.
What makes the output a test dataset is that it carries its own answer. Saved as a zarr directory, a recording contains the movie and the ground truth: the true spatial footprints (A), calcium traces (C), spike trains (S), cell positions and depths, which cells are even physically recoverable, the vasculature mask, and the exact specification that produced it. You can also export a plain grayscale AVI that looks like a real msCam movie, which is what you feed to an analysis pipeline that expects raw Miniscope video.
minisim-notebooks copy build_recording jupyter lab minisim-notebooks/build_recording/build_recording.ipynb
Step 3: Analyze with the MiniAn pipeline
Minian is an analysis pipeline and visualization tool for Miniscope data, inspired by CaImAn and MIN1PIPE. It takes raw msCam video and produces denoised single-cell footprints and deconvolved traces. The pipeline runs in stages, each with interactive HoloViews visualizers for tuning parameters against your own data: preprocessing (removing the smooth glow and background, denoising), motion correction (estimating and undoing the brain's jitter and drift), initialization (seeding candidate cells by peak-to-noise ratio), and CNMF (the spatial and temporal updates that demix overlapping cells into separate footprints and traces). Demixing is the essential step for 1p data: because blur and scatter make neighboring cells overlap, a hand-drawn ROI averages a cell together with its neighbors and the neuropil, whereas CNMF models the movie as a sum of footprints and time courses and solves for both at once.
Install Minian from pypi and copy the pipeline notebook out to a working directory:
pip install minian minian notebooks copy pipeline minian data download pipeline-demo jupyter notebook minian-notebooks/pipeline/pipeline.ipynb
Full command and option details are in the Minian CLI reference and the Minian documentation.
Step 4: Hosted demo data
You do not need your own recording to get started. Minian's demo datasets are hosted on Zenodo (each with a citable DOI), downloaded on first use, cached to your machine, and verified against a checksum on every access, so nothing large lives in the repository and everyone runs the same bytes. The pipeline notebook fetches its demo recording automatically on first run; you can also browse and pull datasets directly:
minian data list # show datasets and sizes minian data download pipeline-demo # fetch + cache (Miniscope V3 CA1, ~688 MB) minian data path pipeline-demo # print the local path (fetching if needed)
The current datasets are pipeline-demo (a mouse hippocampal CA1 recording on a Miniscope V3, which drives the pipeline notebook) and cross-reg-sessions (multiple sessions for cross-registration, below).
Step 5: Close the loop (validate the analysis)
This is the step that turns the two packages into a test suite. Generate a recording with minisim (Step 2), export it as an AVI, run the full MiniAn pipeline on it (Step 3), and then compare what Minian recovered against the ground truth minisim kept: do the estimated footprints land on the true cells, do the recovered traces track the true calcium, were the recoverable cells actually found, and which were missed or merged. Because the truth is exact, you get numbers, not impressions, and you learn where the pipeline and your chosen parameters break down before trusting them on an experiment whose answer you do not know. The same loop lets you stress-test: turn up the density, push cells deeper, add vasculature over a soma, shorten the recording, and watch recovery degrade in a controlled way.
Once a pipeline and parameter set survive this on data that resembles your preparation, you apply the same pipeline to your real recordings with justified confidence. See the minisim documentation for the recovery metrics and the ground-truth fields available for comparison.
The workflow at a glance
- Understand the data: read Anatomy of a Miniscope recording and Data processing considerations; explore with minisim's
01_anatomynotebook. - Simulate a recording that matches your preparation with minisim's
build_recordingstudio, keeping its ground truth. - Install and learn Minian on the hosted demo data (
minian data) using thepipelinenotebook and its interactive visualizers. - Validate: run Minian on the simulated recording and score the recovery against the known truth; tune until it holds.
- Analyze your own recordings with the validated pipeline.
- Register cells across sessions with the cross-registration notebook when you have repeated recordings.
Tools and links
minisim (forward model and synthetic data) — build recordings with known ground truth.
- Code and install: github.com/miniscope/minisim (
pip install "minisim[notebook]") - Documentation: minisim.readthedocs.io
- Notebooks:
minisim-notebooks copy 01_anatomy(understand the data),minisim-notebooks copy build_recording(generate data)
Minian (analysis pipeline) — recover cells and traces from recordings.
- Code: github.com/miniscope/minian (
pip install minian) - Documentation: minian.readthedocs.io
- Notebooks:
minian notebooks copy pipeline,minian notebooks copy cross-registration - Demo data:
minian data list(hosted on Zenodo, fetched on demand)
Related guides
- Anatomy of a Miniscope recording — what each feature of a raw 1p movie is, and what causes it.
- Data processing considerations — the recording and preprocessing choices that shape a good dataset.
- Minian — the MiniAn analysis pipeline overview.
- Minisim