<!-- skill: Jupyter Flow | version: 1.0.0 | description: Run Jupyter kernels, magics, nbconvert schedules, Colab secrets, clean reruns | category: Python & Data -->
## Instructions
- Pick one kernel per notebook (`python3`, `venv` kernel via `ipykernel install --name proj`); record kernel in first cell.
- Start every notebook with: goal (1 line), `%load_ext autoreload` + `%autoreload 2`, imports, `RANDOM_SEED = 42`.
- Restart-and-run-all before sharing: `Kernel → Restart & Run All`; fix any out-of-order `In[*]` execution first.
- Keep cells small and linear: one idea per cell, top-to-bottom flow, no hidden state from deleted cells.
- Use magics deliberately: `%timeit` for perf, `%%bash` for shell, `%matplotlib inline`, `%env` for non-secret config.
- Never hardcode secrets: `os.environ['KEY']` + Colab `userdata.get('KEY')` or `.env` via `python-dotenv`; gitignore `.env`.
- Parameterize with `papermill` (`-p start 2026-01-01`) for scheduled runs; tag parameter cells with `parameters`.
- Schedule headless with `nbconvert --execute --to html report.ipynb --output out.html` in cron; fail loudly on error.
- Save outputs wisely: commit clean `.ipynb` without 50 MB outputs; use `nbstripout` pre-commit hook for repos.
- Version data references: log CSV/Parquet path + mtime + `df.shape` in cell 2 so reruns are auditable.
- Structure analysis as: question → load/inspect → clean → answer (table+chart) → caveats; delete scratch cells after.
- Pair with csv-analysis/pandas-frames skills: explore in notebook, promote the final logic to `analyze.py` script.

## Examples
- EDA notebook → load CSV, profile nulls, 3 charts, conclusion cell, executed via nbconvert nightly to HTML.
- Papermill report → `parameters` cell (date range), scheduled run per client, output saved as `report_2026-10-14.ipynb`.
- Colab share → secrets in Colab Secrets panel, `%pip install` cell pinned, restart-rerun verified before sending link.

## Workflows
1. Setup: create venv kernel, install `ipykernel papermill nbconvert`, gitignore `.env` + checkpoints.
2. Explore: linear cells, autoreload on, seed fixed, inspect shapes after every load.
3. Harden: restart-rerun-all, strip giant outputs, tag `parameters` cell for papermill.
4. Schedule: `papermill in.ipynb out.ipynb -p ...` then `nbconvert --to html`; cron or CI runs it.
5. Promote: copy proven logic to a `.py` script so production does not depend on notebook state.

## Templates
```python
# cell 1 — header: goal + autoreload + seed + safe secrets
# GOAL: monthly revenue by region from orders.csv
%load_ext autoreload
%autoreload 2
import os, random
import numpy as np, pandas as pd
RANDOM_SEED = 42
random.seed(RANDOM_SEED); np.random.seed(RANDOM_SEED)
try:  # Colab secrets, else local .env
    from google.colab import userdata
    API_KEY = userdata.get("API_KEY")
except ImportError:
    from dotenv import load_dotenv
    load_dotenv()
    API_KEY = os.environ.get("API_KEY")
print("pandas", pd.__version__, "| rows next cell will confirm")
```
```bash
# schedule.sh — papermill run + nbconvert to HTML (cron-friendly)
papermill report.ipynb "out/report_$(date +%F).ipynb" -p start 2026-09-01 -p end 2026-10-01
jupyter nbconvert --execute --to html "out/report_$(date +%F).ipynb" --output "report_$(date +%F).html" --no-input
```

## Tools
- Core: `pip install jupyter ipykernel papermill nbconvert python-dotenv nbstripout`.
- Kernel: `python -m ipykernel install --user --name proj --display-name "Python (proj)"`.
- SARAH Run executes `jupyter nbconvert --execute` headless; HTML report saves to `out/`.

## Advanced
- Papermill: tag cell `parameters`, inject `-p` values, read `out.ipynb` + `scrapbook` outputs programmatically.
- Colab: Secrets panel (🔑) → `userdata.get`, `%pip install -q`, `files.download('out.html')` to hand off results.
- nbconvert flavors: `--to html|pdf|script`; `--no-input` hides code for exec reports; `--ExecutePreprocessor.timeout=600`.
- Git hygiene: `nbstripout --install` strips outputs on commit; keep one executed HTML per release, not per commit.
- Debugging kernels: `jupyter kernelspec list`, kill zombies via `jupyter notebook list` + `stop`; pin `ipykernel` version.
- Troubleshooting: stale state → restart-rerun; `ModuleNotFoundError` → wrong kernel — check `sys.executable`.

## Documentation
- Jupyter docs: https://docs.jupyter.org/
- nbconvert: https://nbconvert.readthedocs.io/
- Papermill: https://papermill.readthedocs.io/