Back to work LOCAL-FIRST SYSTEMS ARCHITECTURE, ARCHIVED

Observable

A local-first, privacy-preserving EEG session archive for the Neurosity Crown headset: a single Rust daemon that captures 8-channel brainwave data, durably logs it, and serves it back out through an embedded query engine. Archived: the product direction moved to Entrain, a closed-loop app that talks to the Neurosity SDK directly and doesn't need a separate capture-and-storage layer. The pivot is documented in the repo rather than hidden, and the systems-design work underneath, write-ahead log as source of truth, idempotent materialization, crash recovery, stands on its own.

View repo

Architecture & Trade-offs

What was built, and what was deliberately chosen against.

pipeline.txt

Neurosity Crown
    │  Firebase RTDB SSE (WiFi)
    │  raw EEG (256 Hz, 8ch) · powerByBand (~4 Hz) · awareness (~4 Hz)
    ▼
collector             authenticates, subscribes to all three streams
    │
    ▼
wal_writer             append-only NDJSON, source of truth
                        64 MB / 10-minute rotation
    │
    ▼
parquet_materializer    WAL to Parquet, idempotent, re-runnable
    │
    ▼
axum API on :8765       embedded DuckDB query layer
    │
    ▼
SvelteKit UI on :5173    session browser, timeline charts, journal

startup_reconciler: replays WAL on boot, heals sessions
                     active, interrupted, recovered

The write-ahead log is the system of record, Parquet is derived

Every captured sample is appended to an NDJSON write-ahead log before anything else happens to it. Parquet, the format everything downstream actually queries, is treated as a rebuildable derivative: the materializer that produces it is idempotent and safe to re-run, and it skips ranges it has already processed rather than trusting a one-shot write to have succeeded.

Crash recovery is a startup routine, not an afterthought

A reconciler runs on every boot and walks any session left mid-flight through an explicit state machine, active to interrupted to recovered, by replaying the write-ahead log. A session that was capturing when the process died isn't corrupted or lost, it's just not finished being recovered yet.

Direct-from-cloud capture, with the offline path deliberately left open

The capture pipeline depends on Neurosity's Firebase servers over WiFi, not a local Bluetooth connection, since that's the interface the SDK exposed at the time. The Crown does have a Bluetooth API that could support fully offline capture, and the architecture was kept open to it, but it was never built here: an honest gap rather than an implied capability.

Archived on purpose, not by neglect

The README states plainly that this is a frozen snapshot: issues aren't triaged, pull requests aren't reviewed, and the reason is a real architectural decision, documented in Entrain's own decision log, to talk to the Neurosity SDK directly instead of through a separate capture daemon. The project stays published anyway because the systems-design work holds up independent of whether the product shipped.

Stack

Rust, Tokio, DuckDB, Parquet, Axum, SvelteKit, Plotly.js

Status

Archived, snapshot only. Not actively maintained; issues and pull requests aren't triaged. Superseded by Entrain's direct Neurosity SDK integration.

Public repo. View the repo on GitHub.

Back to work