SkyMiner AI
Automated detection of scientifically novel astronomical objects — built by one person, at survey scale.
What is SkyMiner AI?
Modern astronomical surveys generate more data than any research team can manually review. DESI has catalogued 9 million spectra of galaxies, quasars, and stars. The Zwicky Transient Facility measures how nearly 5 billion objects change in brightness over time. Somewhere in those numbers are objects that behave in ways science has not yet documented.
SkyMiner AI is an independent research project trying to find them — without being told what to look for.
The Approach
Every pipeline in SkyMiner operates without labeled training data. There are no annotated examples of “this is an anomaly” — because genuinely novel objects have no precedent to learn from. Instead, each pipeline learns the structure of normal, and scores how far a candidate deviates from it. All parameters are derived from the data, not assumed.
The architectural decisions involved — which representations to use, how to handle irregular time series, how to define novelty without ground truth — are problems without off-the-shelf answers. This project is, in part, a demonstration of applied AI at a scale and depth that most teams approach with considerably more resources.
Pipelines
Pipeline Dependencies
SkyMiner Fusion requires both Spectral and Temporal complete before it can run. SkyMiner Orbital is fully independent and can proceed in parallel with the other three.
Ecosystem Status
| Pipeline | Primary Dataset | Detection Domain | Status | Current Milestone |
|---|---|---|---|---|
| SkyMiner Spectral | DESI DR1 | Spectral profiles | IN PROGRESS | Ingestion complete — encoder training in progress; scoring architecture redesigned (Persistence Score consensus over 500 HDBSCAN configs) |
| SkyMiner Temporal | ZTF DR24 + NEOWISE | Lightcurve variability | DESIGN COMPLETE | Dataset builder not yet started |
| SkyMiner Fusion | DESI + ZTF | Cross-domain synthesis | BLOCKED | Requires Spectral + Temporal complete |
| SkyMiner Orbital | ZTF DR24 | Solar System Objects | IDEATION | Architecture not yet defined |