Methodology
The observatory evaluates how well spec-driven development approaches work in practice by watching real projects over time — not by quoting marketing pages. Everything on the site comes from one of two places: an automated metrics pipeline that humans never edit, or a curated human assessment that bots never edit. This page explains what we track, where each number comes from, and how to read the ratings.
What we track
Frameworks
Documented methodologies for driving AI-assisted development from specifications. Each page records the framework's approach, workflow, supported tools, maturity, strengths, and limitations.
Projects
Public repositories that use one of those frameworks. Each page tracks how the specs are structured, how actively the code is developed, whether specs and code stay in sync, and how maintenance plays out over time.
Where the data comes from
Every page mixes two kinds of data, and the split is strict:
Automated metrics
Fetched by machines. Humans never edit them.
- What
- Stars, forks, contributors, open issues, repository age, push activity, releases, weekly commits.
- Source
- The GitHub API, via a scheduled job.
- Lives in
data/metrics/- Refreshed
- Daily. Each metrics panel shows its fetch date.
Curated assessments
Written by people. Bots never edit them.
- What
- Framework descriptions, spec structure, drift ratings, defect and rework narratives, maintenance outcomes, timelines.
- Source
- Maintainers and contributors, drawing on framework documentation, release notes, maintainer submissions, and manually verified case studies.
- Lives in
src/content/- Refreshed
- Best-effort review. Each page shows its "last reviewed" date.
Both live in the same public repository, so every change — a bot commit or a human judgment — is reviewable in the git history. Stale assessments are a known failure mode: the "last reviewed" date tells you how fresh a judgment is, and flagging or refreshing one is one of the most useful contributions.
How to read the ratings
Three rating scales appear on cards and pages across the site. All three are curated human judgments, not computed values; the badges below are the exact ones you'll see.
Maturity — frameworks
How settled the framework's methodology is.
| experimental | Early stage; core workflow still changing; little real-world usage. |
|---|---|
| emerging | Real adoption and active development, but conventions still evolving. |
| established | Stable core workflow, meaningful adoption, actively maintained. |
| mature | Widely adopted, stable over multiple release cycles, proven in production use. |
Spec-to-code drift — projects
Whether the project's specs still describe its code, judged by comparing recent code changes against the spec artifacts.
| none | Specs are updated alongside code; spot checks find no divergence. |
|---|---|
| low | Minor gaps; specs trail code slightly but remain a reliable guide. |
| moderate | Specs noticeably lag; parts of the codebase are no longer described. |
| high | Specs are stale or abandoned while the code moves on. |
| unknown | Not yet assessed. |
Tracking status — projects
Whether the project is still being watched.
| active | Under regular development and regular review here. |
|---|---|
| paused | Development, or our tracking of it, is temporarily on hold. |
| archived | Finished or abandoned; kept for the historical record. |
Limitations
- Stars and commit counts measure attention and activity, not quality or productivity.
- Drift ratings are informed human judgment, not a computed diff, and can lag reality between reviews.
- The sample is small and self-selected: submitted projects skew toward teams proud of their process.
- Frameworks differ in what they call a spec, so cross-framework comparisons should be read as directional, not precise.