Year
2026
Client
Personal R&D
Category
AI Infrastructure / Systems Design
Product Duration
In Progress
Modern codebases are stored as repeated file snapshots where every commit duplicates unchanged functions, every RAG pipeline re-embeds the same logic. The result is bloated vector stores, noisy retrieval, and line-based diffs that cannot reason about what actually changed. SCC was built to fix the foundation.
SCC runs every input through a multi-stage pipeline: language/format detection, structure-aware parsing into semantic units (functions, classes, imports, interfaces for code; sections, clauses, tables for documents), and normalization that preserves an exact source for lossless reconstruction while giving each unique unit a stable identity. Each unit is stored once, with its own metadata and embedding.
Every unique unit is stored once and enriched with metadata: language, type, symbol name, relationships in and out, and an embedding. Files aren't stored whole, they're represented as ordered references to those units plus the surrounding glue, keeping reconstruction fully lossless.
On top of the unit store, SCC enables semantic commits. A commit is a manifest where each file is a sequence of unit references plus glue. Diffs surface at the unit level first: unchanged functions are skipped, changed functions are flagged, and merge conflicts scope to the exact function rather than a raw line range giving both a semantic and a textual diff.
For retrieval, SCC acts as a deduplication and indexing layer beneath vector search: repeated logic is embedded once, not many times. Retrieval is hybrid : vector, symbol, path, and relationship-graph expansion combined before ranking and returns unit IDs that are materialized into exact source on demand.
Node.js & TypeScript, Express, PostgreSQL with pgvector, Redis for queuing/caching, and object storage for compressed blobs. Parsing uses Tree-sitter for multi-language extraction.
On a representative mid-size repo: 18,904 semantic chunks found across 4,281 files, with 11,622 unique chunks after deduplication a 38.5% embedding reduction On synthetic repos built to stress-test deduplication, SCC cut duplicate chunk work by ~54–56%.
Built solo as a research and infrastructure project, in progressive phases: a retrieval-focused unit store (current), then semantic diff and function-level history, and a longer-term direction toward native unit-based version storage. The engine is also the core retrieval layer in Gity, a semantic developer tool in developmen
