Lattice Chunking
4/1/2026 - in progress
Retrieval systems chunk text before indexing. Small chunks match precise questions. Large chunks capture broader context. The right size depends on the query, and you don't have the query at ingest time.
The lattice
Don't choose a chunk size. Index all of them.
Start with semantic units. Sentences, paragraphs, whatever the text naturally breaks into. Build overlapping windows: every consecutive pair, every triple, every quadruple, up to some height limit.
A paragraph of text.
The morning was cold. Frost covered the windows. The kettle began to whistle. Steam rose from the cup. The day had started.
Each node is a sliding window of consecutive units. Height 0 is individual sentences. Height 1 is every consecutive pair. Height 2 is every consecutive triple. Each higher-level node connects to the two nodes it was built from.
Every node gets its own embedding. At query time, search runs across all of them. The right granularity emerges from the match.
Does this explode?
The obvious objection: too many chunks. Without a height limit the lattice is a triangle. Growth is .
Cap it at height and it's . At that's roughly total nodes. Drag the slider to see which nodes get pruned.
8 sentences, height capped at 4. Greyed nodes are pruned.
What the lattice captures
A parent node's embedding is not just "more text." It represents the combination. Two sentences about cause and effect produce a vector that neither sentence has alone.
The city was founded in 1847 near the river.
Early settlers built a mill and a general store.
By 1900 the population had reached twelve thousand.
The railroad arrived in 1905 and connected the town to the coast.
Factories opened along the tracks.
A real example
Below is the Bee Movie script chunked into 131 units of 10 lines, built into a lattice of height 30. Each pixel is a node. Color is cosine similarity to the query.
The first query describes events spread across four chunks. The best match is at height 3, not a leaf. The lattice found a granularity that no fixed chunking strategy would have produced.
More on retrieval, embedding bottlenecks, and open questions coming soon.