Grep answers the wrong question. When an agent asks "where is auth middleware configured?", grep can only tell it which lines contain the word "auth" - and then the agent reads three files to find out that none of them were the answer. Every one of those reads costs tokens.

Miru is our open-source attempt to answer the question the agent actually asked. It returns ranked chunks - path, line range, snippet - over MCP, so agents like Claude Code, Cursor and Codex get a short list of places to look instead of a long list of lines to read.

This is a walk through how it works internally, with the real constants. The code is at github.com/takara-ai/miru-code if you want to check any of it.

The unit is the chunk, not the file

Everything downstream depends on getting this right. A chunk that starts halfway through a function embeds badly and reads badly.

We chunk in three tiers, first one that works wins:

Chunking tiers

AST is the default and, for the 22 languages we ship vendored web-tree-sitter grammars for, it is effectively the only tier that runs. We parse the file, then walk the tree greedily packing sibling nodes into groups until the group hits about 1,500 bytes. A node bigger than that on its own gets recursed into. Nodes under 50 bytes aren't worth splitting further, so they stay whole. (Bytes, not characters - tree-sitter reports byte offsets and we convert to character offsets afterwards. Identical for ASCII, slightly different for a file full of CJK comments.) The result is that a chunk boundary almost always lands where a human would put one: end of a function, end of a class body, end of an import block.

Structural only fires if the grammar fails to load - brace and indent heuristics for Python, Go, TypeScript, JavaScript, C and C++. For Python we walk indentation to find block ends and split classes into first-method-plus-header and then one unit per method. For the brace family we count depth from the first {.

Lines catches everything else. Still searchable, but the boundaries are arbitrary, and a chunk of a long Helm values file is a much weaker embedding than a chunk of a Go function. If your repo is mostly YAML, that is the honest limit of what Miru is doing for you today.

Before any of this, the walker honours .gitignore and .miruignore, skips the usual noise directories, and drops files over 1,000,000 bytes. Minified bundles are not worth an embedding.

Every chunk is written into both indexes; the lexical side gets the file path folded into its token stream, the dense side gets windowed, pooled and quantised

Two indexes, because one is always wrong

We build a sparse index and a dense index over the same chunk list.

BM25 is standard Okapi with k1 = 1.5, b = 0.75, stored as inverted postings. Two things about it are not standard.

First, the tokeniser splits identifiers and keeps both forms:

splitIdentifier("getUserById") // ["getuserbyid", "get", "user", "by", "id"] So a query for "user id" hits getUserById without the caller having to guess the casing, and an exact query for getUserById still matches the whole token strongly.

Second, we enrich each document with its own path before tokenising:

return `${chunk.content} ${stem} ${stem} ${dirText}`;

The file stem goes in twice, plus the last three directory names. In practice the path is one of the highest-signal features in a repo - src/auth/middleware.ts tells you more about that file than most of its lines do - and this puts that signal in the lexical index for free.

Scoring runs on the main thread for small corpora and fans out to workers once the corpus passes 256 documents - though only on machines with cores to spare, since worker concurrency is derived from available parallelism minus two. On a 2-core CI box it stays single-threaded at any size.

Dense is where the embeddings live. Each chunk is windowed at 1,300 characters with 120 characters of overlap, every window is embedded, and the windows are mean-pooled and L2-normalised back into one vector per chunk. Windowing matters because a 1,500-character chunk with a long docstring would otherwise have its tail truncated by the model's input limit.

Then we throw away three quarters of the precision on purpose:

const scale = maxAbs > 0 ? maxAbs / 127 : 1;
codes[i] = Math.max(-127, Math.min(127, Math.round(value * inv)));

Symmetric int8 quantisation, one scale per vector. Vectors are already normalised upstream, so the distribution is well-behaved and the recall cost is small. The win is roughly 4x less RAM, which is what makes it reasonable to hold a whole repo's index in a CLI process, and int8 dot products are fast. The inner loop is unrolled 8-wide to give JavaScriptCore something it can auto-vectorise, and top-k comes off a size-k max-heap rather than sorting every candidate.

One detail from the build pipeline that we like more than we probably should: while embed requests are in flight and the pipeline is stalled on backpressure, we spend the idle time tokenising and building BM25 on the main thread. BM25 construction is close to free in wall-clock terms because it happens inside network waits.

Fusion: rank, not score

Semantic scores are cosine distances. BM25 scores are unbounded and corpus-dependent. Normalising them onto a common scale means picking a normalisation, and every choice we tried was wrong on some query shape.

So we don't use the scores. We use the ranks.

const RRF_K = 60;
ranked.forEach(([key], i) => out.set(key, 1.0 / (RRF_K + i + 1)));

Reciprocal rank fusion. Position 1 in either list is worth 1/61 regardless of whether that list produced a score of 0.94 or 31.7. It is crude and it is stable, which is the right trade for a component that has to behave sanely on repos we have never seen.

The two lists are then combined with a weight that depends on what the query looks like:

const ALPHA_SYMBOL = 0.3;
const ALPHA_NL = 0.5;

If the query is shaped like a symbol - UserRepository, auth::verify, _internal - we lean towards BM25, because the user typed the literal string and lexical matching is simply better at literal strings. Anything else gets an even split. We also widen the candidate pool from topK * 5 to topK * 10 for location queries ("where is", "entry point", "bootstrap"), because those need more of the tail to be visible to the reranker.

The two retrievers never compare scores with each other - only positions - which is what stops one corpus-dependent scale from swamping the other.

Reranking is where the opinions are

Fusion gets you a reasonable ordering. The reranker is what turns it into a good first result, and it is unashamedly a pile of heuristics.

File coherence. Every candidate file's best chunk gets a boost of up to 20% of the top score, scaled by that file's share of the total. Files that contributed several strong chunks win; a file with one lucky chunk barely moves.

Definition boosts. For symbol-shaped queries we regex the chunk content for a definition of that symbol - class X, def X, func X, interface X, CREATE TABLE X and about twenty more. Finding one adds 3x the top candidate's score to that chunk, and 4.5x if the file stem matches the symbol too. Note that it's additive and pegged to the leader, not a multiplier on the chunk's own score - a chunk sitting 40th with a real definition in it jumps the whole queue. That is deliberate. A definition beats a call site, every time.

Location signals. For "where is" queries we detect entry-point shapes (#!/usr/bin, function main(, require.main === module) and synthesise a chunk from package.json carrying bin, main and types, so "where does the CLI start?" can return the answer rather than something that mentions starting.

Path penalties. These do more work than any boost:

Test files are the single biggest source of plausible-but-useless results. A test for auth middleware contains every word in the query and is almost never what you wanted.

File saturation. Once a file has contributed a chunk, every further chunk from it is halved again - second chunk at 0.5, third at 0.25, fourth at 0.125. Three results from three files beat three results from one file when the agent's next move is deciding where to read.

We know how this reads. These constants were tuned against our own benchmark set, and a heuristic reranker is a maintenance liability compared to a learned one. It is also, right now, the thing that makes the first result correct often enough to be worth the agent's trust.

Then we cut it down

A ranked chunk is still too much. So we find the line in the chunk that best matches the query terms and return ±15 lines around it. If that actually cut anything, the result carries truncated: true and an anchor_line; a chunk short enough to survive whole carries neither, and the agent works from start_line.

That flag is a contract with the agent. It says: there is more here, and there is a cheap way to get it - call expand with this anchor, don't re-search and don't read the file. Alongside it, find_related takes the same anchor and returns the nearest other chunks in the same language, and locate does exact substring matching over the already-indexed chunks rather than re-walking the disk.

The whole design goal is to make "read the whole file" the last option rather than the first one. Our headline claim is up to 50% fewer tokens, and I'd treat that as indicative rather than a promise - it moves a lot with repo shape and query. Measuring it properly is harder than it sounds. Token savings depend on what the agent does next, and the agent is non-deterministic - two identical runs make different follow-up reads. Benchmarking hybrid search under an LLM deserves its own piece, and we'll write one.

Staying current

The MCP server watches the workspace. On a change we drop chunks for the affected paths, re-chunk and re-embed just those files, carry the surviving vectors across, and rebuild both indexes from the merged set. The on-disk cache is keyed by a SHA-256 of the path and stamped with an epoch derived from the package version.

Two honest caveats there. The epoch is coarse - on 1.x it's the major version, so it protects against a breaking release but not against a chunking change shipped in a minor. And "carry the surviving vectors across" is doing some work: on the int8 path there is no float32 original to carry, so every surviving vector is dequantised, renormalised and requantised, and the whole semantic index plus the whole BM25 index is rebuilt from scratch. At repo scale that is fast enough that we haven't needed to fix it. At monorepo scale it is the next thing that will need fixing.

What it doesn't do

Indexing sends chunk contents to our inference API to be embedded, which is a real consideration if your code is sensitive. There is a self-hosted path that runs the model on a SageMaker endpoint in your own AWS account, which means no egress to us - not no egress at all. And MIRU_WORKSPACE_ROOT fences MCP to a single directory, but only for local paths; a git URL still gets cloned and indexed regardless. Neither changes the basic shape: embeddings come from a model, and the model has to see the code.

The search hooks that block an agent's built-in grep and redirect it to Miru are still experimental. Ranking heuristics are tuned, not learned. And nothing here helps with a query that is genuinely ambiguous - if "handler" is a reasonable name for forty things in your repo, no amount of fusion fixes that.

MIT licensed, bun add -g @takara-ai/miru-code. Most of the ranking constants are in src/ranking/, with the fusion weights in src/search.ts, if you want to argue with them.