Vector Search Logic

Memo's vector search engine runs entirely in-process, backed by SQLite with the sqlite-vec extension. It uses cosine similarity over 768-dimensional float32 embeddings.

Core Algorithm: Cosine Similarity

For a query vector q and document vectors d₁, d₂, ..., dₙ:

similarity(q, d) = (q · d) / (||q|| × ||d||)

Where:

• q · d is the dot product
• ||q|| and ||d|| are the L2 norms

The result is a value between -1.0 (opposite) and 1.0 (identical).

func CosineSimilarity(a, b []float32) float64 {
    var dot, normA, normB float64
    for i := range a {
        dot += float64(a[i]) * float64(b[i])
        normA += float64(a[i]) * float64(a[i])
        normB += float64(b[i]) * float64(b[i])
    }
    if normA == 0 || normB == 0 {
        return 0
    }
    return dot / (math.Sqrt(normA) * math.Sqrt(normB))
}

Top-K Search

The search pipeline returns the K most similar chunks:

1. ANN Pre-filter: The vec0 approximate nearest neighbor index rapidly narrows the search space to ~100 candidates
2. Exact Cosine Scoring: Each candidate is scored against the query vector
3. Sort & Truncate: Results sorted by descending similarity, top K retained
4. Threshold Filter: Any result below min_similarity is discarded

type SearchResult struct {
    Content    string  `json:"content"`
    Source     string  `json:"source"`
    Similarity float64 `json:"similarity"`
    ChunkIndex int     `json:"chunk_index"`
}

func (s *Store) Search(ctx context.Context, queryVector []float32, topK int, minSimilarity float64) ([]SearchResult, error) {
    // 1. ANN pre-filter via vec0
    // 2. Exact cosine scoring
    // 3. Sort, truncate, threshold
}

Parallel Worker Pool

Large search spaces are partitioned and processed in parallel using a goroutine worker pool:

Query Vector
    │
    ├─▶ Worker 1 ── Chunks 0–999   ──▶ Result Set A
    ├─▶ Worker 2 ── Chunks 1000–1999 ──▶ Result Set B
    ├─▶ Worker 3 ── Chunks 2000–2999 ──▶ Result Set C
    └─▶ Worker 4 ── Chunks 3000–3999 ──▶ Result Set D
                                              │
                                        Merge + Top-K
                                              │
                                        Final Results

• Worker count: runtime.NumCPU() by default
• Work distribution: Equal-sized chunks assigned via channel
• Result collection: Each worker writes to a shared results slice protected by sync.Mutex
• Cancellation: Context cancellation stops all workers immediately

Chunked Search Space

The search space grows with ingested documents and past conversations. To maintain fast search as data scales:

Total Chunks	Strategy
< 1,000	Brute-force cosine (single pass)
1,000 – 100,000	vec0 ANN index + exact rescore
> 100,000	ANN index only, Top-200 candidates before rescore

Chunk Metadata

Each chunk carries metadata to enable filtering and attribution:

SELECT content, source, chunk_index, created_at
FROM vec_memory
WHERE embedding MATCH ?  -- KNN search via vec0
ORDER BY distance
LIMIT ?;

Similarity Threshold

The min_similarity threshold (default 0.7) serves two purposes:

1. Prevents noise: Low-similarity chunks add no value and dilute the context window
2. Saves tokens: Fewer chunks injected means more room for actual conversation

Users can adjust the threshold in Settings:

Setting	Effect
0.9	Only nearly identical matches (narrow recall)
0.7	Default — good balance of precision and recall
0.5	Broad recall, may include tangentially relevant content
0.3	Very broad — use only for exhaustive search

Performance Tuning

Parameter	Default	When to Increase	When to Decrease
top_k	5	Need broader context	Context window is full
min_similarity	0.7	Getting irrelevant results	Missing relevant results
chunk_size	512	Documents have long coherent sections	Short Q&A style content
chunk_overlap	10%	Content is dense, needs continuity	Content is already self-contained

---

Cosine similarity computation is O(n) per comparison. With 768-dim vectors, a single comparison takes ~3 microseconds. The worker pool parallelizes this across all available CPU cores.