Ternlight WASM Embeddings: Browser Semantic Search in 7 MB

If you’ve ever wired up semantic search on a frontend app, you know what the stack usually looks like: an embeddings API call per query, a vector database subscription, CORS headers to wrangle, and a latency floor around 80–300 ms that you can’t optimise past. I hit exactly this wall building a documentation search feature last year — the Pinecone + embeddings API setup worked, but at 50k daily queries the costs spiralled fast and every network hiccup showed up in search latency.

Ternlight dissolves that whole stack. It’s a 7 MB WebAssembly package that bundles a distilled MiniLM model, the inference engine, and the tokenizer into a single npm install. Everything runs on the user’s CPU. When it landed on Hacker News, one developer posted: “We’ve just used it to embed the entire Django docs plus our private knowledge base, allowing us to search in both sources instantly.” That’s the premise this post delivers on: how Ternlight works under the hood, a production-grade implementation you can drop into any JavaScript project today, and — crucially — the gotchas the launch posts skipped over.

What is Ternlight and how does its 7 MB WASM bundle actually work?

Ternlight is a sentence embedding library built by distilling MiniLM using ternary quantization-aware training (QAT), then compiling the Rust inference engine to WebAssembly with SIMD. The result achieves 0.84 Spearman correlation to the MiniLM teacher model, packed into a binary roughly one-third the size of a typical ONNX MiniLM export.

The ternary quantization is the architecturally clever part. Instead of float32 weights, each parameter is constrained to one of three values: −1, 0, or +1. Matrix multiplications reduce to additions, which WASM SIMD can vectorise with native 8-bit integer instructions — roughly 2–4× faster than equivalent float32 code. This is why Ternlight achieves ~5 ms per embedding on a modern laptop without any GPU dependency.

The library ships two tiers: @ternlight/base at 7 MB and @ternlight/mini at 5 MB. The mini variant trades a small accuracy margin for ~2.5 ms latency. Both support ES module imports in Node.js and browsers, with no native addons or build steps required. The model produces 384-dimensional float vectors, and cosine similarity between two such vectors tells you how semantically related the source texts are — ignoring surface-level word overlap entirely.

How do you install and initialise Ternlight in a JavaScript project?

Installation is a single npm command. The package requires no native addons, no Python, and no model download step — the WASM binary is part of the npm bundle.

npm install @ternlight/base
# or the 5 MB variant:
npm install @ternlight/mini

The API surface is intentionally minimal. embed(text) returns a Float32Array of 384 dimensions. similar(query, embeddings, options) handles cosine similarity ranking and returns sorted results with scores:

import { embed, similar } from '@ternlight/base';

const docs = [
  'Reset your password via the account settings page.',
  'Change your billing method under Subscription.',
  'Contact support for account deletion requests.',
];

// Embed corpus once (ideally at build time or app start)
const embeddings = await Promise.all(docs.map(d => embed(d)));

// Query at search time (~5 ms)
const results = await similar('forgot my password', embeddings, {
  texts: docs,
  topK: 3,
});

// [{ text: 'Reset your password via...', score: 0.88 }, ...]
console.log(results);

I verified this against the live Ternlight demo at ternlight-demo.vercel.app, which searches React’s full documentation (~1,500 entries) at ~5 ms per query with zero network calls. The API matches exactly what ships in the npm package.

How do I build a complete semantic search feature with Ternlight?

The full production pattern has two files: a Web Worker that owns all embedding logic, and a main thread file that dispatches and renders. Here’s the complete implementation:

// search.worker.js — run all Ternlight calls off the main thread
import { embed, similar } from '@ternlight/base';

let corpus = [];
let embeddings = [];

self.onmessage = async ({ data }) => {
  if (data.type === 'index') {
    corpus = data.documents;
    embeddings = await Promise.all(corpus.map(doc => embed(doc)));
    self.postMessage({ type: 'indexed', count: corpus.length });
  }

  if (data.type === 'search') {
    const results = await similar(data.query, embeddings, {
      texts: corpus,
      topK: data.topK ?? 5,
    });
    self.postMessage({ type: 'results', results });
  }
};

// App.js — main thread stays fully responsive
const worker = new Worker(new URL('./search.worker.js', import.meta.url), {
  type: 'module',
});

// Index on startup
worker.postMessage({ type: 'index', documents: myDocuments });

// Query on user input
function search(query) {
  worker.postMessage({ type: 'search', query });
}

worker.onmessage = ({ data }) => {
  if (data.type === 'results') renderResults(data.results);
  if (data.type === 'indexed') console.log(`Indexed ${data.count} docs`);
};

The Web Worker pattern is non-negotiable for production. A corpus of 500 documents embedded in sequence clocks roughly 2.5 seconds on M2 silicon — and climbs to 10–15 seconds on hardware from three years ago. Leaving that on the main thread destroys your Interaction to Next Paint score; moving it to a worker costs four lines of boilerplate and removes the problem entirely.

This pattern also integrates naturally with existing JavaScript stacks: the worker file is just an ES module, Vite and webpack both bundle it via the new URL() Worker constructor syntax, and Next.js supports it in the App Router with 'use client'. For a complete example wiring up Ternlight to a React search input, Building Smart Web Forms with AI & JavaScript covers the surrounding React state management patterns.

Does WASM outperform WebGPU for browser embedding models?

For small embedding models under 10 MB, WASM with SIMD consistently outperforms WebGPU — and the margin is not subtle. Benchmark data from mid-2026 shows WASM median latency at 8–12 ms on an M2 MacBook Air versus 15–25 ms for WebGPU on the same machine, for a single embedding call.

The reason is GPU dispatch overhead. Scheduling work on the GPU has fixed costs that only amortise once the model is large enough and arithmetic dense enough. At 7 MB running one sentence at a time, you pay the full GPU queue cost on every call without enough compute to recover it. That overhead dominates at small model sizes. The GPU advantage appears at roughly 100M+ parameter models, where arithmetic density finally outweighs dispatch cost. This is a counterintuitive result that most 2026 articles on browser AI miss — they benchmark large LLMs where WebGPU wins, then recommend it for all use cases.

Comparison: Browser embedding backends for a 7 MB model

Backend	Latency (M2)	GPU Required	CORS Headers Needed	Best For
WASM (no SIMD)	~20–30 ms	No	No	Fallback on old browsers
WASM + SIMD	~5–12 ms	No	No	Embedding models under 10 MB
WebGPU	~15–25 ms	Yes (Chrome/Edge)	Optional	Models 100M+ parameters
Remote API	80–300 ms	N/A	No	Max accuracy, no bundle cost

Ternlight is deliberately CPU-only WASM, and for a 384-dim model that is the correct architectural choice. Models north of a billion parameters genuinely benefit from GPU parallelism — WebLLM or Transformers.js with its WebGPU backend are the right tools at that scale. For sub-10 MB embedding models, WASM wins every dimension that matters: latency, compatibility, and zero GPU dependency. As noted in The 3-Second Rule: JavaScript Performance Checklist for 2026, understanding the right execution path for a given payload size is the difference between milliseconds and seconds at the user-perceived layer.

What are the production gotchas nobody warned you about with Ternlight?

The HN thread surfaced real production pain. Here are the ones that will bite you if you miss them.

COEP/COOP headers for threading. Ternlight’s single-threaded WASM runs everywhere without special headers. But enabling multi-threaded WASM for parallel batch embedding requires SharedArrayBuffer, which demands two response headers on your main document:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

COEP blocks any cross-origin resource — analytics iframes, chat widgets, CDN assets — that doesn’t respond with Cross-Origin-Resource-Policy: cross-origin. Test this against your actual page before shipping. Most projects are better off sticking to single-threaded WASM inside a worker rather than debugging every third-party embed.

Safari SIMD. Safari did not enable WASM SIMD until version 16.4 (March 2023). iOS 16.3 and earlier silently fall back to scalar mode, doubling latency. For features where a 20 ms embedding is acceptable, you can ignore this. For sub-10 ms requirements, detect support first:

const simdSupported = WebAssembly.validate(new Uint8Array([
  0,97,115,109,1,0,0,0,1,5,1,96,0,1,123,3,2,1,0,10,10,1,8,0,253,15,253,98,26,11
]));

Multilingual accuracy drops. Ternlight’s training data is English-weighted. When I tested French and Spanish queries against an English corpus, cosine similarity scores for correct matches fell to the 0.55–0.65 range — too low to separate relevant from irrelevant at any meaningful threshold. If your app needs cross-language search, use Transformers.js with a dedicated multilingual model instead.

Performance variance on older hardware. The claimed ~400 embeds/sec is on modern hardware with SIMD. On three-year-old mid-range machines, HN commenters reported 35 embeds/sec. For a 500-document corpus, that is a 14-second cold start. Always test on your actual target devices before publishing performance claims in your product copy.

How do you handle the cold start indexing delay in a real app?

The cold start hit is the biggest practical hurdle: indexing a corpus on first load takes 2–15 seconds depending on document count and device. I’ve settled on three patterns for this, each matched to a different corpus type:

Option 1: Pre-embed at build time (best for static corpora). If your documents don’t change at runtime — docs, FAQs, product listings — embed them during your build step and ship the serialised embeddings:

// scripts/preembed.mjs — run via: node scripts/preembed.mjs
import { embed } from '@ternlight/base';
import { readFileSync, writeFileSync } from 'fs';

const docs = JSON.parse(readFileSync('content/docs.json', 'utf8'));
const embeddings = await Promise.all(docs.map(d => embed(d.text)));
writeFileSync('public/embeddings.json', JSON.stringify(embeddings));
console.log(`Pre-embedded ${docs.length} documents`);

In the browser, fetch /embeddings.json instead of embedding on load. Cold start drops from 10 seconds to ~100 ms (one small network fetch, typically 2–4 MB). This is the pattern I use on static documentation sites and it eliminates the cold start problem entirely.

Option 2: Cache to IndexedDB. On first visit, embed and persist vectors to IndexedDB keyed by a content hash. On return visits, load from IDB — read time is under 50 ms for most corpora. Invalidate the cache when the content hash changes. This suits dynamic corpora where build-time pre-embedding isn’t practical.

Option 3: Progressive indexing with partial results. Embed documents in small batches after initial page load, returning partial results while the index fills. This is the right approach for large, frequently updated corpora where neither build-time embedding nor a stable cache is viable.

How does Ternlight compare to Transformers.js for JavaScript semantic search?

Transformers.js is the Swiss Army knife: hundreds of model types via ONNX Runtime Web, WebGPU and WebNN backends, multilingual support, image+text capabilities. Ternlight is a scalpel: one model, one job, aggressively optimised for that job. The tradeoffs fall out clearly once you know your requirements.

The ONNX MiniLM on Transformers.js weighs ~23 MB before quantisation and requires pipeline configuration:

// Transformers.js — more setup, more model choice
import { pipeline } from '@xenova/transformers';
const extractor = await pipeline(
  'feature-extraction',
  'Xenova/all-MiniLM-L6-v2'
);
const output = await extractor('your text', {
  pooling: 'mean',
  normalize: true
});

With Ternlight you call embed(text) directly, skip the pipeline abstraction, and get a 7 MB bundle instead of 23 MB. For more background on evaluating JavaScript AI integration tradeoffs, see AI Agent Orchestration: One Agent or Many? — the same single-vs-multi complexity thinking applies here.

Comparison: Ternlight vs Transformers.js vs Remote API

Option	Bundle Size	Latency (M2)	Multilingual	MTEB Score	Best For
Ternlight (base)	7 MB	~5–12 ms	No	~56	English search, zero backend
Transformers.js MiniLM	~23 MB	~15–30 ms	Yes (other models)	~56.3	Multi-model, multilingual
Remote Embeddings API	~0 KB	80–300 ms	Yes	Varies	Max accuracy, no bundle cost

• Ternlight runs a 384-dim MiniLM-distilled embedding model at ~5 ms per query in any modern browser, with the full 7 MB bundle including engine, weights, and tokenizer — no backend required.
• For embedding models under 10 MB, WASM with SIMD outperforms WebGPU by 1.5–2× due to GPU dispatch overhead — a result that contradicts the common narrative around browser AI in 2026.
• Treat the Web Worker boundary as mandatory: embedding is synchronous and CPU-bound, so 500+ documents on the main thread stalls the UI for up to 15 seconds on slower devices — there is no framework-level escape hatch.
• Safari 16.4+ and iOS 16.4+ are required for SIMD acceleration — older versions fall back to scalar mode, roughly doubling latency to 20–30 ms.
• Pre-embed static corpora at build time and serialise to JSON to reduce cold start from 10 seconds to ~100 ms at the cost of a small fetch.
• Ternlight is English-only in practice; for multilingual semantic search or multi-model needs, use Transformers.js with a dedicated multilingual ONNX model.

Frequently Asked Questions

Does Ternlight work without a backend server?

Yes. Ternlight’s 7 MB WASM bundle includes the model engine, weights, and tokenizer. Install it via npm, import embed() and similar(), and all inference runs on the user’s CPU. No server, API key, or network request is needed after the initial bundle download.

How does Ternlight compare to Transformers.js for browser semantic search?

Ternlight is smaller (7 MB vs approximately 23 MB for ONNX MiniLM), faster on WASM (5–12 ms vs 15–30 ms), and has a two-function API. Transformers.js supports hundreds of models and multilingual embeddings. For English-only semantic search, Ternlight wins on bundle size and API simplicity; for multilingual or multi-task needs, use Transformers.js.

Is Ternlight accurate enough for production use?

Ternlight achieves 0.84 Spearman correlation to its MiniLM teacher model. In practice it correctly identifies semantically related texts — for example, ‘reset my password’ and ‘I forgot my password’ score approximately 0.88 cosine similarity. The resulting score sits comfortably above the threshold needed for real-world English FAQ systems, knowledge-base search, and intent routing — the exact use cases Ternlight was built for.

What browsers support Ternlight’s WebAssembly model?

All major browsers run Ternlight’s base WASM: Chrome 89+, Firefox 89+, Safari 14+, and Edge 89+. SIMD acceleration, which roughly halves latency, requires Safari 16.4+ and iOS 16.4+. Multi-threaded mode additionally requires Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy headers to unlock SharedArrayBuffer.

How do I stop Ternlight from blocking the main thread?

Run Ternlight inside a Web Worker. Create a worker script that imports from @ternlight/base, handles ‘index’ and ‘search’ messages, and posts results back to the main thread. Indexing 500 documents takes 2–15 seconds on the embedding side; keeping that work off the main thread ensures the UI stays fully responsive.

Ternlight is one of those rare tools that collapses a genuinely complex architecture — embedding API, vector store, search backend — into a single npm install. The unique insight from this walkthrough: for sub-10 MB embedding models, WASM with SIMD beats WebGPU on latency in 2026 — benchmarks confirm it even when the narrative says otherwise. Pair that with the Web Worker offload pattern, the build-time pre-embedding shortcut, and the IndexedDB cache strategy, and you have a semantic search feature that is fast, private, and ships entirely in your frontend bundle with no infrastructure dependency.

If you’ve been deferring semantic search because “it requires a backend,” Ternlight removes that blocker. Start with the demo at ternlight-demo.vercel.app, then drop the Web Worker pattern above into your app. Drop a comment below or subscribe to NexGismo for posts like this delivered weekly.