Rust web service · Skelf-Research
Turn search results into answer-engine chunks.
Polymathy is a small Actix-web service that sits in front of a SearxNG instance,
pulls the top result URLs, and hands them to a content processor for chunking and
embedding. It returns a map of chunk_id → (source_url, text) over
plain HTTP. It is the back of an answer engine, not the front.
/v1/search?q=…
top 10 URLs
chunks + embeddings
What it actually does
The whole service is one endpoint.
GET /v1/search?q={query} hits your configured
SEARXNG_URL, takes the first ten result URLs, and fans them out to
your configured PROCESSOR_URL in parallel. The processor is expected
to return chunks and 384-dim embeddings; Polymathy collects them, assigns sequential
u64 chunk IDs, and returns a JSON map.
That is the entire surface area in v0.2. Everything else — the SearxNG instance, the chunker, the embedding model, the downstream LLM that turns these chunks into a cited paragraph — is your problem. Honestly.
GET /v1/search?q=rust+async+patterns
{
"0": ["https://blog.rust-lang.org/...",
"Async functions in Rust return a Future..."],
"1": ["https://tokio.rs/...",
"Tokio provides an async runtime..."],
"2": ["https://example.com/...",
"When working with async Rust, you'll..."]
}Stack, as shipped
Boring Rust, on purpose.
HTTP
actix-web 4 with apistos for OpenAPI generation.
Built-in Swagger, ReDoc, RapiDoc, and Scalar UIs at /swagger,
/redoc, /rapidoc, /scalar.
Search frontend
SearxNG over reqwest. You point at any instance via
SEARXNG_URL. The service does not implement a crawler or index
of its own — SearxNG is the source of result URLs.
Vector primitives
usearch 2.12 instantiated as 384-dim, inner-product, F32 quantized
(src/index.rs). In v0.2 the index is created per request and held
in memory; the returned payload is the chunk map.
What this isn't
Polymathy is not a RAG product.
- no Embedding model bundled — the content processor handles that. The default request asks for
AllMiniLML6V2at 100-word chunks, but only the processor honours it. - no LLM call — Polymathy returns chunks. Whatever you do with them (rerank, stuff into a prompt, cite) lives downstream.
- no Replacement for Elasticsearch or Tantivy — it sits in front of a metasearch engine, not on top of an inverted index of your corpus.
- no Persistent index — the in-memory USearch index is created per request in the current release.
- no Auth, rate limiting, or multi-tenant isolation — that is your reverse proxy's job.
The README is blunt about this: “it's infrastructure for building answer engines, not an out-of-the-box product.” Treat the rest of the site the same way.
Use it when
Some honest fits.
You already run SearxNG
You have a SearxNG instance for privacy or aggregation reasons and want a small layer that turns its JSON output into reusable chunks for downstream assistants.
You're building a Perplexity-style UI
You need a backend that fetches a handful of pages per query, extracts their content, and returns it shaped for prompt-stuffing — and you want it in Rust because the rest of your stack is.
You're prototyping a chunker
You want a stable HTTP harness that exercises your content-processor service against real-world URLs from arbitrary queries. Polymathy is that harness.