Quickstart

From zero to a running Vectoria server with real product data and a searchable demo storefront.
Time: ~10 minutes, plus ~5 minutes on first run to download the embedding model and ESCI data.

Prerequisites

• Rust 1.80+ — install at rustup.rs
• curl or wget
• python3 (for the webstore frontend)
• ~1.5 GB free disk space (ESCI data + embedding model)

1. Clone and build

git clone https://github.com/gleicon/vectoria
cd vectoria
cargo build --release -p vectoria-server -p vectoria-cli

Or use Make:

make build

2. Configure

A ready-to-use vectoria.toml is included. It sets port 7700, API key vectoria-esci-demo, and in-memory index (data resets on restart).

For a persistent index, edit vectoria.toml:

[index]
vector_backend = "edgestore-hnsw"

[storage]
path = "./data/vectoria.db"

3. Start the server

Option A — Docker Compose (recommended, no Rust required):

# Full image — ONNX model downloaded on first start
VECTORIA_API_KEY=my-secret-key docker compose up

# Run in background
VECTORIA_API_KEY=my-secret-key docker compose up -d

The compose file mounts two named volumes: /data for the index and /root/.cache/fastembed for the model cache. The model (~40 MB) downloads once.

Option B — Docker run:

docker build --target vectoria-full -t vectoria:full .
docker run -p 7700:7700 \
  -v vectoria-data:/data \
  -v fastembed-cache:/root/.cache/fastembed \
  -e VECTORIA_API_KEY=my-secret-key \
  vectoria:full

Option C — from source (foreground):

cargo run --release -p vectoria-server
# or: make server

Option D — from source (background):

make server-bg
# Waits until the server responds at http://localhost:7700/health
# Logs → /tmp/vectoria.log

On first run the server downloads multilingual-e5-small (~40 MB), then prints:

api_key: vectoria-esci-demo
INFO listening on http://0.0.0.0:7700

Verify:

curl http://localhost:7700/health
# {"status":"ok","version":"0.1.8"}

4. Load the ESCI demo dataset

make esci-import

This:

1. Downloads shopping_queries_dataset_products.parquet (~1.1 GB) to data/esci/ — skipped on subsequent runs
2. Downloads shopping_queries_dataset_examples.parquet (~68 MB)
3. Imports 5,000 US-locale products into the running server

To import more products or a different locale:

make esci-import MAX_PRODUCTS=50000 LOCALE=es

5. Open the demo storefront

make webstore

Open http://localhost:8080. The demo store lets you search in three modes — BM25, semantic, and hybrid — and shows per-result score breakdowns.

6. Benchmark search quality

make esci-judges   # writes data/esci/judges.ndjson
make bench         # Recall@K, NDCG@K, MRR across bm25 / semantic / hybrid

Sample output (ESCI, 5,000 US products, E+S labels, 117 judged queries):

── Mode: bm25 ──────────────────────────────
  Coverage:     100.0%
  Recall@10:    0.5347
  NDCG@10:      0.5796
  MRR:          0.6595
  Latency p50:  0.4ms

── Mode: semantic ──────────────────────────
  Coverage:     100.0%
  Recall@10:    0.4407
  NDCG@10:      0.4765
  MRR:          0.5690
  Latency p50:  2.0ms

── Mode: hybrid ────────────────────────────
  Coverage:     100.0%
  Recall@10:    0.4914
  NDCG@10:      0.5635
  MRR:          0.6576
  Latency p50:  2.2ms

All modes reach 100% coverage. Zero-result queries are retried with spell correction (compound splits + typo correction on alphabetic tokens) before falling back to semantic results.

7. Import your own data

NDJSON (one product per line):

{"id":"sku-001","text":"Blue trail running shoe waterproof","metadata":{"title":"Trail X","brand":"Merrell","price":149.99,"in_stock":true}}
{"id":"sku-002","text":"Yoga mat non-slip extra thick 6mm","metadata":{"title":"ProMat","brand":"Manduka","price":89.00,"in_stock":true}}

vectoria import products.ndjson \
  --server http://localhost:7700 \
  --api-key vectoria-esci-demo

CSV (any columns; id, sku, or product_id used as the product ID):

vectoria import catalog.csv --server http://localhost:7700 --api-key vectoria-esci-demo

Parquet (all string and numeric columns mapped to metadata automatically):

vectoria import catalog.parquet --server http://localhost:7700 --api-key vectoria-esci-demo

8. Query the API directly

API_KEY=vectoria-esci-demo

# Hybrid search
curl -X POST http://localhost:7700/search \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"q":"waterproof trail shoes","limit":5,"mode":"hybrid"}'

# Similar products by ID
curl http://localhost:7700/products/sku-001/similar \
  -H "Authorization: Bearer $API_KEY"

# Record a click event — include "query" to activate query-level CTR ranking
# Products clicked for this query will rank higher for future searches of the same query
curl -X POST http://localhost:7700/events \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"event_type":"click","product_id":"sku-001","query":"waterproof trail shoes"}'

# Stats
curl http://localhost:7700/stats -H "Authorization: Bearer $API_KEY"

9. Stop and clean up

make kill       # stop background server
make clean      # delete downloaded ESCI parquet files (not the model cache)

The embedding model cache stays at ~/.cache/fastembed/. Delete it manually to reclaim ~40 MB.

10. Use as an embedded Rust library

No HTTP server required. Add vectoria-core to your Cargo.toml:

[dependencies]
vectoria-core = "0.1.8"

Async (Tokio):

use vectoria_core::{SearchEngineBuilder, model::{SearchRequest, SearchMode}};

let engine = SearchEngineBuilder::new()
    .query_cache(300, 1_000)
    .build()
    .await?;

engine.index(product).await?;

let results = engine.search(SearchRequest {
    q: "running shoes".into(),
    mode: SearchMode::Hybrid,
    limit: 10,
    ..Default::default()
}).await?;

Sync (no runtime required in caller):

use vectoria_core::{SearchEngineSync, model::SearchRequest};

let engine = SearchEngineSync::new()?;
engine.index(product)?;

let results = engine.search(SearchRequest {
    q: "running shoes".into(),
    ..Default::default()
})?;

Defaults: in-memory storage, local multilingual-e5-small ONNX embeddings. Override with builder methods (.storage(), .vector_index(), .embedding(), .weights()). See API reference — Embedded library for the full builder options table.

Make target reference

Override variables on the command line:

make esci-import MAX_PRODUCTS=10000 LOCALE=es SERVER=http://myserver:7700 API_KEY=mykey

Next steps

• API reference — full HTTP endpoint docs and embedded library usage
• crates.io — vectoria-core — embed in your Rust app
• GitHub — source, issues, releases