Ollama
Use a locally-hosted Ollama model as the answer model and entity-extraction LLM for CortexDB.
CortexDB routes its LLM calls through configured routers. Point them at an Ollama instance to keep all inference local — no data leaves your network.
Two lanes: extraction (CORTEX_LLM_*) and answer (CORTEX_ANSWER_*)
CORTEX_LLM_* configures the entity-extraction lane. /v1/answer uses the separate
CORTEX_ANSWER_* lane, which defaults to provider=anthropic / claude-opus-4-6 (baked into the
image). Setting CORTEX_LLM_* alone leaves /v1/answer on the disabled Anthropic default — you
must set both lanes. See Self-hosting defaults.
Deployment configuration
Set these on the cortex server:
# Entity-extraction lane
CORTEX_LLM_URL=http://localhost:11434/v1
CORTEX_LLM_MODEL=llama3.1:70b
CORTEX_ENTITY_API_KEY=ollama # placeholder, but must be non-empty or extraction is disabled
# Answer lane — REQUIRED for /v1/answer to route through Ollama
CORTEX_ANSWER_PROVIDER=ollama
CORTEX_ANSWER_URL=http://localhost:11434
CORTEX_ANSWER_MODEL=llama3.1:70b
CORTEX_ANSWER_API_KEY=ollama # placeholder
# Embedding (for vector recall)
CORTEX_EMBEDDING_PROVIDER=ollama
CORTEX_EMBEDDING_MODEL=nomic-embed-text
CORTEX_EMBEDDING_DIMS=768
CORTEX_EMBEDDING_API_KEY=ollama # non-empty required even for OllamaRestart the service to pick up the new config, then confirm /v1/answer is live:
curl -s http://localhost:3141/v1/admin/ready # degraded:false = real embeddingsChoosing the model
The answer model is the one in CORTEX_ANSWER_MODEL. A per-request answer_model on /v1/answer is
only echoed in diagnostics.answer_model; the configured model answers. To serve a different
model, change CORTEX_ANSWER_MODEL or run a second deployment. See
LLM & Answer Generation.
Pulling models
ollama pull llama3.1:70b
ollama pull nomic-embed-textFor benchmark-grade accuracy use the largest model your GPU fits (Llama 3.1 70B or Qwen 2.5 72B); smaller models lose 5–15 pp on multi-session reasoning.
Embeddings can stay on Ollama or go to OpenAI; pick by hardware
The embedding lane is independent of the answer and extraction lanes. On a high-end machine (spare
CPU and RAM, ideally a GPU) or when text must never leave the box, keep nomic-embed-text on Ollama as
above. On modest hardware (a laptop, no GPU, limited RAM or disk) it is fine, and simpler, to leave
embeddings on OpenAI's text-embedding-3-small (just set CORTEX_EMBEDDING_API_KEY and drop the four
CORTEX_EMBEDDING_* lines above) while the LLM lanes stay on Ollama. Decide before the first write: the
embedding provider is pinned to the data directory, and switching later is an offline rebuild
(see Embeddings).