Groq

Route CortexDB's answer and extraction LLM calls through Groq's fast inference API.

Groq exposes an OpenAI-compatible API, so CortexDB's LLM routers point straight at it for very low-latency answer generation.

Set both the extraction and answer lanes

CORTEX_LLM_* is the entity-extraction lane; /v1/answer uses the separate CORTEX_ANSWER_* lane (default anthropic/claude-opus-4-6). Setting CORTEX_LLM_* alone leaves /v1/answer on the disabled default — set both. See Self-hosting defaults.

Deployment configuration

# Extraction lane
CORTEX_LLM_URL=https://api.groq.com/openai/v1
CORTEX_LLM_MODEL=llama-3.3-70b-versatile
CORTEX_ENTITY_API_KEY=$GROQ_API_KEY

# Answer lane — Groq is OpenAI-compatible, so use the openai provider + Groq URL
CORTEX_ANSWER_PROVIDER=openai
CORTEX_ANSWER_URL=https://api.groq.com/openai/v1
CORTEX_ANSWER_MODEL=llama-3.3-70b-versatile
CORTEX_ANSWER_API_KEY=$GROQ_API_KEY

Embeddings run elsewhere

Groq does not serve embeddings — keep CORTEX_EMBEDDING_* on OpenAI or a local Ollama (nomic-embed-text). See Embeddings.

Restart the service, then confirm the boot log shows Answer LLM router enabled for /v1/answer with the Groq model, and check a test answer in Groq's own usage dashboard (diagnostics.answer_model echoes the configured or requested name and does not prove routing).

Choosing the model

The answer model is the one in CORTEX_ANSWER_MODEL. A per-request answer_model on /v1/answer is only echoed in diagnostics.answer_model; the configured model answers. To serve a different model, change CORTEX_ANSWER_MODEL or run a second deployment. See LLM & Answer Generation.

Deployment configuration

Choosing the model

See also

On this page