Docs

AnythingLLM

AnythingLLM reaches an endpoint it ships no named provider for through the one called Generic OpenAI: a Base URL, a key, and a model id you type yourself. Two of its five fields have defaults that quietly cost you.

AnythingLLM reaches an endpoint it ships no named provider for through the one called Generic OpenAI — Base URL with /v1, a key, and a model id you type yourself, since it fetches no list.

Instance Settings → AI Providers → LLM

LLM Provider           Generic OpenAI
Base URL               https://api.kunavo.com/v1
API Key                sk-kn-...
Selected Model         claude-sonnet-5
Model context window   1000000
Max Tokens             8192

The Base URL keeps the /v1 suffix. AnythingLLM's setup page does not say so in prose — its own shipped server/.env.example does, writing the same setting as GENERIC_OPEN_AI_BASE_PATH='http://proxy.url.openai.com/v1', and the provider source hands that value to the OpenAI SDK as baseURL unmodified, appending only the route. Both read on September 21, 2026.

Kunavo has not run AnythingLLM against this endpoint. This page is a reading of AnythingLLM's own documentation and shipped source on the date printed below, not a test report — a published setup page is not a compatibility result, and nothing here should be read as one. AnythingLLM's own page is blunter still: "This is a developer-focused llm provider - you should not use it unless you know what you are doing."

Two fields are worth filling in even though they look optional. Leave Model context window empty and the provider falls back to 4096, then budgets history, system prompt and user content against that invented window. Max Tokens falls back to 1024, which truncates long answers until you raise it.

This configures the chat half only. AnythingLLM's embedder is a separate provider, set system-wide, and Kunavo serves no embedding model — so do not point it here; its free built-in local embedder, or a dedicated embedding vendor, covers that half. Kunavo serves no speech-to-text or text-to-speech model either, so AnythingLLM's Transcription and Voice & Speech providers are not this endpoint either.

Step by step

Create a key at /app/keys and copy it — it is shown once.
Open Instance Settings → AI Providers → LLM. The path is the same on Desktop and on a self-hosted instance; only the way you reach the settings pane differs.
In LLM Provider, pick Generic OpenAI — described on its own card as "Connect to any OpenAi-compatible service via a custom configuration".
Fill in Base URL, API Key and Selected Model as above. Selected Model is a text field, not a fetched list: the provider runs no pre-flight check on it, so a mistyped id is accepted at save time and fails on the first message.
Set Model context window and Max Tokens rather than leaving them blank, then Save changes.
Send one message in a workspace. A workspace can override the system LLM, so if the answer looks like the old provider, check the workspace's own model setting before touching anything here.

Checked against AnythingLLM's OpenAI (Generic) LLM page on September 21, 2026. Third-party settings move; if a field name here no longer matches what you see, that page is the authority, not this one.

This is the short version. The full walkthrough — model choice, what a real session costs, and the failure modes — is in what an AnythingLLM deployment actually costs.

Verify before you debug the client

One request settles whether a failure is the endpoint, the key, or the configuration file. If this returns JSON, the same base URL and key work in AnythingLLM.

# Settles whether a failure is the endpoint, the key, or the client.
curl -sS https://api.kunavo.com/v1/models \
  -H "Authorization: Bearer sk-kn-..."

Which model id to put in the field

Every text model is reachable as a model id — the live list is GET /v1/models, and the catalog with prices is on the models page. Rates are USD per 1M tokens, input / output.

Model id	Kunavo in / out	Where it fits in AnythingLLM
`claude-sonnet-5`	$2.00 / $10.00	the everyday workspace model for document Q&A
`claude-haiku-4-5`	$0.40 / $2.00	high-volume workspaces where retrieval supplies the facts
`claude-opus-5`	$2.00 / $10.00	an agent or synthesis workspace, where a wrong answer is expensive
`gpt-5-6-sol`	$2.00 / $12.00	a second family for the same key, per workspace
`gemini-3-1-pro`	$0.70 / $4.20	long documents attached whole rather than retrieved

Billing is per token from a prepaid balance with no monthly fee — see billing. On repeated context — which is most of what an editor or a chat client sends — prompt caching moves the bill more than the model choice does.

FAQ

How do I add a custom API endpoint to AnythingLLM?

Open Instance Settings, then AI Providers, then LLM, and choose the provider named Generic OpenAI — AnythingLLM's own card for it reads "Connect to any OpenAi-compatible service via a custom configuration". It gives you five fields: Base URL, API Key, Selected Model, Model context window and Max Tokens, plus a Save changes button. That provider is the supported route for any OpenAI-compatible endpoint AnythingLLM ships no named integration for. Check the provider list first, though: services such as LM Studio, LocalAI and LiteLLM have their own named providers with their own base-URL fields and their own defaults.

Does the AnythingLLM Base URL need /v1 at the end?

Yes, for an OpenAI-compatible endpoint. The setup page itself does not state the form in prose, so the sentence that settles it is AnythingLLM's own shipped server/.env.example, which writes the same setting as GENERIC_OPEN_AI_BASE_PATH='http://proxy.url.openai.com/v1'. Its provider source passes that value to the OpenAI SDK as baseURL without modifying it, so only the route — /chat/completions — is appended. For Kunavo that makes the Base URL https://api.kunavo.com/v1. A missing /v1 shows up as a 404 rather than an authentication error. Both sources read September 21, 2026.

Why does AnythingLLM show no model list for the Generic OpenAI provider?

Because there is no list to show: Selected Model is a text field you type into, and the provider performs no pre-flight validation of what you type. That is by design for a provider meant to reach endpoints AnythingLLM knows nothing about, and it means a typo is accepted when you save and fails only when you send a message. To separate a bad model id from a bad URL or key, run the /v1/models request above against the same pair — JSON back means the endpoint and key are fine and the id is the thing to fix.

Can Kunavo be AnythingLLM's embedder as well as its chat model?

No. Kunavo serves no embedding model, so the embedder is a half of AnythingLLM it does not cover, and AnythingLLM's embedder is a separate provider from the chat one anyway — set system-wide rather than per workspace, and awkward to change later because switching means deleting and re-embedding every document. Use the built-in local embedder that ships with the Desktop build, or a dedicated embedding vendor, and keep this endpoint for the chat provider. Kunavo serves no speech-to-text or text-to-speech model either, so Transcription and Voice & Speech are outside it too.

Has Kunavo tested AnythingLLM against its endpoint?

No. Kunavo has run none of the clients in this section, and this page is not a test report — it is a reading of AnythingLLM's own documentation and shipped source, dated so you can see how stale it is. What it can tell you honestly is which fields exist, what AnythingLLM does with each value, and which defaults cost money. What it cannot tell you is that agent tool calling, streaming or attachments behaved a particular way in your build, because nobody here ran it. Send one message before you move a workspace over.

Can I point AnythingLLM's Anthropic provider at a third-party endpoint instead?

Not through anything AnythingLLM exposes or documents: that provider has no Base URL field in the UI and its source builds the Anthropic SDK client with a key and headers only. The supported way in for a third-party endpoint is Generic OpenAI over chat completions, which is what this page configures. One consequence to plan around: the Generic OpenAI path sends no cache_control marker, so a protocol that requires the client to ask for caching gets no hit through it — whether an endpoint caches without a client marker is a question for that endpoint's own documentation.