Embedding Models

Embedding models convert text into vectors, enabling semantic search in knowledge bases. They are required for document processing and the Retrieval-Augmented Generation (RAG) approach.

Basic Data

Field	Required	Description
Image / Title	Yes	Display name and optional profile image of the model (e.g. “Azure OpenAI - text-embedding-3-small”).
Model Name	Yes	Technical model name (e.g. `text-embedding-3-small`).
Credentials	Yes	Stored credentials for the selected provider (dropdown selection).

Costs

Field	Required	Description
Cost in $ per Million Output Tokens	No	Cost per one million output tokens for cost tracking.

DLP Security Settings

Decide whether texts are pseudonymized before they reach this embedding model – when documents are indexed and on every search. This matters most when the embedding model runs at a cloud provider.

Option	Description
enforced	Documents and search queries are always pseudonymized before they reach the embedding model.
optional (default)	Documents from synced sources and all search queries are pseudonymized; for manual uploads the user decides in the upload dialog.
deactivated	The embedding model receives documents and search queries in clear text.

Changing this setting does not trigger a new indexing run. Documents that are already indexed keep their embeddings; the new setting applies to search queries and to future indexing runs.

This setting only concerns what the embedding model receives. The knowledge base stores its content in clear text, so the full-text search also finds names, IBANs or case numbers. What the language model sees of the search results is set by the Data protection (DLP) setting of each knowledge base: with Pseudonymized and DLP active in the chat it receives pseudonymized results, without DLP in the chat or with Unchecked the clear text.

Documents indexed before version 2026.10 still contain pseudonyms instead of the clear text in their stored text chunks. This only changes once you run one of the following commands:

./flow knowledgebaseembedding:reindex queues documents whose text chunks still contain pseudonyms for a new indexing run. The knowledgebaseembedding:generateembeddings cron job then processes them, which sends the texts to the embedding model again.
./flow knowledgebaseembedding:reindex --restore-tokens restores the clear text directly in the database without indexing the documents again. Documents whose pseudonyms cannot all be resolved are queued for a new indexing run.

With --dry-run, both commands only list the affected documents without changing anything.

ICAP DLP Integration

Connect your data exit with your enterprise DLP solution.

Option	Description
On	ICAP server is used for all content.
Off	ICAP server is not used.

Embedding models have fewer configuration options compared to chat models, as they are used exclusively for text vectorization and do not have direct user interaction.

Running a Local Embedding Model

Instead of using an external provider, you can generate embeddings on your own server. The content of your knowledge bases then stays on the server during vectorization. VARIOS AI connects a local model through OpenAI Compatible credentials. Any model server that provides the OpenAI /v1/embeddings interface is suitable. This guide describes two model servers that run as an additional service in your Docker Compose installation. Both use the multilingual model bge-m3.

Text Embeddings Inference, Ollama and the bge-m3 model are third-party offerings and not part of VARIOS AI. The following configuration is a suggestion that you adapt to your environment. You are responsible for operating, updating and securing the model server and for complying with the license terms of the software and the model.

	Text Embeddings Inference (TEI)	Ollama
Concurrent requests	Batched and processed in parallel.	Processed one after another.
Model download	Automatic on first start from Hugging Face.	Manual with `ollama pull`.
Model in memory	Stays loaded as long as the service runs.	Unloaded after 5 minutes without a request unless configured otherwise.

Use TEI if many users search knowledge bases at the same time or if large amounts of data are embedded.

VARIOS AI stores all embeddings with 1,536 dimensions. Shorter vectors are padded, longer ones are truncated. Therefore, use a model with at most 1,536 dimensions. bge-m3 produces vectors with 1,024 dimensions.

Setting Up the Model Server

Text Embeddings Inference
Ollama

Add the service to docker-compose.yml

Add a tei service under services in your docker-compose.yml. The service must be on the same network as the php service, which is internal in the standard installation.

# docker-compose.yml

services:
  [...]
  tei:
    image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.9
    restart: always
    environment:
      MODEL_ID: "BAAI/bge-m3"
      MAX_BATCH_TOKENS: "2048"
    volumes:
      - "./TEI:/data"
    networks:
      - internal

Hugging Face provides official images for each type of hardware. cpu-1.9 runs without a GPU. Servers with an NVIDIA GPU have their own images, for example 89-1.9 for the Ada Lovelace generation. The update helper runs docker compose pull on every update; with a fixed version, TEI is only updated when you change the version yourself.MODEL_ID sets the model. MAX_BATCH_TOKENS limits how many tokens TEI processes in one pass and therefore also the maximum length of an input; TEI truncates longer inputs. Without this setting, TEI uses 16384 and checks at startup with a request of this size whether the model runs. The memory of a server without a GPU is often not sufficient for this. With the default chunk size of 2,000 characters, 2048 is usually sufficient because a token typically covers several characters. Increase the value if you increase the chunk size.The ./TEI directory on the host stores the downloaded model. Do not publish a port for the service: VARIOS AI reaches TEI over the internal network.

Start the service

docker compose up -d tei
docker compose logs -f tei

On first start, TEI downloads the model from Hugging Face. This can take a few minutes. If the server needs a proxy server for this, read Downloading the Model Through a Proxy Server first.TEI only accepts requests once the model has been downloaded and loaded. Wait until the log shows the Ready message, and stop the output with Ctrl+C. Then check whether the PHP container can reach the model:

docker compose exec php curl -sS http://tei:80/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"BAAI/bge-m3","input":"Test"}'

The response contains a vector under embedding in the data field.

Add the service to docker-compose.yml

Add an ollama service under services in your docker-compose.yml. The service must be on the same network as the php service, which is internal in the standard installation.

# docker-compose.yml

services:
  [...]
  ollama:
    image: ollama/ollama:<version>
    restart: always
    environment:
      OLLAMA_KEEP_ALIVE: "-1"
    volumes:
      - "./Ollama:/root/.ollama"
    networks:
      - internal

Replace <version> with a fixed version from Docker Hub. The update helper runs docker compose pull on every update; with a fixed version, Ollama is only updated when you change the version yourself.OLLAMA_KEEP_ALIVE sets how long Ollama keeps a model in memory after the last request. Without this setting, Ollama unloads the model after 5 minutes. The next request then has to load it again, so the first search in a knowledge base takes longer. The value -1 keeps the model loaded permanently, which keeps the memory occupied. To free the memory during quiet periods, specify a duration instead, for example 24h.The ./Ollama directory on the host stores the downloaded models. Do not publish a port for the service: VARIOS AI reaches Ollama over the internal network.

Start the service and download the model

If the server needs a proxy server for the download, read Downloading the Model Through a Proxy Server first.

docker compose up -d ollama
docker compose exec ollama ollama pull bge-m3

Then check whether the PHP container can reach the model:

docker compose exec php curl -sS http://ollama:11434/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"bge-m3","input":"Test"}'

The response contains a vector under embedding in the data field.

Ollama always processes requests to an embedding model one after another. The OLLAMA_NUM_PARALLEL setting only applies to chat models and has no effect here. If several requests arrive at the same time, for example searches from several chats while indexing is running, Ollama places them in a queue. A search request contains only a short text and is processed accordingly quickly. However, the time spent in the queue counts toward the 30-second timeout that VARIOS AI uses for every request.

Downloading the Model Through a Proxy Server

TEI and Ollama need internet access to download the model. If this access must go through a proxy server, set the proxy server in the HTTPS_PROXY environment variable of the respective service:

    environment:
      [...]
      HTTPS_PROXY: "http://proxy.example.com:8080"

If the proxy server requires authentication, add the user name and password to the address, for example http://user:password@proxy.example.com:8080. The password is then stored in plain text in docker-compose.yml. Do not also set HTTP_PROXY for Ollama. Ollama downloads models over HTTPS only; according to Ollama, HTTP_PROXY can interfere with the connection from ollama pull to its own service.

The proxy server only affects the model server’s connection to the internet. Requests from VARIOS AI to the model server run over the internal Docker network. You configure the proxy server of VARIOS AI itself under General, see Troubleshooting.

Proxy Servers with TLS Inspection

If the proxy server breaks up the TLS connection and re-signs it with its own certificate, the model server must trust the proxy server’s CA certificate. Mount the certificate as for the PHP container and add it to the container’s trust store with update-ca-certificates at startup. The requirements for the certificate file are described in Setting Up Trusted Certificate Authorities.

Text Embeddings Inference
Ollama

# docker-compose.yml

tei:
  [...]
  entrypoint:
    - /bin/sh
    - -c
    - update-ca-certificates && exec text-embeddings-router --json-output
  volumes:
    - "./TEI:/data"
    - "./certs/internal-ca.crt:/usr/local/share/ca-certificates/internal-ca.crt:ro"

# docker-compose.yml

ollama:
  [...]
  entrypoint:
    - /bin/sh
    - -c
    - update-ca-certificates && exec /bin/ollama serve
  volumes:
    - "./Ollama:/root/.ollama"
    - "./certs/internal-ca.crt:/usr/local/share/ca-certificates/internal-ca.crt:ro"

Then restart the service with docker compose up -d tei or docker compose up -d ollama.

Setting Up the Model in VARIOS AI

Create credentials

Under Credentials, create new credentials with the provider OpenAI Compatible.

Field	Text Embeddings Inference	Ollama
API Base URL	`http://tei:80/v1`	`http://ollama:11434/v1`
API Key	Any value, for example `local`	Any value, for example `local`

Neither model server checks the API key in this configuration. However, VARIOS AI requires the field.

Create the embedding model

Under Models, create a new model in the Embedding Models tab. Enter BAAI/bge-m3 (TEI) or bge-m3 (Ollama) as the Model Name and select the credentials you just created under Credentials. You can leave the cost fields empty.

Use the model

Select the model as the AI model for generating embeddings in a knowledge base. To use it for all new knowledge bases, set it as the Default AI model for generating embeddings under General.

If you change the embedding model of an existing knowledge base, VARIOS AI re-embeds all documents of that knowledge base with the new model. Until processing is complete, search only returns results from documents that have already been processed.

Both model servers run without a GPU but require additional memory and CPU capacity. Plan for this in addition to the sizing guide, especially if you embed large amounts of data.

Troubleshooting

Documents remain in the Error or Processing status

Use docker compose ps to check whether the tei or ollama service is running, and docker compose logs tei or docker compose logs ollama to check whether the model has been loaded. For Ollama, docker compose exec ollama ollama list shows the downloaded models. The Model Name in VARIOS AI must match the model name in the model server.

Error: Failed to connect to tei port 80

TEI only opens the port once the model has been downloaded and loaded. Use docker compose logs tei to check whether the Ready message appears. If the download is still running, wait for it to finish. If docker compose ps tei shows that the service keeps restarting, the log names the cause, for example a failed download.

TEI keeps restarting after “Warming up model” (exited with code 137)

Code 137 means that the process was killed, usually because of insufficient memory. TEI uses the most memory during warm-up. Set MAX_BATCH_TOKENS to 2048 as described above and restart the service with docker compose up -d tei. A smaller value reduces memory usage further. Note that MAX_BATCH_TOKENS counts tokens, not characters: if a chunk contains more tokens than this value, TEI truncates it and the remainder is not part of the embedding.

The model cannot be downloaded

Check whether the server reaches the internet directly or through the proxy server set in HTTPS_PROXY. If the log reports a certificate error, the proxy server most likely breaks up the TLS connection. In that case, mount the CA certificate as described in Proxy Servers with TLS Inspection.

The first search after a pause is slow or fails (Ollama)

Ollama has unloaded the model and reloads it for the request. VARIOS AI waits at most 30 seconds for a response. Use docker compose exec ollama ollama ps to check whether the model is loaded and until when it stays loaded. Set OLLAMA_KEEP_ALIVE as described above and restart the service with docker compose up -d ollama.

The connection only fails when a proxy server is active in VARIOS AI

If a proxy server is configured under General, VARIOS AI also routes requests to the model server through the proxy. Add tei or ollama to Proxy exceptions (comma-separated hostnames).

The hostname tei or ollama cannot be resolved

The model server and the php service are not on the same Docker network. Check the networks entry of both services in docker-compose.yml.

Getting Started

Usage

Administration

Installation

Operations

Security

Changelog

Basic Data

Costs

DLP Security Settings

ICAP DLP Integration

Running a Local Embedding Model

Setting Up the Model Server

Downloading the Model Through a Proxy Server

Proxy Servers with TLS Inspection

Setting Up the Model in VARIOS AI

Troubleshooting

​Basic Data

​Costs

​DLP Security Settings

​GDPR-Compliant Data Protection

​ICAP DLP Integration

​Running a Local Embedding Model

​Setting Up the Model Server

​Downloading the Model Through a Proxy Server

​Proxy Servers with TLS Inspection

​Setting Up the Model in VARIOS AI

​Troubleshooting

Basic Data

Costs

DLP Security Settings

GDPR-Compliant Data Protection

ICAP DLP Integration

Running a Local Embedding Model

Setting Up the Model Server

Downloading the Model Through a Proxy Server

Proxy Servers with TLS Inspection

Setting Up the Model in VARIOS AI

Troubleshooting