vLLM and other OpenAI-compatible servers

Any server that speaks the OpenAI chat API (vLLM, LM Studio, Ollama) can write lens's prose captions. Lens supports it alone, or after a local tagger as hybrid:<tagger>+openai-compat.

Used by

Package	What for	Reads
argus-lens (opens in new tab)	`openai-compat` and `hybrid:*+openai-compat` backends. Default model `llava`, at `http://localhost:11434/v1`.	`ARGUS_OPENAI_COMPAT_BASE_URL`, `ARGUS_OPENAI_COMPAT_MODEL`, `ARGUS_OPENAI_COMPAT_API_KEY`, `ARGUS_OPENAI_COMPAT_PROMPT`, `--base-url`, `--model-id`
argus-mint (opens in new tab)	The card bio writer (an OpenAI-style chat endpoint).	`ARGUS_MINT_WRITER_URL`, `ARGUS_MINT_WRITER_KEY`

Official guides

Minimal setup

Serve a vision-language model with an OpenAI-compatible API. For example, with vLLM:
```
vllm serve Qwen/Qwen2.5-VL-7B-Instruct --port 8000
```

Point lens at it. Any of the ARGUS_OPENAI_COMPAT_* variables can be given as a CLI flag instead:

ARGUS_BACKEND=hybrid:wd14+openai-compat
ARGUS_OPENAI_COMPAT_BASE_URL=http://<GPU_HOST>:8000/v1
ARGUS_OPENAI_COMPAT_MODEL=Qwen/Qwen2.5-VL-7B-Instruct
ARGUS_OPENAI_COMPAT_API_KEY=   # only if the server wants one

hybrid:wd14+openai-compat tags locally with WD14 and asks the server for the prose. Plain openai-compat asks the server for everything. ARGUS_OPENAI_COMPAT_PROMPT picks a prompt preset (detailed, straightforward, descriptive, booru, count) or takes a literal prompt.

Darkroom's --expand is the exception: it speaks Ollama's native API only (see Ollama).