# LiteLLM

[LiteLLM](https://litellm.ai/) provides a shared Python interface across model
providers. Use its `openai/` routing prefix to call Aurora's Chat Completions API.

> Available Inference endpoint: `https://ai.aur.lu/v1` (Aurora).

## Install and prepare

Use a supported Python version in a virtual environment:

```bash
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade litellm
export AURORA_API_TOKEN="<your-token>"
```

On Windows, activate with `.venv\Scripts\Activate.ps1` in PowerShell. Create the
token using [Getting started](https://docs.aur.lu/docs/inference/getting-started).

## Send a request

Save as `aurora_litellm.py`:

```python
import os
from litellm import completion

response = completion(
    model="openai/deepseek/deepseek-v4-flash",
    api_base="https://ai.aur.lu/v1",
    api_key=os.environ["AURORA_API_TOKEN"],
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(response.choices[0].message.content)
print(response.usage)
```

```bash
python aurora_litellm.py
```

Expect a greeting and token usage. LiteLLM strips the first `openai/` routing
prefix and sends `deepseek/deepseek-v4-flash` to Aurora. Omitting that prefix
can select LiteLLM's direct DeepSeek provider. `api_base` includes `/v1`.

## Stream a response

Replace the request and print statements with:

```python
stream = completion(
    model="openai/deepseek/deepseek-v4-flash",
    api_base="https://ai.aur.lu/v1",
    api_key=os.environ["AURORA_API_TOKEN"],
    messages=[{"role": "user", "content": "Explain streaming in three sentences."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if getattr(chunk, "usage", None):
        print("\nUsage:", chunk.usage)
print()
```

## Use an existing LiteLLM proxy

If you already run a LiteLLM proxy, append this entry to its `model_list`:

```yaml
model_list:
  - model_name: aurora-deepseek
    litellm_params:
      model: openai/deepseek/deepseek-v4-flash
      api_base: https://ai.aur.lu/v1
      api_key: os.environ/AURORA_API_TOKEN
```

Set `AURORA_API_TOKEN` in the **proxy process** environment and reload/restart
the proxy with the updated configuration. Downstream clients use the proxy URL,
its configured authentication, and the alias `aurora-deepseek`. The Aurora token
belongs on the proxy's upstream connection. For a new proxy deployment, follow
the [LiteLLM proxy guide](https://docs.litellm.ai/docs/simple_proxy).

## Troubleshooting

- **Wrong provider:** Keep `openai/` before the entire Aurora ID.
- **Model not found through a proxy:** Clients use `model_name` (`aurora-deepseek`);
  the upstream model ID belongs under `litellm_params`.
- **Missing token:** The Python SDK reads the environment where the script runs;
  a proxy needs the token in its own environment.
- **Cost estimates unavailable:** Custom model metadata may not be present in
  LiteLLM's catalog. Use Aurora's Portal for actual usage and pricing.

See [shared troubleshooting](https://docs.aur.lu/docs/inference/integrations#troubleshooting) and the
[LiteLLM OpenAI-compatible reference](https://docs.litellm.ai/docs/providers/openai_compatible).