Skip to main content

LiteLLM

LiteLLM provides a shared Python interface across model providers. Use its openai/ routing prefix to call Aurora's Chat Completions API.

Select Inference Endpoint
Choose preferred region.

Install and prepare​

Use a supported Python version in a virtual environment:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade litellm
export AURORA_API_TOKEN="<your-token>"

On Windows, activate with .venv\Scripts\Activate.ps1 in PowerShell. Create the token using Getting started.

Send a request​

Save as aurora_litellm.py:

import os
from litellm import completion

response = completion(
  model="openai/deepseek/deepseek-v4-flash",
  api_base="",
  api_key=os.environ["AURORA_API_TOKEN"],
  messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(response.choices[0].message.content)
print(response.usage)
python aurora_litellm.py

Expect a greeting and token usage. LiteLLM strips the first openai/ routing prefix and sends deepseek/deepseek-v4-flash to Aurora. Omitting that prefix can select LiteLLM's direct DeepSeek provider. api_base includes /v1.

Stream a response​

Replace the request and print statements with:

stream = completion(
  model="openai/deepseek/deepseek-v4-flash",
  api_base="",
  api_key=os.environ["AURORA_API_TOKEN"],
  messages=[{"role": "user", "content": "Explain streaming in three sentences."}],
  stream=True,
  stream_options={"include_usage": True},
)

for chunk in stream:
  if chunk.choices:
      print(chunk.choices[0].delta.content or "", end="", flush=True)
  if getattr(chunk, "usage", None):
      print("\nUsage:", chunk.usage)
print()

Use an existing LiteLLM proxy​

If you already run a LiteLLM proxy, append this entry to its model_list:

model_list:
- model_name: aurora-deepseek
  litellm_params:
    model: openai/deepseek/deepseek-v4-flash
    api_base: 
    api_key: os.environ/AURORA_API_TOKEN

Set AURORA_API_TOKEN in the proxy process environment and reload/restart the proxy with the updated configuration. Downstream clients use the proxy URL, its configured authentication, and the alias aurora-deepseek. The Aurora token belongs on the proxy's upstream connection. For a new proxy deployment, follow the LiteLLM proxy guide.

Troubleshooting​

  • Wrong provider: Keep openai/ before the entire Aurora ID.
  • Model not found through a proxy: Clients use model_name (aurora-deepseek); the upstream model ID belongs under litellm_params.
  • Missing token: The Python SDK reads the environment where the script runs; a proxy needs the token in its own environment.
  • Cost estimates unavailable: Custom model metadata may not be present in LiteLLM's catalog. Use Aurora's Portal for actual usage and pricing.

See shared troubleshooting and the LiteLLM OpenAI-compatible reference.