Runtime and providers¶
Runtime modes¶
| Mode | Label | Backend | Use |
|---|---|---|---|
live |
Backend status | Yes | Use the deployed backend and real service connections. |
replay |
DETERMINISTIC REPLAY |
No | Use a read-only fallback scenario. |
The mode is explicit. Orion does not change live to replay after a failure.
Active inference¶
The production target uses:
INFERENCE_PROVIDER=openrouter
MODEL_NAME=deepseek/deepseek-v4-flash-0731
MODEL_FALLBACK_NAME=qwen/qwen3.7-flash
EMBEDDING_MODEL=openai/text-embedding-3-small
EMBEDDING_DIMENSIONS=1024
The fallback is one bounded request after a permitted transient primary error. It is not a silent model change. The trace reports the model that answered.
Future-only adapters¶
Bedrock/Luna and cloud speech code remain isolated for a future deployment. They are not active fallback paths. A failed OpenRouter request remains a live error.
Provider errors¶
| Error | Meaning |
|---|---|
401 or 403 |
Key or access problem. |
402 |
Credit or billing limit. |
404 |
Model or endpoint is not available. |
429 |
Rate or quota limit. |
408 or 5xx |
Timeout or provider failure. |
| Invalid JSON | Model output does not match the contract. |
| Wrong vector length | Embedding is not exactly 1,024 values. |
The UI shows the error. It keeps learner input when possible.
Cold start¶
The first live load can take several seconds. Lambda may wake from standby. It may load the secret and open a CockroachDB connection. The UI shows a startup state and keeps actions locked until the snapshot is ready.
Cost boundary¶
AWS has a USD 20 monthly budget boundary for the deployed stack. OpenRouter is outside AWS billing. The OpenRouter account or key needs its own credit limit.
Application limits also reduce cost:
- bounded prompt size;
- bounded output tokens;
- 12-second model and embedding timeouts in the deployment script;
- 8-second Managed MCP timeout in the deployment script;
- one schema repair;
- one permitted fallback request.