fuel-price/docs/llm-prediction-providers.md

LLM Prediction Providers

The oil price direction prediction supports multiple LLM backends behind a shared interface. The active provider is selected via environment variable. All providers return the same response shape and fall back to EWMA if not configured or if the API call fails.

Selecting a Provider

Set LLM_PREDICTION_PROVIDER in .env:

LLM_PREDICTION_PROVIDER=anthropic   # default
LLM_PREDICTION_PROVIDER=openai
LLM_PREDICTION_PROVIDER=gemini

Each provider needs its own API key. If the key is missing or empty the provider returns null and EWMA is used instead.

---

Providers

Anthropic (default)

Key: ANTHROPIC_API_KEY
Model: ANTHROPIC_MODEL (default: claude-sonnet-4-6)

Uses tool use with a forced submit_prediction tool call — no JSON parsing, guaranteed schema. Structured output is enforced at the API level via tool_choice: { type: "tool", name: "submit_prediction" }.

Two-phase prediction flow:

1. Context phase — multi-turn web search (web_search_20250305 tool) for recent oil/geopolitical news (up to 5 iterations, pause_turn loop)
2. Submission phase — once searches are complete, forces a submit_prediction tool call with the full conversation context

If the context phase fails, falls back to a single-turn basic prediction (tool use only, no web search).

// Structured output schema (enforced by Anthropic)
'input_schema' => [
    'type' => 'object',
    'properties' => [
        'direction'  => ['type' => 'string', 'enum' => ['rising', 'falling', 'flat']],
        'confidence' => ['type' => 'integer', 'minimum' => 0, 'maximum' => 85],
        'reasoning'  => ['type' => 'string'],
    ],
    'required' => ['direction', 'confidence', 'reasoning'],
],

PredictionSource: llm_with_context (web search succeeded) or llm (basic fallback).

---

OpenAI

Key: OPENAI_API_KEY
Model: OPENAI_MODEL (default: gpt-4o-mini)

Uses response_format: json_schema with strict: true. The schema is sent to the API and the response is guaranteed to match it.

'response_format' => [
    'type' => 'json_schema',
    'json_schema' => [
        'name'   => 'oil_prediction',
        'strict' => true,
        'schema' => [
            'type' => 'object',
            'properties' => [
                'direction'  => ['type' => 'string', 'enum' => ['rising', 'falling', 'flat']],
                'confidence' => ['type' => 'integer'],
                'reasoning'  => ['type' => 'string'],
            ],
            'required'             => ['direction', 'confidence', 'reasoning'],
            'additionalProperties' => false,
        ],
    ],
],

Response is extracted from choices.0.message.content (a JSON string) and decoded.

PredictionSource: llm

---

Gemini

Key: GEMINI_API_KEY
Model: GEMINI_MODEL (default: gemini-2.0-flash)

Uses responseMimeType: application/json and responseSchema in generationConfig. The API key is passed as a query parameter.

'generationConfig' => [
    'responseMimeType' => 'application/json',
    'responseSchema'   => [
        'type' => 'OBJECT',
        'properties' => [
            'direction'  => ['type' => 'STRING', 'enum' => ['rising', 'falling', 'flat']],
            'confidence' => ['type' => 'INTEGER'],
            'reasoning'  => ['type' => 'STRING'],
        ],
        'required' => ['direction', 'confidence', 'reasoning'],
    ],
],

Response is extracted from candidates.0.content.parts.0.text (a JSON string) and decoded.

PredictionSource: llm

---

Confidence Caps

All providers cap confidence at 85 regardless of what the model returns. EWMA is capped at 65.

---

EWMA Fallback

OilPriceService::generatePrediction() always runs EWMA first and stores its result. The LLM provider runs after; its result is stored and returned if non-null. If the provider returns null (key missing, API error, malformed response), EWMA is returned instead.

generatePrediction()
├── generateEwmaPrediction()  → always stored
└── provider->predict()
    ├── on success            → stored and returned (LLM wins)
    └── on null               → EWMA returned

---

Adding a New Provider

1. Create app/Services/LlmPrediction/YourProvider.php implementing OilPredictionProvider
2. Add a case to the match in AppServiceProvider::register()
3. Add key/model config to config/services.php and document the .env vars

The interface requires one method:

public function predict(Collection $prices): ?PricePrediction;

Return null on any failure — the orchestrator handles the fallback.