Local

Source: examples/model_selection/local.py

Introduction

FrugalGPT and RouteLLM both frame model selection as a policy problem balancing capability, latency, and cost, while HELM stresses evaluation rigor across model choices. This example demonstrates local model selection policy execution with observable scoring outputs and traces.

Technical Implementation

  1. Configure Tracer with JSONL + console output so each run emits machine-readable traces and lifecycle logs.

  2. Build the runtime surface (public APIs only) and execute ModelSelector.select(...) with a fixed request_id.

  3. Evaluate model constraints and policy, then expose selector metadata in the traced payload.

  4. Print a compact JSON payload including trace_info for deterministic tests and docs examples.

        flowchart LR
    A["Input prompt or scenario"] --> B["main(): runtime wiring"]
    B --> C["ModelSelector.select(...)"]
    C --> D["policy and constraints resolve one model-selection outcome"]
    C --> E["Tracer JSONL + console events"]
    D --> F["ExecutionResult/payload"]
    E --> F
    F --> G["Printed JSON output"]
    
 1from __future__ import annotations
 2
 3import json
 4from dataclasses import asdict
 5from pathlib import Path
 6
 7from design_research_agents import ModelSelector, Tracer
 8
 9
10def _select_local() -> dict[str, object]:
11    selector = ModelSelector()
12    decision = selector.select(
13        task="Summarize engineering design review findings for stakeholders.",
14        priority="quality",
15        max_cost_usd=0.01,
16        hardware_profile={
17            "total_ram_gb": 16.0,
18            "available_ram_gb": 12.0,
19            "cpu_count": 8,
20            "load_average": (0.2, 0.1, 0.1),
21            "gpu_present": False,
22            "gpu_vram_gb": None,
23            "gpu_name": None,
24            "platform_name": "example",
25        },
26        output="decision",
27    )
28    return asdict(decision)
29
30
31def main() -> None:
32    """Run traced local-first model selection and print decision."""
33    # Fixed request id keeps traces and docs output deterministic across runs.
34    request_id = "example-model-selection-local-design-001"
35    tracer = Tracer(
36        enabled=True,
37        trace_dir=Path("artifacts/examples/traces"),
38        enable_jsonl=True,
39        enable_console=True,
40    )
41    payload = tracer.run_callable(
42        agent_name="ExamplesModelSelectionLocal",
43        request_id=request_id,
44        input_payload={"scenario": "local-selection"},
45        function=_select_local,
46    )
47    assert isinstance(payload, dict)
48    payload["example"] = "model_selection/local.py"
49    payload["trace"] = tracer.trace_info(request_id)
50    print(json.dumps(payload, ensure_ascii=True, indent=2, sort_keys=True))
51
52
53if __name__ == "__main__":
54    main()

Expected Results

Run Command

PYTHONPATH=src python3 examples/model_selection/local.py

Example output captured with DRA_EXAMPLE_LLM_MODE=deterministic (timestamps, durations, and trace filenames vary by run):

{
  "catalog_signature": "440e215f0fee",
  "example": "model_selection/local.py",
  "model_id": "qwen3-14b-instruct-gguf-q4_k_m",
  "policy_id": "default",
  "provider": "llama_cpp",
  "rationale": "priority=quality; selection_reason=local_fit; model_size_b=14.0; ram_budget_gb=10.0; max_cost_us...
  "safety_constraints": {
    "max_cost_usd": 0.01,
    "max_latency_ms": null
  },
  "trace": {
    "request_id": "example-model-selection-local-design-001",
    "trace_dir": "artifacts/examples/traces",
    "trace_path": "artifacts/examples/traces/run_20260222T162207Z_example-model-selection-local-design-001.jsonl"
  }
}

References