Propose Critic

Source: examples/patterns/propose_critic.py

Introduction

Self-Refine and related critique/revise work motivate iterative self-critique loops, and Human-AI collaboration by design explains why critique transparency is critical for trustworthy engineering decisions. This example demonstrates a propose-critic refinement cycle with bounded iterations and structured run output.

Technical Implementation

  1. Configure Tracer with JSONL + console output so each run emits machine-readable traces and lifecycle logs.

  2. Build the runtime surface (public APIs only) and execute ProposeCriticPattern.run(...) with a fixed request_id.

  3. Configure and invoke Toolbox integrations (core/script/MCP/callable) before assembling the final payload.

  4. Print a compact JSON payload including trace_info for deterministic tests and docs examples.

        flowchart LR
    A["Input prompt or scenario"] --> B["main(): runtime wiring"]
    B --> C["ProposeCriticPattern.run(...)"]
    C --> D["proposal and critique turns iterate until stop criteria"]
    C --> E["Tracer JSONL + console events"]
    D --> F["ExecutionResult/payload"]
    E --> F
    F --> G["Printed JSON output"]
    
 1from __future__ import annotations
 2
 3import json
 4from pathlib import Path
 5
 6from design_research_agents import LlamaCppServerLLMClient, Toolbox, Tracer
 7from design_research_agents.patterns import ProposeCriticPattern
 8
 9
10def main() -> None:
11    """Run propose/critique refinement orchestration with tracing."""
12    # Keep request ids deterministic so critique traces are easy to compare run-to-run.
13    request_id = "example-workflow-propose-critic-design-001"
14    tracer = Tracer(
15        enabled=True,
16        trace_dir=Path("artifacts/examples/traces"),
17        enable_jsonl=True,
18        enable_console=True,
19    )
20    # Run the propose/critic pattern using public runtime surfaces. Using this with statement will automatically
21    # shut down the managed client and tool runtime when the example is done.
22    with Toolbox() as tool_runtime, LlamaCppServerLLMClient() as llm_client:
23        workflow = ProposeCriticPattern(
24            llm_client=llm_client,
25            tool_runtime=tool_runtime,
26            # Tracer is threaded through the pattern so proposer/critic turns share one timeline.
27            tracer=tracer,
28        )
29        result = workflow.run(
30            prompt=(
31                "Write and iteratively improve a short engineering design rationale for using "
32                "modular connectors in field-serviceable devices."
33            ),
34            request_id=request_id,
35        )
36
37    # Print the results
38    summary = result.summary()
39    print(json.dumps(summary, ensure_ascii=True, indent=2, sort_keys=True))
40
41
42if __name__ == "__main__":
43    main()

Expected Results

Run Command

PYTHONPATH=src python3 examples/patterns/propose_critic.py

Example output shape (values vary by run):

{
  "success": true,
  "final_output": "<example-specific payload>",
  "terminated_reason": "<string-or-null>",
  "error": null,
  "trace": {
    "request_id": "<request-id>",
    "trace_dir": "artifacts/examples/traces",
    "trace_path": "artifacts/examples/traces/run_<timestamp>_<request_id>.jsonl"
  }
}

References