design-research-problems#
A library of benchmark tasks for design research.
What This Library Does#
design-research-problems provides structured design tasks spanning
ideation, decision-making, optimization, grammar-based design exploration, and
MCP-backed workflows. It is built for recurring research workflows where clear
metadata, reusable evaluation contracts, and domain fidelity all matter.
Stable problem metadata, packaged statements, and explicit family APIs are core features. They make benchmarks easier to compare across agents, experiments, and downstream analyses.
Highlights#
Packaged benchmark families for ideation, decision, optimization, grammar, and MCP-backed workflows
Stable problem metadata and reusable family-specific APIs
Explicit downstream metadata and evaluation contracts for experiments and analysis
Catalog entry points for browsing and loading packaged problems
Runnable examples spanning the major benchmark families
Typical Workflow#
Start from a family API or a catalog entry point.
Load a packaged problem and inspect its metadata, statement, and structured inputs.
Hand the problem to agents or experiments while preserving benchmark metadata.
Capture outputs against the downstream metadata contract for comparison.
Rejoin benchmark context in downstream analysis and reporting.
Note
Start with Quickstart to load a first problem, inspect the public family APIs, and get the package into a reproducible local loop before diving into the broader catalog and reference material.
Guides#
Learn the family model, setup flow, and benchmark-selection patterns that shape a stable problem-research pipeline.
Examples#
Browse runnable examples that show the public APIs across the major problem families.
Reference#
Look up the stable import surface, rendered catalog entry points, and optional dependency guidance for the packaged benchmark families.
Integration With The Ecosystem#
The Design Research Collective maintains a modular ecosystem of libraries for studying human and AI design behavior.
design-research-agents implements AI participants, workflows, and tool-using reasoning patterns.
design-research-problems provides benchmark design tasks, prompts, grammars, and evaluators.
design-research-analysis analyzes the traces, event tables, and outcomes generated during studies.
design-research-experiments sits above the stack as the study-design and orchestration layer, defining hypotheses, factors, conditions, replications, and artifact flows across agents, problems, and analysis.
Together these libraries support end-to-end design research pipelines, from study design through execution and interpretation.
