API

The top-level package exports a curated public API suitable for notebook/script use:

Study
Factor
FactorKind
Level
Constraint
Condition
Block
RunBudget
SeedPolicy
ProblemPacket
RecipeStudyConfig
PromptFramingConfig
OptimizationBenchmarkConfig
AgentArchitectureComparisonConfig
GrammarScaffoldConfig
HumanVsAgentProcessConfig
DiversityAndExplorationConfig
Hypothesis
OutcomeSpec
AnalysisPlan
RunSpec
RunResult
BenchmarkBundle
ideation_bundle
optimization_bundle
grammar_problem_bundle
human_vs_agent_bundle
build_prompt_framing_study
build_optimization_benchmark_study
build_agent_architecture_comparison_study
build_grammar_scaffold_study
build_human_vs_agent_process_study
build_diversity_and_exploration_study
build_design
generate_doe
materialize_conditions
run_study
resume_study
export_analysis_tables
render_markdown_summary
render_methods_scaffold
render_codebook
render_significance_brief
write_markdown_report
validate_study

Public API exports for design-research-experiments.

class design_research_experiments.AgentArchitectureComparisonConfig(study_id=None, title=None, description=None, authors=None, rationale=None, tags=None, hypotheses=None, factors=None, blocks=None, constraints=None, design_spec=None, outcomes=None, analysis_plans=None, run_budget=None, seed_policy=None, output_dir=None, provenance_metadata=None, notes=None, problem_ids=None, agent_specs=None, primary_outcomes=None, secondary_outcomes=None, bundle=None)[source]

Bases: RecipeStudyConfig

Overrides for the agent architecture comparison recipe.

class design_research_experiments.AnalysisPlan(analysis_plan_id, hypothesis_ids, tests, outcomes=(), covariates=(), random_effects=(), filters=<factory>, multiple_comparison_policy='none', plots=(), export_tables=(), notes='')[source]

Bases: object

Mapping from hypotheses to downstream analyses.

analysis_plan_id

covariates

export_tables

filters

hypothesis_ids

multiple_comparison_policy

notes

outcomes

plots

random_effects

tests

class design_research_experiments.BenchmarkBundle(bundle_id, name, description, problem_ids, agent_specs, metadata=<factory>)[source]

Bases: object

Named bundle of agent/problem defaults for a study recipe.

agent_specs

bundle_id

description

metadata

name

problem_ids

class design_research_experiments.Block(name, levels, metadata=<factory>)[source]

Bases: object

Blocking structure for design-of-experiments materialization.

levels

metadata

name

class design_research_experiments.Condition(condition_id, factor_assignments, block_assignments, metadata=<factory>, admissible=True, constraint_messages=<factory>)[source]

Bases: object

One realized treatment combination.

Parameters:

condition_id – Stable condition ID.
factor_assignments – Materialized factor assignments.
block_assignments – Materialized block assignments.
metadata – Optional metadata payload.
admissible – Constraint admissibility flag.
constraint_messages – Constraint warning/error messages.

admissible

block_assignments

condition_id

constraint_messages

factor_assignments

metadata

class design_research_experiments.Constraint(constraint_id, description, expression=None, callable_ref=None, severity=ConstraintSeverity.ERROR)[source]

Bases: object

Admissibility rule over factor and block assignments.

Parameters:

constraint_id – Stable identifier for this constraint.
description – Human-readable description.
expression – Optional safe expression string.
callable_ref – Optional module:callable reference.
severity – Whether violation should fail or warn.

callable_ref

constraint_id

description

evaluate(factors, blocks)[source]: Return whether this constraint passes for one assignment.

expression

severity

class design_research_experiments.DiversityAndExplorationConfig(study_id=None, title=None, description=None, authors=None, rationale=None, tags=None, hypotheses=None, factors=None, blocks=None, constraints=None, design_spec=None, outcomes=None, analysis_plans=None, run_budget=None, seed_policy=None, output_dir=None, provenance_metadata=None, notes=None, problem_ids=None, agent_specs=None, primary_outcomes=None, secondary_outcomes=None, bundle=None)[source]

Bases: RecipeStudyConfig

Overrides for the diversity and exploration recipe.

class design_research_experiments.Factor(name, description, kind=FactorKind.MANIPULATED, levels=(), dtype=None, default=None, metadata=<factory>)[source]

Bases: object

Definition of one experimental factor.

Parameters:

name – Stable factor identifier.
description – Human-readable description.
kind – Factor type.
levels – Allowed level set.
dtype – Optional value type hint.
default – Optional default level value.
metadata – Optional metadata payload.

default

description

dtype

iter_values()[source]: Return all admissible values for cartesian materialization.

kind

levels

metadata

name

class design_research_experiments.FactorKind(*values)[source]

Bases: StrEnum

Classification for an experimental factor.

BLOCKED = 'blocked'

MANIPULATED = 'manipulated'

MEASURED = 'measured'

NUISANCE = 'nuisance'

class design_research_experiments.GrammarScaffoldConfig(study_id=None, title=None, description=None, authors=None, rationale=None, tags=None, hypotheses=None, factors=None, blocks=None, constraints=None, design_spec=None, outcomes=None, analysis_plans=None, run_budget=None, seed_policy=None, output_dir=None, provenance_metadata=None, notes=None, problem_ids=None, agent_specs=None, primary_outcomes=None, secondary_outcomes=None, bundle=None)[source]

Bases: RecipeStudyConfig

Overrides for the grammar scaffold recipe.

class design_research_experiments.HumanVsAgentProcessConfig(study_id=None, title=None, description=None, authors=None, rationale=None, tags=None, hypotheses=None, factors=None, blocks=None, constraints=None, design_spec=None, outcomes=None, analysis_plans=None, run_budget=None, seed_policy=None, output_dir=None, provenance_metadata=None, notes=None, problem_ids=None, agent_specs=None, primary_outcomes=None, secondary_outcomes=None, bundle=None)[source]

Bases: RecipeStudyConfig

Overrides for the human-vs-agent process recipe.

class design_research_experiments.Hypothesis(hypothesis_id, label, statement, kind=HypothesisKind.EFFECT, independent_vars=(), dependent_vars=(), moderators=(), mediators=(), contrast=None, direction=HypothesisDirection.DIFFERENT, minimum_effect_of_interest=None, linked_analysis_plan_id=None, notes='')[source]

Bases: object

Machine-readable research claim.

contrast

dependent_vars

direction

hypothesis_id

independent_vars

kind

label

linked_analysis_plan_id

mediators

minimum_effect_of_interest

moderators

notes

statement

class design_research_experiments.Level(name, value, label=None, metadata=<factory>)[source]

Bases: object

One admissible level/value for a factor.

Parameters:

name – Stable level identifier.
value – Encoded level value.
label – Optional display label.
metadata – Optional metadata payload.

label

metadata

name

value

class design_research_experiments.OptimizationBenchmarkConfig(study_id=None, title=None, description=None, authors=None, rationale=None, tags=None, hypotheses=None, factors=None, blocks=None, constraints=None, design_spec=None, outcomes=None, analysis_plans=None, run_budget=None, seed_policy=None, output_dir=None, provenance_metadata=None, notes=None, problem_ids=None, agent_specs=None, primary_outcomes=None, secondary_outcomes=None, bundle=None)[source]

Bases: RecipeStudyConfig

Overrides for the optimization benchmark recipe.

class design_research_experiments.OutcomeSpec(name, source_table, column, aggregation, derivation=None, primary=False, expected_type='float', missing_data_policy='drop', description='')[source]

Bases: object

Outcome definition mapped to source tables and derivations.

aggregation

column

derivation

description

expected_type

missing_data_policy

name

primary

source_table

class design_research_experiments.ProblemPacket(problem_id, family, brief, payload=<factory>, metadata=<factory>, evaluator=None)[source]

Bases: object

Normalized executable problem payload.

brief

evaluator

family

metadata

payload

problem_id

class design_research_experiments.PromptFramingConfig(study_id=None, title=None, description=None, authors=None, rationale=None, tags=None, hypotheses=None, factors=None, blocks=None, constraints=None, design_spec=None, outcomes=None, analysis_plans=None, run_budget=None, seed_policy=None, output_dir=None, provenance_metadata=None, notes=None, problem_ids=None, agent_specs=None, primary_outcomes=None, secondary_outcomes=None, bundle=None)[source]

Bases: RecipeStudyConfig

Overrides for the prompt framing recipe.

class design_research_experiments.RecipeStudyConfig(study_id=None, title=None, description=None, authors=None, rationale=None, tags=None, hypotheses=None, factors=None, blocks=None, constraints=None, design_spec=None, outcomes=None, analysis_plans=None, run_budget=None, seed_policy=None, output_dir=None, provenance_metadata=None, notes=None, problem_ids=None, agent_specs=None, primary_outcomes=None, secondary_outcomes=None, bundle=None)[source]

Bases: object

Shared typed overrides for recipe study construction.

Any field set to None keeps the recipe default. Any field set to a non-None value replaces that section of the study definition wholesale.

agent_specs

analysis_plans

authors

blocks

bundle

constraints

description

design_spec

factors

hypotheses

notes

outcomes

output_dir

primary_outcomes

problem_ids

provenance_metadata

rationale

run_budget

secondary_outcomes

seed_policy

study_id

tags

title

class design_research_experiments.RunBudget(replicates=1, max_runs=None, parallelism=1, fail_fast=False)[source]

Bases: object

Execution budget controls for one study.

Parameters:

replicates – Number of run replicates for each unit.
max_runs – Optional upper bound on total runs.
parallelism – Local worker count.
fail_fast – Stop after first failure when True.

fail_fast

max_runs

parallelism

replicates

class design_research_experiments.RunResult(run_id, status, outputs=<factory>, metrics=<factory>, evaluator_outputs=<factory>, cost=0.0, latency=0.0, trace_refs=<factory>, artifact_refs=<factory>, error_info=None, provenance_info=<factory>, observations=<factory>, run_spec=None, started_at=None, ended_at=None)[source]

Bases: object

Normalized result bundle for one run.

artifact_refs

cost

ended_at

error_info

evaluator_outputs

latency

metrics

observations

outputs

provenance_info

run_id

run_spec

started_at

status

trace_refs

class design_research_experiments.RunSpec(run_id, study_id, condition_id, problem_id, replicate, seed, agent_spec_ref, problem_spec_ref, execution_metadata=<factory>)[source]

Bases: object

One executable run specification.

agent_spec_ref

condition_id

execution_metadata

problem_id

problem_spec_ref

replicate

run_id

seed

study_id

class design_research_experiments.SeedPolicy(base_seed=0, strategy='condition_replicate', per_run_offset=9973)[source]

Bases: object

Deterministic seed policy for run generation.

Parameters:

base_seed – Study-wide base seed.
strategy – Seed derivation strategy name.
per_run_offset – Numeric offset mixed into each run seed.

base_seed

derive_seed(study_id, condition_id, replicate, salt='')[source]: Derive one deterministic per-run seed.

per_run_offset

strategy

class design_research_experiments.Study(study_id, title, description, authors=(), rationale='', tags=(), hypotheses=(), factors=(), blocks=(), constraints=(), design_spec=<factory>, outcomes=(), analysis_plans=(), run_budget=<factory>, seed_policy=<factory>, output_dir=None, provenance_metadata=<factory>, notes='', problem_ids=(), agent_specs=(), primary_outcomes=(), secondary_outcomes=())[source]

Bases: object

Top-level experiment definition.

agent_specs

analysis_plans

authors

blocks

constraints

description

design_spec

factors

classmethod from_dict(payload)[source]: Construct a study from a loose mapping payload.

classmethod from_json(path)[source]: Load a study from JSON.

classmethod from_yaml(path)[source]: Load a study from YAML.

hypotheses

notes

outcomes

output_dir

primary_outcomes

problem_ids

provenance_metadata

rationale

run_budget

secondary_outcomes

seed_policy

study_id

tags

title

to_dict()[source]: Serialize the study to a stable JSON/YAML-friendly mapping.

to_json(path)[source]: Write the study definition to JSON.

to_yaml(path)[source]: Write the study definition to YAML.

design_research_experiments.build_agent_architecture_comparison_study(config=None)[source]: Build a study comparing agent architecture choices across prompt difficulty.

design_research_experiments.build_design(study, design_spec=None)[source]: Build study conditions for the configured design.

design_research_experiments.build_diversity_and_exploration_study(config=None)[source]: Build a study evaluating diversity and exploration outcomes.

design_research_experiments.build_grammar_scaffold_study(config=None)[source]: Build a study comparing unconstrained and grammar-guided generation.

design_research_experiments.build_human_vs_agent_process_study(config=None)[source]: Build a study comparing human-only, AI-assisted, and hybrid teams.

design_research_experiments.build_optimization_benchmark_study(config=None)[source]: Build a benchmark study for optimization generalization and learning effects.

design_research_experiments.build_prompt_framing_study(config=None)[source]: Build an ideation study with framing and prompt manipulation.

design_research_experiments.export_analysis_tables(study, *, conditions, run_results, output_dir=None, include_sqlite=False, validate_with_analysis_package=False)[source]: Export canonical analysis tables aligned with downstream workflows.

design_research_experiments.generate_doe(*, kind, factors, n_samples=None, seed=0, center_points=0, replicates=1, randomize=True, block_randomization_key=None)[source]: Generate and summarize a DOE table for migration from drcutils-style workflows.

design_research_experiments.grammar_problem_bundle()[source]: Return a default grammar-scaffold benchmark bundle.

design_research_experiments.human_vs_agent_bundle()[source]: Return a default human-vs-agent teaming benchmark bundle.

design_research_experiments.ideation_bundle()[source]: Return a default ideation-focused benchmark bundle.

design_research_experiments.materialize_conditions(factors, blocks=None, constraints=None, *, seed=None, randomize=False, counterbalance=False)[source]: Materialize admissible conditions from factors, blocks, and constraints.

design_research_experiments.optimization_bundle()[source]: Return a default optimization-focused benchmark bundle.

design_research_experiments.render_codebook(study, conditions)[source]: Render a simple codebook of factors, blocks, and condition IDs.

design_research_experiments.render_markdown_summary(study, run_results)[source]: Render a compact markdown summary for one completed study.

design_research_experiments.render_methods_scaffold(study)[source]: Render a methods-section scaffold for manuscript drafting.

design_research_experiments.render_significance_brief(analysis_rows)[source]: Render a short significance/effect-size brief from analysis outputs.

design_research_experiments.resume_study(study, *, conditions=None, agent_factories=None, problem_registry=None, parallelism=None, checkpoint=True, include_sqlite=False)[source]: Resume a study from checkpointed run results.

design_research_experiments.run_study(study, *, conditions=None, agent_factories=None, problem_registry=None, parallelism=None, dry_run=False, resume=False, checkpoint=True, include_sqlite=False)[source]: Run a study end-to-end and export canonical artifacts.

design_research_experiments.validate_study(study)[source]: Validate cross-object references and study consistency.

design_research_experiments.write_markdown_report(output_dir, filename, content)[source]: Write markdown report content into the study artifacts directory.