Scoring
trialdesignbench.scoring
Versioned scoring rules. Pure functions; no I/O.
Changing any rule here must bump SCORING_VERSION so reports never mix
scores computed under different rules.
dimension_means(questions, config=None)
Weighted pass fraction per rubric dimension, pooled across questions.
Extraction questions have no dimension; they are pooled under extraction.
group_means(questions, attr)
Mean question score grouped by a QuestionGrade attribute.
majority_verdict(votes)
Majority across judge votes. Any error vote, or a tie, is not a pass.
mean(values)
Arithmetic mean, or None for no values.
reward_score(rubric_score, blocking_failures)
Reward is the rubric score unless any gate failed, then 0.
score_question(results, config=None)
Weighted pass fraction over Add criteria.
unclear and error verdicts earn nothing. Returns score=None when the
question has no Add criteria (nothing to score).
task_rubric_score(questions)
Mean score over scored questions.
verdict_counts(results)
Count verdicts by kind.