ACRE Report -- All Contests Review & Evaluation

San Francisco, California, 2024 General Election

Audit Engine version:'v2.32.X'

Run Date: 2026-11-07 18:10:05.845691 Copyright (c) Citizens' Oversight

827 ballots selected (93 hand-marked, 734 BMD), grouped into review-priority rounds. See the Statistics section below for what each round guarantees, and for full run parameters/methodology.

Open TargetMapper to review/correct this job's style mapping.

Ballot-by-Ballot Review

Open the ACRE Review App to compare each selected ballot's image against its CVR record, filter to specific rounds or contests, mark each contest as matching or not (with a reason if not), and export your review. Any mismatch is a direct, ballot-level finding -- no cross-referencing required.

Round Summary

ACRE Statistics Report

San Francisco, California, 2024 General Election

Audit Engine version:'v2.32.X'

Run Date: 2026-11-07 18:09:59.235379 Copyright (c) Citizens' Oversight

Run Parameters & Characteristics

Review Tiers

Ballots are grouped into review-priority rounds.

Round 1 alone covers every contest option, undervote, overvote, and write-in-used check, on at least one contest-based style. That's enough to catch an error affecting every style.

Later rounds add coverage of the remaining contest-based style variants.

A reviewer can stop after any round once that round's coverage guarantee is met, or split rounds across reviewers.

Rounds are not a separate sample. They relabel the same selected ballots used everywhere else in this report.

(11 raw coverage tiers combined into 4 rounds of ~300 ballots each -- see tier_round_target_size.)

Raw Tier Distribution

Rounds above bundle these finer-grained raw tiers into a simpler presentation layer. See compute_acre_tiers() for how tiers are computed.

Contest-based styles this tier: how many distinct contest-based styles this tier's picks touched. This count varies enormously by election. A small jurisdiction might have a dozen total contest-based styles. A large county can have several hundred. Use this count, not raw ballot counts, to compare this run's shape against a different election's.

Dominant contest(s): the contest with the most explicit picks in that tier (ties listed together). A contest with many style variants naturally dominates the tiers right before its coverage runs out. That's why the long tail often shows one or two contests repeating across several tiers in a row. This is expected, not a bug or an unusually large sample.

Votable options covered % and Strata coverage %: running totals through the end of that tier. See the chart below for the full-resolution shape.

Coverage vs. Ballots

Two set-cover progress curves share the same x-axis: cumulative ballots selected, at full per-pick resolution, not just one point per tier.

Votable options covered % tracks every contest option, undervote, overvote, and write-in-used check, seen on at least one contest-based style. It ramps up fast across Tier 1's picks, since Tier 1 samples the rarest contest-based style/option first, then sits flat at 100%. That's why Round 1 alone is disproportionately valuable.

Strata coverage % only reaches 100% once every contest-based style variant of every votable option has also been verified. It climbs more slowly, across the remaining tiers and rounds.

Each point on the strata curve is labeled with its tier number, matching the Raw Tier Distribution table above.

Both curves show general set-cover progress. They are not specific to any one contest or election, so this shape is what to expect when comparing across different elections' runs.

Full per-pick data, not just tier-end snapshots, is in acre_coverage_curve.csv for building your own visualization.

Contest Coverage

• Margin (votes) / Margin (%): the CVR's own reported margin of victory for that contest, not computed from the ACRE sample.
• RLA n: simplified risk-limiting-audit sample size (gamma=0, no-error case, risk limit 5%), computed across all ballot types together -- not from the ACRE sample. Uses the DILUTED margin (margin / total ballots with this contest on their style, including genuine blanks/undervotes) as an RLA's sampling universe requires, not the Margin (%) column above (which is undiluted -- margin / votes actually cast, excluding undervotes). Blank if the margin is zero/unknown, or if the contest is out of scope (include_uncontested_contests=False and this contest is uncontested).
• BMD ballots / Non-BMD ballots: selected ballots covering this contest, by type.
• BMD contest-based styles / Non-BMD contest-based styles: distinct contest-based styles, across the FULL CVR (not just the selected sample), that carry this contest -- the real denominator behind how many covering ballots to expect (contest-based styles x options is the strata minimum), since a contest can share a rare style family with several other rare contests.
• BMD primary picks / Non-BMD primary picks: of the covering ballots, how many were selected specifically because of THIS contest's own stratum, as opposed to being covered incidentally (cascade) by a pick made for some other contest on the same contest-based style. Ballots column can run well above contest-based styles x options while primary picks stays near it -- that's cascade coverage from sharing styles with other rare contests, not a bug.
• BMD write-ins / Non-BMD write-ins: of those, how many happen to carry a write-in in this contest -- not a coverage target (include_writeins default is FALSE), just an observed count.
• Round reaching RLA n: the earliest review round whose cumulative ballots covering this contest first reach the RLA n column's sample size. Blank if RLA n is unknown, or never reached even at full coverage (common for high-margin contests, where ACRE naturally needs very few ballots). This is a visibility/comparison metric only -- ACRE selects ballots for votable-option coverage, not by the simple random sampling an RLA's risk-limit math requires, so matching the ballot COUNT is not the same as having a valid RLA sample.
• No summed TOTAL row: a single ballot covers many contests at once (the cascade-coverage mechanism), so summing any column down would count most ballots many times over. True totals are in the parameters table above.