Discrepancy Report

2021 August Special Primary, Summit County, OH

Audit Engine version:'bd43070'

Run Date: 2023-06-28 18:59:13.672388 Copyright (c) Citizens' Oversight

Table of Contents

• Introduction
• Metadata Summary
• Summary of Discrepancy Records
• Contest Discrepancy Report
• Precinct Report
• Unprocessed Ballots Report

Introduction

This report is the result of comparing the official Cast Vote Records (CVR) with the result by AuditEngine.

One of the key differences between ballot image audits, most specifically with the analysis provided by AuditEngine, and other types of auditing, is that the comparison between the results from AuditEngine and the official results occurs for all ballots at the ballot level, and we also compare with any adjudications as well.

Definitions

• Sheets -- Each ballot image comprises a single paper sheet, including both sides. This is the case even if multiple sheets are included in the logical ballot. Individual ballot sheets that include a number of contests. The number of sheets in tables is not necessarily a direct sum because a single sheet may include multiple contests. Also, depending on the number of sheets included in a single logical ballot, the number of ballots cast may be less than the total number of sheets, and it may be hard to predict because subsequent sheets are commonly not included in the logical ballot cast. Counting the first sheet will usually produce a pretty accurate number, but even then, some voters do not include the first sheet either.

• Contest or Ballot-Contest -- On each sheet, there are a number of contests. A single contest on a single ballot is a "ballot-contest" and sometimes just "contest" in this context. This is not the entire contest with all votes from all ballots summed, but simply the evaluation of that single contest on that sheet. Frequently, these can be called "votes", except that sometimes a single contest will allow multiple votes.

• Write-in -- If the write-in oval is marked or if the write-in area is filled in, then it is considered that the voter intent is to write-in the name for some other candidate.
  How this is handled is specific to different states, but most often, the written-in name must also be from a list of qualified write-in candidates.
  Write-ins on BMD ballots do not need human-eye adjudication at this stage because the names are keyed in.
  The name written-in should be examined in close contests even if there were no candidates that were qualified for this contest, because the name written-in may be of a listed candidate, and if so, then the vote is accepted for that listed candidate.
  Write-ins normally account for about 80% of Contest Variants. At this stage, most jurisdictions do not fully filter whether the name written-in is a qualified write-in candidate, but may do so during adjudication.

• Overvotes -- One overvote occurs when more than the "vote-for" number are selected. Overvotes do not happen on BMD ballots. Very frequently, overvotes are misinterpreted by the voting system and should be fully reviewed in close contests. If a contests has an overvote, it will be first classified as an overvote, even when the write-in target is selected. Some election departments will mark overvotes as an undervote when they are confirmed as an overvote. AuditEngine understands this form of adjudication and does not regard it as a disagreement.
  AuditEngine will correctly interpret many marks that would be considered undervotes by voting systems, such as when ovals are circled or if a checkmark does not actually get inside the oval.

• Adjudicated -- For some voting systems, such as Dominion, the CVR may also provide not only the original evaluation of the ballot sheet, but also a 'modified' record which includes all contests on the ballot. The 'modified' record may modify zero or many contests after human-eye review by election staff. If the modified record does not show a change in a given contest, it is not possible to know from that record if the contest was inspected and confirmed or not.

• Disagreed -- a contest is considered "disagreed" unless the voting system and AuditEngine fully agree on the outcome, including whether it was overvoted, undervoted, or had write-ins. Since write-ins and overvotes are frequently reviewed and adjudicated by election staff, disagreed write-ins and overvotes are treated together in terms of analysis. "Normal Disagreed" are the rest of the disagreed, which are not write-ins or overvotes.

• Fully Agreed Sheets -- A ballot sheet is classified as Fully Agreed if ithas no write-ins, overvotes, and gray-flags, and it also every contest fully agrees between the evaluation of the voting system and AuditEngine. Such sheets are maintained as a single record but keep track of the number and which contests are on them.

• Partially Agreed Sheets -- If a ballot sheet has at least one contest-variant detected, then that contest variant is logically pulled from this comparison record, and what remains are the rest of the non-variant contests. Note that if all contests are considered contest-variants, then it will be an empty shell with no contests left in it, and all contest variants will be moved to the contest-variants set.

• Contest Variant -- A contest variant is a ballot-contest which has write-ins, overvotes, is gray-flagged, or if it is 'disagreed', i.e. if there is any disagreement between the evaluation of the vote by AuditEngine and the official result. (Undervotes are not included in the set unless they are disagreed.) Contest Variants can normally be summed by contest. Each contest variant has a separate record for each ballot-contest instance.

• Unprocessed -- Unprocessed sheets are those that were NOT successfully processed by AuditEngine. These are generally due to images that are damaged due to 'stretching' of the image when the sheet is not evenly fed through the scanner, barcodes that were corrupted and unreliable, or BMD ballots that were not perfectly read using OCR. Cleaning the scanner rollers can help to avoid corrupted images. AuditEngine does not track contests on unprocessed sheets.

• Ballot Variant -- If a complete sheet is blank or if the image was detected as corrupted, then this is classified as a ballot-variant.

• Images Missing -- Sometimes we can't access all the images and so these are accounted for as images missing. AuditEngine does not attempt to sum the contests on ballot images that are missing.

Record Types Used by AuditEngine

• Fully Agreed: One record per sheet which has all the contests on that sheet, when they are Fully Agreed and there are no contest variants at all.

• Partially Agreed: One record per sheet that includes the contests that have not been pulled and "moved" to the contest-variant list because they are disagreed, or have overvotes, write-ins, or gray-flags.

• Contest-Variants: A separate record for each ballot-contest that is disagreed, or has overvotes, write-ins, or gray-flags.

• Ballot-Variants: One record per sheet that is completely blank or is corrupted. These may be partially included in the Fully Agreed set (when they are agreed and fully blank).

The Discrepancy Report includes columns for both Sheets and Contests. Please note that a given sheet may include several variant contests, and one sheet may be included in different categories. Thus, the total number of sheets is not always a direct sum of the sheets in each category, and in some cases it is listed as a direct sum and that is only an estimate rather than an exact value.

These comparison records can then be used to produce the discrepancy report.

High Level Reconciliation

Metadata Summary

Summary of Discrepancy Records

This first section is an election-wide review of the ballots that are considered

High Level Reconciliation

AuditEngine Flagged

If no CVR is available for comparison, then these groups are used to drive adjudication to fine tune the result from AuditEngine. These sets only include ballots with images. The group 'ae_uncategorized' were classified as variants based on cvr comparison and not included in the cases detected without any cvr comparison. It is common for this to be a fairly large number of variants. These are analyzed separately below.

Contest Variants Breakdown

This high-level breakdown is based on AuditEngine evaluation.

Non additive Groups

Ballot Variants

Completely blank ballots and any that we find had different style in CVR from the ballot.

Gray-flagged agreed votes.

These contests are marked as variants only because AuditEngine noted ambiguous votes or heuristic vote resolution.

Variant list for group:'ov_rest_unadjudicated': first (upto) 50 of 15.

Relevant Settings

Contest Discrepancy Report

This report is the most revealing in terms of possible changes in the outcome of any contests. Please see the more thorough description of each term in the introduction in this report.

• Total: Total ballots cast which included this contest with images that were processed by AuditEngine.

• NonVariant: Ballots with this contest where the official outcome and the evaluation by AuditEngine agreed, and did not include write-ins, overvotes, and were not flagged as 'gray'.

• Agreed Overvotes: Ballots where both AuditEngine and the voting system detected an overvote.

• Agreed Write-ins: Ballots which included write-ins in terms of a marked target, but where the name written-in may not be a qualified write-in candidate, and the write-in may be correctly attributed as a vote for a listed candidate.

• Agreed Undervotes: Undervotes are very numerous and we do not break those down here, and are not included in the discrepancy report unless they are disagreed or gray flagged.

• Disagreed: These are ballot-contests which were not initially evaluated as overvotes or write-ins, and where the evaluation by AuditEngine disagrees with the voting system.

• Gray Only: Ballot-contests where AuditEngine detected an ambiguous mark on this contest or used heuristics to decide voter-intent. This column omits any ballots which are in the columns for write-ins, overvotes, or disagreed ballots, even if AuditEngine internally flags them as gray.

• All Variants: Ballots with Agreed Overvotes, Agreed Write-ins, Disagreed, or Gray Flagged. The number of ballots cast should equal the sum of "Plain Agreed", and "All Variants". The components of this column are highlighted.

• Disagreed% of Margin This provides a good measure of whether the variants may have any impact on the outcome, and the highest five values are highlighted. Further analysis is still required to see if the disagreements will reduce the current margin of victory.

• Variant% of Margin This provides a maximum measure of whether the variants may have any impact on the outcome, and the highest five values are highlighted. Typically, the vast majority (perhaps 90%) of All Variants are Agreed Write-ins and Agreed Overvotes which may only rarely result in any changes in the outcome.

• Vote Margin: This is the margin of victory, i.e. gap between votes for runner-up and winner (lowest winner if contest has multiple winners) among the ballots processed, and may be a subset of the total margin for the entire district if AuditEngine did not receive or process all ballot images.

To reduce the report size, precinct details are limited to the first few contests or those contests that were highlighted or are included in the setting 'cmpcvr_report_include_contests'. If you are interested in a specific contest, include it in the setting 'cmpcvr_report_include_contests', which is a list.

Contest 'Representative to Congress 11th District DEM'

The following results are from the CVR but only including those ballots included in the audit. Missing ballot images are not included in these totals.

Summary of the comparison of the results of AuditEngine with the CVR.

The following contest variant groups not included in the detailed list below

Gray-flagged agreed votes.

These contests are marked as variants only because AuditEngine noted ambiguous votes or heuristic vote resolution.

Contest 'Representative to Congress 11th District DEM', group: 'x_x_gry'

Unadjudicated agreed contests with votes, marked gray_eval

(Total of 259 cmpcvr records; this report limited to a maximum of 10 ballot sheets)

group: 'x_x_gry' (#1 of 259 records)

abs_1_0052

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#2 of 259 records)

abs_1_0053

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#3 of 259 records)

abs_1_0084

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#4 of 259 records)

abs_1_0090

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#5 of 259 records)

abs_1_0116

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#6 of 259 records)

abs_1_0120

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#7 of 259 records)

abs_1_0125

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#8 of 259 records)

abs_10_0017

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#9 of 259 records)

abs_10_0018

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

group: 'x_x_gry' (#10 of 259 records)

abs_10_0025

• Return to contest 'Representative to Congress 11th District DEM'

• Return to contest 'Representative to Congress 11th District DEM' group: 'x_x_gry'

  ---

Contest 'Representative to Congress 11th District REP'

The following results are from the CVR but only including those ballots included in the audit. Missing ballot images are not included in these totals.

Summary of the comparison of the results of AuditEngine with the CVR.

The following contest variant groups not included in the detailed list below

Precinct Report

Please note that the figures in this table are by ballot, rather than by contest, and the %Disagree is the number of cases among the ballots in the group rather than of a margin, since these cases are among all the contests in the precinct.

To reduce the report size, precinct details are limited to those precincts that were highlighted or are included in the setting 'cmpcvr_report_include_precincts'.