candidates

Every statement both blind runs produced (3,819), with its full trail: script check → verifier verdict and reason → adjudicator ruling and reason → whether it is in the final set. Most rejections are duplicates between the two runs.

Data license: Not yet licensed — internal review only · Data source: Ahmed ElBatanony, Innopolis University

4 rows where adjudicator = "corrected", app_id = "F2-019" and verifier_verdict = "corrected"

✎ View and edit SQL

This data as json, CSV (advanced)

run 2

verifier_verdict 1

  • corrected · 4 ✖

in_final_set 1

app_id 1

  • Xero · 4 ✖

adjudicator 1

  • corrected · 4 ✖
candidate_id ▼ app_id run statement excerpt source_url source_file script_check verifier_verdict verifier_reason verifier_correction adjudicator adjudicator_reason in_final_set
F2-019-A010 Xero F2-019 a The system lets a user track sales tax and manage contractor details for W-9 and 1099 reporting. Xero helps you track sales tax and stay ready at filing time, and it makes managing contractor details for W-9 and 1099 reporting straightforward. https://www.xero.com/us/accounting-software/ 01.txt passed corrected Two capabilities in one statement; contractor W-9/1099 details covered separately. The system lets a user track sales tax. corrected Split two capabilities; W-9/1099 contractor details are covered separately. 1.0
F2-019-A025 Xero F2-019 a The system lets a manager assign roles and permissions to control who can create and approve bills. Assign roles and permissions to control who can create and approve bills, improving security and accuracy. https://www.xero.com/us/accounting-software/pay-bills/ 04.txt passed corrected Source states no manager role; actor should be a user. The system lets a user assign roles and permissions to control who can create and approve bills. corrected The source names no manager role. 1.0
F2-019-A034 Xero F2-019 a The system lets a user track jobs for project planning, budgeting, quoting and invoicing, and record time and costs. Use Xero’s job tracking tools for project planning, budgeting, quoting and invoicing. Record time and track costs. https://www.xero.com/us/accounting-software/all-features/ 05.txt passed corrected Time and cost recording is a separate capability. The system lets a user use job tracking tools for project planning, budgeting, quoting and invoicing. corrected Split two capabilities; time and cost recording is covered separately. 1.0
F2-019-B068 Xero F2-019 b The system lets an admin give team members specific permission levels. you can give your team members specific permission levels, so everyone only sees what they need to. https://www.xero.com/us/accounting-software/send-invoices/ 06.txt passed corrected Source states no admin role; actor should be a user. The system lets a user give team members specific permission levels. corrected The source names no admin role; general permission levels differ from the bill-specific roles statement. 1.0

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [candidates] ([candidate_id] TEXT PRIMARY KEY, [app_id] TEXT, [run] TEXT, [statement] TEXT, [excerpt] TEXT, [source_url] TEXT, [source_file] TEXT, [script_check] TEXT, [verifier_verdict] TEXT, [verifier_reason] TEXT, [verifier_correction] TEXT, [adjudicator] TEXT, [adjudicator_reason] TEXT, [in_final_set] REAL, FOREIGN KEY ([app_id]) REFERENCES apps(app_id));
CREATE INDEX [idx_candidates_app_id] ON [candidates]([app_id]);
CREATE INDEX [idx_candidates_run] ON [candidates]([run]);
CREATE INDEX [idx_candidates_verifier_verdict] ON [candidates]([verifier_verdict]);
CREATE INDEX [idx_candidates_adjudicator] ON [candidates]([adjudicator]);