candidates

Every statement both blind runs produced (3,819), with its full trail: script check → verifier verdict and reason → adjudicator ruling and reason → whether it is in the final set. Most rejections are duplicates between the two runs.

Data license: Not yet licensed — internal review only · Data source: Ahmed ElBatanony, Innopolis University

5 rows where adjudicator = "corrected", app_id = "F1-005" and in_final_set = "1.0"

✎ View and edit SQL

This data as json, CSV (advanced)

run 2

verifier_verdict 1

in_final_set 1

app_id 1

  • DeskcommCRM · 5 ✖

adjudicator 1

  • corrected · 5 ✖
candidate_id ▼ app_id run statement excerpt source_url source_file script_check verifier_verdict verifier_reason verifier_correction adjudicator adjudicator_reason in_final_set
F1-005-A033 DeskcommCRM F1-005 a The system lets a user export and redact personal data on request export and redact via workers, cascading anonymization, audited consent https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Two capabilities in one statement; redaction/anonymization is covered by A032. The system lets a user export personal data on request. corrected The statement bundled export with redaction, and redaction/anonymization is already covered by A032. 1.0
F1-005-A059 DeskcommCRM F1-005 a The system lets a user integrate with an external e-commerce platform with native Nuvemshop integration https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Names a vendor and no kind of exchange; the import is stated in the setup guide. The system lets a user connect an e-commerce platform to import orders, products and customers into the CRM. corrected The original named a vendor and no kind of exchange; the setup guide states the integration imports orders, products and customers. 1.0
F1-005-B004 DeskcommCRM F1-005 b The system lets an AI agent trigger automations trigger automations https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Automation form needs an event; the source says agents trigger automations. The system automatically triggers automations when an AI agent handles a conversation. corrected An AI agent is not a user role, so the automation form fits better, and the source says agents trigger automations. 1.0
F1-005-B007 DeskcommCRM F1-005 b The system lets an AI agent execute skills on its own in the middle of a conversation skills the agent executes on its own mid-conversation https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Describes an automation, not a user action. The system automatically runs skills for an AI agent during a conversation when the agent decides to use them. corrected The source describes autonomous agent behaviour, which is an automation and not a user action. 1.0
F1-005-B008 DeskcommCRM F1-005 b The system lets an AI agent keep an operational memory operational memory https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Describes an automation; the 'lets an AI agent' form misstates the actor. The system automatically keeps an operational memory that AI agents use when answering. corrected This is an automation; the 'lets an AI agent' form gives the wrong actor. 1.0

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [candidates] ([candidate_id] TEXT PRIMARY KEY, [app_id] TEXT, [run] TEXT, [statement] TEXT, [excerpt] TEXT, [source_url] TEXT, [source_file] TEXT, [script_check] TEXT, [verifier_verdict] TEXT, [verifier_reason] TEXT, [verifier_correction] TEXT, [adjudicator] TEXT, [adjudicator_reason] TEXT, [in_final_set] REAL, FOREIGN KEY ([app_id]) REFERENCES apps(app_id));
CREATE INDEX [idx_candidates_app_id] ON [candidates]([app_id]);
CREATE INDEX [idx_candidates_run] ON [candidates]([run]);
CREATE INDEX [idx_candidates_verifier_verdict] ON [candidates]([verifier_verdict]);
CREATE INDEX [idx_candidates_adjudicator] ON [candidates]([adjudicator]);