candidates

Every statement both blind runs produced (3,819), with its full trail: script check → verifier verdict and reason → adjudicator ruling and reason → whether it is in the final set. Most rejections are duplicates between the two runs.

Data license: Not yet licensed — internal review only · Data source: Ahmed ElBatanony, Innopolis University

22 rows where adjudicator = "corrected", in_final_set = "1.0" and run = "b"

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: source_url, source_file

verifier_verdict 2

run 1

in_final_set 1

  • 1.0 · 22 ✖

adjudicator 1

  • corrected · 22 ✖
candidate_id ▼ app_id run statement excerpt source_url source_file script_check verifier_verdict verifier_reason verifier_correction adjudicator adjudicator_reason in_final_set
F1-002-B017 It's a Plan F1-002 b The system lets a user embed files in a documentation page and link the page to the issues it describes favorites, embedded files, and links to the issues they describe https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/README.md 01.txt passed corrected Combines two capabilities; the link to issues is already covered by A022. The system lets a user embed files in a documentation page. corrected The original combined two capabilities and linking pages to issues already survives separately. 1.0
F1-002-B052 It's a Plan F1-002 b The system lets a user set an owner, a priority, a target date and a health status on an initiative It has an owner, a priority, a target date and a health status, and its progress counts the issues under it by state. https://itsaplan.dev/features/initiatives 05.txt passed rejected duplicate of F1-002-A030   corrected The health status is advertised separately in the sources and is absent from A030, so only that part is not a duplicate. 1.0
F1-005-B004 DeskcommCRM F1-005 b The system lets an AI agent trigger automations trigger automations https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Automation form needs an event; the source says agents trigger automations. The system automatically triggers automations when an AI agent handles a conversation. corrected An AI agent is not a user role, so the automation form fits better, and the source says agents trigger automations. 1.0
F1-005-B007 DeskcommCRM F1-005 b The system lets an AI agent execute skills on its own in the middle of a conversation skills the agent executes on its own mid-conversation https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Describes an automation, not a user action. The system automatically runs skills for an AI agent during a conversation when the agent decides to use them. corrected The source describes autonomous agent behaviour, which is an automation and not a user action. 1.0
F1-005-B008 DeskcommCRM F1-005 b The system lets an AI agent keep an operational memory operational memory https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Describes an automation; the 'lets an AI agent' form misstates the actor. The system automatically keeps an operational memory that AI agents use when answering. corrected This is an automation; the 'lets an AI agent' form gives the wrong actor. 1.0
F1-011-B037 Asp.Net Core Inventory Order Management System (Indotalent WMS) F1-011 b The system automatically generates document numbers. Auto-generated document numbers. https://indotalent.com/free-wms.html 03.txt passed corrected Automation lacked a triggering event; the source states auto-generated document numbers for sales orders. The system automatically generates a document number when a user creates a sales document. corrected The automation needed a triggering event, and the source places auto-generated numbers under sales order management. 1.0
F1-047-B049 Mayan EDMS F1-047 b The system automatically expires workflow states and escalates workflows. Comply with regulations using automatic state expiration and workflow escalation. https://docs.mayan-edms.com/chapters/features.html 02.txt passed corrected State expiration duplicates A076; escalation is not otherwise covered. The system automatically escalates workflows corrected State expiration is already covered by A076; escalation is a separate capability with no other statement. 1.0
F1-047-B077 Mayan EDMS F1-047 b The system lets a user scope each quota to specific document types, users, or groups. Scope each quota to specific document types, users, or groups, and turn it on or off as needed. https://docs.mayan-edms.com/chapters/features.html 02.txt passed corrected Scoping is not covered by A111; actor should be an administrator. The system lets an administrator scope each usage quota to specific document types, users, or groups corrected Quota scoping is not covered by A111, and quotas are an administrator function. 1.0
F1-047-B079 Mayan EDMS F1-047 b The system lets a user view statistics as line, pie, and doughnut charts and refresh any statistic on demand. View the results as line, pie, and doughnut charts, and refresh any statistic on demand. https://docs.mayan-edms.com/chapters/features.html 02.txt passed corrected Chart display duplicates A113; on-demand refresh is not otherwise covered. The system lets a user refresh any statistic on demand corrected Chart display is already covered by A113; on-demand refresh has no other statement. 1.0
F2-007-B053 Userback F2-007 b The system lets a user assess sentiment, generate titles, and detect when users are waiting for a response in feedback using AI. It can automatically classify feedback, generate titles, assess sentiment, and even detect if users are waiting for a response. https://support.userback.io/en/articles/10198801-automate-your-feedback-management 09.txt passed corrected Compound; sentiment assessment is covered by A059, response detection is listed as missed. The system automatically generates a title for incoming feedback using AI corrected The statement joins several capabilities: sentiment is covered by A059 and response detection is a separate missed item, so only title generation remains. 1.0
F2-007-B062 Userback F2-007 b The system lets a user comment on, edit, change the status of, or unpublish feedback published to the portal. where you can add comments, edit the original feedback, change its status, or unpublish it if needed. https://support.userback.io/en/articles/6103650-manage-your-ideas-portal 11.txt passed corrected Compound; unpublishing is covered by A042, the rest is routine editing. The system lets a user comment on feedback published to the portal corrected The statement joins several capabilities; unpublishing is covered by A042, so commenting on portal feedback is the part left to cover. 1.0
F2-007-B085 Userback F2-007 b The system lets a user let customers sort the roadmap. Customers can sort the roadmap from the dropdown above the columns. https://support.userback.io/en/articles/10672753-create-a-public-roadmap 15.txt passed corrected Actor is the customer viewing the roadmap. The system lets a customer sort the roadmap in the ideas portal corrected The source names the customer as the person who sorts the roadmap. 1.0
F2-015-B025 Pipedrive F2-015 b The system lets a user hold video calls, meetings and conferences with remote team members and contacts. Communicate with remote team members and contacts through video calls, meetings and conferences with Zoom, Microsoft Teams and more https://www.pipedrive.com/en/features 01.txt passed corrected Delivered through a third-party integration, not native. The system lets a user communicate with remote team members and contacts through a video conferencing integration. corrected Delivered via integrations; correction names the kind of exchange. 1.0
F2-015-B028 Pipedrive F2-015 b The system lets a user fill the pipeline with new leads from websites, social media and events. Fill your pipeline with new leads from websites, social media and events by using marketing and sales force automation https://www.pipedrive.com/en/features 01.txt passed corrected Delivered through integrations; wording tightened. The system lets a user capture new leads from websites, social media and events through lead generation integrations. corrected Delivered via integrations; wording tightened. 1.0
F2-015-B029 Pipedrive F2-015 b The system lets a user keep track of goals, projects and individual tasks. Assist salespeople and sales teams by keeping track of goals, projects and individual tasks https://www.pipedrive.com/en/features 01.txt passed corrected Delivered through integrations. The system lets a user keep track of goals, projects and individual tasks through task management integrations. corrected Delivered via integrations. 1.0
F2-019-B068 Xero F2-019 b The system lets an admin give team members specific permission levels. you can give your team members specific permission levels, so everyone only sees what they need to. https://www.xero.com/us/accounting-software/send-invoices/ 06.txt passed corrected Source states no admin role; actor should be a user. The system lets a user give team members specific permission levels. corrected The source names no admin role; general permission levels differ from the bill-specific roles statement. 1.0
F2-025-B007 Salesforce Field Service F2-025 b The system automatically guides a technician through each troubleshooting step. guide technicians through each troubleshooting step for faster resolution https://www.salesforce.com/service/field-service-management/ 01.txt passed corrected Not an automation triggered by an event; user-action form is correct. The system lets a technician follow guided troubleshooting steps from an AI agent. corrected No triggering event, so the user-action form fits. 1.0
F2-025-B056 Salesforce Field Service F2-025 b The system lets a technician get step-by-step guidance from an expert using Visual Remote Assistant from a mobile device. a technician can get step-by-step guidance from an expert using Visual Remote Assistant, reference knowledge base articles, or check inventory in an asset service management system https://www.salesforce.com/service/field-service-management/mobile-workforce/ 04.txt passed rejected duplicate of F2-025-A030   corrected Technician-side expert remote assistance differs from the customer-side video guidance in F2-025-A030. 1.0
F2-025-B058 Salesforce Field Service F2-025 b The system lets a user integrate asset data from any source with prebuilt connectors and monitor real-time telemetry data. Integrate asset data from any source with prebuilt connectors to drive calculated insights. Monitor and act on comprehensive telemetry data in real-time. https://www.salesforce.com/service/field-service-management/asset-management/ 05.txt passed corrected Integration is covered by A041; this keeps only the monitoring capability. The system lets a user monitor real-time asset telemetry data. corrected Integration is kept in F2-025-A041; this keeps the distinct monitoring capability. 1.0
F2-043-B161 ServiceNow IT Service Management F2-043 b The system automatically populates the requester and the assignee as recipients when a user emails a change record Both the user who requested the change and the user who is assigned to the change are automatically populated in the list of recipients. https://www.servicenow.com/docs/api/khub/maps/eH2O6EPUpf09wX0~2cZBFg/topics/uXk8LEj8iwoJoS8~QkyrIA/content 21.txt passed corrected Source describes a user-initiated email action, not an automatic population on its own. The system lets a user email a change record, with the requester and the assignee prefilled as recipients corrected The source describes a user-initiated email action with prefilled recipients, not a standalone automation. 1.0
F2-061-B046 Vanta F2-061 b The system lets a user support security and privacy frameworks such as SOC 2, ISO 27001 and HIPAA, or customize their own like SOC 2, ISO 27001, HIPAA, and more (or customize your own) https://www.vanta.com/products/automated-compliance 05.txt passed corrected Badly formed; the actor and action are missing. The system lets a user manage compliance against security and privacy frameworks such as SOC 2, ISO 27001 and HIPAA, or a custom framework corrected The original lacked a proper action; the correction is supported by the excerpt. 1.0
F2-073-B021 Success.co F2-073 b The system lets a team run weekly meetings that follow the EOS meeting flow with a built-in timer Each section follows the EOS meeting flow exactly, with a clean interface and a built-in timer that gently keeps teams on track. https://www.success.co/platform/meetings 04.txt passed corrected The timer part duplicates A027; the meeting-flow capability is kept. The system lets a team run weekly meetings that follow the EOS meeting flow corrected The timer is already covered by A027, so only the meeting-flow capability is kept. 1.0

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [candidates] ([candidate_id] TEXT PRIMARY KEY, [app_id] TEXT, [run] TEXT, [statement] TEXT, [excerpt] TEXT, [source_url] TEXT, [source_file] TEXT, [script_check] TEXT, [verifier_verdict] TEXT, [verifier_reason] TEXT, [verifier_correction] TEXT, [adjudicator] TEXT, [adjudicator_reason] TEXT, [in_final_set] REAL, FOREIGN KEY ([app_id]) REFERENCES apps(app_id));
CREATE INDEX [idx_candidates_app_id] ON [candidates]([app_id]);
CREATE INDEX [idx_candidates_run] ON [candidates]([run]);
CREATE INDEX [idx_candidates_verifier_verdict] ON [candidates]([verifier_verdict]);
CREATE INDEX [idx_candidates_adjudicator] ON [candidates]([adjudicator]);