candidates

Every statement both blind runs produced (3,819), with its full trail: script check → verifier verdict and reason → adjudicator ruling and reason → whether it is in the final set. Most rejections are duplicates between the two runs.

Data license: Not yet licensed — internal review only · Data source: Ahmed ElBatanony, Innopolis University

55 rows where in_final_set = "1.0", run = "a" and verifier_verdict = "corrected"

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: source_file

verifier_verdict 1

  • corrected · 55 ✖

run 1

in_final_set 1

  • 1.0 · 55 ✖

adjudicator 1

candidate_id ▼ app_id run statement excerpt source_url source_file script_check verifier_verdict verifier_reason verifier_correction adjudicator adjudicator_reason in_final_set
F1-001-A091 Operately F1-001 a The system lets an administrator onboard members. Onboard members, manage permissions, and maintain company structure https://raw.githubusercontent.com/operately/operately/HEAD/README.md 01.txt passed corrected Actor "administrator" is not stated in the source; use "a user". The system lets a user onboard members. corrected The source lists onboarding members under Team Management without naming an administrator role. 1.0
F1-002-A038 It's a Plan F1-002 a The system lets an administrator control access by role. - Auto-archive, a notification inbox, role-based access control, and more https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/README.md 01.txt passed corrected Actor "administrator" is not stated in the source; use "a user". The system lets a user control access by role. corrected The source names role-based access control without stating an administrator actor. 1.0
F1-002-A067 It's a Plan F1-002 a The system lets a user use the interface in 13 languages. - Interface in English, Ukrainian, Russian, Simplified Chinese, Traditional Chinese (Taiwan), Arabic, Persian, French, Portuguese (Brazil), Bahasa Indonesia, Spanish (Spain), Korean, and Turkish https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/README.md 01.txt passed corrected The source gives both 12 and 13 languages; the count is not reliably supported. The system lets a user use the interface in several languages. corrected The sources give both 12 and 13 languages, so the count is dropped. 1.0
F1-005-A033 DeskcommCRM F1-005 a The system lets a user export and redact personal data on request export and redact via workers, cascading anonymization, audited consent https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Two capabilities in one statement; redaction/anonymization is covered by A032. The system lets a user export personal data on request. corrected The statement bundled export with redaction, and redaction/anonymization is already covered by A032. 1.0
F1-005-A059 DeskcommCRM F1-005 a The system lets a user integrate with an external e-commerce platform with native Nuvemshop integration https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Names a vendor and no kind of exchange; the import is stated in the setup guide. The system lets a user connect an e-commerce platform to import orders, products and customers into the CRM. corrected The original named a vendor and no kind of exchange; the setup guide states the integration imports orders, products and customers. 1.0
F1-029-A003 Wukong HRM F1-029 a The system lets a user manage employees and the organization Employee Organization Management, Optimal Configuration of Talent Resources https://raw.githubusercontent.com/WuKongOpenSource/Wukong_HRM/HEAD/README_EN.md 02.txt passed corrected Statement bundled employee management with organization management; organizational planning is already A001. The system lets a user manage employees corrected The original bundled two capabilities; organizational planning is already covered by A001, so the statement is reduced to employee management. 1.0
F1-037-A019 FOSSBilling F1-037 a The system lets a client receive files or digital downloads after purchase Deliver files or digital downloads after purchase. https://docs.fossbilling.org/admin-guide/product-types/ 05.txt passed corrected Describes an automation, not a user action. The system automatically delivers files or digital downloads to a client after purchase. corrected Delivery after purchase is an automation, so the verifier's correction is right. 1.0
F1-043-A044 QloApps F1-043 a The system lets a user notify guests when the inventory of rooms is low Notify your guests when the inventory of your rooms is low. https://qloapps.com/booking-engine/ 03.txt passed corrected Describes an automation, not a user action. The system automatically notifies guests when the inventory of rooms is low. corrected The low-inventory notice is triggered by an event, so the automation form is correct. 1.0
F1-046-A004 Probo F1-046 a The system lets a user keep a vendor inventory and track data processing agreements Vendor inventory, automated website risk assessment, DPA/BAA tracking, subprocessor discovery https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed corrected Compound statement; vendor inventory is the single capability here, contract tracking is covered by A020. The system lets a user keep an inventory of vendors. corrected Compound statement; agreement tracking is already covered by the surviving third-party contract-tracking statement. 1.0
F1-046-A005 Probo F1-046 a The system lets a user record processing activities and maintain a data inventory processing activity records, data inventory https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed corrected Compound statement; processing activities are covered separately by A026. The system lets a user maintain a data inventory. corrected Compound statement; processing activity records are covered by a separate surviving statement. 1.0
F1-046-A006 Probo F1-046 a The system lets a user collect evidence as files and URLs and track implementation state Evidence collection (files and URLs), implementation state tracking, task assignment https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed corrected Compound statement; implementation-state tracking is not a separate evidenced capability. The system lets a user collect evidence as files and URLs. corrected Compound statement; splitting keeps one capability per statement and the bare table cell gives no further detail on implementation-state tracking. 1.0
F1-046-A013 Probo F1-046 a The system lets an auditor scope audits, map controls, track findings, and generate reports Audit scoping, control mapping, finding tracking, report generation https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed corrected Compound statement; finding tracking is covered by A024, report generation is listed as missed. The system lets an auditor scope audits and map controls to them. corrected Compound statement; finding tracking is covered separately and report generation is accepted as a missed item. 1.0
F1-046-A014 Probo F1-046 a The system lets a user receive task and comment lifecycle events through webhooks Use GraphQL, the `prb` CLI, MCP, n8n, and webhooks instead of limiting work to the console. https://www.probo.com/docs/product/getting-started/product-overview.md 03.txt passed corrected Describes an automation, not a user action. The system automatically sends task and comment lifecycle events through webhooks when tasks or comments change. corrected The source (09.txt: 'Webhooks Receive task and comment lifecycle events') supports it, and event delivery is an automation. 1.0
F1-046-A041 Probo F1-046 a The system lets an administrator invite and role-manage members in the organization Invite and role-manage members in the organization https://www.probo.com/docs/product/identity-and-access 08.txt passed corrected Invitation is a duplicate of A017; only the role-management part is new. The system lets an administrator assign roles to organization members. corrected Inviting members is already covered by A017; only role management is new. 1.0
F1-047-A007 Mayan EDMS F1-047 a The system lets a user work with multiple page PDF, office, and TIFF files Multiple page PDF, office, and TIFF files are supported. https://docs.mayan-edms.com/chapters/features.html 02.txt passed corrected Format-support claim restated as the navigation capability the source names. The system lets a user navigate the pages of multi page PDF, office, and TIFF documents corrected Format support is not a capability; the adjacent source line names page navigation, which no other statement covers. 1.0
F1-047-A010 Mayan EDMS F1-047 a The system automatically restricts printing of documents to users who hold the printing permission Printing is controlled by its own permission, separate from viewing and https://docs.mayan-edms.com/chapters/features.html 02.txt passed corrected Access-control capability, not an event-driven automation. The system lets an administrator control printing of documents with a permission separate from viewing and downloading corrected A separate permission is an access-control setting, not an event-driven automation. 1.0
F1-047-A021 Mayan EDMS F1-047 a The system automatically prevents users without the dedicated permission from viewing the content beneath redactions A dedicated permission controls who, if anyone, may view the content https://docs.mayan-edms.com/chapters/features.html 02.txt passed corrected Access-control capability, not an event-driven automation. The system lets an administrator restrict, with a dedicated permission, who may view the content beneath redactions corrected A dedicated permission is an access-control setting, not an event-driven automation. 1.0
F1-047-A061 Mayan EDMS F1-047 a The system lets a workflow number documents from the next sequence value as they progress Consume the next value directly from a workflow to number documents as https://docs.mayan-edms.com/chapters/features.html 02.txt passed corrected Actor was a workflow; this is an automation. The system automatically numbers documents with the next sequence value as they progress through a workflow corrected A workflow is not a role, so this is an automation. 1.0
F2-001-A055 Asana F2-001 a The system lets an administrator set session duration limits and password strength requirements The security features you can enforce for user login include Google SSO, SAML authentication, two-factor authentication (2FA), session duration limits, and password strength requirements. https://asana.com/features/admin-security 06.txt passed corrected Compound: session limits and password strength are two capabilities; password strength is listed as missed. The system lets an administrator set session duration limits corrected The statement joins two capabilities; password strength is kept as a separate missed item. 1.0
F2-007-A036 Userback F2-007 a The system lets a user provide a centralized portal where users can view, vote, and comment on upcoming features Provide a centralized portal where users can view, vote, and comment on upcoming features. https://userback.io/features 06.txt passed corrected Actor is the portal visitor; the statement described an administrator providing the portal. The system lets a user view, vote on, and comment on upcoming features in a portal corrected The capability belongs to the person visiting the portal, not to someone providing it. 1.0
F2-015-A002 Pipedrive F2-015 a The system lets a user record and transcribe calls. It records and transcribes your calls so you don’t have to. https://www.pipedrive.com/en/features 01.txt passed corrected Describes an automation, not a user action. The system automatically records and transcribes a user's calls when a call is held. corrected Automation, and recording and transcription are two capabilities; transcription is kept separately via F2-015-B003. 1.0
F2-015-A003 Pipedrive F2-015 a The system automatically generates a recap and drafts CRM updates that a user can review and approve after a call. It generates a recap and drafts CRM updates you can review and approve in one click https://www.pipedrive.com/en/features 01.txt passed corrected Combined two capabilities; draft-and-approve CRM updates is covered separately. The system automatically generates a recap of a call after the call. corrected Split two capabilities; review-and-approve of drafted CRM updates is covered by a surviving statement. 1.0
F2-015-A007 Pipedrive F2-015 a The system lets a user turn prompts into emails and summarize email threads. Streamline sales outreach by turning prompts into emails and summarize lengthy threads quickly https://www.pipedrive.com/en/features 01.txt passed corrected Two capabilities in one statement; thread summarising is covered separately. The system lets a user turn prompts into emails. corrected Split two capabilities; thread summarising is covered separately. 1.0
F2-015-A011 Pipedrive F2-015 a The system lets a user set up automated lead qualification flows and customer nurturing sequences. Set up automated lead qualification flows and customer relationship nurturing sequences https://www.pipedrive.com/en/features 01.txt passed corrected Two capabilities in one statement; nurturing sequences covered separately. The system lets a user set up automated lead qualification flows. corrected Split two capabilities; nurturing sequences are covered separately. 1.0
F2-015-A012 Pipedrive F2-015 a The system lets a user set up SSO and 2FA. then set up SSO and 2FA for further security https://www.pipedrive.com/en/features 01.txt passed corrected Two capabilities in one statement; 2FA covered separately. The system lets a user set up single sign-on. corrected Split two capabilities; 2FA is covered separately. 1.0
F2-015-A019 Pipedrive F2-015 a The system lets a user exchange records with an external accounting system. Find sales CRM integrations to automate your finance processes and pipeline management https://www.pipedrive.com/en/features 01.txt passed corrected Excerpt does not support record exchange, only integration for finance automation. The system lets a user connect the CRM with accounting and invoicing apps to automate finance processes. corrected The excerpt, under the 'Accounting and invoicing' heading, supports an integration for finance automation, not record exchange as such. 1.0
F2-015-A031 Pipedrive F2-015 a The system lets a user trigger follow-ups and reminders automatically based on pipeline stage or activity. Schedule or automatically trigger follow-ups and reminders based on pipeline stage or activity. https://www.pipedrive.com/en/features/email-integration 04.txt passed corrected Describes an automation, not a user action. The system automatically triggers follow-ups and reminders when a deal reaches a pipeline stage or an activity occurs. corrected Describes an automation. 1.0
F2-015-A045 Pipedrive F2-015 a The system lets a user import leads and deals and assign tasks to reps. Import leads and deals, assign tasks to reps and follow high-value deals as they progress through your pipeline https://www.pipedrive.com/en/features/sales-reporting 05.txt passed corrected Two capabilities in one statement; task assignment covered separately. The system lets a user import leads and deals. corrected Split two capabilities; task assignment is covered separately. 1.0
F2-015-A050 Pipedrive F2-015 a The system lets a user schedule an activity after a deal has been moved to a specific stage. Schedule an activity after a deal has been moved to a specific stage in your pipeline https://www.pipedrive.com/en/features/workflow-automation 06.txt passed corrected Describes an automation, not a user action. The system automatically schedules an activity after a deal is moved to a specific stage. corrected Workflow-automation example, so an automation. 1.0
F2-019-A010 Xero F2-019 a The system lets a user track sales tax and manage contractor details for W-9 and 1099 reporting. Xero helps you track sales tax and stay ready at filing time, and it makes managing contractor details for W-9 and 1099 reporting straightforward. https://www.xero.com/us/accounting-software/ 01.txt passed corrected Two capabilities in one statement; contractor W-9/1099 details covered separately. The system lets a user track sales tax. corrected Split two capabilities; W-9/1099 contractor details are covered separately. 1.0
F2-019-A025 Xero F2-019 a The system lets a manager assign roles and permissions to control who can create and approve bills. Assign roles and permissions to control who can create and approve bills, improving security and accuracy. https://www.xero.com/us/accounting-software/pay-bills/ 04.txt passed corrected Source states no manager role; actor should be a user. The system lets a user assign roles and permissions to control who can create and approve bills. corrected The source names no manager role. 1.0
F2-019-A034 Xero F2-019 a The system lets a user track jobs for project planning, budgeting, quoting and invoicing, and record time and costs. Use Xero’s job tracking tools for project planning, budgeting, quoting and invoicing. Record time and track costs. https://www.xero.com/us/accounting-software/all-features/ 05.txt passed corrected Time and cost recording is a separate capability. The system lets a user use job tracking tools for project planning, budgeting, quoting and invoicing. corrected Split two capabilities; time and cost recording is covered separately. 1.0
F2-025-A007 Salesforce Field Service F2-025 a The system lets a dispatcher use natural language to find appointments that need attention and take action such as filling a gap in the schedule or locating missing parts. Dispatchers can speak to Agentforce using natural language to quickly find appointments that need attention and take action like filling a gap in the schedule or even locating missing parts. https://www.salesforce.com/service/field-service-management/ 01.txt passed corrected Two capabilities in one statement; taking action is covered separately. The system lets a dispatcher use natural language to find appointments that need attention. corrected Split two capabilities; taking action on gaps or missing parts is covered separately. 1.0
F2-025-A019 Salesforce Field Service F2-025 a The system automatically schedules the workforce and ensures compliance with service-level agreements. automates workforce scheduling, reduces travel time, and ensures compliance with service-level agreements https://www.salesforce.com/service/field-service-management/ 01.txt passed corrected Tightened to the single scheduling automation the excerpt supports. The system automatically schedules the workforce in line with service-level agreements. corrected SLA compliance is a quality of the scheduling automation, not a second capability. 1.0
F2-025-A022 Salesforce Field Service F2-025 a The system lets a manager monitor service outcomes and create preventive maintenance plans based on asset use, condition, and specific criteria. Monitor service outcomes and create preventive maintenance plans based on asset use, condition, and specific criteria. https://www.salesforce.com/service/field-service-management/ 01.txt passed corrected Two capabilities in one statement; monitoring service outcomes is covered separately. The system lets a manager create preventive maintenance plans based on asset use, condition and specific criteria. corrected Split two capabilities; monitoring service outcomes is covered separately. 1.0
F2-025-A024 Salesforce Field Service F2-025 a The system lets a manager predict the time and reason for an asset failure. predict the time of — and reason for — an asset failure https://www.salesforce.com/service/field-service-management/ 01.txt passed corrected Describes an automation, not a user action. The system automatically predicts the time and reason for an asset failure. corrected Describes an automation. 1.0
F2-025-A041 Salesforce Field Service F2-025 a The system lets a user integrate asset data from any source with prebuilt connectors and monitor telemetry data in real time. Integrate asset data from any source with prebuilt connectors to drive calculated insights. Monitor and act on comprehensive telemetry data in real-time. https://www.salesforce.com/service/field-service-management/asset-management/ 05.txt passed corrected Two capabilities in one statement; telemetry monitoring covered separately. The system lets a user integrate asset data from any source with prebuilt connectors. corrected Split two capabilities; telemetry monitoring is kept via F2-025-B058. 1.0
F2-025-A059 Salesforce Field Service F2-025 a The system lets a dispatcher ask a conversational AI assistant to find and schedule unscheduled appointments, summarize service appointments and answer from knowledge base articles. They can simply ask the assistant to quickly find and schedule appointments that haven’t been scheduled yet, create summaries of service appointments, and generate answers from knowledge base articles to resolve customer questions faster. https://www.salesforce.com/service/field-service-management/what-is-dispatch-management/ 07.txt passed corrected Three capabilities in one statement; narrowed to the first. The system lets a dispatcher ask a conversational AI assistant to find and schedule unscheduled appointments. corrected Three capabilities in one; the other two are accepted as missed items. 1.0
F2-031-A053 Salesforce Service Cloud F2-031 a The system automatically categorizes and prioritizes tickets with AI uses AI to categorize, prioritize, and resolve simple issues automatically https://www.salesforce.com/service/support-ticketing-tools-systems/ 09.txt passed corrected The statement bundled categorizing and prioritizing; prioritizing is covered separately. The system automatically categorizes tickets with AI corrected The original bundled two capabilities; prioritizing is kept separately as B064. 1.0
F2-037-A005 RUN Powered by ADP F2-037 a The system automatically imports information from an uploaded document into the system immediately import information into the system https://mediacenter.adp.com/adp-run-new-features-top-tool-for-small-business 02.txt passed corrected Automation statement lacked its triggering event. The system automatically imports information from an uploaded document into the system when a user uploads it corrected The automation needs its triggering event, which the source context (upload) supplies. 1.0
F2-043-A057 ServiceNow IT Service Management F2-043 a The system automatically generates an event to prevent incidents from going unnoticed The inactivity monitor generates an event to prevent incidents from going unnoticed https://www.servicenow.com/docs/api/khub/maps/eH2O6EPUpf09wX0~2cZBFg/topics/Uc8KCF0LjG~p2cz7LcOYiw/content 10.txt passed corrected Original was vague about the trigger and effect. The system automatically triggers email notifications or scripts when an incident has no updates over a specified time corrected The source (10.txt line 24) states the trigger and effect that the original statement left vague. 1.0
F2-055-A117 MaintainX F2-055 a The system lets a user search asset manuals, create work tasks and troubleshoot issues with an AI maintenance assistant. MaintainX Assist is an AI-powered maintenance assistant that helps you search your asset manuals, create work tasks, troubleshoot issues, and share insights directly within MaintainX. https://help.getmaintainx.com/ai-basics 10.txt passed corrected Bundles three capabilities; creating work tasks and troubleshooting are covered by other statements, so only manual search remains. The system lets a user search asset manuals with an AI maintenance assistant using natural language prompts. corrected The original bundles three capabilities; creating work tasks and troubleshooting are covered by A007 and A040, which leaves searching manuals. 1.0
F2-061-A011 Vanta F2-061 a The system lets a user auto-approve document access requests and collect NDAs through integrations with CRM and contract management systems Speed up document access with auto-approvals and NDA collection with Salesforce, HubSpot, DocuSign, and Ironclad integrations. https://www.vanta.com/products/trust-center 02.txt passed corrected Automation described as a user action. The system automatically approves document access requests and collects NDAs when a request is made, through integrations with CRM and contract management systems corrected The source describes an automation, so the verifier's automation form is correct. 1.0
F2-061-A018 Vanta F2-061 a The system lets a user automate reminders and workflows to assess, review, and approve risks Vanta automates reminders and workflows to regularly assess, review, and approve risks https://www.vanta.com/products/risk-management 03.txt passed corrected Automation described as a user action. The system automatically sends reminders and runs workflows for regular assessment, review and approval of risks corrected The source says Vanta automates the reminders and workflows. 1.0
F2-061-A034 Vanta F2-061 a The system lets a user pull findings from vendor SOC2 reports, DPAs, and questionnaires automatically Vanta automatically pulls findings from your vendors SOC2 reports, DPAs, and questionnaires https://www.vanta.com/products/vendor-risk-management 04.txt passed corrected Automation described as a user action. The system automatically pulls findings from vendors' SOC 2 reports, DPAs, and questionnaires corrected The source says Vanta pulls findings automatically. 1.0
F2-061-A036 Vanta F2-061 a The system lets a user discover newly adopted vendors through integration with existing procurement systems Discovers newly adopted vendors and integrates with your existing procurement systems to eliminate shadow IT and AI blind spots. https://www.vanta.com/products/vendor-risk-management 04.txt passed corrected Automation described as a user action. The system automatically discovers newly adopted vendors through integration with procurement systems corrected Discovery is performed by the system, not by the user. 1.0
F2-061-A037 Vanta F2-061 a The system lets a user retrieve vendor documentation from first- and third-party Trust Centers Retrieves verified documentation directly from first- and third-party Trust Centers https://www.vanta.com/products/vendor-risk-management 04.txt passed corrected Automation described as a user action. The system automatically retrieves vendor documentation from first- and third-party Trust Centers corrected Retrieval is performed by the system, not by the user. 1.0
F2-061-A038 Vanta F2-061 a The system lets a user identify SLAs, DPAs, and custom security terms in SOC 2 reports Identifies high-stakes SLAs, DPAs, and custom security terms in SOC 2 reports https://www.vanta.com/products/vendor-risk-management 04.txt passed corrected Automation described as a user action. The system automatically identifies SLAs, DPAs, and custom security terms in vendors' SOC 2 reports corrected Identification is performed by the system, not by the user. 1.0
F2-061-A040 Vanta F2-061 a The system lets a user manage document requests and follow-ups with vendors using reminders Manages document requests and follow-ups with persistent reminders https://www.vanta.com/products/vendor-risk-management 04.txt passed corrected Automation described as a user action. The system automatically sends document requests and follow-up reminders to vendors corrected The source describes the system managing requests and reminders. 1.0
F2-061-A043 Vanta F2-061 a The system lets a user add vendor findings directly to the risk register Vendor findings feed directly into your risk register and compliance posture https://www.vanta.com/products/vendor-risk-management 04.txt passed corrected Automation described as a user action. The system automatically feeds vendor findings into the risk register corrected The source says findings feed into the register, an automation. 1.0

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [candidates] ([candidate_id] TEXT PRIMARY KEY, [app_id] TEXT, [run] TEXT, [statement] TEXT, [excerpt] TEXT, [source_url] TEXT, [source_file] TEXT, [script_check] TEXT, [verifier_verdict] TEXT, [verifier_reason] TEXT, [verifier_correction] TEXT, [adjudicator] TEXT, [adjudicator_reason] TEXT, [in_final_set] REAL, FOREIGN KEY ([app_id]) REFERENCES apps(app_id));
CREATE INDEX [idx_candidates_app_id] ON [candidates]([app_id]);
CREATE INDEX [idx_candidates_run] ON [candidates]([run]);
CREATE INDEX [idx_candidates_verifier_verdict] ON [candidates]([verifier_verdict]);
CREATE INDEX [idx_candidates_adjudicator] ON [candidates]([adjudicator]);