candidates

Every statement both blind runs produced (3,819), with its full trail: script check → verifier verdict and reason → adjudicator ruling and reason → whether it is in the final set. Most rejections are duplicates between the two runs.

Data license: Not yet licensed — internal review only · Data source: Ahmed ElBatanony, Innopolis University

175 rows where in_final_set = "1.0" and run = "b"

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: source_file

verifier_verdict 3

adjudicator 3

run 1

in_final_set 1

  • 1.0 · 175 ✖
candidate_id ▼ app_id run statement excerpt source_url source_file script_check verifier_verdict verifier_reason verifier_correction adjudicator adjudicator_reason in_final_set
F1-001-B026 Operately F1-001 b The system lets a user see all spaces the user has access to Spaces: See all spaces you have access to. https://operately.com/help/quick-tour 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B028 Operately F1-001 b The system lets a user see a timeline of the latest activities within a space Activity feed: A timeline for the latest activities within this space https://operately.com/help/quick-tour 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B036 Operately F1-001 b The system lets a user view a champion’s profile and all of that champion’s work from a work map Click on a champion’s name to see their profile and all their work. https://operately.com/help/work-map-overview 09.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B044 Operately F1-001 b The system lets a user configure baseline access for company members and space members on a project Permissions — Configure baseline access for Company members and Space members. https://operately.com/help/create-project 05.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B056 Operately F1-001 b The system lets a user adjust the deadline of a milestone Create or reorder milestones and tasks, adjust milestone deadlines and descriptions, update task status, assignees, or due dates, and open task details for notes and discussion without losing context. https://operately.com/help/quick-tour 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B057 Operately F1-001 b The system lets a user adjust the description of a milestone Create or reorder milestones and tasks, adjust milestone deadlines and descriptions, update task status, assignees, or due dates, and open task details for notes and discussion without losing context. https://operately.com/help/quick-tour 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B061 Operately F1-001 b The system lets a user open task details for notes and discussion Create or reorder milestones and tasks, adjust milestone deadlines and descriptions, update task status, assignees, or due dates, and open task details for notes and discussion without losing context. https://operately.com/help/quick-tour 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B071 Operately F1-001 b The system lets a user see all project resources, action items, and retrospectives in one place See all project resources, action items, and retrospectives in one place https://operately.com/help/features-overview 03.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B077 Operately F1-001 b The system lets a champion read the latest check-in inline while writing a new check-in While writing your check-in, click Show previous check-in to read the latest update inline. https://operately.com/help/check-in-goal 14.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B090 Operately F1-001 b The system automatically notifies everyone in the space about a new document by default By default, everyone in the space will be https://operately.com/help/create-document 10.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B105 Operately F1-001 b The system automatically notifies all members of the space or goal about a new discussion by default By default, all members of the Space (or Goal) will be notified. https://operately.com/help/start-discussion 13.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B108 Operately F1-001 b The system lets a user edit the content or change the publish date of a scheduled discussion until it publishes Until it publishes, you can reopen the discussion to edit the content or change the publish date. https://operately.com/help/start-discussion 13.txt passed confirmed     (not sent: confirmed)   1.0
F1-001-B116 Operately F1-001 b The system automatically collapses completed milestones on the Kanban board by default Completed milestones are collapsed by default under N completed milestone(s). https://operately.com/help/kanban-board-introduction 11.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B017 It's a Plan F1-002 b The system lets a user embed files in a documentation page and link the page to the issues it describes favorites, embedded files, and links to the issues they describe https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/README.md 01.txt passed corrected Combines two capabilities; the link to issues is already covered by A022. The system lets a user embed files in a documentation page. corrected The original combined two capabilities and linking pages to issues already survives separately. 1.0
F1-002-B018 It's a Plan F1-002 b The system lets a user mark documentation pages as favorites favorites, embedded files, and links to the issues they describe https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/README.md 01.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B029 It's a Plan F1-002 b The system automatically starts an agent run when the agent is mentioned in a comment, when an issue is assigned to it, or on a schedule A run starts on an @mention in a comment, on an assignment, or on a schedule https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/README.md 01.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B052 It's a Plan F1-002 b The system lets a user set an owner, a priority, a target date and a health status on an initiative It has an owner, a priority, a target date and a health status, and its progress counts the issues under it by state. https://itsaplan.dev/features/initiatives 05.txt passed rejected duplicate of F1-002-A030   corrected The health status is advertised separately in the sources and is absent from A030, so only that part is not a duplicate. 1.0
F1-002-B057 It's a Plan F1-002 b The system lets a user pan and zoom an infinite canvas of notes Notes live on an infinite pane you pan and zoom. https://itsaplan.dev/features/notes 06.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B068 It's a Plan F1-002 b The system lets a user jump to any view from a command menu Jump to any view from a fast command menu. https://itsaplan.dev/features/views 07.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B070 It's a Plan F1-002 b The system automatically creates an account when a user signs in through a single sign-on provider for the first time The first sign-in creates the account, and the registration mode under **Authentication** https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/docs/self-hosting.md 08.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B072 It's a Plan F1-002 b The system automatically expires an invite after the number of days set by an administrator Invites expire after the days set under **Authentication** https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/docs/self-hosting.md 08.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B073 It's a Plan F1-002 b The system lets an administrator turn off email and password sign-in once a single sign-on provider works Once a provider works, you can turn off **Authentication → Email and password**. The sign-in https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/docs/self-hosting.md 08.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B074 It's a Plan F1-002 b The system lets an identity provider create, deactivate and delete user accounts over a provisioning protocol An identity provider can create, deactivate and delete the accounts of the instance over https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/docs/self-hosting.md 08.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B075 It's a Plan F1-002 b The system lets an identity provider grant project access to users through its groups SCIM 2.0, and grant project access through its groups. Provisioning is set up in god mode by https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/docs/self-hosting.md 08.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B076 It's a Plan F1-002 b The system automatically prevents a deactivated account from signing in and refuses its sessions and API keys until it is reactivated account cannot sign in, and its sessions and API keys are refused until it is reactivated. https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/docs/self-hosting.md 08.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B078 It's a Plan F1-002 b The system lets a user enable the MCP server for a project Enable MCP for the project first, in Settings, MCP Server. It is off by default. https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/docs/runner.md 09.txt passed confirmed     (not sent: confirmed)   1.0
F1-002-B079 It's a Plan F1-002 b The system lets a user send an external agent's tasks to an endpoint that they run The instance can also send the tasks to an endpoint that you run. https://raw.githubusercontent.com/croffasia/itsaplan/HEAD/docs/runner.md 09.txt passed confirmed     (not sent: confirmed)   1.0
F1-005-B004 DeskcommCRM F1-005 b The system lets an AI agent trigger automations trigger automations https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Automation form needs an event; the source says agents trigger automations. The system automatically triggers automations when an AI agent handles a conversation. corrected An AI agent is not a user role, so the automation form fits better, and the source says agents trigger automations. 1.0
F1-005-B007 DeskcommCRM F1-005 b The system lets an AI agent execute skills on its own in the middle of a conversation skills the agent executes on its own mid-conversation https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Describes an automation, not a user action. The system automatically runs skills for an AI agent during a conversation when the agent decides to use them. corrected The source describes autonomous agent behaviour, which is an automation and not a user action. 1.0
F1-005-B008 DeskcommCRM F1-005 b The system lets an AI agent keep an operational memory operational memory https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed corrected Describes an automation; the 'lets an AI agent' form misstates the actor. The system automatically keeps an operational memory that AI agents use when answering. corrected This is an automation; the 'lets an AI agent' form gives the wrong actor. 1.0
F1-005-B045 DeskcommCRM F1-005 b The system lets a user manage follow-ups for AI agents **Follow-ups** https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 01.txt passed confirmed     (not sent: confirmed)   1.0
F1-005-B046 DeskcommCRM F1-005 b The system lets a user manage routers for AI agents **Routers** https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-005-B048 DeskcommCRM F1-005 b The system lets a user manage the memory of AI agents **Memory** https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-005-B049 DeskcommCRM F1-005 b The system lets a user manage skills of AI agents **Skills** https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 01.txt passed confirmed     (not sent: confirmed)   1.0
F1-005-B050 DeskcommCRM F1-005 b The system lets a user view the runs of AI agents **Runs** https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-005-B051 DeskcommCRM F1-005 b The system lets a user view AI usage and budget **Usage & budget** https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-005-B052 DeskcommCRM F1-005 b The system lets a user view the health of a messaging connection, reconnect it and manage templates **Connections** (QR or Meta's official channel, with health, reconnect and templates) https://raw.githubusercontent.com/melgarafael/DeskcommCRM/HEAD/README.en.md 04.txt passed confirmed     (not sent: confirmed)   1.0
F1-005-B067 DeskcommCRM F1-005 b The system automatically identifies a customer’s history, orders and prior agreements when a message arrives Before any reply, the system already knows who this is: history, orders, what was agreed last time. https://deskcomm.com.br/en 02.txt passed confirmed     (not sent: confirmed)   1.0
F1-011-B037 Asp.Net Core Inventory Order Management System (Indotalent WMS) F1-011 b The system automatically generates document numbers. Auto-generated document numbers. https://indotalent.com/free-wms.html 03.txt passed corrected Automation lacked a triggering event; the source states auto-generated document numbers for sales orders. The system automatically generates a document number when a user creates a sales document. corrected The automation needed a triggering event, and the source places auto-generated numbers under sales order management. 1.0
F1-017-B013 ZenTao (now branded Sanplex) F1-017 b The system lets a user designate a product manager to collect and explain stories, make release plans and review stories Product manager: collect and explain stories, make release plans and review stories. https://www.sanplex.com/book/zentaomanual/free-open-source-project-management-software-createproducts-33 03.txt passed confirmed     (not sent: confirmed)   1.0
F1-017-B031 ZenTao (now branded Sanplex) F1-017 b The system lets a project manager see the number of tasks a story has been decomposed into You can also check the number of tasks decomposed. https://www.sanplex.com/book/zentaomanual/free-open-source-project-management-software-decomposetasks-37 06.txt passed confirmed     (not sent: confirmed)   1.0
F1-017-B041 ZenTao (now branded Sanplex) F1-017 b The system lets a tester leave a bug unassigned when it is unclear who will fix it A bug can be assigned to a certain person or leave it null if it is not sure who will fix it. https://www.sanplex.com/book/zentaomanual/free-open-source-project-management-software-bugmanagement-27 07.txt passed confirmed     (not sent: confirmed)   1.0
F1-017-B047 ZenTao (now branded Sanplex) F1-017 b The system lets a user write a note on a closed bug Edit a bug: You can write a note for the bug closed. https://www.sanplex.com/book/zentaomanual/free-open-source-project-management-software-bugmanagement-27 07.txt passed confirmed     (not sent: confirmed)   1.0
F1-017-B050 ZenTao (now branded Sanplex) F1-017 b The system lets a tester set estimations for each test case step You can set estimations for each step so to facilitate managing test results and creating bugs. https://www.sanplex.com/book/zentaomanual/free-open-source-project-management-software-createtestcase-85 08.txt passed confirmed     (not sent: confirmed)   1.0
F1-023-B004 Helper F1-023 b The system lets a user view a road map of projects Add _Road Map_ feature https://raw.githubusercontent.com/devaslanphp/project-management/HEAD/README.md 01.txt passed confirmed     (not sent: confirmed)   1.0
F1-023-B009 Helper F1-023 b The system lets a user edit the epic details of a ticket Edit ticket epic details https://raw.githubusercontent.com/devaslanphp/project-management/HEAD/README.md 01.txt passed confirmed     (not sent: confirmed)   1.0
F1-034-B018 WeCMDB F1-034 b The system lets a user view the roles a user belongs to and the menu and data permissions of those roles Clicking _user name_(e.g. admin) can show all roles the user belongs to, and display the menus and data permissions which those roles have. https://raw.githubusercontent.com/WeBankPartners/we-cmdb/HEAD/wiki/docs/wecmdb_user_guide_en.md 02.txt passed confirmed     (not sent: confirmed)   1.0
F1-034-B034 WeCMDB F1-034 b The system lets a user view the raw data of an integrated enquiry message Clicking _raw data_ behide a message can display detail information of the message. https://raw.githubusercontent.com/WeBankPartners/we-cmdb/HEAD/wiki/docs/wecmdb_user_guide_en.md 02.txt passed confirmed     (not sent: confirmed)   1.0
F1-037-B013 FOSSBilling F1-037 b The system automatically generates invoice PDFs. FOSSBilling generates invoice PDFs with Dompdf, which renders HTML and CSS into a printable document. https://docs.fossbilling.org/admin-guide/invoice-pdf/ 06.txt passed confirmed     (not sent: confirmed)   1.0
F1-037-B016 FOSSBilling F1-037 b The system lets an administrator view every installed email template with its subject, owning module, and enabled status. You’ll see every installed email template, along with its subject, owning module, and enabled status. https://docs.fossbilling.org/admin-guide/email-templates/ 07.txt passed confirmed     (not sent: confirmed)   1.0

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [candidates] ([candidate_id] TEXT PRIMARY KEY, [app_id] TEXT, [run] TEXT, [statement] TEXT, [excerpt] TEXT, [source_url] TEXT, [source_file] TEXT, [script_check] TEXT, [verifier_verdict] TEXT, [verifier_reason] TEXT, [verifier_correction] TEXT, [adjudicator] TEXT, [adjudicator_reason] TEXT, [in_final_set] REAL, FOREIGN KEY ([app_id]) REFERENCES apps(app_id));
CREATE INDEX [idx_candidates_app_id] ON [candidates]([app_id]);
CREATE INDEX [idx_candidates_run] ON [candidates]([run]);
CREATE INDEX [idx_candidates_verifier_verdict] ON [candidates]([verifier_verdict]);
CREATE INDEX [idx_candidates_adjudicator] ON [candidates]([adjudicator]);