candidates

Every statement both blind runs produced (3,819), with its full trail: script check → verifier verdict and reason → adjudicator ruling and reason → whether it is in the final set. Most rejections are duplicates between the two runs.

Data license: Not yet licensed — internal review only · Data source: Ahmed ElBatanony, Innopolis University

61 rows where adjudicator = "rejected (duplicate upheld)" and app_id = "F1-046"

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: source_url, source_file

run 2

  • b 55
  • a 6

verifier_verdict 1

in_final_set 1

app_id 1

  • Probo · 61 ✖

adjudicator 1

  • rejected (duplicate upheld) · 61 ✖
candidate_id ▼ app_id run statement excerpt source_url source_file script_check verifier_verdict verifier_reason verifier_correction adjudicator adjudicator_reason in_final_set
F1-046-A009 Probo F1-046 a The system lets a user manage cookie banners and keep consent records Cookie banner management, tracker detection, consent records https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A077   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-A031 Probo F1-046 a The system lets a user keep records of enrolled devices and their reported posture Device records represent enrolled endpoints and their reported posture. https://www.probo.com/docs/product/getting-started/core-concepts.md 04.txt passed rejected duplicate of F1-046-A016   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-A051 Probo F1-046 a The system automatically creates the next task when the deadline of a recurring task passes When the deadline passes, Probo creates the next task: https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A022   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-A058 Probo F1-046 a The system automatically updates the other side when a linked task or issue is edited Once a task is linked, edits on either side update the other: https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A023   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-A069 Probo F1-046 a The system lets a user create a compliance portal with a public URL Creating a portal assigns it a public URL and opens its configuration page. https://www.probo.com/docs/product/compliance-portal/quickstart 11.txt passed rejected duplicate of F1-046-A008   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-A082 Probo F1-046 a The system lets a user store a consent record with the action, category choices, banner version, client context, and timestamp A consent record stores the action, category choices, banner version, client context, and timestamp reported to Probo. https://www.probo.com/docs/product/cookie-banner/overview 12.txt passed rejected duplicate of F1-046-A078   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B001 Probo F1-046 b The system lets a user maintain a risk register with inherent and residual scoring and treatment strategies. Risk register, inherent/residual scoring, treatment strategies (mitigate, accept, avoid, transfer), threat-based risk assessments https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A001   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B002 Probo F1-046 b The system lets a user maintain a control library with maturity levels. Control library with maturity levels, custom framework import/export, Statement of Applicability (SoA) https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A002   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B003 Probo F1-046 b The system lets a user keep a vendor inventory and track data processing agreements. Vendor inventory, automated website risk assessment, DPA/BAA tracking, subprocessor discovery https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A004   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B004 Probo F1-046 b The system lets a user record data inventory and processing activity records. DPIA, Transfer Impact Assessments, processing activity records, data inventory, rights requests (SAR/erasure) https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A005   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B005 Probo F1-046 b The system lets a user scope audits, map controls, track findings, and generate reports. Audit scoping, control mapping, finding tracking, report generation https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A013   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B006 Probo F1-046 b The system lets a user collect evidence as files and URLs and track implementation state. Evidence collection (files and URLs), implementation state tracking, task assignment https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A006   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B007 Probo F1-046 b The system lets a user manage versioned documents with approval quorums, electronic signatures, and PDF export. Versioned documents, approval quorums, electronic signatures, PDF export, bulk operations https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A007   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B008 Probo F1-046 b The system lets a user manage NDAs and publish certifications on a public compliance portal with a custom domain. Public compliance portal, NDA management, certification publishing, custom domain support https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A008   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B009 Probo F1-046 b The system lets a user manage cookie banners and keep consent records. Cookie banner management, tracker detection, consent records https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A077   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B010 Probo F1-046 b The system lets a user access every entity through a command-line client. A fully-featured command-line client for scripting, automation, and CI/CD integration. https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A010   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B011 Probo F1-046 b The system lets an external AI agent read and write compliance data through an agent protocol interface. Any MCP-compatible LLM agent (Claude, Cursor, Continue, and others) can connect directly and interact with your compliance data. https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A011   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B012 Probo F1-046 b The system lets a user automate compliance tasks in an external no-code automation tool. brings Probo into n8n workflows for no-code automation of compliance tasks over the GraphQL API. https://raw.githubusercontent.com/getprobo/probo/HEAD/README.md 01.txt passed rejected duplicate of F1-046-A012   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B013 Probo F1-046 b The system lets a user model scopes, systems, threats, scenarios, and resulting risks in risk assessments. Maintain a risk register and model scopes, systems, threats, scenarios, and resulting risks. https://www.probo.com/docs/product/getting-started/product-overview.md 03.txt passed rejected duplicate of F1-046-A035   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B014 Probo F1-046 b The system lets a user receive task and comment lifecycle events through webhooks. Webhooks Receive task and comment lifecycle events https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A014   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B015 Probo F1-046 b The system lets a user invite users as members of an organization. Users are invited as members of an organization. https://www.probo.com/docs/product/getting-started/core-concepts.md 04.txt passed rejected duplicate of F1-046-A017   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B019 Probo F1-046 b The system lets a user assign work to named users. Assign work to named users and use the audit log to investigate important changes. https://www.probo.com/docs/product/compliance-program 05.txt passed rejected duplicate of F1-046-A032   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B020 Probo F1-046 b The system lets a user export risk assessment scope diagrams as Mermaid. Scope diagrams can be exported as Mermaid for use in technical review. https://www.probo.com/docs/product/risk-management 06.txt passed rejected duplicate of F1-046-A036   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B021 Probo F1-046 b The system lets a user link risks to internal controls, obligations, third-party assessments, and risk-assessment scenarios. Risks can be linked to: https://www.probo.com/docs/product/risk-management 06.txt passed rejected duplicate of F1-046-A019   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B022 Probo F1-046 b The system lets a user record data protection impact assessments and transfer impact assessments as records linked to processing, data, vendors, risks, and internal controls. Link them to the relevant processing, data, vendors, risks, and internal controls so reviewers can follow the decision. https://www.probo.com/docs/product/privacy-management 07.txt passed rejected duplicate of F1-046-A027   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B023 Probo F1-046 b The system lets a user track rights requests such as access or erasure through their lifecycle. Rights requests track requests such as access or erasure through their operational lifecycle. https://www.probo.com/docs/product/privacy-management 07.txt passed rejected duplicate of F1-046-A037   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B024 Probo F1-046 b The system lets a visitor submit a rights request from a public site. To let visitors submit a request from your public site, see Rights requests in the Compliance Portal. https://www.probo.com/docs/product/privacy-management 07.txt passed rejected duplicate of F1-046-A038   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B025 Probo F1-046 b The system lets an administrator authenticate members through single sign-on with SAML. use SAML single sign-on (SSO) to authenticate them through your identity provider https://www.probo.com/docs/product/identity-and-access 08.txt passed rejected duplicate of F1-046-A039   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B026 Probo F1-046 b The system lets an administrator create, update, and deactivate user access from an identity provider with SCIM. use SCIM to create, update, and deactivate their access. https://www.probo.com/docs/product/identity-and-access 08.txt passed rejected duplicate of F1-046-A040   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B027 Probo F1-046 b The system lets an administrator invite members and manage their roles in an organization. Invite and role-manage members in the organization https://www.probo.com/docs/product/identity-and-access 08.txt passed rejected duplicate of F1-046-A041   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B028 Probo F1-046 b The system lets an administrator map membership roles from an identity provider with a SAML role attribute. Keep membership roles in People, or map them from your identity provider with the SAML Role Attribute. https://www.probo.com/docs/product/identity-and-access 08.txt passed rejected duplicate of F1-046-A042   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B029 Probo F1-046 b The system lets an administrator investigate who changed people, roles, and organization records in an audit log. Audit log Investigate who changed people, roles, and organization records https://www.probo.com/docs/product/identity-and-access 08.txt passed rejected duplicate of F1-046-A034   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B030 Probo F1-046 b The system lets an owner or administrator create tasks with a name, description, state, priority, assignee, internal control, deadline, and time estimate. Owners and admins can create tasks. Viewers can read tasks, comments, and activity. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A044   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B031 Probo F1-046 b The system lets a user search and filter the task list by name, state, or assignee. The list can be searched by name and filtered by state or assignee. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A046   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B032 Probo F1-046 b The system lets a user reorder tasks by dragging them. Drag a task to change its order. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A047   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B033 Probo F1-046 b The system lets a user set a recurrence interval on a task with a deadline. Set a recurrence interval together with a deadline. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A022   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B034 Probo F1-046 b The system automatically creates the next task when the deadline of a recurring task passes. When the deadline passes, Probo creates the next task: https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A022   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B035 Probo F1-046 b The system lets a user comment on tasks. The task page has a comment thread and an activity log. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A052   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B036 Probo F1-046 b The system automatically records the previous and new value of each change to a task in an activity log. Each entry stores the previous value and the new value. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A055   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B037 Probo F1-046 b The system lets a user publish a task as a new issue in an external issue tracker. Publish creates a new Linear issue in the team you pick. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A056   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B038 Probo F1-046 b The system lets a user link a task to an existing issue in an external issue tracker. Link attaches the task to an issue that already exists in that team. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A057   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B039 Probo F1-046 b The system automatically keeps a linked task and an external issue tracker issue in sync when either is edited. Once a task is linked, edits on either side update the other: https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A023   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B040 Probo F1-046 b The system lets a user stop syncing a task with an issue tracker. Unlink on the task removes the connection. https://www.probo.com/docs/product/tasks/overview 09.txt passed rejected duplicate of F1-046-A059   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B041 Probo F1-046 b The system lets a user create access sources and access review campaigns. Create a campaign and attach the sources that define its scope. https://www.probo.com/docs/product/access-review/overview 10.txt passed rejected duplicate of F1-046-A063   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B042 Probo F1-046 b The system automatically fetches and snapshots available access entries when an access review campaign is started. Start the campaign to fetch and snapshot available access entries. https://www.probo.com/docs/product/access-review/overview 10.txt passed rejected duplicate of F1-046-A060   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B043 Probo F1-046 b The system lets a reviewer add flags and record decisions on access entries. Review entries, add flags where useful, and record decisions. https://www.probo.com/docs/product/access-review/overview 10.txt passed rejected duplicate of F1-046-A061   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B044 Probo F1-046 b The system lets a user complete an access review campaign after every entry has a decision. Complete the campaign after every entry has a decision. https://www.probo.com/docs/product/access-review/overview 10.txt passed rejected duplicate of F1-046-A062   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B045 Probo F1-046 b The system lets a user paste an exported account list as a CSV access source for a system that cannot be connected directly. Paste an exported account list for a system Probo cannot connect to directly. https://www.probo.com/docs/product/access-review/overview 10.txt passed rejected duplicate of F1-046-A064   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B046 Probo F1-046 b The system lets a user create a compliance portal with a public URL. Creating a portal assigns it a public URL and opens its configuration page. https://www.probo.com/docs/product/compliance-portal/quickstart 11.txt passed rejected duplicate of F1-046-A008   rejected (duplicate upheld) duplicate upheld 0.0
F1-046-B047 Probo F1-046 b The system lets a user add a description, website URL, contact email, headquarters address, and logos to a portal public profile. Add a description, website URL, contact email, and headquarters address. https://www.probo.com/docs/product/compliance-portal/quickstart 11.txt passed rejected duplicate of F1-046-A070   rejected (duplicate upheld) duplicate upheld 0.0

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [candidates] ([candidate_id] TEXT PRIMARY KEY, [app_id] TEXT, [run] TEXT, [statement] TEXT, [excerpt] TEXT, [source_url] TEXT, [source_file] TEXT, [script_check] TEXT, [verifier_verdict] TEXT, [verifier_reason] TEXT, [verifier_correction] TEXT, [adjudicator] TEXT, [adjudicator_reason] TEXT, [in_final_set] REAL, FOREIGN KEY ([app_id]) REFERENCES apps(app_id));
CREATE INDEX [idx_candidates_app_id] ON [candidates]([app_id]);
CREATE INDEX [idx_candidates_run] ON [candidates]([run]);
CREATE INDEX [idx_candidates_verifier_verdict] ON [candidates]([verifier_verdict]);
CREATE INDEX [idx_candidates_adjudicator] ON [candidates]([adjudicator]);