Beyond the Questionnaire: Evidence-Based Vendor Security Assessment for Security Leaders

Introduction

Most organisations can produce hundreds of completed vendor security questionnaires. They sit in a GRC platform or a shared drive, scored and signed off. Far fewer organisations can say which of those vendors could take the business offline tomorrow, which ones hold standing administrative access to production systems, or whether the answers they received are actually true.

That gap is why third-party exposure now ranks as the second-highest near-term global concern for CISOs, behind only cyber threats themselves. Protiviti's 2026 research, based on a survey of 1,540 global executives, links this to expanding vendor ecosystems and "the need for greater assurance beyond traditional compliance checks" (Protiviti).

Regulators are moving in the same direction. In the UK, the Cyber Security and Resilience Bill brings managed service providers into regulatory scope and gives authorities power to designate "critical suppliers" who become subject to security duties and incident reporting in their own right (Mayer Brown). In the EU, NIS2 requires in-scope organisations to assess and manage the security risks of their suppliers, and financial services firms face additional ICT third-party requirements under DORA.

A questionnaire tells you what a supplier says. Assurance comes from what you can verify. This guide sets out a practical, evidence-based model for vendor security assessment that security leaders can put in place this year, along with the metrics to show the board that third-party risk is genuinely under control.

Why Questionnaires Alone No Longer Work

Questionnaires were designed for a world with fewer suppliers, simpler integrations and slower change. Today they struggle for four reasons.

They measure intent, not reality. A "yes" to "Do you enforce MFA for all administrative access?" tells you what the supplier believes or wants you to believe. It does not tell you whether a forgotten legacy account still sits outside the policy.

They are point-in-time. Most questionnaires are completed at onboarding and refreshed annually at best. A supplier's security posture can change in weeks through an acquisition, a cloud migration, staff turnover or a new sub-processor.

They treat every supplier the same. When the same 200-question form goes to a catering provider and a managed security provider, effort is wasted on low-risk vendors and stretched thin on the ones that matter.

They ignore access. The most damaging supplier breaches usually travel through access you granted: remote management tools, shared accounts, API integrations and OAuth connections. Questionnaires rarely map that access in any detail.

None of this means questionnaires should be scrapped. They remain useful for low-risk suppliers and as a starting point for higher tiers. The point is that they should be one input among several, not the whole programme.

Step 1: Tier Vendors by Business Impact, Not Spend

Contract value is a poor proxy for risk. A low-cost managed service provider with domain admin rights can do far more damage than a high-value marketing agency with no system access. Start by tiering every vendor on two questions:

1. How badly would an outage or compromise at this vendor disrupt our critical operations?
2. How much access does this vendor have to our data, networks and systems?

The answers place each vendor in a tier that determines how deeply you assess it.

In most organisations the critical tier is a small fraction of the vendor list. That is where effort should go. Tiering also gives you a defensible answer when the board or a regulator asks why some suppliers receive more scrutiny than others.

To tier accurately, involve the business. Security teams often underestimate how dependent a department is on a niche platform, or overestimate the importance of a well-known brand. Ask service owners which suppliers they could not operate without for a week, and which hold data that would trigger regulatory notification if exposed.

Step 2: Map the Access Every Critical Vendor Holds

Once you know which vendors matter most, find out exactly how they connect to you. For each critical vendor, record:

• Named and shared accounts in your environment, including service accounts and API keys.
• Remote access routes such as VPN, remote monitoring and management (RMM) tools, jump hosts and support portals.
• OAuth and SaaS integrations connected to your Microsoft 365, Google Workspace, CRM and finance tenants.
• Data flows, including what data the vendor receives, where it is stored and who can access it.
• Fourth parties the vendor relies on to deliver your service, such as hosting providers and their own subcontractors.

This exercise almost always reveals surprises: accounts for staff who left the supplier years ago, integrations no one remembers approving, and remote access tools installed for a single project and never removed.

Then apply the same rules you apply internally. Every vendor account needs a named owner on your side, MFA, the least privilege required and a review date. Remote access should be brokered, logged and time-bound rather than always on. Where possible, replace shared accounts with individual, attributable identities.

Step 3: Ask for Evidence, Then Validate It

For critical and high-tier vendors, replace yes or no answers with requests for proof. Useful evidence includes:

1. Certifications with scope statements. ISO 27001, SOC 2 Type II and Cyber Essentials Plus are valuable, but only if the service you buy is within scope. Always read the scope statement and the statement of applicability or system description.
2. Independent testing. Ask for recent penetration test summaries and, more importantly, evidence that critical and high findings were remediated.
3. Incident response arrangements. Request the supplier's incident response plan, including how and when they will notify you, who your contact will be, and what support they will provide during an investigation.
4. Backup and recovery results. For suppliers that host or process critical data, ask for evidence of restore testing and recovery time performance.
5. Identity controls. Ask how privileged access to your environment and your data is controlled, reviewed and logged on their side.

Evidence still needs validating. Supplement what vendors provide with an outside-in view:

• External attack surface scanning shows exposed services, unpatched systems and misconfigurations that attackers can see today.
• Leaked credential monitoring reveals whether supplier staff credentials are circulating on criminal markets.
• Threat intelligence shows whether the vendor is being discussed, targeted or claimed as a victim by threat actors.
• Technical assessment of the integration itself, such as testing the security of an API connection or remote access route, closes the loop.

When evidence and validation disagree, the conversation with the supplier becomes far more productive. You are no longer debating a questionnaire score. You are discussing a specific exposed service or a specific leaked credential.

Step 4: Move from Annual Reviews to Continuous Assurance

Annual reassessment leaves long blind spots. For critical vendors, build lightweight continuous assurance:

• Monitor external posture and threat intelligence continuously, with alerts for significant changes.
• Require suppliers to notify you of material changes, such as acquisitions, new sub-processors, major incidents or changes to hosting.
• Review vendor access quarterly, removing accounts and integrations that are no longer needed.
• Hold short service reviews that include security alongside performance, so security does not become a once-a-year conversation.

Continuous assurance does not need to mean more paperwork. It means fewer, better signals that arrive when they are useful rather than when the calendar says so.

Step 5: Strengthen the Contract

Many supplier breaches become harder to manage because the contract is silent on what matters most. For critical vendors, make sure agreements include:

• A defined breach notification timeframe that allows you to meet your own regulatory obligations. The UK's proposed regime, for example, would require an initial notification within 24 hours and a full report within 72 hours for regulated entities.
• Rights to request evidence, conduct assessments and receive summaries of independent testing.
• Requirements for MFA, logging and least privilege on any access to your environment.
• Obligations to flow security requirements down to subcontractors.
• Cooperation duties during an incident, including access to logs and forensic findings.
• Exit and data return or deletion arrangements.

Procurement and legal teams are natural allies here. A short, standard security schedule for critical suppliers saves time on every negotiation.

Step 6: Rehearse a Supplier Breach

The real test of third-party cyber risk management is how you respond when a key supplier is compromised. Many organisations have incident response plans that assume the attacker is inside their own network. Fewer have rehearsed the scenario where the attacker is inside a supplier that holds keys to their environment.

Run a tabletop exercise with security, IT, legal, procurement, operations and communications. Build the scenario around a realistic event, such as a compromised managed service provider pushing malicious updates through its RMM tool. Work through questions such as:

• How would we find out? Would the supplier tell us, or would we detect it ourselves?
• Which credentials, integrations and remote access routes do we revoke first, and who has authority to do it?
• Which services stop, and what is our manual fallback for each?
• What do our contracts require the vendor to tell us, and how quickly?
• What must we tell regulators, customers and insurers, and by when?
• How do we decide when it is safe to reconnect?

The gaps you find become concrete improvements: contract changes, access reductions, new detection rules and continuity plans. Repeat the exercise at least annually for your most critical suppliers.

Getting Started: The First 90 Days

Transforming a questionnaire-led process can feel like a large programme. In practice, most organisations can make meaningful progress within a single quarter by focusing on the critical tier first.

Days 1 to 30: Build the picture. Consolidate your vendor lists from procurement, finance and IT into one register. Agree the tiering criteria with business owners and tier every supplier. Identify your critical vendors and confirm a named internal owner for each one.

Days 31 to 60: Close the obvious gaps. Map access for every critical vendor and remove anything dormant or excessive. Enforce MFA on remaining vendor accounts. Request evidence packs from critical suppliers and start external posture monitoring so you have an independent baseline.

Days 61 to 90: Embed and rehearse. Agree a standard security schedule with legal and procurement for new and renewing critical contracts. Run your first supplier breach tabletop exercise. Present the initial metrics to the executive team, along with a plan for the high-tier vendors in the following quarter.

Resourcing is often the deciding factor. Few security teams have spare capacity to assess dozens of critical suppliers in depth. Automation helps with external monitoring and evidence collection, while specialist support can accelerate tiering, technical validation and exercise design. The aim is to concentrate your own team's judgement on decisions, not on chasing documents.



It also helps to agree from the start what happens when a critical supplier falls short. Define in advance who can accept a residual risk, who can require remediation and in what timeframe, and when the business should consider an alternative provider. A clear escalation path turns findings into action instead of leaving them in a register.

Measuring and Reporting Third-Party Risk

Boards do not need to see every supplier score. They need confidence that the most important third-party risks are understood, owned and reducing.

A concise report should show the percentage of vendors that have been tiered, demonstrating programme coverage; the proportion of critical vendors that have completed an evidence-based assessment, showing assurance where it matters most; and the percentage of critical vendor access accounts protected by MFA with a named owner, addressing a common attack path.

It should also highlight open high-risk findings involving critical vendors and how long they have remained unresolved, indicating whether issues are being fixed, alongside the number of supplier-breach scenarios exercised in the past 12 months to demonstrate response readiness. Pair these measures with a short narrative on the three most significant supplier risks and the actions underway for each. This gives the board meaningful assurance without drowning it in detail.

Common Mistakes to Avoid

• Assessing everyone the same way. Uniform assessments waste effort and miss the vendors that matter.
• Accepting certificates without reading the scope. A certificate that covers a different service gives you no assurance.
• Forgetting fourth parties. Your critical vendor's hosting provider or subcontractor may be the weakest link.
• Leaving vendor access always on. Persistent, unmonitored remote access remains one of the most common routes into organisations.
• Treating onboarding as the finish line. Risk changes during the relationship, and assurance needs to keep pace.
  

A Quick Checklist for Critical Vendors

☐ Vendor tiered by business impact and access

☐ All vendor access mapped, owned and protected with MFA

☐ Certification scope confirmed against the services you use

☐ Penetration test remediation evidence reviewed

☐ External posture and threat intelligence monitored continuously

☐ Breach notification timeframe written into the contract

☐ Fourth parties identified for critical services

☐ Supplier breach scenario exercised in the last 12 months

FAQs

What is an evidence-based vendor security assessment?

It is an approach that verifies a supplier's security through documents, technical validation and external monitoring, rather than relying only on self-reported questionnaire answers.

How often should critical vendors be assessed?

Critical vendors should have a full assessment at least annually, supported by continuous monitoring and quarterly access reviews in between.

Who should own third-party cyber risk?

Ownership is usually shared. Security sets the standard and performs assessments, procurement and legal embed requirements in contracts, and business service owners remain accountable for the risk of the suppliers they rely on.

Summary

Cybergen's risk consultants help security leaders tier their supply chain, map vendor access, validate supplier controls with threat intelligence and technical assessment, and run realistic supplier breach exercises. We help you replace paperwork with evidence, so you can show the board and regulators that third-party risk is understood and actively managed.

Talk to our team about turning vendor questionnaires into real assurance.

Ready to strengthen your security posture? Contact us today for more information on protecting your business.