➽Explainer Article

A Critical Evaluation of Security Risk Scoring Systems: Methodologies and Limitations

May 8, 2025
|
by Cyber Analyst
A Critical Evaluation of Security Risk Scoring Systems: Methodologies and Limitations

➤Summary

Introduction

Cybersecurity risk scoring systems have become essential for organizations evaluating their security posture or assessing vendor risks. These systems typically assign ratings through external, passive assessments. This paper critically examines the methodologies, technical details, limitations, and market adoption rationale of such scoring systems.

Methodology Overview

Cybersecurity risk scoring systems typically assign ratings ranging from low risk (A) to critical risk (F) based on several core factors:

  1. Network Security
  2. DNS Health
  3. Patching Cadence
  4. Endpoint Security
  5. IP Reputation
  6. Application Security
  7. Security Best Practices
  8. Hacker Chatter
  9. Information Leak
  10. Social Engineering

Each factor is scored individually, contributing to an aggregate rating representing the organization’s overall cybersecurity posture.

Detailed Technical Analysis and Critical Review

Passive Enumeration Techniques

Passive enumeration involves collecting publicly available data without interacting directly with the organization’s internal networks. Techniques include:

  • Discovering DNS records: Utilizing platforms such as DNSDumpster (a DNS enumeration tool that provides DNS records, IP addresses, MX records, and subdomains), Shodan, Censys, Zoomeye, FOFA, and others, these tools can identify public domain names and subdomains. Analysts then filter these results to focus on Fully Qualified Domain Names (FQDN) that resolve to IP addresses, ensuring the relevance and accuracy of live systems identified.
  • Resolving domain names to IP addresses: Using passive DNS databases like VirusTotal, SecurityTrails, and PassiveTotal, which store historical DNS resolution data.
  • Querying public databases: IP addresses are cross-referenced against platforms like Shodan, Censys, Zoomeye, FOFA, and BinaryEdge. These platforms store extensive data about internet-facing services and banners, helping identify potentially vulnerable or misconfigured services.

Network Security

Technical Methodology: Passive scanning identifies open ports and analyzes service banners (HTTP headers, SMTP banners, FTP banners, etc.) extracted from public scanning databases.

Critical Review: This methodology depends heavily on service banners, which can be misleading, outdated, or deliberately obfuscated. Without active verification or internal confirmation, these results can generate substantial inaccuracies, leading to misinterpretation of an organization’s actual security posture.

DNS Health

Technical Methodology: This involves verifying the presence and correctness of DNS configurations such as SPF (Sender Policy Framework), DKIM (DomainKeys Identified Mail), and DMARC (Domain-based Message Authentication, Reporting & Conformance) records using passive DNS queries.

Critical Review: Although important for email security, DNS records represent a minimal portion of overall cybersecurity vulnerabilities. The absence or misconfiguration of these records alone is insufficient to accurately gauge an organization’s comprehensive security posture.

Patching Cadence

Technical Methodology: Software versions publicly revealed through banners are cross-checked against known vulnerabilities (CVEs) using databases like the National Vulnerability Database (NVD).

Critical Review: Banners alone are unreliable indicators of internal patch status. Outdated banners can incorrectly indicate vulnerabilities, resulting in misleading risk assessments about an organization’s real-time vulnerability exposure.

Endpoint Security

Technical Methodology: Passive collection of endpoint metadata from user-agent strings, browser headers, and plugin versions exposed through HTTP requests captured by public scanning infrastructures.

Critical Review: Metadata collection frequently produces outdated and inaccurate snapshots. It offers limited visibility into real-time endpoint security conditions and overlooks internal controls, patch management practices, and endpoint protection mechanisms.

Hacker Chatter

Technical Methodology: Systematic monitoring of hacker forums, dark web discussions, and paste sites for explicit mentions of the organization.

Critical Review: Mentions alone rarely indicate actionable threats without substantial context. Consequently, monitoring chatter can produce excessive false positives and inadequate evidence of real threats or ongoing malicious activities.

Information Leak

Technical Methodology: Monitoring surface-web forums, Telegram channels, and repositories for daily leaks of credentials, log files, and other sensitive data. Often inaccurately marketed as “darknet data analysis,” most of these sources are actually accessible via the surface web or deep web. Collected leaks are aggregated into searchable databases allowing queries against specific organizational domains.

Critical Review: Leaked data typically consists of historical, previously exploited information. Without active validation of credentials, their relevance or operational impact remains uncertain, limiting the practical use of these findings to assess immediate risks.

Limitations of Passive Enumeration

  • Reliance on Public Data: High likelihood of inaccuracies and incomplete data.
  • No Internal Visibility: Passive techniques miss internal security controls and internal network configurations.
  • Historical Bias: Often reflects outdated vulnerabilities rather than current threats.
  • Inaccuracy in Risk Correlation: Passive enumeration can produce misleading or incorrect conclusions due to lack of active verification.

Market Adoption

Despite notable limitations, passive scoring systems remain popular due to:

  • Quick and scalable assessments.
  • Their non-intrusive nature is attractive for vendor assessments and compliance purposes.
  • Regulatory frameworks mandating standardized external risk evaluations.
  • Simplified scoring models appealing to non-technical stakeholders.

Who Needs Security Risk Scores?

  • Cybersecurity Insurance Providers: They require quantifiable measures to evaluate the cybersecurity posture of clients, determining insurance premiums based on assessed risk.
  • Supply Chain Risk Management: Organizations utilize risk scores to evaluate and manage the security posture of vendors and partners. Supply chain risk management involves systematically identifying and mitigating cybersecurity risks within the extended vendor ecosystem to prevent breaches via third-party integrations or dependencies.

The Kaduu Threat Intelligence Approach

Kaduu Threat Intelligence has developed its own security risk scoring system, building upon all lessons learned from traditional passive scoring models while significantly enhancing depth, accuracy, and contextual relevance.

Key Enhancements in Our Model:

  • Contextual Risk Weighting:
    • Industry type: Some industries (e.g., finance, healthcare) face inherently greater exposure.
    • Company size: Larger organizations tend to have more publicly available assets and thus greater exposure.
    • Country: Geographic risk factors are considered, reflecting local threat actors, regulatory standards, and digital infrastructure maturity.
  • Benchmarking Capabilities:
    • Organizations can be benchmarked against similar peers based on sector, size, and geography.
    • Useful for internal reporting and comparative analysis within supply chains or competitive landscapes.
  • Deeper Data Validation and Contextualization:
    • For leaked credentials, we assess not only volume but risk impact:
      • How recent is the leak?
      • Are passwords reused or aligned with the target’s policy complexity requirements?
      • Are exposed accounts tied to first-party infrastructure (e.g., internal mail, VPN) or third-party services?
      • If third-party: Are there concerning patterns (e.g., use of corporate email on private accounts like Netflix or gaming sites)?

These improvements allow us to generate detailed, accurate, and contextualized risk metrics, whether for vendor assessments, security ratings of product vendors, or tailored insights for end clients.

Conclusion

Passive cybersecurity scoring systems provide basic indicators of publicly observable security postures. However, they heavily rely on external, surface-level assessments, severely limiting their accuracy in capturing internal and operational security realities. While these assessments are common for compliance (“compliance dark web monitoring”), they predominantly serve as liability protection measures rather than effective preventive controls against security incidents. A truly robust cybersecurity strategy demands comprehensive internal assessments, penetration testing, active vulnerability management, and contextualized threat intelligence.

References

  • National Vulnerability Database (NVD): https://nvd.nist.gov
  • Shodan: https://www.shodan.io
  • Censys: https://censys.io
  • Zoomeye: https://www.zoomeye.org
  • DNSDumpster: https://dnsdumpster.com
  • CVSS Scoring System, FIRST.org: https://www.first.org/cvss