Spry LogoOpsfolio

Introduction

Fleetfolio — an operational security platform that moves beyond traditional penetration testing by enabling continuous Operational Acceptance Testing (OAT) and real-world threat exposure validation in a compliance-ready workflow.

Fleetfolio

Fleetfolio is a specialized service layer inside the broader Opsfolio and Opsfolio Compliance-as-a-Service (CaaS) ecosystem. It is built for organizations that need a repeatable, evidence-driven approach to discovering, evaluating, and operationally validating their external-facing digital assets — compressing traditional penetration testing timelines from weeks to hours, while producing audit-ready evidence at every step.

Acceptance Testing in Fleetfolio

Fleetfolio supports both User Acceptance Testing (UAT) and Operational Acceptance Testing (OAT), applied based on context:

  • User Acceptance Testing (UAT): Validates deterministic application behavior such as UI flows, APIs, and expected feature outputs.
  • Operational Acceptance Testing (OAT): Validates how systems behave under real-world conditions, including security exposure, exploitability, and operational risk.

Fleetfolio primarily focuses on OAT for security — ensuring that findings are not just technically correct, but meaningful and actionable in real-world environments.

Fleetfolio is built around two tightly integrated capabilities:

  • Fleetfolio Pentest — The technical assessment engine that runs structured penetration testing and asset discovery workflows using 34+ open-source tools.
  • Fleetfolio Threat Exposure — The reporting, dashboards, and communications layer that transforms raw findings into actionable workflows and executive insights.

Together, they form the foundation for continuous threat exposure management through Operational Acceptance Testing (OAT) — making it accessible to both non-technical users running assessments and security professionals interpreting, validating, and acting on results.

  • Fleetfolio Pentest provides the technical assessment engine — containerized, portable, and schedulable.
  • Fleetfolio Threat Exposure delivers real-time reporting, dashboards, and workflow automation.

Both feed directly into Opsfolio CaaS POA&M workflows, ensuring every vulnerability is traceable, actionable, and tied to compliance obligations.


Fleetfolio Pentest

Fleetfolio Pentest is the core assessment capability of the platform. It provides a reusable, auditable, and compliance-aligned engine for discovering, evaluating, and operationally validating external-facing enterprise assets — running continuously or on-demand without significant operational overhead.

Fleetfolio Pentest contributes to both UAT and OAT:

  • Supports UAT by validating application behavior, workflows, and expected outputs
  • Enables OAT by generating real-world security findings that require analyst validation for exploitability and operational impact

What Makes It Different

  • Non-technical friendly: Even users without a security background can trigger assessments. Security professionals then review, validate, and interpret results — with automated report generation reducing manual effort and human error.
  • Fast: Traditional penetration testing can take weeks. Fleetfolio reduces this to hours while still producing comprehensive, structured outputs.
  • Portable: Docker-based deployment means it runs anywhere — on-premises, in CI/CD pipelines, or scheduled via cron jobs.
  • Open and extensible: Leverages 34+ open-source security tools, reducing dependency on expensive proprietary solutions. Results from third-party or paid tools can also be imported and unified in the same dashboard.

How It Works

Deploy the container

  • Based on kalilinux/kali-last-release:latest with Spry runbook installed.
  • Mounts /{pwd}/results as the working evidence directory.

Configure targets

  • Provide domains, IP ranges, and key URLs/APIs via environment variables.
  • Excludes may be set to respect customer scoping and authorization boundaries.

Run the assessment

  • A pre-configured Spry runbook (eaa-pentest-lite.spry.md) orchestrates a sequence of lightweight, authorized tests.
  • Outputs are generated consistently under /{pwd}/results/<timestamp>/<tool>/… in JSON, JSONL, XML, or text formats.

Review artifacts

  • Each run generates a SQLite database consolidating all findings.
  • Results are visualized through the Surveilr web UI for easy analysis and decision-making.
  • Analysts can also directly inspect structured outputs or feed them into the Fleetfolio reporting pipeline.

Result Structure

Every assessment produces a consistent, timestamped directory of artifacts:

results/
├── 2026-03-04T10-20-10/
│   └── *.sqlite.db
├── 2026-03-04T11-05-30/
│   └── *.sqlite.db

Each run generates a SQLite database storing all findings, which powers visual dashboards via the Surveilr web UI. External findings from paid or open-source tools can be imported by placing outputs in markdown, JSON, JSONL, or TXT format into a timestamped folder and regenerating the database — no complex pipelines required.

Why use Fleetfolio Pentest?

  • Consistent structure: Every run produces predictable artifacts, making it easy to compare results over time.
  • Audit-ready: JSON/JSONL-first evidence formats are machine-readable and correlatable across tools, stored with timestamps for traceability.
  • Schedulable: Supports cron-based scheduling for continuous, unattended security assessments.
  • Import-friendly: Bring in findings from third-party tools by dropping them into a timestamped folder — Fleetfolio handles the rest.
  • Compliance fit: Findings feed directly into Opsfolio CaaS, linking evidence to compliance controls and POA&M workflows.
  • Scope Authorization: Fleetfolio Pentest assumes all provided targets have been explicitly authorized for testing by the customer.
  • Lightweight by Default: The included runbook runs "lite" tests suited for continuous assessments. Deeper scans should be handled by specialized teams.
  • Artifacts Are Evidence, Not Conclusions: Raw outputs support Operational Acceptance Testing (OAT) and require analyst interpretation to determine real-world risk and exploitability.
  • Performance Controls: Apply rate limiting, exclusions, and scheduling thoughtfully to avoid unintended impact on production systems.

Assessment Tools

The Spry runbook (fleetfolio-eaa-pentest-lite.spry.md) orchestrates 34+ tools across the following stages:

  1. Subfinder → JSONL — Discovered subdomains and sources.
  2. dnsx → JSONL — DNS resolution results. Confirm live hosts and prune dead subdomains.
  3. httpx → JSONL — HTTP probing. Identify live web services, technologies, and response metadata.
  4. WhatWeb → JSON — Web technology fingerprinting. Review CMS, frameworks, and server components.
  5. Naabu → JSONL — Fast port scan results. Feed into Nmap for deeper service enumeration.
  6. Nmap (+ xq) → XML + JSON — Service and version detection. Use for attack surface and exposure mapping.
  7. OpenSSL → text — Raw certificate details. Review expiry, issuer, and certificate chain issues.
  8. Nuclei → JSONL — Template-based vulnerability findings. Sort by severity and validate high-impact issues.
  9. Katana (optional) → JSONL — Crawled endpoints. Look for hidden APIs, admin paths, and sensitive resources.
  10. Dirsearch → JSON — Discovered directories and files. Review response codes for exposed paths.
  1. wafw00f → text — Detected WAFs.
  2. Testssl → JSON — TLS configuration analysis. Identify weak ciphers, protocols, and compliance gaps.
  3. sqlmap → text — SQL injection findings.
  4. Subzy → text — Subdomain takeover candidates.
  5. Corsy → text — CORS misconfiguration analysis.
  6. Nikto → text — Web server misconfigurations and outdated components.
  7. WPScan → JSON — WordPress core, plugin, and theme vulnerabilities.
  8. RustScan → text — High-speed port discovery.
  9. Amass → text — Comprehensive subdomain and DNS discovery.
  10. DNSEnum → XML — DNS record enumeration and AXFR attempts.
  1. TheHarvester → JSON — OSINT data (emails, hosts, subdomains).
  2. Paramspider → text — URL parameters for injection testing.
  3. Ghauri → text — Advanced SQL injection detection.
  4. cdncheck → JSONL — CDN identification. Look for exposed origin IPs.
  5. ffuf → JSON — Fuzzing results for endpoints and parameters.
  6. Dalfox → JSONL — XSS findings and reflection points.
  7. SSLyze → JSON — SSL/TLS posture assessment.
  8. SMTP-User-Enum → text — SMTP user enumeration.
  9. Fierce → text — DNS reconnaissance.
  10. VirusTotal → JSON — Domain and file intelligence.
  11. VulnAPI → text — API security findings.
  12. Commix → text — Command injection testing.
  13. WAF-Bypass → text — WAF evasion attempts.
  14. Trivy → text — Vulnerabilities in code, dependencies, images, and IaC.

Fleetfolio Threat Exposure

Fleetfolio Threat Exposure is the Operational Acceptance Testing (OAT) layer of the platform.

It ensures that security findings are validated in real-world operational contexts — transforming raw technical results into actionable insights, risk-informed decisions, and measurable outcomes.

Security assessments are only valuable when validated through Operational Acceptance Testing (OAT) — ensuring findings translate into real-world action and risk reduction.

Fleetfolio Threat Exposure solves this with a real-time, evidence-driven reporting and communications layer that transforms technical findings into actionable dashboards, automated workflows, and executive insights.

From Feature Validation to Operational Reality

Traditional approaches focus heavily on UAT — verifying that systems behave as expected.

Fleetfolio extends this by enabling OAT — validating whether systems are secure, resilient, and reliable under real-world conditions.

This shift ensures organizations move beyond feature correctness to true operational security.

How Pentest & Threat Exposure Work Together

Ingest Evidence

Fleetfolio Pentest executes authorized penetration testing workflows. All findings are normalized into the evidence warehouse via Surveilr-based ingestion pipelines, structured into SQL tables.

Centralize & Visualize

SQLPage dashboards surface evidence consistently across domains, assets, and runs. Analysts get an Evidence Explorer, Findings Viewer, and Runbook Log Viewer — no more hunting through files or spreadsheets.

Deliver & Automate

Real-time dashboards show vulnerabilities as soon as they're found. Findings are automatically routed into IT workflows using severity, ownership, and asset-type rules — delivered to Jira, ServiceNow, Slack, or Teams.

Remediate & Track

Issues follow a consistent lifecycle: Open → In Progress → Remediated → Validated → Closed. Stakeholders are notified and remediation progress is visible to both technical teams and leadership.

Validate & Close the Loop

When a POA&M item is marked remediated, Fleetfolio Threat Exposure automatically triggers Pentest retests to confirm closure. Results are archived for compliance audits.

Customer Benefits

AudienceWhat They Get
CISOs & Security LeadersDashboards for exposure trends, MTTR, and compliance posture across environments
DevOps & IT OperationsPentest findings delivered directly to Jira/ServiceNow — no tool switching
Compliance & Audit TeamsAuto-generated POA&M items mapped to SOC2, ISO, CMMC, and more
MSPs / MSSPsMulti-tenant support with standardized dashboards per customer

Relationship to Opsfolio

  • Fleetfolio Pentest contributes structured artifacts (JSON, JSONL, XML, SQLite) to the Opsfolio evidence warehouse.
  • Fleetfolio Threat Exposure provides the customer-facing UI/UX for reporting, dashboards, and workflow automation.
  • Opsfolio CaaS uses both to generate Plans of Actions and Milestones (POA&M) for regulated industries — linking threat evidence directly to compliance frameworks including SOC2, ISO, CMMC, HIPAA, and FedRAMP.

How is this guide?

Last updated on

On this page