Qualityfolio Operational Truth™ Quick Start Guide

This guide outlines the step-by-step setup of Qualityfolio, which delivers Operational Truth™ through Test-Management-as-Code (TMaC) system powered by Spry’s Axiom-based execution pattern and Surveilr’s ingestion, transformation, and ETL pipeline.

Welcome to Qualityfolio - a Markdown-native Test Management system backed by Spry’s ontology engine and Surveilr’s data ingestion pipeline. Qualityfolio follows a Test Management as Code (TMaC) approach, where test artifacts, execution evidence, and dashboards are all derived from structured files under version control.

Objectives

Prerequisites

Ensure the following tools and files are available on your system before proceeding:

1. Deno – runtime required by Spry
2. Homebrew (brew) – package manager for installing dependencies
3. Spry – runbook and SQLPage orchestration engine
4. Surveilr – ingestion and transformation engine
5. Run Spry – Visual Studio Code Extension/ Cursor IDE Extension which simplifies editing Evidence Block values
6. qualityfolio-json-etl.sql – SQL-based ETL script for Qualityfolio data
7. qualityfolio.md – Markdown file containing database configuration and SQLPage queries
8. sqlpage.json – SQLPage runtime configuration file (under ./sqlpage/)

Installation

To work with Spry and the Surveilr-based Qualityfolio framework, you need to install Deno, Homebrew, Spry, and Surveilr, and ensure the required Qualityfolio files are available in your workspace.

cd $HOME/workspaces

curl -fsSL https://deno.com/install.sh | sh

deno --version

deno upgrade

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

brew --version

brew install programmablemd/packages/spry

brew uninstall spry
brew untap programmablemd/packages
brew install programmablemd/packages/spry

spry --version

which surveilr

sudo rm -rf <surveilr_path_with_file_name>

wget https://github.com/surveilr/packages/releases/download/3.23.0/surveilr_3.23.0_x86_64-unknown-linux-gnu.tar.gz

tar -xzf surveilr_3.23.0_x86_64-unknown-linux-gnu.tar.gz

sudo mv surveilr /usr/local/bin/

surveilr --version
sudo surveilr upgrade -v 3.23.0

cd $HOME/workspaces

curl -fsSL https://deno.com/install.sh | sh

deno --version

deno upgrade

curl -fsSL -o install.sh https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh

brew --version

brew install programmablemd/packages/spry

brew uninstall spry
brew untap programmablemd/packages
brew install programmablemd/packages/spry

spry --version

which surveilr

sudo rm -rf <surveilr_path_with_file_name>

brew tap surveilr/tap && brew install surveilr

surveilr --version
sudo surveilr upgrade -v 3.23.0

Install Run Spry Extension (for Visual Studio Code/ Cursor IDE)

To install Run Spry extension, download and install the pre-packaged extension for Visual Studio Code or Cursor IDE from the links below:

For more details on Run Spry extension usage and updates, refer the documentation.

Folder Structure

Qualityfolio expects the following directory structure:

ASSURANCE-PRIME/
├── support/
│   └── assurance/
│       └── qualityfolio/
│           ├── evidence/
│           │   ├── TC-GLUE-001/
│           │   │   └── 1.1/
│           │   │       ├── result.auto.json
│           │   │       └── run.auto.md
│           │   └── TC-GLUE-002/
│           │       └── 1.1/
│           │           ├── loginButtonClick.png
│           │           ├── result.auto.json
│           │           └── run.auto.md
│           ├── sqlpage/
│           │   └── sqlpage.json                  # runtime configuration file for SQLPage, auto-generated
│           ├── test-artifacts/
│           │   └── example-artifact.md
│           ├── qualityfolio-json-etl.sql         # SQL ETL script for Qualityfolio data
│           ├── qualityfolio.md                   # SQLPage Markdown page (DB config + queries)
│           └── resource-surveillance.sqlite.db   # Database generated

Key Directories and Files

• test-artifacts/ – Authoring location for test cases, plans, suites, and metadata
• evidence/ – Location for test execution evidence (results, runs, attachments)
• qualityfolio.md – Core logic file that defines database creation and SQLPage queries
• qualityfolio-json-etl.sql – ETL script that transforms ingested JSON into relational tables
• sqlpage/sqlpage.json – SQLPage runtime configuration
• resource-surveillance.sqlite.db – Generated SQLite database containing test data

Authoring Tests in Markdown

Qualityfolio tests follow a structured Markdown hierarchy to ensure full traceability and machine-readable execution. Qualityfolio projects can range from simple to highly complex structures. You can explore ready-made examples based on real-world implementations.

An example test artifact file (example-artifact.md) for the project, OWASP - GLUE UP (with Qualityfolio's Small pattern) is available here.

This example demonstrates:

• Header-based classification
• Hierarchical test design
• Evidence block patterns

Generate SQLite Database from Test Artifacts

To generate a queryable SQLite database from your test artifacts and execution evidence, run the following command from the qualityfolio directory:

spry rb run qualityfolio.md

What This Does

• Parses test artifacts under test-artifacts/
• Ingests execution evidence under evidence/
• Applies the ETL logic defined in qualityfolio.md and qualityfolio-json-etl.sql
• Produces a clean, ready-to-use SQLite database: resource-surveillance.sqlite.db

You can open this database using any SQLite-compatible SQL editor for ad-hoc analysis or reporting.

Generate Test Management Dashboard

The Test Management Dashboard provides a live, browser-based view of Qualityfolio metrics.

spry rb run qualityfolio.md
spry sp spc --fs dev-src.auto --destroy-first --conf sqlpage/sqlpage.json --md qualityfolio.md
spry sp spc --fs dev-src.auto --destroy-first --conf sqlpage/sqlpage.json --md qualityfolio.md --watch

sqlpage

HTML UI Generation (Live Preview)

You can launch the server using the following command in a terminal from the qualityfolio directory:

spry axiom web-ui test-artifacts/example-artifact.md

Running this command loads the parsed test artifact(s) in your browser at: http://127.0.0.1:9876/

• Real-time ontology parsing
• Automatic live reload whenever you save a file
• A structured HTML UI showing headings, roles, metadata, hierarchy, cases, and evidence
• Built-in validation of classification rules and document structure

This feature allows you to visually inspect and validate your Qualityfolio pattern directly from HTML UI.

Next Steps

Once your database and dashboard are running, you can:

• Query resource-surveillance.sqlite.db for custom analytics
• Extend SQLPage views by updating qualityfolio.md
• Add new test artifacts and execution evidence
• Integrate Qualityfolio into CI/CD pipelines