Using Validators in Paramify

Validators (Automated)

Automated validators use regular expressions (regex) to continuously check evidence artifacts pulled from your connected data sources. When an artifact is collected, the validator runs its regex pattern against the artifact's content and applies defined rules to determine a Pass or Fail result.

In this article

• Overview of Automated Validators
• Viewing Evidence Artifacts and Validation Results
• Understanding Validation Rules
• Other Types of Validation (Manual Attestations)
• Regex Basics (ECMAScript/JavaScript Flavor)
• Common Regex Patterns Reference
• Real-World Validator Examples
• Testing Your Regex
• Importing and Exporting Validators via CSV

Overview of Automated Validators

Each automated validator is composed of two parts:

• Regex Pattern – A regular expression that searches the evidence artifact text for specific patterns, values, or structures.
• Validation Rules – Logic that interprets what the regex found (match count, captured group value, etc.) and decides whether the result is a Pass or Fail.

Paramify uses the ECMAScript (JavaScript) flavor of regular expressions.

Viewing Evidence Artifacts and Validation Results

To review how an automated validator evaluated a specific evidence artifact:

1. Navigate to Implementation

Select Implementation in the top navigation bar.

2. Select the desired evidence set

Choose your evidence set (e.g., GitLab Merge Request Summary) from the Evidence list.

3. Review the Result Summary tab

The Result Summary tab shows all collected artifacts with their validation results.

Each row in the table shows:

• Artifact ID – Unique identifier for the evidence artifact (e.g., gitlab_merge_request_summary_project_1)
• Effective Date – When the evidence was collected
• Source Type – How data was retrieved (e.g., API via Evidence Fetcher)
• Validation Result – Pass or Fail

Understanding Validation Rules

Click the Validation Rules tab to see exactly how the regex interprets each artifact and what constitutes a pass or fail. The Validation Rules tab shows the regex pattern and the criteria that determine a Pass or Fail disposition.

Regex Operation Types

Criteria Options

Regex capture groups can also be named with the following syntax: ?<name>

Other Types of Validation (Manual Attestations)

Not all evidence is validated automatically. Some evidence sets use manual questionnaires that are answered periodically (typically quarterly) to attest to compliance.

Manual attestation evidence sets use a Questionnaire tab. Each Yes/No response maps to a Pass or Fail, with support for nested conditional sub-questions.

Manual validators support:

• Top-level Yes/No questions mapping directly to Pass or Fail
• Nested sub-questions (a, b, c) that unlock based on parent answers
• Based On Nested disposition — the parent passes only when all child questions also pass

Regex Basics (ECMAScript/JavaScript Flavor)

Character Classes

Quantifiers

Capture Groups

Use (...) to define capture groups. Groups are numbered left-to-right starting at 1. Paramify's Match Group operation references these by number.

• Example pattern:

  "rotation_percentage"\s*:\s*(\d+)

Group 1 captures the numeric value after the key — e.g., 94 from "rotation_percentage": 94.

Spanning Multiple Lines

Because the m flag is not applied, use [\s\S]*? to match across lines between two patterns:

"rds_total":\s*(\d+)[\s\S]*?"rds_ssl_enforced":\s*(\d+)

Lookaheads

Use (?!...) (negative lookahead) to exclude specific values from matching:

"GroupName":\s*"(?!Administrators")([^"]+)"

This matches any group name that is not "Administrators".

Common Regex Patterns Reference

Real-World Validator Examples

These examples come directly from Paramify's validator library.

Example 1 – Change Management Active

What it checks: All four change management flags are true in the same artifact.

"change_management_active":\s*true[\s\S]*?"approval_process_enabled":\s*true[\s\S]*?"review_process_active":\s*true[\s\S]*?"meets_minimum_review":\s*true

If the full pattern matches exactly once (all four flags present and true in sequence), the artifact passes. A match count of 0 means a flag is missing or false — the artifact fails.

Example 2 – KMS Key Rotation Percentage

What it checks: At least 90% of KMS keys have rotation enabled.

"rotation_percentage"\s*:\s*(\d+)

Group 1 captures the percentage number. If the captured value is 90 or higher, the artifact passes.

Example 3 – RDS SSL Enforcement (Comparing Two Groups)

What it checks: The count of SSL-enforced RDS instances equals the total.

"rds_total":\s*(\d+)[\s\S]*?"rds_ssl_enforced":\s*(\d+)

Group 1 = total RDS instances, Group 2 = SSL-enforced count. When they match, 100% of instances are compliant. The same pattern works for S3, ALB, NLB, and other resources by swapping the key names.

Example 4 – AWS Config Rule Compliance Status

What it checks: The AWS Config rule for KMS key rotation shows COMPLIANT.

"ConfigRuleName": "cmk-backing-key-rotation-enabled-conformance-pack[^\]]*"ComplianceType": "([^\]]*)"

Example 5 – Load Balancers Using FIPS & TLS 1.3 (Multiple Rules)

What it checks: Every SSL policy must reference both FIPS and TLS 1.3.

\s*"ssl_policy":\s*"([\s\S]*?)"

Both rules must pass. Each match's captured policy name is checked for both substrings. A policy with only TLS13 but not FIPS would fail rule 02.

Example 6 – IAM Users Not in Unexpected Groups

What it checks: No IAM groups exist with names other than "Administrators".

"GroupName":\s*"(?!Administrators")([^"]+)"

The negative lookahead skips the Administrators group. If any other group is found (count > 0), the artifact fails.

Example 7 – Vulnerability Scan Has No Critical Findings

What it checks: A CSV vulnerability report contains zero rows with "Critical" severity.

,"Critical",

A simple literal string match against CSV data. Each row containing the Critical column value increments the count. A related validator uses the same pattern but checks for "High" severity.

Example 8 – WAF Rules Configured

What it checks: At least one WAF rule is configured.

"Rules":\s*\[\s*\{

Matches the opening of a non-empty JSON array under the "Rules" key. An empty array ("Rules": []) would not match.

Testing Your Regex

Before saving a validator, always test your regex against real artifact content. Recommended tool: regex101.com — select ECMAScript / JavaScript flavor. Paste your artifact content as the test string and your pattern in the expression field. Enable the g (global) and s (dotAll) flags to replicate Paramify's behavior.

Testing Checklist

• Set flavor to ECMAScript (JavaScript) and enable the g and s flags.
• Paste a real artifact payload as the test string.
• Verify match count matches your expectation.
• If using capture groups, confirm each group contains the correct value.
• Test a failing case to confirm the pattern does not match when it shouldn't.
• Test edge cases: missing keys, null values, extra whitespace.

Importing and Exporting Validators via CSV

Validators can be managed in bulk via CSV — useful for backups, environment migrations, or building validators in a spreadsheet before uploading.

Exporting Validators

1. Navigate to Implementation → Evidence

Select Implementation in the top navigation bar, then choose Evidence.

2. Open the more options menu

Click the ⋮ (more options) menu in the Validators section.

3. Select Export as CSV

The file downloads with all validator definitions.

Importing Validators

1. Prepare your CSV file

Use the column structure shown below. The easiest starting point is to export existing validators and modify them.

2. Navigate to Implementation → Evidence and select Import CSV

Select Implementation in the top navigation bar, choose Evidence, then click Import CSV.

3. Upload and confirm

Upload the file and review any validation warnings before confirming the import.

CSV Column Reference

Validation Rules JSON Structure

[
  {
    "index": 1,
    "regexOperation": "MATCH_COUNT",        // or "MATCH_GROUP"
    "regexOperationGroupNumber": 0,          // group number for MATCH_GROUP; 0 for MATCH_COUNT
    "criteria": "EQUALS",                   // EQUALS | GREATER_THAN_OR_EQUAL_TO | CONTAINS
    "value": "CUSTOM_TEXT",                 // CUSTOM_TEXT | MATCH_GROUP
    "valueGroupNumber": 0,                  // group number when value is "MATCH_GROUP"
    "valueCustomText": "1"                  // literal value when value is "CUSTOM_TEXT"
  }
]