Creating a New Fetcher

The evidence-fetchers repository includes a library of scripts that automatically collect compliance evidence from your infrastructure and upload it to Paramify. If the service or check you need isn't already covered, you can write a new fetcher and register it in a few steps. This guide walks you through the full process — from copying a template to validating your script and seeing it appear as a selectable fetcher in the catalog.

Prerequisites

• The evidence-fetchers repository cloned locally
• Python 3 and the required packages installed (pip install -r requirements.txt)
• A configured .env file (copy .env.example and fill in your values)
• The CLI tools your new fetcher will use (e.g., aws-cli, kubectl, or the relevant SDK)
• Read access to the service you want to collect evidence from

1. Choose a template

Two starter templates live in the 6-add-new-fetcher/ folder — one for Bash scripts and one for Python scripts. Choose whichever language fits the service you're integrating with:

Copy your chosen template into the appropriate category folder under fetchers/, and give it a descriptive name that reflects what it collects:

For a Bash script:

cp 6-add-new-fetcher/new_script_template.sh fetchers/aws/my_new_check.sh
chmod +x fetchers/aws/my_new_check.sh

For a Python script:

cp 6-add-new-fetcher/new_script_template.py fetchers/okta/my_new_check.py
chmod +x fetchers/okta/my_new_check.py

The existing category folders are aws, k8s, okta, knowbe4, sentinelone, gitlab, rippling, and checkov. If you're targeting a service that doesn't have a folder yet, you can create a new one.

2. Write the script

Open your new file and replace the placeholder values throughout the template. Both templates are structured the same way at a high level — a header comment block that documents what the script does, followed by the collection logic, followed by JSON output.

Here's what to fill in:

Configuration and environment variables

You don't need to add any argument parsing to your script. Both templates automatically load all configuration from your .env file by sourcing a shared helper at startup:

• Bash — the first line of logic sources env_loader.sh, which populates $PROFILE, $REGION, and $OUTPUT_DIR for you:
• Python — call parse_fetcher_args() at the top of main() to get the same three values:

You can also override these at run time without modifying the script (useful for testing):

bash fetchers/aws/my_new_check.sh --profile my-profile --region us-east-1 --output-dir /tmp/evidence

Output format

Your script must write its results as a valid JSON file to the output directory. The Bash template initializes the file for you with a standard metadata wrapper:

{
  "metadata": {
    "profile": "...",
    "region": "...",
    "datetime": "...",
    "account_id": "...",
    "arn": "..."
  },
  "results": [],
  "summary": {}
}

Append your collected records into the results array. The Python template follows the same pattern and saves the file using json.dump().

3. Register the fetcher in the catalog

The system tracks all available fetchers in a central catalog file. You need to add your new script to it before it will appear as a selectable option. The easiest way to do this is with the interactive registration tool:

python 6-add-new-fetcher/add_evidence_fetcher.py --interactive

The tool will prompt you for:

• The path to your script file (e.g., fetchers/aws/my_new_check.sh)
• The category (e.g., aws, okta, k8s)
• A name and description (it will try to extract these from your header comments automatically — you can confirm or override)

The tool also auto-generates a unique ID for your fetcher in the format EVD-[CATEGORY]-[SCRIPT-NAME] (e.g., EVD-AWS-S3-ENCRYPTION) and adds it to both the catalog and the customer configuration template.

python 6-add-new-fetcher/add_evidence_fetcher.py \
  --script-file fetchers/aws/my_new_check.sh \
  --category aws \
  --name "My New Check"

4. Validate the catalog

After registering, run the validation script to confirm everything is wired up correctly:

python 6-add-new-fetcher/validate_catalog.py

This checks that:

• All required metadata fields are present for your new entry
• The script file exists at the path recorded in the catalog
• Your fetcher's ID is unique across all categories
• The customer configuration template is in sync with the catalog

If validation reports errors, the output will tell you exactly which field or file is the problem. Fix the issue and run validation again before moving on.

5. Test the fetcher

First, run your script directly to confirm it collects evidence and writes a valid JSON file:

# Bash
bash fetchers/aws/my_new_check.sh

# Python
python fetchers/okta/my_new_check.py

Check the output directory (default: ./evidence/) for a .json file with your script's name. Open it and confirm the structure looks correct before proceeding.

Next, verify that the fetcher appears correctly when generating evidence sets:

python generate_evidence_sets.py

Your new fetcher should now be selectable in the menu and included when customers configure their evidence sets.

6. Contribute your fetcher (optional)

If your new fetcher covers a check that would be useful to other Paramify customers, consider submitting it back to the shared library. Commit your script and the updated catalog files, then open a pull request against the main branch of the paramify/evidence-fetchers repository on GitHub. The developer guide in 6-add-new-fetcher/DEVELOPER_GUIDE.md has more detail on contribution conventions and how to add custom categories or dependency types.

Troubleshooting