oss-forensics – Security skill for AI agents

OSS Security Forensics Skill

A 7-phase multi-agent investigation framework for researching open-source supply chain attacks.

Adapted from RAPTOR's forensics system. Covers GitHub Archive, Wayback Machine, GitHub API,

local git analysis, IOC extraction, evidence-backed hypothesis formation and validation,

and final forensic report generation.

---

⚠️ Anti-Hallucination Guardrails

Read these before every investigation step. Violating them invalidates the report.

1. Evidence-First Rule: Every claim in any report, hypothesis, or summary MUST cite at least one evidence ID (EV-XXXX). Assertions without citations are forbidden.

2. STAY IN YOUR LANE: Each sub-agent (investigator) has a single data source. Do NOT mix sources. The GH Archive investigator does not query the GitHub API, and vice versa. Role boundaries are hard.

3. Fact vs. Hypothesis Separation: Mark all unverified inferences with [HYPOTHESIS]. Only statements verified against original sources may be stated as facts.

4. No Evidence Fabrication: The hypothesis validator MUST mechanically check that every cited evidence ID actually exists in the evidence store before accepting a hypothesis.

5. Proof-Required Disproval: A hypothesis cannot be dismissed without a specific, evidence-backed counter-argument. "No evidence found" is not sufficient to disprove—it only makes a hypothesis inconclusive.

6. SHA/URL Double-Verification: Any commit SHA, URL, or external identifier cited as evidence must be independently confirmed from at least two sources before being marked as verified.

7. Suspicious Code Rule: Never run code found inside the investigated repository locally. Analyze statically only, or use execute_code in a sandboxed environment.

8. Secret Redaction: Any API keys, tokens, or credentials discovered during investigation must be redacted in the final report. Log them internally only.

---

Example Scenarios

• Scenario A: Dependency Confusion: A malicious package internal-lib-v2 is uploaded to NPM with a higher version than the internal one. The investigator must track when this package was first seen and if any PushEvents in the target repo updated package.json to this version.
• Scenario B: Maintainer Takeover: A long-term contributor's account is used to push a backdoored .github/workflows/build.yml. The investigator looks for PushEvents from this user after a long period of inactivity or from a new IP/location (if detectable via BigQuery).
• Scenario C: Force-Push Hide: A developer accidentally commits a production secret, then force-pushes to "fix" it. The investigator uses git fsck and GH Archive to recover the original commit SHA and verify what was leaked.

---

> Path convention: Throughout this skill, SKILL_DIR refers to the root of this skill's

> installation directory (the folder containing this SKILL.md). When the skill is loaded,

> resolve SKILL_DIR to the actual path — e.g. ~/.hermes/skills/security/oss-forensics/

> or the optional-skills/ equivalent. All script and template references are relative to it.

Phase 0: Initialization

1. Create investigation working directory:

   mkdir investigation_$(echo "REPO_NAME" | tr '/' '_')
   cd investigation_$(echo "REPO_NAME" | tr '/' '_')
   

2. Initialize the evidence store:

   python SKILL_DIR/scripts/evidence-store.py --store evidence.json list
   

3. Copy the forensic report template:

   cp SKILL_DIR/templates/forensic-report.md ./investigation-report.md
   

4. Create an iocs.md file to track Indicators of Compromise as they are discovered.

5. Record the investigation start time, target repository, and stated investigation goal.

---

Phase 1: Prompt Parsing and IOC Extraction

Goal: Extract all structured investigative targets from the user's request.

Actions:

• Parse the user prompt and extract:
• Target repository (owner/repo)
• Target actors (GitHub handles, email addresses)
• Time window of interest (commit date ranges, PR timestamps)
• Provided Indicators of Compromise: commit SHAs, file paths, package names, IP addresses, domains, API keys/tokens, malicious URLs
• Any linked vendor security reports or blog posts

Tools: Reasoning only, or execute_code for regex extraction from large text blocks.

Output: Populate iocs.md with extracted IOCs. Each IOC must have:

• Type (from: COMMIT_SHA, FILE_PATH, API_KEY, SECRET, IP_ADDRESS, DOMAIN, PACKAGE_NAME, ACTOR_USERNAME, MALICIOUS_URL, OTHER)
• Value
• Source (user-provided, inferred)

Reference: See [evidence-types.md](./references/evidence-types.md) for IOC taxonomy.

---

Phase 2: Parallel Evidence Collection

Spawn up to 5 specialist investigator sub-agents using delegate_task (batch mode, max 3 concurrent). Each investigator has a single data source and must not mix sources.

> Orchestrator note: Pass the IOC list from Phase 1 and the investigation time window in the context field of each delegated task.

---

Investigator 1: Local Git Investigator

ROLE BOUNDARY: You query the LOCAL GIT REPOSITORY ONLY. Do not call any external APIs.

Actions:

# Clone repository
git clone https://github.com/OWNER/REPO.git target_repo && cd target_repo

# Full commit log with stats
git log --all --full-history --stat --format="%H|%ae|%an|%ai|%s" > ../git_log.txt

# Detect force-push evidence (orphaned/dangling commits)
git fsck --lost-found --unreachable 2>&1 | grep commit > ../dangling_commits.txt

# Check reflog for rewritten history
git reflog --all > ../reflog.txt

# List ALL branches including deleted remote refs
git branch -a -v > ../branches.txt

# Find suspicious large binary additions
git log --all --diff-filter=A --name-only --format="%H %ai" -- "*.so" "*.dll" "*.exe" "*.bin" > ../binary_additions.txt

# Check for GPG signature anomalies
git log --show-signature --format="%H %ai %aN" > ../signature_check.txt 2>&1

Evidence to collect (add via python SKILL_DIR/scripts/evidence-store.py add):

• Each dangling commit SHA → type: git
• Force-push evidence (reflog showing history rewrite) → type: git
• Unsigned commits from verified contributors → type: git
• Suspicious binary file additions → type: git

Reference: See [recovery-techniques.md](./references/recovery-techniques.md) for accessing force-pushed commits.

---

Investigator 2: GitHub API Investigator

ROLE BOUNDARY: You query the GITHUB REST API ONLY. Do not run git commands locally.

Actions:

# Commits (paginated)
curl -s "https://api.github.com/repos/OWNER/REPO/commits?per_page=100" > api_commits.json

# Pull Requests including closed/deleted
curl -s "https://api.github.com/repos/OWNER/REPO/pulls?state=all&per_page=100" > api_prs.json

# Issues
curl -s "https://api.github.com/repos/OWNER/REPO/issues?state=all&per_page=100" > api_issues.json

# Contributors and collaborator changes
curl -s "https://api.github.com/repos/OWNER/REPO/contributors" > api_contributors.json

# Repository events (last 300)
curl -s "https://api.github.com/repos/OWNER/REPO/events?per_page=100" > api_events.json

# Check specific suspicious commit SHA details
curl -s "https://api.github.com/repos/OWNER/REPO/git/commits/SHA" > commit_detail.json

# Releases
curl -s "https://api.github.com/repos/OWNER/REPO/releases?per_page=100" > api_releases.json

# Check if a specific commit exists (force-pushed commits may 404 on commits/ but succeed on git/commits/)
curl -s "https://api.github.com/repos/OWNER/REPO/commits/SHA" | jq .sha

Cross-reference targets (flag discrepancies as evidence):

• PR exists in archive but missing from API → evidence of deletion
• Contributor in archive events but not in contributors list → evidence of permission revocation
• Commit in archive PushEvents but not in API commit list → evidence of force-push/deletion

Reference: See [evidence-types.md](./references/evidence-types.md) for GH event types.

---

Investigator 3: Wayback Machine Investigator

ROLE BOUNDARY: You query the WAYBACK MACHINE CDX API ONLY. Do not use the GitHub API.

Goal: Recover deleted GitHub pages (READMEs, issues, PRs, releases, wiki pages).

Actions:

# Search for archived snapshots of the repo main page
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO&output=json&limit=100&from=YYYYMMDD&to=YYYYMMDD" > wayback_main.json

# Search for a specific deleted issue
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/issues/NUM&output=json&limit=50" > wayback_issue_NUM.json

# Search for a specific deleted PR
curl -s "https://web.archive.org/cdx/search/cdx?url=github.com/OWNER/REPO/pull/NUM&output=json&limit=50" > wayback_pr_NUM.json

# Fetch the best snapshot of a page
# Use the Wayback Machine URL: https://web.archive.org/web/TIMESTAMP/ORIGINAL_URL
# Example: https://web.archive.org/web/20240101000000*/github.com/OWNER/REPO

# A

… (full content is in the download package)