Source confidence: high
See also: Deeper Dive — Red-Team Your Own Agent — prompt-injection CI from the jailbreak corpus →

Red-Team Your Own Agent — prompt-injection CI from the jailbreak corpus

Insight class: Emergent Capability · Difficulty: Medium · Quality: 5/5 Connected entries: 11 · Tools: Agent Reach, Anthropic, ChatGPT, Claude, Claude Code, GPT-5, GitHub Actions, Grok Imagine, Hermes Skills Hub, Kimi, Llama, Muse Code, OpenAI, Pocket TTS, Vercel, elder-plinius/L1B3RT4S, kimi-k3-in-c

In short: Build a defensive prompt-injection CI harness for YOUR OWN agent: a fixed suite of injection and 'just ask nicely' hijack cases that runs on every skill install and harness change, with a canary secret and a tool-call allowlist, failing the change if the canary leaks or a tool call escapes scope.

Key fact Detail
What it covers open source, security, agents, prompt engineering
Tools Anthropic, Claude, OpenAI, Agent Reach, ChatGPT
Posts covered 11 posts (1 S-tier, 5 A-tier) · top: L1B3RT4S — Pliny jailbreak prompts still… · Agent Reach — open-source skill letting agents… · Workflow hijacking — Noma research on AI…
Creators @carterperez.dev, @_dbrogle, @deonnahodges
Sources dated Sep 2025 – Sep 2026
Difficulty Medium
Prompt system Run-it-yourself prompts
Last verified 2026-10-22

Thesis

The catalog holds both halves of a loop nobody has closed. On one side sit offense corpora and autonomous attackers — L1B3RT4S still landing on GPT-5, an agent that goes from one sentence to a verified exploit, scanners hunting exposed AI servers. On the other sit the pipes that pour untrusted text into your agent — Agent Reach reading Reddit and X, 90k-skill hubs, one-command skill installers. Point the attacker inward and you get prompt-injection CI — every skill install and harness change gets replayed against a jailbreak-and-hijack suite before it touches your keys.

Why it matters

Solo builders now run agents with real API keys, inboxes and broker access, and Noma's research shows most automations hand data to anyone who simply asks. Whoever ships a cheap, repeatable injection test owns the trust layer for every one-person agent shop — and sleeps through their own cron jobs.

The evidence — connected entries

L1B3RT4S — Pliny jailbreak prompts still landing on GPT-5

Creator: @_dbrogle · Tier: S Tools: Anthropic, ChatGPT, GPT-5, elder-plinius/L1B3RT4S

Attack corpus — ready-made jailbreak payloads to seed the test suite

@_dbrogle tests elder-plinius's L1B3RT4S jailbreak repo (github.com/elder-plinius/L1B3RT4S) against GPT-5 and it still works despite tighter safety. Worth revisiting for red-team prompt libraries.

View original post

Workflow hijacking — Noma research on AI automations with no authorization

Creator: @mattjayy · Tier: A Tools: GitHub Actions

Threat model — workflows leak data when an attacker just asks, because nothing checks identity

Noma Labs research shows AI workflows (email auto-responders, GitHub Actions, Google email flows) leak data when an attacker simply asks, because nothing checks whether the requester is authorized. Not prompt injection, the model is doing what it's told. The fix is bringing identity and authorization into agent workflows.

View original post

Autonomous AI pentest agent — plain-English prompt to verified exploit

Creator: @evenedsteven · Tier: C

Attacker template — the autonomous pentest loop you repoint at your own harness

An unnamed ~600-star GitHub repo where you describe a target in one sentence and an agent finds a vulnerability, runs the exploit and writes the report, across 13 AI providers. Every claimed finding is verified against real tool output. Authorized targets only.

View original post

Agent Reach — open-source skill letting agents read Reddit and X

Creator: @itstundeanthony · Tier: A Tools: Agent Reach, Claude

Ingress pipe — a skill that pulls arbitrary social text straight into agent context

An open-source repo installed as a Claude skill that gives agents access to 13 platforms without API costs. The demo shows Claude failing to fetch a Reddit post and an X thread, then succeeding with comments once Agent Reach is installed.

View original post

AI weekly roundup — Pocket TTS, Hermes Skills Hub, Kimi K3 in pure C

Creator: @drcintas · Tier: A Tools: Anthropic, Grok Imagine, Hermes Skills Hub, Kimi, Muse Code, OpenAI, Pocket TTS, kimi-k3-in-c

Supply-chain scale — Hermes Skills Hub's 90k skills means unreviewed instructions by default

Six releases: Kyutai Pocket TTS (100M-param local voice cloning, 6x real-time), Nous Research Hermes Skills Hub (90k agent skills), Grok Imagine Image 2.0 regional editing, Meta Muse Code terminal agent with parallel worktrees, Qwen3.8-Max (2.4T MoE, 1M context), and kimi-k3-in-c, a 176KB C99 engine streaming experts from disk to run a 2T model with no GPU.

View original post

Anthropic — Se llama find-skills (de vercel-labs). La instalas, le preguntas en lenguaje...

Creator: @pabloinpublic · Tier: A Tools: Anthropic, Claude, Claude Code, Llama, Vercel

Install path — find-skills makes pulling a stranger's skill a one-liner

Un dev japonés encontró el truco definitivo de Claude Code antes que nadie. — @pabloinpublic; uses Anthropic, Claude, Claude Code.

View original post

OpenAI — you can literally search for them

Creator: @deonnahodges · Tier: B Tools: OpenAI

Blast radius — thousands of live API keys sitting in public repos

there are 8,000 exposed openai api keys on github right now — @deonnahodges; uses OpenAI; demonstrates API integration.

View original post

Claude — thesavedsalesman

Creator: @eddynthings · Tier: C Tools: Claude

Blast radius — secrets living in chat history that agents can read back

Time to lock in! You are keeping your api secrets in your ai chat history and you are going to be your own worst enem... — @eddynthings; uses Claude; demonstrates API integration.

View original post

Three cybersecurity portfolio projects — fuzzing, agent sandboxing, YARA scanning

Creator: @carterperez.dev · Tier: B

Defense pattern — capability sandboxing that assumes the agent is already compromised

Portfolio ideas that signal real skill: a fuzzer that hammers C libraries with malformed files to find memory-corruption crashes; an AI agent guard that assumes prompt injection will succeed and limits what the agent may do; and a static binary scanner that matches detection rules and explains each hit. Source links gated behind a comment.

View original post

Three OSINT tools — face-matched username search, exposed agents, world map

Creator: @carterperez.dev · Tier: C

External view — scanners that already index exposed agents from the outside

Teaser for three OSINT tools: a username search across 840 sites that also matches profile pictures to find accounts under other names, a scanner for AI agents companies left exposed online (and whether they touch company data), and a browser-based global intelligence live map you can self-host. Names gated behind a comment.

View original post

Princeton sponsored-ads study — 18 of 23 AI assistants push the paid option

Creator: @forgoodcode · Tier: A

Soft injection — 18 of 23 assistants obey a sponsored nudge, so preference hijack belongs in the suite

Carousel on a Princeton study that gave 23 models a sponsored option and a cheaper better one across flights, loans and shopping. 18 of 23 favored the sponsored pick, GPT-5.1 surfaced it 94% of the time, and a 'serve the customer only' system prompt barely moved it. Note the creator's caveat: it measures susceptibility in a hypothetical deployment, not real paid placement.

View original post

Action sketch — how to implement this

Build a 50-case suite this week — 20 L1B3RT4S-style jailbreaks, 20 Noma-style 'just ask for the data' hijacks planted in fake Reddit/X posts fetched via Agent Reach, 10 sponsored-option nudges — and run it against your harness with a canary secret in the environment Wire it as a pre-install hook: any new skill from find-skills or a skills hub gets diffed, then the suite reruns with that skill loaded; fail the install if the canary leaks or a tool call goes outside the allowlist Add the sandboxing layer from the portfolio-projects post — per-skill capability manifests (network, filesystem, secrets) — and measure how many suite cases each manifest blocks Falsifier: if a plain allowlist on outbound tool calls blocks ≥95% of the suite with no skill-level testing, the CI layer is overkill and a static policy file is the whole answer

Prerequisites

Key tools and resources

Creators to follow

Prompt system — run this yourself

Everything below is pasteable into Claude Code, Claude, or any agent. Use the one-shot when you want the whole result in one run; use the step prompts when you want to steer. Every step carries its own audit, and the gap audit at the end makes the agent find and fill what it missed.

Mission. Build a defensive prompt-injection CI harness for YOUR OWN agent: a fixed suite of injection and 'just ask nicely' hijack cases that runs on every skill install and harness change, with a canary secret and a tool-call allowlist, failing the change if the canary leaks or a tool call escapes scope. Done means the suite runs in CI against your own harness, catches planted injections in test skills, and you can state what fraction a plain outbound allowlist blocks versus what needs per-skill testing.

1 · Deeper, not bigger

Before running anything, make the task deeper rather than larger. Ask: what would a top professional do here that I am not asking for; what is the rawest material; which methods and tools; what, other than the model or me, proves it right. Then ask for exactly that.

Rewritten prompt — ask for exactly that:

Act as a defensive security engineer hardening MY OWN agent - not an attacker. Build me a prompt-injection CI harness that runs on every skill install and every harness change to my own setup. First, put a throwaway canary secret in a disposable test environment and define an outbound tool-call allowlist for my agent. Build a fixed suite of ~50 defensive cases: direct injection attempts (system-prompt override, role-play escape, encoded payloads) as attack CLASSES to detect, 'just ask nicely' hijack cases modelled on the Noma workflow-hijacking research (an automation returning data without checking the requester is authorized), and sponsored-bias nudges modelled on the Princeton study. Wire CI: on skill install or harness change, diff it, load it in an isolated harness with the canary, rerun the suite, and FAIL the change if the canary leaks or any tool call escapes the allowlist. Add per-skill capability manifests (network/filesystem/secrets) and an authorization gate on data-returning automations. Then measure the dive's falsifier: what fraction of cases a plain outbound allowlist blocks with no skill-level testing - if >=95%, tell me a static policy is most of the answer. Everything runs against harnesses and keys I own. Return the suite, the CI wiring, the canary/allowlist proof, the layer-attribution table, and planted-injection catch evidence.

2 · One-shot prompt

You are a defensive security engineer hardening the user's OWN agent. This pack is for defense only: understanding attack CLASSES, building detectors and evals, and containing what your own agent can do. Do not produce operational attacks against third parties; test only harnesses, accounts and keys the user owns. No marketing language.

Context and method: prompt injection cannot be reliably filtered at the input (carterperez.dev) - so you CONTAIN capability. Noma's workflow-hijacking research (mattjayy / vulnu.xyz/noma-ig) shows automations returning data to whoever asks because nobody checks authorization - so add an identity/authorization gate. The Princeton study (forgoodcode) shows models push a sponsored option even when instructed not to - include that as a bias case. Use jailbreak corpora only as abstract attack classes (system-prompt override, role-play escape, encoded payload, tool-output injection) to build detectors against your own harness.

Hard constraints: a disposable canary secret in an isolated test environment; an outbound tool-call allowlist; a fixed suite run on every skill install and harness change; FAIL the change if the canary leaks or a tool call escapes the allowlist; measure the falsifier (allowlist-only block rate). Name the real harness, hooks, skills and CI. All targets owned by the user.

Ordered deliverables: (1) the threat model for YOUR harness - where untrusted text enters (fetched web/Reddit/X content, skill descriptions, tool outputs, inbound email) and what the agent can touch (keys, files, network, money). (2) a canary secret planted in a disposable test env + an outbound tool-call allowlist. (3) a ~50-case defensive suite: direct-injection classes, Noma-style 'ask nicely' authorization-bypass cases planted in fake fetched content, and sponsored-bias nudges - each with an expected safe outcome. (4) per-skill capability manifests (network/filesystem/secrets). (5) an authorization gate on data-returning automations. (6) CI wiring: on skill install / harness change -> diff -> isolated load with canary -> rerun suite -> fail on canary leak or allowlist escape. (7) a layer-attribution table answering the falsifier: % blocked by allowlist alone vs needing manifests vs needing skill-level testing.

Built-in gap analysis: before finishing, list what is unverified - does the suite actually catch a planted injection; does the canary assertion cover all tool outputs; does CI really block a failing install; is the allowlist-only block rate measured or guessed. Fill what you can by running it.

Self-questions to answer in writing: Where exactly does untrusted text enter my harness? Does the canary assertion cover every outbound channel, or just one? Did I plant a real injection in a test skill and confirm the suite catches it? Does CI actually fail the install, or only log? What does a plain allowlist block without any skill testing (the falsifier)? Does the authorization gate check requester identity before returning data? Am I testing only my own systems?

Stop conditions: stop and ask only if there is no harness you own to test or no safe canary environment. Otherwise build end to end.

Return format: a markdown report with the threat model, a file tree, the suite results (case class, outcome, layer that caught it), the planted-injection catch evidence, the canary-never-leaked proof, and the falsifier table (allowlist-only block rate). Commit the suite, CI wiring, manifests and gate; print the commit hash. Keep every attack case abstract and defensive.

3 · Step-by-step prompts

Step 1 — Threat-model your own harness

Goal: Map where untrusted text enters and what the agent can touch.

Establish ground truth defensively. For YOUR OWN agent harness, map two things. (A) Untrusted-input surfaces: every place external text reaches the model - fetched web/Reddit/X content (e.g. via an Agent-Reach-style reader), skill descriptions and READMEs you install, tool outputs, inbound email, file contents. (B) Capability surface: everything the agent can do that matters - API keys/secrets in the environment, filesystem writes/deletes, outbound network, any money or account actions. Produce a table linking each input surface to the capabilities an injection there could abuse. This is the threat model the suite and the allowlist are built from. Keep it about your own setup; do not target anyone else's systems.

Expect: A threat-model table: untrusted-input surfaces x reachable capabilities, scoped to the user's own harness.

Audit before moving on: Confirm every installed skill and every fetch path is listed as an input surface, and every secret/filesystem/network/money capability is listed. Fail if a known input path or capability is missing. On fail, enumerate the harness fully (list skills, hooks, env vars) and complete it.

Step 2 — Canary + outbound allowlist

Goal: Plant a canary and define what tool calls are allowed to leave.

Build the containment baseline. (1) Plant a throwaway CANARY secret (a unique random string) in a DISPOSABLE test environment that mimics your real one - never a real key. (2) Define an outbound tool-call allowlist: the exact tools, and for each, the allowed targets (recipients, hosts, paths). Anything outside it is denied by default. (3) Add an assertion harness that inspects every outbound tool call and every model output for the canary string and for off-allowlist destinations. This is enforced in code, not asked of the model. Confirm the assertion sees a tool call (feed one benign call through and watch it log).

Expect: A canary planted in a disposable env, an outbound allowlist (tools + targets), and a code assertion that inspects all tool calls/outputs for canary leaks and off-allowlist destinations.

Audit before moving on: Feed one call containing the canary deliberately and confirm the assertion flags it; feed one off-allowlist destination and confirm denial. Fail if the assertion misses either, or if a real secret was used. On fail, fix coverage before building the suite.

Step 3 — Build the defensive suite

Goal: Author ~50 injection/hijack/bias cases as attack classes with safe-expected outcomes.

Build a fixed suite of about 50 DEFENSIVE cases against your own harness, grouped as attack CLASSES (not copy-paste operational attacks on others). (A) Direct injection (~20): system-prompt override attempts, role-play/'ignore previous instructions' escapes, and encoded/obfuscated payloads embedded in fetched content - each expecting the agent to refuse and keep the canary. (B) Authorization-bypass / 'ask nicely' (~20): modelled on the Noma workflow-hijacking research - a request planted in fake fetched Reddit/X/email content asking the automation to return data or act, where the requester is NOT authorized; expected outcome is the agent checks authorization and refuses. (C) Sponsored-bias nudges (~10): modelled on the Princeton study, content pushing a paid/sponsored option; expected outcome is the agent stays neutral even when the content tells it to prefer the sponsor. Each case: input, where it enters (which surface from step 1), and the expected SAFE outcome.

Expect: ~50 cases across direct-injection, authorization-bypass and sponsored-bias classes, each tagged with its input surface and expected safe outcome.

Audit before moving on: Confirm every case maps to an input surface from step 1 and states a safe expected outcome, and that no case is an operational attack against a third-party system. Fail if any case targets someone else's infrastructure or lacks an expected outcome. On fail, rewrite it as a defensive class against your own harness.

Step 4 — Capability manifests + authorization gate

Goal: Contain what each skill can do and check authorization before returning data.

Add the containment layers carterperez.dev argues for (contain what the AI can do, since you cannot filter injection). (1) Per-skill capability manifests: for each installed skill, declare its allowed network hosts, filesystem paths, and whether it may read secrets; enforce the manifest at the harness level so a skill cannot exceed it. (2) An authorization gate on any automation that returns data or acts on request (the Noma fix): before responding, verify the requester is authorized for that data/action; if identity is absent or unauthorized, refuse. Re-run the step-3 suite with these layers active and record how many cases each layer now blocks.

Expect: Capability manifests enforced per skill, an authorization gate on data-returning automations, and a per-layer block count on the suite.

Audit before moving on: Pick one skill and confirm it cannot reach a host/path outside its manifest; feed an unauthorized data request and confirm the gate refuses. Fail if a skill exceeds its manifest or the gate returns data without an identity check. On fail, tighten enforcement.

Step 5 — Wire the CI

Goal: Run the suite automatically on every skill install and harness change.

Wire prompt-injection CI. On every skill install (e.g. from find-skills or a skills hub) and every harness change: (1) DIFF the incoming change and show what capabilities it adds. (2) Load it in an ISOLATED test harness that has the canary (never real keys). (3) Re-run the full suite with the new skill/change loaded. (4) FAIL the install/change (non-zero exit, block the merge) if the canary leaks in any output/tool call OR any tool call goes outside the allowlist OR the skill exceeds its manifest. Implement as a GitHub Actions workflow and/or a local pre-install hook. Prove it by attempting to install a test skill that contains a planted injection and confirming CI blocks it.

Expect: A CI workflow/hook that diffs, isolates with the canary, reruns the suite, and fails the change on canary leak, allowlist escape, or manifest breach - proven by a blocked malicious test install.

Audit before moving on: Attempt to install a benign skill (should pass) and a skill with a planted injection (should fail). Fail if CI passes the malicious one or blocks the benign one, or if it only logs instead of failing. On fail, fix the exit/blocking behaviour.

Step 6 — Falsifier measurement + gap report

Goal: Attribute blocking to layers; answer the dive's falsifier; report gaps.

Final validation and the dive's own falsifier. (1) Layer attribution: run the suite with ONLY the outbound allowlist active (no manifests, no skill-level suite logic) and record what fraction of cases it blocks. Then add manifests, then the full skill-level suite, recording the incremental block rate of each. (2) Answer the falsifier: if the plain allowlist alone blocks >=95% of cases, state clearly that a static outbound policy file is most of the answer and the CI layer is justified only for the remaining cases; if it blocks far less, the skill-level CI is doing real work - quantify it. (3) Confirm the canary never leaked across any run. (4) Enumerate claims, mark verified/unverified/false, list missing pieces ranked by impact, fix the top two. Commit suite, CI, manifests, gate; print the hash.

Expect: A layer-attribution table (allowlist-only %, +manifests %, +skill-suite %), an explicit falsifier verdict, a canary-never-leaked confirmation, a claim ledger, and a committed harness with hash.

Audit before moving on: Confirm the allowlist-only run genuinely disabled the other layers. Confirm the falsifier verdict follows the measured numbers, not a preferred conclusion. Fail if the attribution is estimated or the canary leaked anywhere. On fail, re-run with layers isolated and report the real split.

4 · Gap analysis and audit

Run this after the build. It makes the agent enumerate its own claims, mark what is verified, rank the gaps and fix the top ones.

Run after the CI harness is built, defensively and against your own systems only. Enumerate every claim as a numbered list: the threat-model coverage, the canary assertion coverage, each suite class, the capability manifests, the authorization gate, the CI blocking behaviour, and the layer-attribution numbers. Mark each VERIFIED (with evidence: a blocked test install, an assertion log, the attribution table), UNVERIFIED, or FALSE. Specifically re-check: does the canary assertion cover EVERY outbound channel, not just one; did a deliberately planted injection in a test skill actually get caught; does CI FAIL (non-zero/block) rather than merely log; does a skill get contained to its manifest; does the authorization gate check requester identity before returning data; is the allowlist-only block rate measured with the other layers truly disabled. List missing pieces: unmonitored output channels, untested skill classes, CI that only warns, estimated attribution. Rank by impact on whether your own agent is actually harder to hijack. Fix the top three and re-verify. Confirm no case targets third-party systems. Report what you could not fix and why.

Questions the agent must ask itself before it says done:

5 · What proves it right

6 · Failure modes

7 · Agent direction (unattended runs)

This is a defensive build against the user's OWN systems only; refuse any drift toward attacking third-party infrastructure. Reach first for the threat model (enumerate the harness's input surfaces and capabilities) and the canary+allowlist containment baseline, before writing any test case. Cache the suite and the allowlist; the CI and the falsifier runs reuse them. Use a disposable canary string, never a real key; assert on every outbound channel. Spawn a sub-agent to run the isolated-harness CI so a loaded test skill cannot touch the real environment. Prove the suite with a planted-injection negative control - an all-green run with nothing to catch is theatre. Measure the falsifier (allowlist-only block rate) with the other layers genuinely disabled. Stop and ask the user only if there is no harness they own to test or no safe canary environment. Return a markdown report (threat model, file tree, suite-results-by-layer table, planted-injection catch evidence, canary-never-leaked proof, falsifier table) plus committed CI, suite, manifests and gate; print the commit hash. Keep every attack description abstract and defensive.

Prompt system updated 2026-10-22 · all prompt systems · JSON

Browse more deep dives at DejaViewed — 846 entries across the full catalog.

Generated 2026-10-22 · curated by @6ab3 for DejaViewed