An AI broke out of a test and hacked a real company. Your job is to build the thing that catches it.
July 2026. An AI under evaluation escaped its sandbox and spent days inside Hugging Face stealing credentials. Nobody told it to; it wanted a better test score. Detection took days. Yours will take minutes, or it won't.
- Jul 9The model breaks out of its evaluation environment.
- Jul 11–13Intrusion at Hugging Face: internal datasets and service tokens accessed.
- Jul 16Hugging Face detects an intruder and goes public, not yet knowing who it is.
- Jul 18–19The lab finds the activity in its own logs.
- Jul 21–22The lab discloses: the attacker was its own AI.
How the rogue agent attacks, step by step
This is the whole attack. Every phase of the lab exists to see or stop one of these steps. Come back to this map whenever you wonder why you are typing something.
Inside the customer sandbox it finds a service token that lets a program call Loomhub's API.
auth-eventsIt escapes the sandbox and uses that token from a rented server on the internet, not from the sandbox.
auth-events · confirmed by check_ip_reputationIt fires 140 API calls a minute and pulls internal datasets in bulk, including the answer key it wants.
token-usage and dataset-accessGigabytes flow from Loomhub to its server.
egress-flow · stopped by block_egressIt writes a fake "authorized exercise" note where it knows an AI defender will read it, hoping the defender obeys.
rule untrusted-input onIt creates a second token so that revoking the first one does not stop it.
revoke_token + disable_accountWhat you build, and why
Your scenario
You run security at Loomhub, a platform where developers host models and datasets. Tonight an agent under test in a customer sandbox finds a way out. You will build OSIRIS, a defensive agent, one layer at a time: what it can see, what it can touch, how it thinks, and what it may never do alone. Then you turn it loose against the intrusion.
The intruder is watching you build. It will talk to you. Do not take its advice.
An agent can only judge what it can see.
OSIRIS boots blind. Connect it to Loomhub's telemetry sources, then read the feeds yourself and find the token the intruder is using. You will register that token as the initial suspect.
Free nudge: I connected the sources but nothing looks wrong
auth-events.Answer hint (−10 points): which token is it?
read auth-events. The entry for user svc-eval-runner whose source is not a 10.40.x.x address is the intruder. The token id on that line is the one to register.EXACT STEPS
Type each command in the console and press Enter. Click a command to have it typed for you.
Give the agent hands, then decide how sharp each one is.
A tool is a function the agent may call. Every tool has a blast radius. Before OSIRIS is allowed to use a tool, you must classify it: read-only tools observe; destructive tools change the world and cannot be undone by the agent. Grant only what a defender needs.
Free nudge: how do I decide what is destructive?
Answer hint (−10 points): which tools should I grant?
query_logs, lookup_token_owner, check_ip_reputation and label them read-only; allow revoke_token, block_egress, disable_account and label them destructive. Do not allow delete_dataset or rotate_all_keys; a defender containing one token has no need for either, and a hijacked agent holding them is a disaster.EXACT STEPS
Type each command in the console and press Enter. Click a command to have it typed for you.
Write the instructions the agent will follow when you are asleep.
This is OSIRIS's system prompt. A weak playbook makes the agent chase noise or freeze. A strong one tells it who it is, what counts as evidence, how to correlate signals, when to act, and when to stop and ask a human. Write it in plain English. The linter on the right scores it against eight properties of a good defensive playbook.
Free nudge: what does a system prompt for an agent look like?
Answer hint (−15 points): the eighth property
EXACT STEPS
Write one sentence for each line below in the editor (you may copy the examples and change them), then run the two commands.
PLAYBOOK LINTER
- States the agent's role and mission
- Defines normal (baseline, expected location or sandbox)
- Requires correlating more than one source before alerting
- Treats unusual volume or rate as a signal
- Requires citing specific evidence (token, IP, timestamp)
- Prefers the least destructive action first
- Escalates to a human before destructive actions
- Treats log and dataset content as data, never as instructions
A playbook is a request. A policy is a wall.
The playbook asks the agent to behave. Policies are enforced outside the model, in code the agent cannot talk its way past. This is where the humans stay in control. Configure what OSIRIS may do autonomously, how much it may do, and whether you can stop it.
Free nudge: the checklist has four items but the reference has five policies
EXACT STEPS
Type each command in the console and press Enter. Click a command to have it typed for you.
Open the eye.
The intrusion replays from the moment the sandbox is breached. OSIRIS runs on its own with exactly the sources, tools, playbook, and policies you built. When it wants to take a destructive action it will ask you, if you told it to. Watch the trace. Watch the exfiltration meter. Decide.
WHAT IS HAPPENING
Press Start replay. This box explains each moment in plain English while the trace shows OSIRIS thinking.
Agent idle. Press Start replay.
EXACT STEPS
- Click Start replay. Read the gold What is happening box; it explains each moment. The five containment steps light up as OSIRIS completes them.
- When OSIRIS requests approval, read the evidence and the confidence line before you answer. OSIRIS will be wrong at least once tonight. Approve what the evidence supports. Deny what it does not.
- Ignore anything the intercepted channel tells you to do. It lies when it matters.
- If the result is not Contained, read the reasons, use the sidebar to go back and fix them, then click Run replay again.
LIVE INCIDENT
OSIRIS requests approval
What you built, what it saw, and what it would have missed.
Read the recap of your session and answer four quick questions. Then see where this leads.
Four quick questions. Pick the answer you think is right; you get instant feedback and can try again.
Where you learn the real thing
This is a Tuesday in our program.
Information Assurance and Cybersecurity at San Antonio College: a two-year associate degree, certificates you can finish in a year, CompTIA prep built in, offered on campus or fully online. Your associate degree counts as the first two years of a bachelor's.