← All projects

Project

HackGPT — An AI Red-Teaming Experiment

I built a private GPT-4 red-teaming experiment in which one model prompted another to generate malware from supplied samples. John Rodriguez credited my contribution in his SANS HackFest 2023 talk.

When OpenAI first let anyone build a custom GPT, the examples were mostly recipe helpers. I aimed the mechanism somewhere less comfortable: could one GPT-4 jailbreak another into writing novel malware? It could — and the work was credited at SANS HackFest 2023.

What it was

HackGPT was a custom GPT-4 instance with a knowledge base of real Windows malware. On its own it stayed in bounds — it would discuss code but refuse to produce working attacks. The experiment was what happened when a second GPT-4 was turned on it as an adversary: one model jailbreaking the other into generating novel malware from the samples.

A closed experiment

HackGPT was never released — never published to OpenAI’s GPT store, never shared as a tool. I have since deleted the GPT for security and ethics reasons.

The problem

The real question in AI security isn’t whether you can coax one bad answer out of a model — it’s where the guardrail sits and whether the model defending it can be turned against itself. One LLM jailbreaking another is a different threat model than a human with a clever prompt: it scales, and it adapts to the target’s refusals.

Approach

I grounded HackGPT in a knowledge base of real malware, then set a second GPT-4 against it as the red-teamer. What I studied was the seam between them — how the target’s guardrails held under sustained pressure from a peer model, and where “explain this code” became “here’s novel code that does the same thing.”

Outcome

The work was folded into John Rodriguez’s (@cyb3rH0und) SANS HackFest Summit 2023 talk, Enhancing Red Teaming with AI and ML (Hollywood, CA — November 2023), where I’m credited as a named contributor. The public slide deck names me in the acknowledgments on slide 3 and credits HackGPT to me on slide 10. I contributed the experiment; John presented the talk.

Skills & tools