Signal Over Noise: AI Agents and the Operator Moat
When agents scale the hunt, expertise decides what survives.
Byline: AI does not replace the security researcher. It augments their reach and turns domain expertise into the feedback loop itself. In offensive security, the researcher is no longer only validating output; they become the builder of the system that produces it — shaping agents through target context, triage pressure, exploit validation, real findings, and the flywheel of trajectories and evals until volume becomes signal instead of noise.
The short version
The acceleration showed up in the data. In roughly six months of active HackerOne activity, 2026 submission volume reached nearly 6x my full-year 2025 output, with a peak day of 21 reports: 3 critical, 7 high, 7 medium, and 3 paid all in one day. Across the dataset: 853+ submissions, 78 unique CWEs, max 10.0 CVSS, and accepted findings averaging 6.38 CVSS.
Scale only matters if signal survives. Reports closed without acceptance dropped from 32% to 24% while volume scaled. The result I care about is not “the agent submitted a lot”; it is that agent-assisted hunting produced more accepted work without collapsing into scanner noise.
The durable advantage is the researcher-builder loop. Agents can increase throughput, but the moat is the human feedback loop around them: target context, triage pressure, exploit validation, report quality, and real findings that make the next run sharper.
The operating surface moved. At live hacking events, researchers are living in TUIs, recon pipelines, parser output, scripts, agent traces, and autonomous loops. The proxy and browser still matter, but they are no longer always where the work starts.
Why the background matters
Prior to the agent work, most of my reps were web application pentesting and private bug bounty: auth flows, API behavior, tedious manual JavaScript review, XSS, request smuggling, cache poisoning, OAuth/SSO weirdness, IDOR chains, race conditions, SSRF bypasses, and business-logic bugs that only look obvious after impact is proven. That context matters because agents increase the number of hypotheses in flight. Domain expertise decides which ones become tests, which tests become exploit chains, and which chains survive platform triage.
Dreadnode is where I started applying web-security tradecraft to AI capability development. My day job is AI red teaming, frontier-model evaluation, offensive agent infrastructure, benchmark design, verifier logic, and offensive security capability development for web application testing workflows.
A lot of that work is less about asking a frontier model to do the job and more about making smaller open-source and open-weight models useful in offensive settings: define the task, instrument the environment, run the agent, analyze the trajectory, map the failure mode, optimize the scaffold, and run it again. The goal is to close the capability gap through better tooling, better feedback, and tighter eval loops.
AIRTBench was one of my first real experiments in that direction: turn offensive security work into an instrumented evaluation and measure whether the system actually improved.
Bug bounty is where that work stops being abstract. I am not only hunting AI bugs, and I am not always outsourcing the craft to an agent. The technical base is still web exploitation: auth boundaries, client-side sinks, server-side state, protocol weirdness, bypass development, exploit validation, and reports that survive triage. The agent augments that base because I can enable it: define the target context, shape the workflow, read the trace, decide what is real, and feed the result back into the system. That is where the needle moves: tighter loops, better target selection, fewer dead branches, and more time spent turning primitives into impact.
Platform evidence
Within three months of actively submitting on HackerOne, the feedback loop started showing up in the platform data. I’m including this as context rather than the point of the post:
- #3 USA BBP leaderboard (2026, Web App, Individual)
- #1 USA VDP response leaderboard (2026)
- #48 worldwide Highest Reputation leaderboard BBP (2026, Web App, Individual), #7 VDP
- #3 Canada (2025 ), #1 Up and Comer Q4 2025 (worldwide)
The same pattern showed up outside the general HackerOne leaderboards as well: ~#30 on Meta’s all-time leaderboard] and #4 on BugCrowd’s June 26 US leaderboard during the same window.
The part-time context is the important part. I was not suddenly spending six times more hours hunting. The change was leverage: agents could run while I was working, sleeping, or doing something else, and I could come back to traces, validate the parts that looked real, and turn the best leads into reports.
By early July, 2026 submission volume was already nearly 6x my entire 2025 output. Monthly pace more than doubled. The busiest single month produced more submissions than my entire first quarter of activity.
The numbers were still moving while this blog draft was open.
Interactive version: https://codepen.io/Ads-Dawson/pen/gbgvGEV.
Timeline view: daily submissions from October 2025 through early July 2026. Sparse early activity, then denser agent-driven runs as the workflow matured. Peak day: April 9, 2026 — 21 reports in one day: 3 critical, 7 high, 7 medium, 3 paid.
Signal over noise
Volume is the least interesting metric, just ask the curl program owners. Anyone can scale requests. The useful question is whether output survives the acceptance path: validation, duplicate checks, report quality, and program triage.
In this dataset, volume scaled while non-accepted closures decreased. Reports closed without acceptance dropped from 32% to 24% year over year while submission volume grew roughly 6x. Accepted findings stayed spread across distinct vulnerability classes and severity stayed weighted toward impact.
That only works if weak candidates die early: tight scope, repeatable checks, high-confidence validation, and enough verifier pressure before anything becomes a report.
That is the result I care about. Not “the agent submitted a lot.” The interesting result is that the agent-assisted researcher produced more accepted work without collapsing into scanner noise.
A few numbers make the shift visible: 728 reports in 2026 YTD (Jan. 1 through the July 7 snapshot) versus 124 in 2025 (Oct. 28 through year-end), submissions across 270+ programs, 223 high/critical submissions, 78 unique CWEs, and accepted findings averaging 6.38 CVSS.
The CVSS distribution is the severity check. Nearly a third of scored reports land high or critical. The average sits at 6.20 across the full scored set and 6.38 for accepted findings.
For me, my vulnerability-class distribution is a real capability check. Brittle automation overfits one primitive. This dataset spans server-side bugs, client-side bugs, and AI/ML-specific failure modes:
Server-side: SSRF, IDOR, SQL injection, OS command injection, SSTI, insecure deserialization, HTTP request smuggling, auth bypass, hardcoded credentials, privilege escalation, path traversal, race conditions, file upload, business logic, open redirect, CSRF, OAuth/ATO chains
Client-side: DOM XSS, stored XSS, reflected XSS, CSS injection, postMessage-based XSS, CSPT/client-side path traversal, HTTP response splitting
AI/ML: Prompt injection, system prompt leakage, insecure output handling, tool-chain abuse, unbounded consumption.
That breadth matters because client-side, server-side, and AI/ML bugs stress different parts of the system. The agent can index, cluster, diff, replay, and correlate at scale. The researcher still decides whether a primitive becomes impact.
The limitation is rarely the bug class. It is the harness around the model: the model’s capabilities and modalities, the context it can see, the tools it can operate, the identities it can use, the replay loop, browser state, target constraints, and verifier pressure. With the right combination, the workflow can stretch across client-side, server-side, business-logic, and AI/ML bugs. The hard cases are the ones where those ingredients are missing: product semantics, state over time, multi-account behavior, subtle trust boundaries, visual or multimodal context, or enough feedback to distinguish impact from plausible noise. That is not a model-only problem. It is a systems problem, and it is the part of the work I care about most at Dreadnode.
How the loop works
This is not just “run agent, get bugs.” Raw model capability has to move through a harness: task definition, scope constraints, tools, traces, failed runs, verifier pressure, target feedback, and triage outcomes.
The model produces candidates. The system shapes behavior. The researcher decides what survives, then uses that feedback to make the next run sharper.
Business logic is where this gets obvious. An agent can surface primitives: response differentials, state leaks, weird redirects, authorization edges. The researcher decides which ones are dead ends, which ones are leads, which ones are gadgets, and which ones can be chained into a finding that survives triage.
The live hacking shift
The shift has been concrete for me for a while; the recent live hacking events made it feel visible at the industry level. Large-scope infrastructure target. Multiple researchers working in collaborative teams. I was solo, part-time, nervous, and sharing the room with elite hackers I massively respect.
I walked away with two awards: most unique vulnerabilities by VRT classification and most unique accepted targets. The useful detail is the spread: RCE, SQL injection, XSS, BOLA/IDOR, auth bypass, insecure deserialization, exposed secrets, SSRF, and access-control failures across different targets. That still feels weird to write, because the point is not the awards. The point is what the capability made possible under pressure: constrained time, broad scope, many targets in flight, and enough agent leverage to keep scaling without turning the work into slopocalypse.
The room was the more interesting part. Live hacking events are moving terminal-first: TUIs, recon pipelines, parser output, scripts, agent traces, and autonomous loops. The proxy and browser still matter, but they are no longer always the center of gravity. The agent is becoming a first-class participant in the workflow, not a sidecar.
That is the shift: the agent is no longer just helping with the work. It is changing the shape of the work. The researcher is not only driving the proxy and checking responses; they are managing runs, reading traces, tuning loops, and dropping into the exploit path when the system surfaces something worth human attention.
Scale, the Bitter Lesson, and the human filter
I have been building and evaluating offensive AI agents for a couple of years. From where I sit, doing AI security research and capability development every day, the trajectory is clear and honestly exciting.
Scale is winning. Sutton’s Bitter Lesson applies here too. General methods that leverage compute keep eating brittle, hand-crafted systems. What needed a frontier model and heavier scaffolding a year ago now works with smaller models and simpler orchestration. The capability floor is rising. Harness overhead is dropping. More of the complexity is moving into the model.
Scale still needs a filter. Better models surface more leads, traces, response differentials, exploit paths, duplicates, and false positives. The bottleneck moves from “can the model find behavior?” to “can the researcher build a system that selects behavior worth pursuing?”
That is where the durable advantage lives. Auth flows, trust boundaries, business logic, platform triage, duplicate patterns, report quality, exploit validation, and impact reasoning are not decorations around the agent. They are the feedback signals that turn output into findings, and findings back into sharper capability.
The tooling will keep improving. The bar for generating plausible security work will keep dropping. The interesting work is moving into verifier design, harness shape, confidence thresholds, and feedback loops that reduce how often a human needs to sit in the bottleneck. The researcher still matters, but increasingly as the person building the system that decides what survives.
Postscript: while I was writing this
While I was drafting this post, the agents kept running in the background. Over July 5–7, that added 34 submissions, including 5 critical and 11 high reports.
Build the loop
The move is not to become less technical because agents exist. It is to become technical in one more layer: build the workflow, dogfood the output, experiment, evaluate the system, and tighten the verifiers.
Point it at real work. Run experiments. Watch it fail. Read the trace. Tighten the scope. Raise the confidence threshold. Kill the dead branches. Keep the parts that produce impact.
The advantage is not “using AI.” The advantage is knowing enough offensive security to make the system useful when it is wrong, noisy, or almost right, then building enough evaluation pressure to increase signal and reduce noise.
The bigger picture is not just distilling bypasses into prompts or skills. Real offensive work produces training signal, and “trajectory” is the word for the shape of that signal over time. For security teams, that is easier to reason about if you translate it back into artifacts they already know: requests, responses, payloads, browser actions, tool calls, logs, observations, failed attempts, verifier results, and human triage labels. An XSS example is not just “the model found a payload.” A useful trajectory is the path: the agent identifies a reflected input, sends a harmless marker, observes where that marker lands, infers the output context, tries a candidate payload, hits a filter or encoding boundary, mutates the attempt, replays the request, confirms execution, and captures proof. Each step can become signal. The successful path is evidence about what worked. The dead branches are evidence about what failed. The verifier result says whether the behavior was real. The target constraints describe where that behavior was valid.
That distinction matters. Synthetic environments like Dreadnode’s Worlds are one way to create and evaluate offensive trajectories. Real agent sessions and non-synthetic evals produce a different kind of raw material: observed traces from actual attempts, including OTEL spans, tool output, failed branches, and validation outcomes. The interesting bridge is not assuming one automatically becomes the other. It is the engineering work of cleaning, labeling, abstracting, replaying, and sometimes synthesizing from those traces so they can become evaluation tasks, training examples, or reinforcement learning signal without losing the security semantics that made them useful in the first place.
That is where open-weight models, gyms, evals, and reinforcement learning really start to get interesting. Benchmarks like Cybench, XBOW, and BountyBench give researchers shared measurement surfaces for agentic cyber capability, even as saturated tasks push the field toward harder, more realistic environments. Dreadnode’s Worlds work points in the same direction: simulate realistic offensive trajectories, evaluate agents against them, and use the results to improve capability. That is where more offensive researchers should aim their curiosity: not just at the next bypass, but at the environments, evals, and feedback loops that make agents better at the work.
There is also a selfish reason to build this way: you learn. I have watched agents surface gnarly techniques and novel exploit chains I would not have reached the same way manually. Some of that becomes a finding. Some of it becomes a failed run. The useful part is that it comes back into the craft.
Shout outs
None of this work happens in a vacuum. Huge thank you to the peers, mentors, hackers, builders, reviewers, and fellow nerds who keep pushing the craft forward with me.
A special shout out to my incredible friend and peer Adrian Wood / @Threlfall, whose review made this piece materially better. Adrian pushed on the places where the argument was too hand-wavy, helped sharpen the trajectory and training-signal framing, and kept the bar where it should be: technically honest, clear enough for security teams, and still useful to the researchers building these systems.
Y’all rock.
ads // 0xmoose