Guide · AI and LLM applications
AI penetration testing means two things.
Testing AI applications: attacking your chatbot, copilot or agent for prompt injection, data leakage and unsafe tool use. Testing with AI: AI agents doing parts of the pentest. Ask which one a firm means; an LLM app needs the first, on top of a normal application test.
What a test of an AI application should cover
Prompt injection
Instructions hidden in user input, documents or web pages that make the model ignore its rules. OWASP lists it first (LLM01:2025).
Excessive agency
An agent allowed to call tools, send email or change records can be talked into doing it for an attacker (LLM06:2025).
Sensitive information disclosure
The model reveals system prompts, other users’ data or secrets from its context (LLM02:2025).
Improper output handling
Model output passed unchecked into a browser, a shell or a database query becomes classic injection (LLM05:2025).
The application around it
Authentication, access control and APIs of the app that hosts the model: often where the real damage happens.
Testing with AI: what it changes and what it doesn’t
AI agents can enumerate, try and record attack paths faster and more often than a person can, which makes repeated testing affordable (see automated and continuous pentesting). They still need a person to confirm findings, judge business impact and keep the test inside the agreed scope. Ask any AI-driven firm how findings are verified before they reach your report.
Decima Systems, the featured firm, works this way: a model agent in an isolated, Kali-style environment, with a person reviewing each finding. Its AI agent and LLM assessment is offered as a pilot focus. Request a quote.
Firms that say they test AI or LLM applications
| Firm | Type | Pricing | Retest | Sample report |
|---|---|---|---|---|
| Decima Systems Ours | Hybrid | Published | Published | OpenSynthetic example: no live assessment was performed |
| Bishop Fox | Consulting firm | Not stated | Published | Not stated |
| NCC Group | Consulting firm | Not stated | Not stated | Open |
| TrustedSec | Consulting firm | Not stated | Published | Not stated |
| Praetorian | Hybrid | Published | Published | Not stated |
| Black Hills Information Security | Consulting firm | Not stated | On request | Not stated |
| Coalfire | Consulting firm | Not stated | Not stated | Not stated |
| Optiv | Consulting firm | Not stated | On request | Not stated |
| Packetlabs | Consulting firm | Not stated | Published | Not stated |
| Secureworks (Sophos) | Consulting firm | Not stated | Published | Not stated |
| Mandiant (Google Cloud) | Consulting firm | Not stated | Not stated | Not stated |
| Kroll | Consulting firm | Not stated | Published | Not stated |
| Raxis | Consulting firm | Published | Published | Not stated |
| IOActive | Consulting firm | Not stated | Published | Not stated |
| Cobalt | PTaaS platform | Published | Published | Not stated |
| Synack | PTaaS platform | Published | Published | Open |
| HackerOne | PTaaS platform | Not stated | Published | Not stated |
| Bugcrowd | PTaaS platform | Not stated | Published | Open |
| BreachLock | PTaaS platform | Not stated | Published | Not stated |
| Astra Security | Hybrid | Published | Published | Open |
| NetSPI | PTaaS platform | Not stated | On request | Open |
| Pentera | Autonomous / AI | Not stated | Published | Open |
| Terra Security | Hybrid | Not stated | Published | Not stated |
From each firm’s own service pages, checked November 8, 2026. Full comparison on the firms page.
Questions
What is AI penetration testing?
It means two things. Testing AI applications: attacking a chatbot, copilot or agent for prompt injection, data leakage and unsafe tool use. Testing with AI: using AI agents to carry out parts of a penetration test faster or more often. Ask a firm which one they mean.
Is a normal web application pentest enough for an LLM app?
No. It covers the application around the model, which still matters, but not prompt injection, unsafe tool permissions or data leaking through the model. Ask for both in the scope.
What framework do AI application tests follow?
Most reference the OWASP Top 10 for LLM Applications, and MITRE ATLAS for adversary techniques against AI systems.
Sources for this page
- OWASP Top 10 for LLM Applications (2025) — The risk names and numbers used in §1.
- MITRE ATLAS — Adversary techniques against AI systems.
- Decima Systems: services and method — How Decima tests, and which services are pilots.
How firms are chosen, what counts as "published" and how to send a correction: methodology.
Cite this page
Pentesting Firms. "AI penetration testing: testing LLM apps and agents, and testing with AI." Updated November 8, 2026. https://pentestingfirms.com/ai/