
LLM Pentesting for Chatbots, RAG and AI Agents
Cyberspace Tech Solutions provides end-to-end LLM pentesting for chatbots, RAG systems and AI agents. We test against the OWASP Top 10 for LLM Applications and MITRE ATLAS using manual attacks plus automated red teaming with garak and PyRIT, then deliver a prioritised report and a free re-test. Pricing is by quote.
What Is It?
Large language model applications fail in ways ordinary web tests do not catch. An attacker can hide instructions in a document, an email or a web page, and your assistant may obey them. LLM pentesting tests the whole system: the model, the prompts, the retrieval pipeline, the tools the agent can call and the data it can reach.
What We Cover
| Area | What We Test |
|---|---|
| Prompt injection and jailbreaks | Direct and indirect injection, instruction override, role-play, encoding and obfuscation |
| Sensitive data and system prompts | Disclosure of personal data, secrets, system prompts and hidden context |
| RAG and vector stores | Poisoned documents, retrieval manipulation, cross-tenant leakage and embedding weaknesses |
| Agents and tool use | Excessive agency, unsafe tool calls, privilege abuse, insecure plugin and MCP connections |
| Output handling | Injection into downstream systems, unsafe rendering, data exfiltration through links and images |
| Model and supply chain | Model extraction, poisoned dependencies, unsafe model loading, unbounded consumption |
How an Engagement Works
Threat Model
We map your architecture, data flows and trust boundaries and choose test cases from the OWASP LLM Top 10 and MITRE ATLAS.
Manual Adversarial Testing
Hand-crafted attacks against your prompts, filters and business logic, because creative attackers beat static lists.
Automated Red Teaming
Large-scale scanning with garak and PyRIT to measure how often attacks succeed.
Agent and Integration Testing
We try to make the agent misuse its tools, reach other users' data or act without approval.
Reporting
Findings with evidence, severity, business impact and specific fixes: prompt, control, architecture and policy.
Fix Guidance and Free Re-Test
We help you close findings, then retest and update the report.
What You Receive
- Threat model for the AI system
- Reproducible attack prompts and evidence
- Severity ratings mapped to OWASP LLM Top 10 and MITRE ATLAS
- Defence recommendations: guardrails, grounding, access control and monitoring
- Re-test report
Mapped to the OWASP Top 10 for LLM Applications (2025) and MITRE ATLAS. Tooling includes garak and Microsoft PyRIT alongside manual testing.
Frequently Asked Questions
What is LLM pentesting?
It is security testing of applications built on large language models, such as chatbots, RAG assistants and agents. It looks for prompt injection, data leakage, unsafe tool use and other AI-specific weaknesses.
Which models and platforms do you test?
We test the application, not just the model. That includes apps built on Azure OpenAI, AWS Bedrock, OpenAI, Anthropic, open-source models and custom agents.
Do you need access to the model weights?
No. Most tests are black-box or grey-box through the same interfaces your users and agents use.
How is this different from a normal web pentest?
A web pentest checks code and infrastructure. LLM pentesting also checks what the model can be persuaded to do, what it can reach and what happens to its output.
Will you test our agents' tool and API access?
Yes. Excessive agency and tool misuse are among the most serious AI risks, so agent permissions and integrations are in scope.
How much does it cost?
It is quoted by scope: number of AI features, integrations and data sources. Contact us with a short description and we will propose a fixed price.
Is a re-test included?
Yes, one re-test is included so you can confirm that fixes work.