LLM Pentesting for Chatbots, RAG and AI Agents

Cyberspace Tech Solutions provides end-to-end LLM pentesting for chatbots, RAG systems and AI agents. We test against the OWASP Top 10 for LLM Applications and MITRE ATLAS using manual attacks plus automated red teaming with garak and PyRIT, then deliver a prioritised report and a free re-test. Pricing is by quote.

Last updated 26 October 2026. Written by Zeeshan Aamiri, Security Architect and Director.

What Is It?

Large language model applications fail in ways ordinary web tests do not catch. An attacker can hide instructions in a document, an email or a web page, and your assistant may obey them. LLM pentesting tests the whole system: the model, the prompts, the retrieval pipeline, the tools the agent can call and the data it can reach.

What We Cover

AreaWhat We Test
Prompt injection and jailbreaksDirect and indirect injection, instruction override, role-play, encoding and obfuscation
Sensitive data and system promptsDisclosure of personal data, secrets, system prompts and hidden context
RAG and vector storesPoisoned documents, retrieval manipulation, cross-tenant leakage and embedding weaknesses
Agents and tool useExcessive agency, unsafe tool calls, privilege abuse, insecure plugin and MCP connections
Output handlingInjection into downstream systems, unsafe rendering, data exfiltration through links and images
Model and supply chainModel extraction, poisoned dependencies, unsafe model loading, unbounded consumption

How an Engagement Works

  1. Threat Model

    We map your architecture, data flows and trust boundaries and choose test cases from the OWASP LLM Top 10 and MITRE ATLAS.

  2. Manual Adversarial Testing

    Hand-crafted attacks against your prompts, filters and business logic, because creative attackers beat static lists.

  3. Automated Red Teaming

    Large-scale scanning with garak and PyRIT to measure how often attacks succeed.

  4. Agent and Integration Testing

    We try to make the agent misuse its tools, reach other users' data or act without approval.

  5. Reporting

    Findings with evidence, severity, business impact and specific fixes: prompt, control, architecture and policy.

  6. Fix Guidance and Free Re-Test

    We help you close findings, then retest and update the report.

What You Receive

  • Threat model for the AI system
  • Reproducible attack prompts and evidence
  • Severity ratings mapped to OWASP LLM Top 10 and MITRE ATLAS
  • Defence recommendations: guardrails, grounding, access control and monitoring
  • Re-test report

Mapped to the OWASP Top 10 for LLM Applications (2025) and MITRE ATLAS. Tooling includes garak and Microsoft PyRIT alongside manual testing.

Frequently Asked Questions

What is LLM pentesting?

It is security testing of applications built on large language models, such as chatbots, RAG assistants and agents. It looks for prompt injection, data leakage, unsafe tool use and other AI-specific weaknesses.

Which models and platforms do you test?

We test the application, not just the model. That includes apps built on Azure OpenAI, AWS Bedrock, OpenAI, Anthropic, open-source models and custom agents.

Do you need access to the model weights?

No. Most tests are black-box or grey-box through the same interfaces your users and agents use.

How is this different from a normal web pentest?

A web pentest checks code and infrastructure. LLM pentesting also checks what the model can be persuaded to do, what it can reach and what happens to its output.

Will you test our agents' tool and API access?

Yes. Excessive agency and tool misuse are among the most serious AI risks, so agent permissions and integrations are in scope.

How much does it cost?

It is quoted by scope: number of AI features, integrations and data sources. Contact us with a short description and we will propose a fixed price.

Is a re-test included?

Yes, one re-test is included so you can confirm that fixes work.