Attacking LLMs and Possible Mitigations – Part 2

In Part 1, we explored several techniques researchers use to challenge Large Language Model (LLM) safety mechanisms. While the specific techniques vary, most exploit a familiar security principle:

Different layers interpret the same information differently.

This raises an important question:

How do we defend against these attacks?

As with traditional security, there is no single control that solves the problem. Effective AI security requires multiple layers working together.

1. Normalize Before Inspection

The Problem

Many attacks rely on presenting information in an unusual form:

  • Alternative character sets
  • Encoded content
  • Modified spellings
  • Unicode tricks

Different security components may interpret these inputs differently.

The Mitigation

Normalize the content before inspection.

Examples include:

  • Converting Unicode variants to canonical forms
  • Decoding supported encodings
  • Removing unnecessary formatting
  • Standardizing character representations

Security Analogy

This is similar to URL normalization in WAFs.

Before making a security decision, all layers should see the same representation of the data.

2. Inspect Meaning, Not Just Keywords

The Problem

Simple pattern matching works well against known terms but struggles against creative variations.

Attackers continuously find new ways to express the same intent.

The Mitigation

Move beyond keyword detection.

Evaluate:

  • User intent
  • Semantic meaning
  • Context
  • Requested outcome

Rather than asking:

“Does this contain a forbidden word?”

Ask:

“What is the user trying to achieve?”

Security Analogy

The transition from signature-based detection to behavioral analysis.

3. Use Multiple Independent Safety Layers

The Problem

A single point of failure creates a single point of bypass.

If one classifier makes a mistake, the entire system may fail.

The Mitigation

Use multiple independent controls.

Examples:

  • Input classifiers
  • Behavioral analysis
  • Output validation
  • Policy engines
  • Risk scoring

Each layer should make its own assessment.

Security Analogy

Defense in depth.

No single control should be trusted to stop every attack.

4. Validate Outputs, Not Just Inputs

The Problem

Many AI systems focus heavily on inspecting incoming prompts.

However, the most important question is often:

“What is the model about to produce?”

The Mitigation

Analyze generated responses before they leave the system.

Evaluate:

  • Risk
  • Policy compliance
  • Sensitive information exposure
  • Safety violations

Security Analogy

Egress filtering.

Many organizations inspect outbound traffic because compromise may already have occurred.

5. Adversarial Testing

The Problem

Attackers continuously discover new techniques.

Static testing quickly becomes outdated.

The Mitigation

Create dedicated red-team programs.

Continuously challenge the model with:

  • Novel prompts
  • New attack patterns
  • Unexpected edge cases
  • Emerging jailbreak techniques

Security Analogy

Penetration testing and continuous security assessments.

6. AI Defending Against AI

The Problem

Attackers increasingly use AI to automate discovery.

Thousands of variations can be tested in minutes.

The Mitigation

Use defensive AI systems.

Examples include:

  • Automated prompt analysis
  • Risk classification
  • Pattern discovery
  • Attack clustering
  • Adaptive defense recommendations

Security Analogy

Modern SOC platforms already use automation to process attack volumes beyond human scale.

7. Context Isolation

The Problem

Large context windows create opportunities for instruction conflicts and context manipulation.

The model may receive:

  • System instructions
  • Developer instructions
  • User instructions
  • Retrieved knowledge
  • Conversation history

all within a single reasoning space.

The Mitigation

Separate trust domains.

Treat each source differently.

Examples:

  • System instructions remain isolated
  • User content is clearly identified
  • Retrieved content is marked as untrusted
  • Tool outputs are validated independently

Security Analogy

Privilege separation.

Applications should not treat all data as equally trusted.

8. Minimize Exposure of Internal Instructions

The Problem

The more information attackers learn about internal architecture, the easier it becomes to target weaknesses.

The Mitigation

Reduce unnecessary disclosure.

Avoid:

  • Revealing internal prompts
  • Exposing policy structures
  • Returning implementation details
  • Providing debugging information to untrusted users

Security Analogy

Security through obscurity is not sufficient, but excessive transparency can increase attack surface.

9. Risk-Based AI Operations

The Problem

Not all interactions carry the same risk.

A weather question and a high-impact operational decision should not receive identical treatment.

The Mitigation

Apply risk scoring.

Examples:

Risk Level Controls
Low Standard safety checks
Medium Enhanced validation
High Human review
Critical Restricted execution

Security Analogy

The same principle used in vulnerability management and incident response.

Resources are focused where risk is highest.

10. Architectural Security Beyond Prompts

The Problem

Many discussions focus entirely on prompts.

However, prompts are only one layer of the system.

The Mitigation

Security must exist throughout the architecture:

  • Identity controls
  • Authorization
  • Data isolation
  • Monitoring
  • Audit logging
  • Tool permissions
  • Network controls
  • Resource restrictions

Security Analogy

A WAF alone does not secure an application.

Similarly, prompt engineering alone cannot secure an AI system.

Final Thoughts

The most important lesson from modern LLM security research is that AI security is rapidly becoming a traditional security problem.

The technologies are new.

The principles are not.

Security engineers have spent decades dealing with:

  • Parser differentials
  • Input validation failures
  • Canonicalization issues
  • Trust boundary violations
  • Defense-in-depth architectures

LLMs introduce new implementations of these challenges, but the underlying concepts remain familiar.

As AI systems become increasingly integrated into business operations, organizations will need to stop viewing AI safety as a prompt-engineering problem and start treating it as a full security engineering discipline.

The future of AI security will likely belong to the teams that combine traditional security principles with a deep understanding of how language models reason, interpret, and interact with the world around them.