What Is an AI Model Harness?
An AI model harness is the application layer that connects an AI model to users, business systems, APIs, databases and external tools. While the AI model performs the reasoning, the harness controls authentication, permissions, data retrieval, tool execution and workflow orchestration. Securing the harness is critical because attackers are often more likely to exploit these surrounding systems than the model itself.
AI security has largely centred on the models themselves. Organisations have spent the past two years discussing prompt injection, hallucinations, model theft and data leakage, while vendors have invested heavily in improving the security of large language models.
The Australian Signals Directorate’s latest guidance shifts that conversation in an important direction.
Rather than focusing exclusively on protecting the model, the ASD is encouraging organisations to secure the AI model harness. This reflects how enterprise AI is actually being deployed today. Most organisations aren’t simply providing employees with access to a standalone chatbot. They’re integrating AI into business applications, connecting models to internal knowledge bases, cloud services, APIs and automation platforms that allow AI to retrieve information and perform tasks.
From a security perspective, that changes everything.
The model itself is only one component of a much larger ecosystem. The surrounding orchestration layer, APIs, retrieval systems, identity services and business logic often present a broader attack surface than the language model itself. As organisations continue integrating AI into operational workflows, understanding and securing these components is becoming just as important as protecting the model.
AI Models Aren’t the Biggest Attack Surface
When people hear the term “AI security”, they often think about protecting the model from manipulation or preventing sensitive information from appearing in responses.
Those are legitimate concerns, but they’re only part of the picture.
In most enterprise environments, the AI model doesn’t operate independently. It sits behind an application that receives prompts, authenticates users, retrieves relevant business information, interacts with APIs and determines what actions the AI is allowed to perform.
This surrounding infrastructure is commonly referred to as the model harness.
A typical enterprise AI deployment might include:
- Identity providers for authentication and authorisation
- Retrieval-Augmented Generation (RAG) pipelines
- Vector databases storing indexed business information
- API gateways connecting business systems
- Workflow automation platforms
- Logging and monitoring systems
- Prompt management and guardrail frameworks
Every one of these components introduces additional trust relationships.
If an attacker can manipulate the harness, they may never need to attack the model itself.
For example, compromising an API that retrieves sensitive documents, abusing excessive permissions assigned to an AI agent or exploiting weaknesses within an orchestration platform may provide significantly greater value than attempting to manipulate the language model directly.
This is why the ASD’s guidance is important.
The focus is shifting from “How secure is our AI model?”
To:
“How secure is everything connected to our AI model?”
What Does an AI Model Harness Actually Do?
The easiest way to understand a model harness is to think of it as the operational layer that sits between users and the AI model.
When someone submits a prompt, the request rarely travels directly to the language model.
Instead, the harness performs several tasks before the model even begins generating a response.
Depending on the deployment, it may:
- Authenticate the user.
- Check authorisation levels.
- Retrieve relevant business information from internal systems.
- Query vector databases for contextual information.
- Apply prompt templates and safety guardrails.
- Decide which external tools the model can access.
- Execute API calls.
- Log activity for monitoring and compliance.
Once the model generates a response, the harness may perform additional processing before anything is presented to the user. It may filter sensitive information, remove prohibited content or trigger downstream workflows based on the response.
The harness is therefore responsible for much more than simply forwarding prompts.
It’s effectively the control plane for the entire AI application.
That’s precisely why it deserves the same level of security scrutiny as any other critical business application.
Where We See the Greatest Security Risks
One of the biggest misconceptions surrounding AI security is that attackers primarily target the language model.
In practice, they’re often more interested in the surrounding application architecture.
During security assessments, we generally focus on how the AI system behaves as a complete solution rather than treating the model as an isolated component.
Some of the most common areas of concern include:
Prompt Injection
Prompt injection remains one of the most widely discussed AI security risks, but the impact often depends on what the model is connected to.
If the AI only generates text, a successful prompt injection may produce an incorrect or unexpected response.
If the same AI can query internal systems, retrieve confidential documents or execute business workflows, the consequences become significantly more serious.
The vulnerability isn’t always the prompt itself.
It’s what the prompt enables the surrounding application to do.
Excessive Permissions
AI agents frequently inherit permissions from service accounts or connected business applications.
If these permissions aren’t carefully managed, attackers may be able to retrieve information far beyond what an individual user should legitimately access.
Applying the principle of least privilege to AI systems is just as important as it is for human users.
Retrieval-Augmented Generation (RAG) Risks
Many enterprise AI platforms use Retrieval-Augmented Generation to improve the quality of responses by accessing internal documentation.
This introduces another security consideration.
If sensitive information enters the retrieval pipeline without appropriate controls, AI may expose data that wasn’t intended to be available to certain users.
The issue isn’t that the model has been compromised.
It’s that the surrounding retrieval system has provided information it shouldn’t have accessed.
Insecure Tool Calling
Modern AI platforms increasingly interact with external tools through APIs.
These tools might create support tickets, retrieve customer information, update databases or trigger business workflows.
Every connected tool expands the potential attack surface.
Without strong authentication, input validation and access controls, attackers may attempt to manipulate these integrations rather than the AI model itself.
Why Traditional Security Controls Aren’t Enough
Many organisations already have mature security controls protecting their infrastructure. Multi-factor authentication is enabled, endpoint protection is deployed, privileged access is monitored and network traffic is inspected. These controls remain essential, but AI introduces new interactions that traditional security programmes weren’t designed to address.
An AI application doesn’t simply authenticate a user and display information. It reasons over data, retrieves content from multiple sources, interacts with APIs and, in some cases, executes actions on behalf of users. This creates a dynamic environment where permissions, business logic and context become just as important as the underlying infrastructure.
Consider an AI assistant integrated with Microsoft 365, SharePoint, a CRM platform and an internal ticketing system. Each integration may be individually secure, but together they create new pathways for information to move between systems. A prompt that appears harmless may trigger multiple API calls, retrieve documents from several repositories and combine information in ways the original application designers never anticipated.
This is why AI security is becoming an architectural challenge rather than simply an infrastructure challenge. Organisations need visibility into how information flows through the entire AI application, not just whether the language model itself has known vulnerabilities.
Security teams should therefore be reviewing:
- How identities are authenticated and authorised.
- Which APIs the AI can access.
- What permissions service accounts inherit.
- How sensitive information is retrieved and filtered.
- Whether user actions and AI actions are independently logged.
- How guardrails are enforced before and after model responses.
Strong security controls remain important, but they need to extend across the entire AI ecosystem rather than stopping at the model itself.
Lessons From AI Red Teaming
One of the reasons AI security is evolving so quickly is that many risks only become visible when systems are tested in realistic attack scenarios.
Traditional penetration testing is designed to identify exploitable vulnerabilities within networks, applications and infrastructure. AI systems require an additional layer of testing because the objective isn’t always to compromise the application. Sometimes it’s to influence how the application behaves.
At Intrix, we’ve performed AI red teaming engagements where the objective wasn’t to attack the language model itself, but to evaluate how the surrounding application responded when exposed to realistic adversarial techniques.
During one engagement for a large Australian organisation, our consultants assessed an enterprise AI deployment that integrated multiple internal data sources and business applications. Rather than focusing solely on prompt injection, we evaluated how the orchestration layer handled malicious prompts, whether sensitive information could be extracted through indirect requests, and how effectively permissions were enforced when the AI accessed connected systems.
The assessment included scenarios such as:
- Attempting prompt injection to bypass business rules.
- Testing for unintended disclosure of sensitive information.
- Assessing whether indirect prompts could influence downstream actions.
- Evaluating exposure to model inversion techniques.
- Reviewing API interactions between the harness and connected services.
- Validating whether excessive permissions increased the potential impact of compromise.
What these engagements consistently reinforce is that the greatest risks often exist around the model rather than inside it.
In many cases, the language model behaved exactly as expected.
The findings instead related to overly permissive integrations, insecure workflows, excessive access rights or assumptions within the orchestration layer that could be abused under the right conditions.
That’s precisely why AI red teaming has become such an important complement to traditional penetration testing. It helps organisations understand how an attacker would interact with the complete AI application rather than assessing each component in isolation.
Securing the AI Harness: Where to Start
Securing an AI model harness doesn’t require organisations to reinvent their cyber security programme. Many of the same security principles already applied to cloud platforms, APIs and enterprise applications remain relevant. The difference is understanding how those controls apply within AI-driven workflows.
A practical starting point includes:
Apply Least Privilege to AI
Treat AI agents like privileged users. Only grant access to the systems, APIs and data sources required to perform their intended function. Avoid broad permissions simply because they’re easier to configure.
Secure API Integrations
Every API connected to an AI application should require strong authentication, appropriate authorisation and input validation. APIs should be monitored for unusual behaviour and protected using the same standards as any other internet-facing service.
Review Retrieval-Augmented Generation (RAG)
If your AI platform retrieves information from internal documentation or knowledge bases, regularly review what content is indexed, who can access it and whether sensitive information has been unintentionally exposed through retrieval pipelines.
Monitor AI Activity
Logging shouldn’t stop at user authentication. Organisations should record AI interactions, API calls, prompt history where appropriate, tool usage and administrative changes. This provides valuable forensic evidence if suspicious behaviour occurs.
Validate Before Production
AI applications should undergo independent security testing before deployment and after significant architectural changes. This includes testing not only the language model, but the surrounding harness, integrations and business workflows that determine how the system operates in practice.
These activities won’t eliminate every AI risk, but they’ll significantly reduce the likelihood that attackers exploit weaknesses outside the model itself.
AI Security Is Becoming an Engineering Discipline
As enterprise AI adoption accelerates, security programmes need to evolve alongside it.
The focus is no longer limited to whether a language model can be manipulated through prompt injection or whether sensitive information appears in responses. Organisations must also consider how AI applications authenticate users, retrieve business data, interact with APIs and enforce security controls across increasingly complex workflows.
That’s why the ASD’s guidance on AI model harnesses is so significant.
It recognises that securing AI means protecting the entire application architecture, not just the intelligence at its centre.
For security teams, architects and developers, this represents an important shift in mindset. AI should be treated like any other enterprise application, subject to secure design principles, least privilege, continuous monitoring and independent security testing.
The organisations that adopt this approach early will be far better positioned to deploy AI confidently while reducing the risks introduced by increasingly connected and autonomous systems.
Frequently Asked Questions
What is an AI model harness?
An AI model harness is the application layer that connects an AI model to users, enterprise systems and external tools. It manages authentication, permissions, prompt handling, API integrations, retrieval pipelines and workflow orchestration. In most enterprise deployments, the harness determines what the AI can access and what actions it can perform, making it a critical component of the overall security architecture.
Why is securing the AI model alone not enough?
The language model is only one part of an enterprise AI application. Attackers are often more likely to target APIs, retrieval systems, orchestration platforms or identity services connected to the model than the model itself. If these components are compromised, an attacker may be able to retrieve sensitive information or manipulate business workflows even if the underlying model remains secure.
What is AI red teaming?
AI red teaming is the process of simulating realistic attacks against AI systems to identify weaknesses before they can be exploited. Unlike traditional penetration testing, AI red teaming evaluates how the model, model harness, APIs and connected systems respond to techniques such as prompt injection, indirect prompt manipulation, data extraction and model inversion.
When should organisations perform an AI security assessment?
AI security assessments should be conducted before enterprise AI solutions are deployed into production and repeated whenever significant changes are made to the application architecture. New integrations, retrieval systems, external tools or business workflows can introduce additional risks that should be independently validated before they become part of day-to-day operations.
Conclusion
Enterprise AI is evolving far beyond standalone chatbots. Today’s AI applications retrieve information from internal systems, interact with business applications, automate workflows and support operational decision-making. As these capabilities expand, so does the attack surface.
The ASD’s guidance on AI model harnesses highlights an important shift in how organisations should think about AI security. Protecting the language model remains important, but it’s only one layer of a much larger architecture. Identity services, orchestration frameworks, APIs, retrieval pipelines and business logic all influence how securely an AI application operates.
For security practitioners, the priority should be building AI systems that are secure by design. That means applying the same engineering discipline used for other critical business applications: least privilege, secure architecture, continuous monitoring and independent security validation. By treating the model harness as a critical security boundary rather than an implementation detail, organisations can significantly reduce the risks associated with enterprise AI deployments.
How Intrix Can Help
As Australian organisations continue integrating AI into business operations, ensuring those systems are secure requires more than reviewing the language model alone. At Intrix, we help organisations assess the complete AI ecosystem, identifying risks across the model harness, orchestration layer, APIs, retrieval systems and cloud infrastructure before they can be exploited.
Our team has practical experience conducting AI red teaming engagements that evaluate enterprise AI applications under realistic attack scenarios. Rather than focusing solely on model behaviour, we assess how AI systems respond to prompt injection, indirect prompt manipulation, data extraction attempts, model inversion techniques and insecure tool integrations. This provides organisations with a clearer understanding of how their AI deployments would perform against real-world adversarial techniques.
Through our AI Security and Cloud Security services, Intrix helps organisations:
- Assess AI applications before they enter production.
- Conduct AI red teaming exercises against enterprise AI deployments.
- Review AI model harness architecture and orchestration security.
- Evaluate Retrieval-Augmented Generation (RAG) implementations and data access controls.
- Assess API security, identity management and least privilege configurations.
- Strengthen cloud security controls supporting AI workloads.
- Provide practical remediation guidance aligned with secure-by-design principles.
AI adoption is accelerating across every industry, but secure deployment requires more than selecting the right model. By validating the security of the systems surrounding AI, organisations can adopt new technologies with greater confidence while reducing the likelihood of data exposure, workflow manipulation and unintended business risk.
To learn more about securing enterprise AI deployments, contact Intrix today for our services or to speak with our consultants about an AI security assessment tailored to your organisation’s environment.
