AI models such as ChatGPT, Claude and Gemini have become an integral part of the daily working lives of many IT professionals. Generating code, summarising documentation, developing concepts. The range of applications is vast. However, those working in offensive IT security quickly come up against limitations with cloud-based AI services – not technical ones, but practical, legal and ethical ones.
The topic of on-premises AI is increasingly occupying our industry. In this article, we aim to honestly assess the arguments for and against it and spark a discussion: What are your views on on-premises AI in security research?
Why Cloud AI is a problem for Pentesters
Anyone carrying out penetration tests or red-team engagements inevitably works with sensitive client data: API documentation, source code, network diagrams, internal configurations. None of this should be uploaded to a AI provider’s Cloud. Not only because it would be contractually problematic, but because it would constitute a breach of trust towards the client.
No matter how well a provider’s privacy policy is worded, the risk that data might be used for training or exposed through security vulnerabilities can never be completely ruled out.
Added to this is a very practical problem: cloud-based AI models refuse to generate malicious code. At first glance, this sounds reasonable. For an IT security firm that needs precisely this code for authorised attacks, however, it is a significant obstacle. Anyone needing a working reverse-shell payload or a Phishing template for a red team engagement must first convince ChatGPT and the like that they are authorised, or pretend they are currently taking part in a CTF (Capture the Flag) tournament.
What local AI can achieve
Locally operated models resolve precisely these friction points. Four areas of application have proven particularly valuable in practice:
Digital sparring partner
If you get stuck whilst analysing a web application, you can show the model the relevant section of code and develop hypotheses together. Where might a race condition exist? Is there insufficient input validation that you’ve overlooked? AI doesn’t replace your own thinking, but it reveals blind spots and acts as a second pair of eyes.
Support in DFIR
There is also real potential in digital forensics and Incident Response. Parsing log files, correlating artefacts from memory dumps, merging timelines from various sources. Particularly in time-critical IR scenarios, a local AI assistant can make all the difference. Not as a substitute for experience, but as an accelerator.
Code review and tooling
Checking source code for security vulnerabilities is time-consuming. Local models can analyse large codebases in advance and flag potential vulnerabilities. The hit rate isn’t perfect, but it’s certainly useful as an initial filter before a manual review. And as the code never leaves your own infrastructure, confidentiality is maintained.
Integration into your own toolchains
Local models can be seamlessly integrated into your own workflows. Frameworks such as Ollama or vLLM make operation technically feasible, whilst open models such as Llama, Mistral or DeepSeek now offer impressive quality.
Digital sovereignty
Anyone who relies entirely on Cloud-based AI services makes themselves dependent. Dependent on pricing models that can change overnight. Dependent on terms of service that may suddenly exclude certain use cases. And dependent on geopolitical decisions. Export restrictions can mean that the availability of a service is simply no longer ensured for certain countries or sectors.
Local models ensure independence. Once downloaded and configured, they continue to run even if a provider changes its directives. And what is processed locally stays local. No logging by third parties, no possibility of input data reappearing in future training runs.
The other side: costs and limitations
As attractive as the picture may seem so far, local AI is not a sure-fire success. There are three points one should realistically take into account.
Hardware
Powerful local models require powerful hardware. Anyone wishing to run a 70B-parameter model smoothly will need GPUs with sufficient VRAM. A single professional GPU can easily cost several thousand euros, and for really large models, you may need several of them. Especially now, as hardware prices continue to rise due to increasing demand for AI accelerators, this represents a significant investment. Smaller models in the 7B to 14B range may run on consumer hardware, but they do not always deliver the quality required for complex tasks.
Quality and effort
Even though open-source models have made enormous progress, the large Cloud-based models still lead the way in certain areas. The gap is narrowing, but it still exists. Added to this is the effort involved in setting up, updating and evaluating new model versions. This takes time and requires specialist knowledge.
AI is no substitute for researchers
AI is no substitute for thinking for oneself. Anyone who blindly trusts AI-generated results without validating them will, sooner or later, come unstuck. Hallucinations are a real risk, context is lost, and a model does not understand the nuances of a specific customer environment. Security research requires creativity, intuition and a deep technical understanding. A language model does not provide this.
Mint Secure GmbH
How Mint Secure supports you
We help you integrate AI sensibly and securely into your offensive and defensive processes:
AI Strategy & Setup
We provide consulting services for the selection, sizing and configuration of on-premises models, from hardware right through to toolchain integration with Ollama or vLLM.
Pentest & Red Teaming
Authorised attack simulations carried out by experienced analysts, supported by AI but underpinned by genuine expertise.
Data Protection-Compliant Workflows
We design processes in which sensitive customer data never leaves your own infrastructure and confidentiality is maintained.
Ready to get started? We offer a free initial consultation.
Get in touch.
Conclusion
Local AI is more than just a trend in offensive security research. It solves real-world problems: the confidentiality of customer data, the lack of flexibility in Cloud models, and digital dependency. At the same time, it is not a free, all-inclusive, worry-free package. Hardware, effort and the limitations of the AI models must be factored in honestly.
Ultimately, AI remains a tool, not a substitute for the experience, creativity and intuition of seasoned researchers. Those who use it in this way gain independence without compromising on quality.
What are your views on local AI in security research? Mint Secure supports you in building secure, sovereign AI workflows.
Talk to us.
