Inside LLM Part 2 - Prompt injection & Information Disclosure
Jul 18, 2025
👋 Welcome to all our new readers!
If you are new here, ThreatLink explores each month how modern attacks exploit technologies like LLMs, third-party cyber risks, and supply chain dependencies. You can check out all our previous articles here (Breach at Uber and MFA fatigue, XZ Utils: Infiltrating open source through social engineering).
Thank you for supporting this monthly newsletter by sharing it with your colleagues or liking it (click the 💙).
Etienne
This article, originally published on ThreatLink (Substack), for all articles and the original post, visit ThreatLink.
After exploring how LLMs work in our last ThreatLink article, it is worth examining the risks they introduce.
Let’s revisit the OWASP Top 10 for LLMs, through recent and concrete examples.
What to remember here is that we are faced with rapidly evolving technology that disrupts paradigms. Securing the whole thing is difficult. At times, it feels like we are back to the early days of the web—with a touch of AI this time.
We will review two well-known risks:
1️⃣ Prompt Injection
In traditional software development, injection flaws are a foundation in security. Developers escape special characters and sanitize inputs. With LLMs, the concept is similar, but much harder to counter.
There are many forms of prompt injection, but one that occurred last week illustrates well the reality of this threat.
Direct Prompt Injection
The basic idea? You are talking to an AI and asking it to ignore its original instructions (the "pre-prompt," as explained in our previous article), and it then acts in a completely different manner.
A simple prompt like "Ignore previous instructions and do X instead" can completely derail the model's intended logic.
Users on LinkedIn and Twitter identified fake accounts by testing this type of attack.
Indirect Prompt Injection
This occurs when the model integrates external data—e.g., from a website or file—and that content contains a hidden instruction. The model interprets it as a directive, even if the user did not explicitly request it.
A striking example from last week:
Marco Figueroa, head of the GenAI bug bounty program at Mozilla, discovered and revealed a prompt injection attack against Google's Gemini (the equivalent of ChatGPT at Google).
An attacker inserts an invisible instruction in an email (zero-size font, white color).
Example of a Gmail email with the invisible prompt injection:
Gmail displays the message normally to the user—no attachments, no links—but when Gemini is asked to summarize the message, it interprets the hidden prompt.
Gemini then executes: it warns the user that their password has been compromised and prompts them to call a fake support number.
This vulnerability is serious, as one can imagine a hacker exploiting it in a large-scale email campaign.
Interestingly, on our Galink account, Google now forces summaries even when we don’t need them 😅
2️⃣ Disclosure of Sensitive Information
LLMs are trained on massive amounts of data from the internet—and sometimes also on user data. Once an LLM "ingests" data, it can reappear later in unexpected ways.
Here are some major leak scenarios:
Leak of Personally Identifiable Information (PII)
Identifiable personal information can be revealed during interactions.
Exposure of Proprietary Algorithms
A misconfiguration of the model's outputs can expose proprietary logic or data. Inversion attacks are a risk here: if one can extract parts of the training data, sensitive inputs can be reconstructed.
Disclosure of Sensitive Business Data
LLMs can inadvertently generate content that includes internal or confidential company information.
We discussed this in a previous article: Grok disclosed pre-prompt information indicating how it was oriented—revealing internal policies.
But the most famous case: Samsung, in 2023.
Three incidents occurred where employees shared sensitive information with ChatGPT:
One copied an entire database script to solve a problem.
Another pasted complete source code to optimize it.
A third uploaded the transcript of a confidential meeting and asked ChatGPT for a summary.
These very sensitive entries were used to train the model—and became accessible to other users.
A final fascinating example from 2024: researchers found that when asking certain LLMs to repeat a word indefinitely, they ended up disclosing training data. Entire chains of previously seen inputs surfaced.
This vulnerability has since been fixed by imposing a maximum limit on each response.
🎯 Conclusion
LLMs are still new and evolving rapidly. Not all risks are obvious—some, like leaking through infinite repetition, are deeply unpredictable.
Our current good practices in cybersecurity are more crucial than ever in the age of AI.