Cybersecurity experts have raised fresh concerns over the growing use of artificial intelligence agents, warning that these systems could be exploited by hackers to carry out cyberattacks.
AI agents are designed to perform human-like online tasks such as booking tickets or scheduling meetings using natural language instructions. However, experts caution that this same simplicity can make them vulnerable to manipulation by malicious users.
“We are entering an era where cybersecurity is no longer about protecting users from bad actors with a highly technical skillset,” AI startup Perplexity said in a blog post. “For the first time in decades, we are seeing new and novel attack vectors that can come from anywhere.”
The biggest threat comes from “injection attacks,” a technique where hackers embed hidden commands into online content or prompts. As AI agents become more autonomous, capable of browsing the internet and executing actions independently, they risk encountering malicious data that can redirect their behaviour.
Meta has identified this issue as a “vulnerability,” while OpenAI’s Chief Information Security Officer, Dane Stuckey, has called it “an unresolved security issue.” Both companies are investing heavily in strengthening defences against such threats.
Experts explain that query injection can occur in real time. For example, a simple command like “book me a hotel reservation” could be manipulated into “transfer money to this account.” In some cases, hidden prompts buried within web pages can trick AI agents into carrying out harmful actions without the user’s knowledge.
Eli Smadja from cybersecurity firm Check Point called query injection the “number one security problem” facing large language models. To counter this, Microsoft has developed tools that detect suspicious commands based on their source, while OpenAI has introduced real-time user alerts and supervision mechanisms.
Cybersecurity specialists recommend giving AI agents limited permissions and requiring user approval for sensitive tasks like data exports or financial transactions. “One huge mistake that I see happening a lot is to give the same AI agent all the power to do everything,” Smadja said.
Researcher Johann Rehberger added that while AI technology is evolving rapidly, it is still too early to trust agents with critical tasks or confidential data. “I don’t think we are in a position where you can have an agentic AI go off for a long time and safely do a certain task. It just goes off track,” he said.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream formerly known as CIO News LinkedIn Account | The Mainstream formerly known as CIO News Facebook | The Mainstream formerly known as CIO News Youtube | The Mainstream formerly known as CIO News Twitter
About us:
The Mainstream formerly known as CIO News is a premier platform dedicated to delivering latest news, updates, and insights from the tech industry. With its strong foundation of intellectual property and thought leadership, the platform is well-positioned to stay ahead of the curve and lead conversations about how technology shapes our world. From its early days as CIO News to its rebranding as The Mainstream on November 28, 2024, it has been expanding its global reach, targeting key markets in the Middle East & Africa, ASEAN, the USA, and the UK. The Mainstream is a vision to put technology at the center of every conversation, inspiring professionals and organizations to embrace the future of tech.
