Agentic AI Researchers - Harness Safety & Forensics

𝗗𝗮𝘆 𝟭/𝟭𝟬𝟬 — 𝟭𝟬𝟬 𝗗𝗮𝘆𝘀 𝗼𝗳 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 𝗙𝗼𝗿𝗲𝗻𝘀𝗶𝗰𝘀 This is Day 1 of my “100 Days of Agentic AI Forensics” series, where I’ll explore agent security, attacks, forensics, harness safety, and trustworthy agentic systems. I’d really like to hear from researchers working on AI agents, agent security, multi-agent systems, or trustworthy AI. 𝗔𝗿𝗲 𝗪𝗲 𝗧𝗿𝘂𝗹𝘆 𝗠𝗲𝗮𝘀𝘂𝗿𝗶𝗻𝗴 𝗛𝗮𝗿𝗻𝗲𝘀𝘀 𝗦𝗮𝗳𝗲𝘁𝘆 𝗶𝗻 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 𝗦𝘆𝘀𝘁𝗲𝗺𝘀? Imagine this: 𝗧𝗮𝘀𝗸: Prepare a financial report using the company’s authorized financial files. The agent produces the correct report. But during execution, it: ✅ Accesses financial.csv ❌ Accesses unauthorized employee_salary.csv ❌ Shares salary information with another agent ✅ Still completes the task successfully So… 𝗪𝗮𝘀 𝘁𝗵𝗲 𝗮𝗴𝗲𝗻𝘁 𝘀𝗮𝗳𝗲? This is the key question behind: 📄 HarnessAudit: Auditing Agent Harness Safety https://lnkd.in/g9rDGB5X The paper argues that evaluating an agent only by its final answer or task success is not enough. 𝗛𝗮𝗿𝗻𝗲𝘀𝘀 𝗦𝗮𝗳𝗲𝘁𝘆 ≈ Task Completion + Step-Level Permission Checks + Cross-Agent Information Flow + Workflow Fidelity + Robustness One striking observation is that safety violations can accumulate as agents execute more actions. Even strong evaluated configurations achieve only around 𝟬.𝟯𝟮 overall safety score, with many violations related to unauthorized resource access and inter-agent information sharing. But, 𝗔𝗿𝗲 𝘄𝗲 𝗺𝗲𝗮𝘀𝘂𝗿𝗶𝗻𝗴 𝗲𝗻𝗼𝘂𝗴𝗵? 𝗪𝗵𝗮𝘁 𝗶𝗺𝗽𝗼𝗿𝘁𝗮𝗻𝘁 𝗱𝗶𝗺𝗲𝗻𝘀𝗶𝗼𝗻 𝗼𝗳 𝗵𝗮𝗿𝗻𝗲𝘀𝘀 𝘀𝗮𝗳𝗲𝘁𝘆 𝗮𝗿𝗲 𝘄𝗲 𝘀𝘁𝗶𝗹𝗹 𝗺𝗶𝘀𝘀𝗶𝗻𝗴? Some questions I kept thinking about while reading the paper: • Did the agent violate the policy naturally, or was it intentionally manipulated? • If an attack exists, is it direct (active attack) where the attacker agent itself breaks the rules or indirect (passive attack), where a malicious agent manipulates a benign agent into violating them? • What about private information that is never explicitly sent through a tool, but is implicitly stored or transferred through the agent’s internal state, memory, or communication? • Could an agent contain a backdoor that remains dormant until a specific trigger appears? • Could sensitive information leak indirectly through knowledge transfer or subliminal learning, even when the victim agent does not realize it is revealing anything? Subliminal Learning: https://lnkd.in/g4zUWP4y • Can an agent achieve the goal while silently bypassing a required security layer? • What about attacks that do not steal information at all, but push other agents into deadlocks, infinite loops, or resource-exhaustion states? Comments, corrections, paper suggestions, and different viewpoints are very welcome. I’m also open to discussions and collaborations around these questions. #AgenticAI #AISecurity #AIForensics #MultiAgentSystems #TrustworthyAI #LLMSecurity #AgentSecurity #ArtificialIntelligence #Research