AI-generated Code Could Be a Disaster for the Software Supply Chain. Here’s Why.
AI-generated code poses significant risks to the software supply chain by introducing “hallucinated” package dependencies that don't exist. Research shows that 440,000 out of 576,000 AI-generated samples contained these misleading references, heightening vulnerability to dependency confusion attacks. These attacks can lead to malicious packages being installed if developers trust erroneous code without verification. Open-source models exhibited higher hallucination rates than commercial ones, particularly in JavaScript. The prevalence of persistent hallucinations creates exploitable patterns for attackers, highlighting the untrustworthiness of AI output in coding.