Amazon Employees Share Concerns On Q Chatbot: “Severe Hallucinations, Leaking Confidential Data”
Concerns have emerged among Amazon (NASDAQ: AMZN) employees regarding accuracy and privacy issues surrounding the recently unveiled AI chatbot, Q. Just three days after its high-profile announcement at the annual Amazon Web Services developer conference, leaked documents obtained by Platformer suggest that Q is experiencing “severe hallucinations” and leaking confidential data, including sensitive information about the location of AWS data centers, internal discount programs, and unreleased features.
The severity of the situation prompted an employee to mark the incident as “sev 2,” indicating a problem significant enough to necessitate engineers working overtime to address the issues. This development comes at a crucial time for Amazon, as the tech giant strives to dispel the perception that rivals like Microsoft and Google have outpaced it in the race to develop cutting-edge tools and infrastructure leveraging generative artificial intelligence.
Despite the internal concerns, Amazon downplayed the significance of the discussions among employees. A spokesperson stated, “Some employees are sharing feedback through internal channels and ticketing systems, which is standard practice at Amazon. No security issue was identified as a result of that feedback.” The company expressed appreciation for the feedback received and affirmed its commitment to refining Q as it transitions from a product in preview to being generally available.
In a subsequent statement, the spokesperson pushed back on employees’ claims, asserting, “Amazon Q has not leaked confidential information.”
Q, currently available as a free preview, was introduced as an enterprise-software version of ChatGPT, designed to answer developers’ questions about AWS, edit source code, and cite sources. Amazon executives positioned Q as a competitive offering priced lower than similar tools from Microsoft and Google. The company emphasized its commitment to security, positioning Q as a more secure alternative to consumer-grade tools.
Adam Selipsky, CEO of Amazon Web Services, addressed concerns raised by companies that had banned AI assistants due to security and privacy worries. Selipsky told the New York Times that Amazon built Q with enhanced security and privacy features, aiming to overcome the reservations associated with consumer chatbots.
However, leaked internal documents revealed a different side of Q, indicating that the chatbot can hallucinate and provide harmful or inappropriate responses. The document highlighted instances where Q could return outdated security information, potentially putting customer accounts at risk. Such risks were noted to be typical of large language models, acknowledging that they may produce incorrect or inappropriate responses at times.
Information for this story was found via Platformer and the sources mentioned. The author has no securities or affiliations related to the organizations discussed. Not a recommendation to buy or sell. Always do additional research and consult a professional before purchasing a security. The author holds no licenses.
