LLMMedium
Microsoft-Powered New York City Chatbot Advises Illegal Practices
- Occurred
- Mar 29, 2024
- Published
- Mar 29, 2024
- Source
- AI Incident Database
Summary
New York City's chatbot, launched under Mayor Eric Adams's plan to assist businesses, has been reportedly providing dangerously inaccurate legal advice. The Microsoft-powered bot allegedly informed users that landlords can refuse Section 8 vouchers and that businesses can operate cash-free, among other falsehoods. The city acknowledges the chatbot is a pilot program and commits to improvements while the errors are addressed.
Developers
- Microsoft
- New York City Office of Technology and Innovation
Deployers
- New York City Government
- Eric Adams administration
Harmed parties
- New York City small business owners
- New York City landlords and tenants
- New York City employers and employees
- Eric Adams administration
Classification
Category confidence: 90%
Severity confidence: 85%
Rationale: Microsoft-powered chatbot provided inaccurate legal advice, representing a language model failure with potential legal harm.
Classifier: Qwen/Qwen3.5-35B-A3B-FP8