Skip to content
LLMMedium

Microsoft-Powered New York City Chatbot Advises Illegal Practices

Occurred
Mar 29, 2024
Published
Mar 29, 2024
Source
AI Incident Database

Summary

New York City's chatbot, launched under Mayor Eric Adams's plan to assist businesses, has been reportedly providing dangerously inaccurate legal advice. The Microsoft-powered bot allegedly informed users that landlords can refuse Section 8 vouchers and that businesses can operate cash-free, among other falsehoods. The city acknowledges the chatbot is a pilot program and commits to improvements while the errors are addressed.

Developers

  • Microsoft
  • New York City Office of Technology and Innovation

Deployers

  • New York City Government
  • Eric Adams administration

Harmed parties

  • New York City small business owners
  • New York City landlords and tenants
  • New York City employers and employees
  • Eric Adams administration

Classification

Category confidence: 90%
Severity confidence: 85%

Rationale: Microsoft-powered chatbot provided inaccurate legal advice, representing a language model failure with potential legal harm.

Classifier: Qwen/Qwen3.5-35B-A3B-FP8

Visit source