Why We Should Have Seen That Coming

Comments on Microsoft’s Tay “Experiment,” and Wider Implications

  • K.W Miller University of Missouri-St. Louis
  • Marty J Wolf Bemidji State University
  • F.S. Grodzinsky Sacred Heart University
Keywords: learning software development, responsibility, AI, technologies of humility, software profession


In this paper we examine the case of Tay, the Microsoft AI chatbot that was launched in March, 2016. After less than 24 hours, Microsoft shut down the experiment because the chatbot was generating tweets that were judged to be inappropriate since they included racist, sexist, and anti-Semitic language. We contend that the case of Tay illustrates a problem with the very nature of learning software (LS is a term that describes any software that changes its program in response to its interactions) that interacts directly with the public, and the developer’s role and responsibility associated with it. We make the case that when LS interacts directly with people or indirectly via social media, the developer has additional ethical responsibilities beyond those of standard software. There is an additional burden of care.

How to Cite
Miller, K., Wolf, M. J., & Grodzinsky, F. (2017). Why We Should Have Seen That Coming. ORBIT Journal, 1(2). https://doi.org/10.29297/orbit.v1i2.49