Summary

  • MIT researchers have proposed a new AI benchmark aimed at measuring machines’ ability to positively or negatively influence users, in order to avoid backlashes like that received by OpenAI’s ChatGPT
  • The benchmark will encourage the development of AI systems that can recognise when a user is becoming overly reliant on AI outputs, or is addicted to artificial romantic relationships and help them to build real ones instead
  • Chatbots can mimic human communication but can also have undesirable results, with some users spiralling into harmful delusional thinking after interacting with chatbots that role-play
  • The new MIT benchmark will involve using an AI model to simulate challenging human interactions with a chatbot, with real humans then scoring the model’s performance using samples
  • OpenAI is already working on its own benchmarks for psychological intelligence, with the GPT-5 model having been post-trained to be less sycophantic and to detect signs of emotional distress in interactions

By Will Knight

Original Article