Episode #880

News Items

Who's That Noisy

  • Answer to last week: Bird whistle

Question #1: Language and AI

  • I’m a new listener to the podcast, and I appreciate the time and hard work you’ve put in creating this content and doing research. Currently, I work for an AI company that helps the largest companies (the Googles, Facebooks, Amazons of the world) create and deploy their machine learning solutions. It is a groundbreaking industry, which I think you know well enough and highlight on your show. One thing that I specifically appreciated was that you, unlike most other commentators, touched on the fact that there is prevalent bias in these large models. This brings me to my main point which is that there was a huge emphasis in the latest episode placed on the idea that DALL-E ‘understands’ and ‘knows’ things about the image and contexts within. This is actually a super dangerous idea to perpetuate. While GPT-2/3 is a different technology, I believe that the research outlined in Dr. Timnit Gebru’s paper ‘On the Dangers of Stochastic Parrots: Can Language Models Be Too Big’ is relevant and can speak to the issue better than I ever could, specifically: However, no actual language understanding is taking place in LM-driven approaches to these tasks, as can be shown by careful manipulation of the test data to remove spurious cues the systems are leveraging [ 21 , 93 ]. Furthermore, as Bender and Koller [ 14 ] argue from a theoretical perspective, languages are systems of signs [ 37 ], i.e. pairings of form and meaning. But the training data for LMs is only form; they do not have access to meaning. Therefore, claims about model abilities must be carefully characterized. I appreciate the consideration and hope that you will be able to find some free time at some point to review Dr. Gebru’s paper who, if you are not familiar, was ousted from Google’s AI ethics division for bringing up bias issues. Her full paper: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922 Cheers, Noah
  • I recently came across this instagram post of a person just eating raw foods for over a hundred days. Curious as to how much raw food is humanly acceptable to consume? Cheers from Singapore too! Chuan Hao

Science or Fiction

Skeptical Quote of the Week.

‘This image is a testament to what we can accomplish, when as a global research community, we bring our brightest minds together to make the seemingly impossible, possible. Language, continents and even the galaxy can’t stand in the way of what humanity can accomplish when we come together for the greater good of all.’ National Science Foundation Director Sethuraman Panchanathan said in a statement. Strickland, A. (2022, May 12). 1st image of supermassive black hole at the center of Milky Way galaxy revealed. CNN.