CS 370 - Spring 2025. 4/23/2025


[Home]

Welcome to CS 370!

Video of the Day

Meet NEO, Your Robot Butler in Training | Bernt Børnich | TED See Bernt Børnich The home is the source of rich and diverse training data for robots.

A New Philosophy on Artificial Intelligence | Kristian Hammond | TEDxNorthwesternU See Kristian Hammond

Canvas Quiz of the Day (need daily password)

Most days, there will be a simple canvas quiz related to the lecture. You need a password to activate the quiz, which I will provide in class. These quizzes will count toward your class participation grade. The quiz is available only during class. You get full credit for class participation by completing half of the quizzes.

Click for today's quiz.

Lecture 24: Natural Language Processing, Ethics, and Farewell.

Administrivia

  • I have office hours Wednesdays from 4-6 p.m. via Zoom, meeting ID 459 434 2854. This is the last week of office hours for me.

  • The TF's office hours are posted on Ed Discussion.

  • I am available for lunch on Mondays at 1 pm in Morse. No more this term.

  • Homework assignments: [Assignments]. Projects due next Monday, April 28th on gradescope. Papers due May 6th on canvas.

    Grades should be posted shortly.

    Guest Lecture: Alexandra Slade

    You may contact Alexandra directly: aslade@google.com. She would be happy to hear your thoughts and ideas.

    Asides from previous lectures

  • The Waluigi Effect (mega-post) A literary criticism view of LLMs. Waluigi.
  • Google’s AI Recommended Adding Glue To Pizza And Other Misinformation—What Caused The Viral Blunders? Hallucinations.
  • Cooperative principle, aka, Grice's Maxims. Good approach for LLMs. Language Models in Dialogue: Conversational Maxims for Human-AI Interactions

  • Hapax legomenon.
  • Eskimo words for snow. Sapir-Whorf Hypothesis. See The great Eskimo vocabulary hoax Geoffrey Pullum, 1989.

    AI in the news

  • *** Learning to Recognize Dialect Features submitted by Deja Dunlap.
  • Alafia AI Puts a Supercomputer on Your Desk Bloomberg, April 23, 2025.

  • Access to the Atlantic
  • Access to Economist (Economist.com)
  • Access to Financial Times
  • Access to Wall Street Journal from Yale.
  • Q and AI Bloomberg.
  • Access to Bloomberg.com from Yale.

    NLP and LLMs

  • AIMA Slides:
  • *** Can an AI program be fair? Equity Deborah Stone version of Doug Rae's cake problem.
  • *** Impacts of Artificial Intelligence Scientific, Technological, Military, Economic, Societal, Cultural, and Political, R. Trappl (editor), 1985. Includes a chapter by Schank and Slade. I had forgotten I had written this 40 years ago.
    Clearly there is a technological aspect of AI that drives much of the research. However, there is a very important scientific aspect of AI that involves the study of the mind. What is the organizational structure of human memory? How do people learn? How do people make decisions? How do people think?

    ...

    As indicated above, the areas of learning and knowledge organization are of central importance. These touch on many of the subfields within AI, including natural language processing, vision, speech recognition, expert systems, and robotics. We have also mentioned the reasons for developing a psychological methodology for modeling cognitive phenomena.

    One main requirement for productive and useful AI research is the capability to work on large, real-world problems. There is a broad consensus that no major breakthrough in AI will emerge from a small, concise program or from toy domains. AI programs should be explorations in complexity and should tackle the world as it exists.

    One extension to this argument is that AI researchers should look at problems that span more than one subfield. Thus, there should be efforts in integrating expert systems and natural language, or speech recognition and vision. Researchers tend to shy away from these large problems, saying that there are still many small problems that have yet to be solved. That may be, but by looking at larger problems, we may gain important insights into the nature of intelligence and cognition.

    Vanilla Natural Language Processing

  • NLP Progress Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks. (BLEU for machine translation.)

    AIMA Jupyter notebooks:

  • text.html
  • nlp.html
  • nlp_apps.html
  • WordNet open-source, hand-curated dictionary in machine readable form. provides the categories used by ImageNet.
  • Penn Treebank.
  • Natural Language Corpus Data Norvig, Beautiful Code (the actual data)

    Deep Learning for Natural Language Processing

  • Word embeddings: fastText, word2vec (Google TensorFlow), and GloVe (Stanford).
  • Moore's Law vs the More Law.
  • Hands-on large language models : language understanding and generation Jay Alammar, O'Reilly, 2024. (Yale library online book).
  • C4 (Colossal Clean Crawled Corpus)
  • *** Efficient Estimation of Word Representations in Vector Space the word2vec paper, by Jeff Dean and the Google guys. 2011. github with paper and code
  • *** Attention is All You Need Vaswani et al. 2017. Introduced the transformer architecture.

    LinkedIn

    Many of you may have social media accounts on Facebook, X, Instagram, and others. You may also use Github as a code repository and portfolio to show to prospective employers.

    All good. I recommend that you also create a LinkedIn account as an online resume. I invite all of my Yale students to connect to me on LinkedIn to leverage my network, such as it is. I also am interested in following your trajectory, but not in a creepy way.

    On Monday, my daughter Alexandra spoke to the class about her current work. Down the road, if you have some ideas or thoughts you would like to share with future generations of Yalies, let me know. I would be happy to welcome you back, if not for a lecture, maybe a cup of coffee.

    Farewell and Gratitude

    From a commencement address by David Foster Wallace, 2005.
    Twenty years after my own graduation, I have come gradually to understand that the liberal arts cliché about teaching you how to think is actually shorthand for a much deeper, more serious idea: learning how to think really means learning how to exercise some control over how and what you think. It means being conscious and aware enough to choose what you pay attention to and to choose how you construct meaning from experience. Because if you cannot exercise this kind of choice in adult life, you will be totally hosed.

    If you think about it, that thought could apply to LLMs as well.

    Valedictory Cavafy: Ithaka. Life is a journey. Pay attention to the ride, not just the destination. read by James Bond, sort of.

    More appropriate: Billy Collins just in time for Mother's Day, May 11th. youtube

    PS

    I mentioned Jim Meehan in the first lecture. Jim is an accomplished pianist. He and I used to perform original songs about Yale AI at CS parties. See Yale Songbook. (Pardon my LaTeX.) See Yale AI Songbook (pdf)
    [Home]