Let's Talk ChatGPT
DeWitt District Library
AI Progress
What really is ChatGPT?
- ChatGPT is a natural language processing tool driven by generative AI technology that allows you to have human-like conversations and much more with the chatbot.
- The language model can answer questions and assist you with tasks, such as composing emails, essays, and code.
Poem...by ChatGPT
How does it work?
What is the guessing game?
- Shannon's Game: guess the next letter
- D_____
- De____
- DeW___
- DeWi__
- DeWit_
- DeWitt
What do you need to know to win the guessing game?
Michigan State University is located in _____, Michigan.
Answer: trivia
What do you need to know to win the guessing game?
I put _____ fork down on the table.
Answer: syntax
What do you need to know to win the guessing game?
The woman walked across the street, checking for traffic over ___ shoulder.
Answer: coreference
What do you need to know to win the guessing game?
I went to the ocean to see the fish, turtles, seals, and _____.
Answer: lexical semantics/topic
What do you need to know to win the guessing game?
Overall, the value I got from the two hours watching it was the sum total of the popcorn and the drink. The movie was ___.
Answer: sentiment
What do you need to know to win the guessing game?
Iroh went into the kitchen to make some tea. Standing next to Iroh, Zuko pondered his destiny. Zuko left the ______.
Answer: some reasoning - this is harder
What do you need to know to win the guessing game?
I was thinking about the sequence that goes 1, 1, 2, 3, 5, 8, 13, 21, ____
Answer: some basic arithmetic; they don't learn the Fibonnaci sequence
ChatGPT Overview
Step 1: Large Language Model
- Trained on copious amounts of textual data: gleaned from internet text, wikipedia, books, articles, websites, and other publicly available textual information.
- So, it is learning from historically available data.
- The training data is extensive and covers a wide array of topics and writing styles to make the model versatile in its responses.
Initial Language Model: GPT Overview
Initial Language Model: GPT Training Process
Initial Language Model: GPT Language Generation
How much data are ChatBots trained with?
How much compute are ChatBots trained with?
Step 2: Aligning ChatGPT with Human Values
- To ensure responses are safe, acceptable, and unbiased, we use Human Feedback
- Human AI trainers generate conversations and also rank multiple response of the model fro quality and appropriateness.
Training ChatGPT with Human Feedback
Training ChatGPT Reward Model
Chatbot Development
How useful is current ChatGPT?
Ghost of the Internet
- If you ask the internet for the answer to a question on the best forum available and get an answer, it might be in the ballpark of as useful and as correct as that which GPT4 provides notably, in seconds.
- There are pluses and minuses:
- Super human for tasks like text summarization and style.
- Solid performance for commonly discussed content.
- Sketchy for less commonly discussed content.
- Probably should not trust its math, reasoning and logic.
- Driving a car is out of the question, since it cannot be described.
Image Generation from Text
Video Generation from Text
Imagined Dangers of AI?
- Behavior in LLMs is a reflection of our collective behavior.
- Will AI kill everyone?
- Short Term: definitely not in the next few years.
- Medium Term: possible to have very lethal drone warfare.
- Long Term: development of intelligence is inevitable, at least for competitive or capitalistic reasons.
- Will we have superintelligent beings?
- Don't think this will happen in my lifetime.
- Even if it does, it may be possible to have some control over them (e.g., through cryptography).
Drone Capability
Real Dangers of AI
- Job creation typically follows mechanization induced job losses.
- Loss of many jobs in the next couple of decades is very plausible.
- Most humans may not be able to skill up.
- Our society may not be equipped for such a future.
- Jobs that become easy with AI are the ones to disappear first.
Bias and Stereotype in Generative AI
Stereotype in Female Professions
Skin color stereotype for occupation
AI Exacerbates Societal Bias
Feb. 09, 2018
- Boulamwini and Gebru, "Gender Shades:Intersectional Accuracy Disparities in Commercial Gender Classification," FAT 2018
July 04, 2023
October 10, 2023
What to watch out for?
- Misinformation and inaccuracies
- Today's chat bots cannot consider the truth of the text they generate.
- Bias and unintentional harm. Inappropriate content
- Today's chat bots absorb the worldview and biases of their human creators.
- Divulging proprietary information
- Lack of proper citations. Source of information.
- Utilizing copyrighted material.
Inspiration or Plagiarism?
What can we do?
- Embrace the change and adapt.
- Kaizen: continuous improvement, continuous learning, continuous adaptation.
- Try to be as high on the skill (cognitive or physical) ladder as possible, such jobs are harder to automate.
- Talk with your legislators and seek action.
- At least think about potential harms and if regulations are needed.
- Bring together diverse stakeholders and design regulations.
- Prepare for new types of jobs
The dark art of "prompt engineering"
The dark art of "prompt engineering"
How to access Chatbots?
Q & A
