$ whomai

• "Principal Data Scientist/Engineer" at Slido (now part of Cisco)

$ whomai

• "Principal Data Scientist/Engineer" at Slido (now part of Cisco)

• Lecturer at Matfyz (ML, NLP)

$ whomai

• "Principal Data Scientist/Engineer" at Slido (now part of Cisco)

• Lecturer at Matfyz (ML, NLP)

• RoboCupJunior Exec

Short history of Neural Network approaches to sequence processing

• 2001: Neural Language Models
• 2013: Word Embeddings
• 2014: Sequence-to-Sequence models
• 2015: Attention
• 2016: Neural Machine Translation boom
• 2017: Transformers
• 2018: Pretrained Contextualized Word Embeddings (ELMo)
• 2019: Massive Transformer Models (BERT, GPT-2, ...)
• 2020: GPT-3
• 2021: Large Language Models trained on Code (Codex)
• 2022: ChatGPT?
• 2023+: Current Frontiers

Short history of Neural Network approaches to sequence processing

• 2001: Neural Language Models
• 2013: Word Embeddings
• 2014: Sequence-to-Sequence models
• 2015: Attention *
• 2016: Neural Machine Translation boom
• 2017: Transformers *
• 2018: Pretrained Contextualized Word Embeddings (ELMo)
• 2019: Massive Transformer Models (BERT, GPT-2, ...)
• 2020: GPT-3 *
• 2021: Large Language Models trained on Code (Codex)
• 2022: ChatGPT? *
• 2023+: Current Frontiers

* Our today's (loose) agenda

History of Deep Learning Milestones

From Deep Learning State of the Art (2020) by Lex Fridman at MIT

Attention visualized

See nice demo at https://distill.pub/2016/augmented-rnns/

What's Self Attention?

The animal didn't cross the street because it was too tired.

What does "it" refer to?

What's Self Attention?

The animal didn't cross the street because it was too tired.

What does "it" refer to?

Self Attention mechanics

Multi-headed Self Attention

The full Transformer seq2seq process I

The full Transformer seq2seq process II

Intermezzo: implement it yourself

To actually understand what's going on, there is no better approach.

Intermezzo: implement it yourself

To actually understand what's going on, there is no better approach.

• A walkthorugh of the Transformer architecture

Intermezzo: implement it yourself

To actually understand what's going on, there is no better approach.

• A walkthorugh of the Transformer architecture

• The Annotated Transformer

Intermezzo: implement it yourself

To actually understand what's going on, there is no better approach.

• A walkthorugh of the Transformer architecture

• The Annotated Transformer

• Transformers from scratch [video]

Big Transformers Wins: GPT-2

Try it yourself at https://transformer.huggingface.co/doc/gpt2-large

Big Transformer Wins: Huggingface transformers

• A very nicely done library that allows anyone with some Python knowledge to play with pretrained state-of-the-art models (more in the docs).

Big Transformer Wins: Huggingface transformers II

• A small example: English to Slovak translator in about 3 lines of Python code: *

from transformers import pipeline
en_sk_translator = pipeline("translation_en_to_sk")
print(en_sk_translator("When will this presentation end ?"))

Works with many other languages as well -- the full list is here

Attention and Transformers: Recap

Attention and Transformers: Recap

• Attention was a fix for sequence models that did not really work too wel

Attention and Transformers: Recap

• Attention was a fix for sequence models that did not really work too wel

• It turned out it was all that was needed for (bounded) sequence processing

Attention and Transformers: Recap

• Attention was a fix for sequence models that did not really work too wel

• It turned out it was all that was needed for (bounded) sequence processing

• Transformer is an encoder-decoder architecture that is "all the rage" now

Attention and Transformers: Recap

• Attention was a fix for sequence models that did not really work too wel

• It turned out it was all that was needed for (bounded) sequence processing

• Transformer is an encoder-decoder architecture that is "all the rage" now

• It has no time-depencency due to self-attention and is therefore easy to paralelize

Attention and Transformers: Recap

• Attention was a fix for sequence models that did not really work too wel

• It turned out it was all that was needed for (bounded) sequence processing

• Transformer is an encoder-decoder architecture that is "all the rage" now

• It has no time-depencency due to self-attention and is therefore easy to paralelize

• Well known models like BERT and GPT-* took the world of NLP by storm

Attention and Transformers: Recap

• Attention was a fix for sequence models that did not really work too wel

• It turned out it was all that was needed for (bounded) sequence processing

• Transformer is an encoder-decoder architecture that is "all the rage" now

• It has no time-depencency due to self-attention and is therefore easy to paralelize

• Well known models like BERT and GPT-* took the world of NLP by storm

• Very helpful in many tasks, easy to play with thanks to the Huggingface transformers library

GPT-2 vs GPT-3

GPT-2 vs GPT-3

GPT3

• Basically the same architecture as GPT2

GPT3

• Basically the same architecture as GPT2

• The sheer size is astounding (power-law of model/dataset/computation size)

GPT3

• Basically the same architecture as GPT2

• The sheer size is astounding (power-law of model/dataset/computation size)

• It would take 355 years of Tesla V100 GPU time to train

• The training would cost about $4.6M at retail prices to train

GPT3

• Basically the same architecture as GPT2

• The sheer size is astounding (power-law of model/dataset/computation size)

• It would take 355 years of Tesla V100 GPU time to train

• The training would cost about $4.6M at retail prices to train

• It was so expensive to train they didn't even fix the bugs they themselves found:

GPT3

• Basically the same architecture as GPT2

• The sheer size is astounding (power-law of model/dataset/computation size)

• It would take 355 years of Tesla V100 GPU time to train

• The training would cost about $4.6M at retail prices to train

• It was so expensive to train they didn't even fix the bugs they themselves found:

Language Models Are Few-Shot Learners

Q: Ok, so is ChatGPT simply a wrapper around GPT-3?

• Very short A: It depends

Q: Ok, so is ChatGPT simply a wrapper around GPT-3?

• Very short A: It depends

• Short A: It depends on who you ask

Q: Ok, so is ChatGPT simply a wrapper around GPT-3?

• Very short A: It depends

• Short A: It depends on who you ask

• A: It depends on who you ask. OpenAI's Docs probably wouldn't agree.

Q: Ok, so is ChatGPT simply a wrapper around GPT-3?

• Very short A: It depends

• Short A: It depends on who you ask

• A: It depends on who you ask. OpenAI's Docs probably wouldn't agree.

• Actual A: We don't really know. It's behind an API, we don't really have ways of proving this one way or the other.

The "potential" ChatGPT training procedure

InstructGPT: Training language models to follow instructions with human feedback (2022)

Supervised FineTuning (SFT) Model

• The compilation of prompts from the OpenAI API and hand-written by labelers resulted in 13,000 input / output samples to leverage for the supervised model.

Reward Model Training

Reward Model Training II

Illustrating Reinforcement Learning from Human Feedback (RLHF)

Fine-tuning with RL

Illustrating Reinforcement Learning from Human Feedback (RLHF)

The "potential" ChatGPT training procedure

InstructGPT: Training language models to follow instructions with human feedback (2022)

InstructGPT: Summary

• The outputs generated by a small (1.3B) InstructGPT model were prefered to those of GPT3

InstructGPT: Summary

• The outputs generated by a small (1.3B) InstructGPT model were prefered to those of GPT3

• The rewards model was also "rather small" (6B)

InstructGPT: Summary

• The outputs generated by a small (1.3B) InstructGPT model were prefered to those of GPT3

• The rewards model was also "rather small" (6B)

• We don't know how large the model behind ChatGPT is, but chances are it's this "small"

https://old.reddit.com/r/ChatGPT/comments/117gtom/my_friend_is_in_university_and_taking_a_history/

https://mobile.twitter.com/goodside/status/1598253337400717313

https://mobile.twitter.com/goodside/status/1599873570431434752

https://www.linkedin.com/posts/chatgpt-generative-ai_the-art-of-chatgpt-prompting-a-guide-to-activity-7036549105573060608-Xy1O/

https://www.jailbreakchat.com/

https://greshake.github.io/

https://www.linkedin.com/posts/petercotton_timeseries-prediction-activity-7036475253652381696-4u1

https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/

Three rules for using (things like) ChatGPT

1. Let it do things you will do a manual check on anyway

2. Have it draft things you'll rewrite anyway

3. Assume the first response will be far from final

Inspired by https://vickiboykis.com/2023/02/26/what-should-you-use-chatgpt-for/

https://epochai.org/blog/compute-trends