Prof by Lex AI Labs — home
Profby Lex AI
EARLY ACCESS

Shipping daily.

Last shippedMAY 20

New curriculum map — see how all 7 courses build on each other, search any lesson with /.

View what’s new→
COMING NEXT
  • Adaptive quizzes
  • Tutor v2.0
  • More company interview guides
  • Voting board for content
Request a feature or report a bug→
CurriculumPracticeFor OrganizationsPricing
Sign InStart Learning
FoundationsVisionSequencesTransformersAgentic AISystem DesignPracticeInterview

Transformers

Attention Is All You Need0/11
  • Problem with RNNs and LSTMs
  • Positional embeddings & positional encodings
  • Attention
  • Self-attention
  • Multi-headed attention
  • Cross-attention
  • The encoder stack
  • Encoder–decoder transformer
  • From encoder–decoder to GPT
  • Interview Readiness, Attention Is All You Need
Build and Train Your Own GPT-20/12
  • Problem with RNNs and LSTMs
  • Token Embeddings
  • Positional Embeddings / Positional Encodings
  • Attention + Multi-Head Attention
  • Causal Masking
  • Residual Connections
  • Layer Normalization
  • Feed Forward Neural Networks
  • Generation of Next Tokens
  • Decoder Only Transformer
  • Interview Readiness, Build and Train Your Own GPT-2
Courses/Transformers/Build and Train Your Own GPT-2/Decoder Only Transformer
Reading progress

0% read

Course progress
0 / 23 lessons

Transformers

On this page

No sections yet.

Up next

Interview Readiness, Build and Train Your Own GPT-2

Continue