training

9 problems

• Warmup + cosine decay LR scheduleEasy
• Early stopping on validation lossEasy
• Exponential moving average of weightsEasy
• Exponential LR schedulerEasy
• Gradient checkpointing forwardEasy
• Clip gradients by global L2 normEasy
• He weight initializationEasy
• Learning-rate range testMedium
• Step LR schedulerEasy