Interpreting Behaviour Emergence in Large Language Models via Training Dynamics

NLP has undergone major advancements in recent years, and there are no signs of this
trend slowing down. The key to these improvements has been scale: LLMs are growing
larger, training for longer, and being fed more data. While this has unlocked groundbreaking performance
in numerous domains, it has also rendered the inner workings of LLMs much…