Induction Heads and In-Context Learning Mechanisms
Researchers trace in-context learning to a simple copying circuit in transformers.
Dmitri Voloshyn
Contributing Editor, Mechanistic Interpretability
Dmitri Voloshyn
Dmitri Voloshyn spent a decade as a research engineer working on compiler optimization before redirecting his technical writing toward the inner workings of neural networks, a transition he made after co-authoring an open-source toolkit for probing transformer attention heads. At LLM Risk Review he translates circuit-level and activation-analysis research for a technically literate but non-specialist audience.
2 stories
Activation-Based Hallucination Detection in Transformer Decoders
Reading hidden neuron activity reveals hallucinations before language models finish speaking.
Dmitri Voloshyn