Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, Ivan Titov

Journal: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Publisher: Association for Computational Linguistics

Citations are the number of DOI-registered works in Crossref that cite this paper; references are how many works it cites. Full text is on the publisher site via the DOI link.