Expressivity of Transformers: Logic, Circuits, and Formal Languages
Jon Rawski
(San Jose State University)
Registration form:
Register here
Course summary:
A major advancement in language modeling is the use of the transformer architecture. But what problems can transformers solve, what problems can they not solve, and how can we prove it? This course examines the expressivity of transformers through the lens of computability and complexity theory. We situate transformers within the landscape of automata, boolean circuits, and formal logics. We discuss what is currently known about transformers’ capabilities and limitations, address practical implications for NLP, and identify directions for future work.
About the lecturer:
Jon Rawski is Assistant Professor of Linguistics at San Jose State University. He teaches and studies the mathematics of human language and learning. His work places mathematical boundary conditions on the grammars underlying human language and how they can be learned. His work spans linguistics, theoretical computer science, and cognitive science. he holds a Phd in Linguistics from Stony Brook University.
Location and schedule:
| Wednesday, June 11 in 4420
|
| 16:15 - 17:45 | Lecture |
|
| Thursday, June 12 in 4420
|
| 14:15 - 15:45 | Lecture |
| 16:15 - 17:45 | Tutorial |
|
| Friday, June 13 in 4420
|
| 14:15 - 15:45 | Lecture |
| 16:15 - 17:45 | Tutorial |
|
