“A testament to what an earnest, incredibly hardworking technical expert can accomplish when they commit to public service… he has materially influenced many of the bold efforts that the UK has been making to invest in frontier AI Safety.” – Ian Hogarth, Chair of UK AISI
“Few people deserve more credit than Nitarshan for the UK’s enormous strides in state capacity in AI over the last 18 months: an incredibly capable and dedicated civil servant.” – Matt Clifford, Chair of ARIA
“Nitarshan is the shadowy cofounder of UK AISI and all-round cool person”
“Nitarshan is a PhD student in Cambridge, previously a student at MILA, a top machine learning lab in Canada. He is interested in topics of AI safety among others. He also has an impressive repertoire of about five things he can say in Hungarian (and maybe even more).”
“Beware of Nitarshan in general… He asks thought-provoking questions that will make you rethink your whole research agenda”
“Quite fun and interesting. Those are his parameters.”
Bookshelf
Darwin among the Machines (1863)
Progress and Poverty (1879)
The Strenuous Life (1899)
Citizenship in a Republic (1910)
Dust of Snow (1923)
Economic Possibilities for our Grandchildren (1928)
The Human Use of Human Beings (1950)
Computing Machinery and Intelligence (1950)
Can We Survive Technology? (1955)
Man-Computer Symbiosis (1960)
All Watched Over by Machines of Loving Grace (1967)
Spring Snow (1969)
See How It Flies (1995)
Sidewinder: Creative Missile Development at China Lake (1999)
Why the Future Doesn’t Need Us (2000)
The Dream Machine (2001)
Strategic Computing: DARPA and the Quest for Machine Intelligence, 1983-1993 (2002)
Underpaid Labour
GitHub | Google Scholar | Semantic Scholar | OpenReview
Open Problems in Technical AI Governance
arXiv
Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Alexandra Sasha Luccioni, Nitarshan Rajkumar, Nicolas Moës, Jeffrey Ladish, Neel Guha, Jessica Newman, Yoshua Bengio, Tobin South, Alex Pentland, Sanmi Koyejo, Mykel J Kochenderfer, Robert Trager
IDs for AI Systems
arXiv
Alan Chan, Noam Kolt, Peter Wills, Usman Anwar, Christian Schroeder de Witt, Nitarshan Rajkumar, Lewis Hammond, David Krueger, Lennart Heim, Markus Anderljung
Visibility into AI Agents
arXiv FAccT 2024
Alan Chan, Carson Ezell, Max Kaufmann, Kevin Wei, Lewis Hammond, Herbie Bradley, Emma Bluemke, Nitarshan Rajkumar, David Krueger, Noam Kolt, Lennart Heim, Markus Anderljung
Reclaiming the Digital Commons: A Public Data Trust for Training Data
arXiv AIES 2023 Blog
Alan Chan, Herbie Bradley, Nitarshan Rajkumar
Harms from Increasingly Agentic Algorithmic Systems
arXiv FAccT 2023
Too many authors tbh
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics
arXiv Code ICLR 2023 ICML 2022 Workshop (DataPerf)
Shoaib Ahmed Siddiqui, Nitarshan Rajkumar, Tegan Maharaj, David Krueger, Sara Hooker
Evaluating the Text-to-SQL Capabilities of Large Language Models
arXiv Code
Nitarshan Rajkumar, Raymond Li, Dzmitry Bahdanau
Myriad: A Real-World Testbed to Bridge Trajectory Optimization and Deep Learning
arXiv Code NeurIPS 2022
Nikolaus Howe, Simon Dufort-Labbé, Nitarshan Rajkumar, Pierre-Luc Bacon
Self-Supervision for Data Interpretability in Image Classification and Sample Efficiency in Reinforcement Learning
MSc Thesis
Nitarshan Rajkumar
Pretraining Representations for Data-Efficient Reinforcement Learning
arXiv Code NeurIPS 2021 ICLR 2021 Workshop (SSL-RL)
Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, Devon Hjelm, Phil Bachman, Aaron Courville
In Search of Robust Measures of Generalization
arXiv Code NeurIPS 2020
Karolina Dziugaite, Alex Drouin, Brady Neal, Nitarshan Rajkumar, Ethan Caballero, Linbo Wang, Ioannis Mitliagkas, Dan Roy