A New Semester
Since graduating from university last year and starting work right after, I’ve done a few things to improve my knowledge of deep learning: read through research papers often and implement simpler ones infrequently, present a few papers to our company ML reading group, and attend NIPS for a week of inspirational talks and workshops. However, I think my lack of a structured or consistent plan for learning has resulted in me not really gaining a deeper understanding of this field. I’m going to try something different over the next two to three months by emulating a university semester; I’ll use three full courses freely available on the internet for studying, and I’ll also understand and implement a paper (new or old) at least every two weeks. For both of these things, I intend to write up summaries of what I learn and to post those here frequently.
Only one of the courses I’ll be taking is focused on deep learning: Phil Blunsom’s Deep Natural Language Processing. I’ve generally focused on vision and control applications of deep learning and have been wanting to learn more about its uses for NLP, so this should be interesting to go through. The second course is David MacKay’s Information Theory, Pattern Recognition and Neural Networks, which I stumbled across while looking for resources on learning bayesian inference. It isn’t entirely devoted to that topic, but MacKay was a key figure at the intersection of bayesian methods and neural networks so I’m sure there will be lots to learn from his teaching. The last course is Rachel Thomas’s Computational Linear Algebra; this is a shorter and more applied course than the other two, but I think the basic algorithms and complexities of matrix operations are important to know for a deep learning practitioner.
During my undergrad, academic deadlines were the greatest motivators to getting work done, and I hope that setting somewhat public commitments in this manner will also help motivate me to be consistent in my independent studies. I’m looking forward to sharing my progress on this over the coming weeks.