Lecture Note for “NLP with Distributed Representation” on arXiv Now

On the same day I moved to NYC at the end of August, I had coffee with Hal Daume III. Among many things we talked about, I just had to ask Hal for advice on teaching, as my very first full-semester course was about to start then. One of the first questions I asked was whether he had some lectures slides all ready now that it’s been some years since he’s started teaching. 

His response was that there was no slide! No slide? I was shocked for a moment. Though, now that I think about it, most of the lectures I attended during my undergrad were in fact given as a chalkboard lecture. 
I can understand that there are many advantages in chalkboard lectures. And, most of them to students. The slow pace of chalkboard lectures likely (but not necessarily) fits better with the pace of understanding what’s going on in the lecture room, than simply flipping through slides. Also, it becomes nearly impossible for a lecturer to skip anything, as any board starts empty. 
I took this as a challenge (though, I’m sure Hal never meant it to be a challenge in the first place.) Also, I naively thought that the amount of time I need to spend in preparing 100 slides would be much larger than the amount of time I prepare for a chalkboard lecture. After all I’ve been talking about this NLP with DL over and over, and those talks successfully landed me a job. 
One advice from Hal was that it is better to keep the record or note of what I will teach or have taught so that I can reuse this note over and over. In hindsight, it was perhaps not an advice but simply his personal regret (+ a hint that I shouldn’t do chalkboard lectures..)
Sticking to this advice I decided to write a lecture note of roughly 10 pages each week. Since I cannot even remember when it was the last time I hand-wrote any text, I decided to use latex. So far so good, except that it turned out to be an amazingly time-consuming job. Writing 10 pages each week felt never so difficult before (and I used the default latex article class which has gigantic margins..) 
After about a month since the beginning of the semester, I found this amazing review article (or lecture note, I’d say) by Yoav Goldberg. Only if Yoav uploaded this to arXiv a 1.5 month earlier! The course was already more than a third way into the semester, and I couldn’t suddenly ask the students to switch from my (ongoing) lecture note to Yoav’s. Why? Two reasons: (1) my lecture note had deviated quite a far from Yoav’s and (2) my ego wouldn’t let me declare my failure at making a lecture note myself in front of the whole class.
Anyways, I continued on writing the lecture note, and this Monday had the last lecture. I thought of cleaning it up quite significantly, adding more materials and even putting some exercises, but you know.. I’m way too exhausted to do even one of them now. I decided to put the latest version, as of evening Monday, on arXiv, and it’s showed up today: http://arxiv.org/abs/1511.07916.
I must confess that this lecture note is likely to be full of errors (both major and minor.) Also, I had to skip quite many exciting, new stuffs due to time constraint (only if I had twice longer a semester! nope.) I kindly ask your understanding.. I mean, it’s been rough.
Any future plan for this lecture note? Hopefully I will convince the Center for Data Science at NYU the importance of this course, and they’ll let me teach the very same course next year. In that case I will likely clean it up more, fix all those errors, update some of the later chapters, and this time for real, add some exercise problems. Wish me luck!
Oh, right! Before finishing this post, I’d like to thank all the students and non-students who came to the lectures, and two TA’s, Kelvin and Sebastien, who’ve been awesome help.

Leave a Reply