Note: i wrote this on December 9 2024 but could not dare posting it because i did not want to and could not believe what just happened then. my heart still aches so much to even think about it, but i’m posting it on the last day of 2024 to remember Felix.
It was sometime early summer in 2014. I was a postdoc in Montreal under supervision of Yoshua Bengio, and Felix was a visiting student who just arrived in Montreal then.
I was struggling with building a neural machine translation system that can handle long source/target sentences, and in doing so, was trying every possible idea i could come up with (and attention wasn’t one of these, and I had to wait until Dima Bahdanau arrived at Montreal as an intern the same summer.) Among those (futile) ideas was to build a gated convolutional encoder rather than using a recurrent net based encoder. With a proper constraint on the gates, I was able to train this model and make it even somewhat interpretable. When Felix stopped by at my desk and introduced himself to me as a linguist/computer scientist, I could not wait to show him this interpretable structure uncovered by this new model. I showed him:
With an extreme level of confidence, Felix told me “Kyunghyun, syntax isn’t a thing” (with perhaps the best pronunciation of my name by a non-Korean speaker.) I could immediately tell Felix would be one of my best friends (and he indeed has been one ever since,) and this quote became one of the regulars in my slide decks over the next few years.
Between all those fun hangouts and deep philosophical (but still light-hearted and fun) conversations, Felix and I were able to do a bit of research together as well. Many interesting findings we made together aside, our biggest legacy may be a relatively weird one; creating a trend that lasted about 3-5 years since 2016, where everyone was compelled to put a gigantic table filled up with unnecessarily and unimaginably many numbers. When we were writing <Learning distributed representations of sentences from unlabelled data>, we weren’t thinking too much about it, but ended up putting two gigantic tables:
Early 2018 Felix noticed “several that include The Really Enormous Table” from the ICLR proceedings, and we could only blame ourselves. Even in serious research, we could not stop having fun together.
Fast forward almost a decade from thereon, as one of the organizers of Khipu 2023 (the AI summer school in south america), felix invited me as a speaker. he excitedly told me about things and places we’d do and visit, such as seeing a football match, and visiting Buenos Aires. When I got to Khipu in MonteVideo in March 2023, I noticed Felix hadn’t arrived yet. I was told by other organizers that he couldn’t make it due to some health issues. i didn’t know then that i would not meet him again in person:
This past June (2024) after so many years, I finally had a chance to stop by in London and messaged Felix if he’d be down for lunch. Although I knew it was a long shot, I really, really wanted to meet, chat and hang out with him; the last time we met in person was pre-pandemic and since then it had only been far-and-apart zoom calls. I also was secretly hoping to be surprised by how much he had recovered. a couple of months later (Aug 2024), Felix replied and apologized for the late reply, which was utterly and completely unnecessary. he also sent a picture of us (yes, this picture was not AI-generated but Felix-generated,) reminding both of us of the last time we hung out in London.
Last friday, I got a Whatsapp message from Douwe. I got to know Douwe and became one of the irreplaceable and best friends with him, thanks to Felix’s intro in 2014. I immediately tried to reach out to Felix. I called him on Whatsapp, and I called him on his mobile. None of those lines worked.
Felix (1984-2024), I hope you’re feeling well now and are with your mum.