A day in the life of an AI researcher
Kensho AI researcher Rik Koncel-Kedziorski pulls back the curtain on the daily grind behind the breakthroughs, from morning stand-ups to late afternoon model runs.
This blog is part of our career stories series, which gives a glimpse of what it’s like to be a part of our team and share the career stories of Kenshins. Stay tuned for more articles on life at Kensho!
Artificial intelligence research seems to progress by leaps and bounds, but those great leaps are made possible by the consistent daily grind of thousands of AI researchers. What does a typical workday look like for them? In this article, I will take you through a day in my life as an AI researcher, which probably looks similar to a few or even as many as several other AI researchers’ work lives.
My job at Kensho consists of three main activities: learning, innovating, and reflecting. I will describe these in more detail below but an example of the “learning” phase of research would be reading research papers, whereas the “innovating” phase mostly involves talking to people, but sometimes requires writing math equations. I would say I spend roughly 85% of my time learning and the other 15% innovating, but that ignores the significant portion of my day spent reflecting: digesting or recuperating from the work of learning and innovating. I cannot stress enough the importance of unstructured reflection time in the psychic life of a researcher! In what follows, I will discuss these three components of the workday of an AI researcher. Of course, other researchers will follow different patterns, and what works for me won’t necessarily work for you. Take what you like and leave the rest!
Learning
The most important trait of an AI researcher is that they know a lot about AI research. Ideally, an AI researcher should have some breadth; that is, they should know at least something about a wide-ranging set of problems in multiple subdisciplines e.g. language, vision, robotics, time-series data, planning, etc. But the real value that a work-a-day AI researcher provides for the world is in their expertise in a focused area of research.
I am a language researcher (alternately known as “Natural Language Processing” or “Computational Linguistics”), and within that subdiscipline I currently primarily specialize in “Question answering,” or trying to get neural networks to learn to answer questions better. Even in this narrow scope, I see hundreds of papers come out each year that have been vetted by our illustrious, if flawed, peer review process, and hundreds more that are not peer reviewed but still very important. I need to be aware of as much of this body of research as possible, but of course it’s not possible for me as a finite being to read so many papers. Each day, I do a few things to stay up to speed with this massive volume of information:
I read at least one hard paper in detail, following references as needed to gain a deep understanding of the work. Hard papers for me are the ones that challenge my intuition about what should work, or that bring some obscure statistical method to bear on a problem I thought I had a handle on, or that take me a bit outside my field but are so impressive that I need to learn everything about what they’re doing over there so I can replicate it over here.
I quickly scan a handful of easier papers, paying special attention to their “related work” sections as it is often only here where researchers first reveal the grim truth of the limited contribution that their paper makes to the field (we must often write the introduction as a slick “sales pitch” to wow our peer reviewers and mislead journalists rather than an honest description of the work that was done).
I talk to at least one researcher working in another area to get some cross-pollination of ideas and increase my breadth of knowledge. If you are a researcher who would like to have such a conversation, drop me a line!
About once a week, I deep dive on a fundamental mathematical concept that I don’t fully understand. These topics are farther afield from the typical math I use daily, but studying almost any math seems to make me smarter in complex and diffuse ways. And of course, whenever a conference happens, I am obliged to read as many of the new papers whose preprints I had missed as I can manage. You can never read too much in this exciting and interdisciplinary line of work!
Innovating
Innovating in the world of AI involves searching for new insights into the data and math that make AI work, as well as developing new techniques for better approximating an increasing set of intelligent behaviors. It can be a slow and painful process due to its extreme difficulty. If I do my job well, I can invent something exciting and useful not only to the AI research community but to the world at large. If I’m doing the research part of my job well, I will gain more insight into what doesn’t work than what does. The process of innovation is fraught with failure. Even supposing I find easy success, I won’t know the boundaries of my contribution until I’ve met with many of the ways that it fails.
The basic ingredients for innovation for me are:
Recent learnings: I need to be well aware of what peers and industry colleagues have been doing before I can do anything they haven’t already done, hence all the reading.
“Research Code” aka hacky, badly written code: I’m a bit of a boomer, and I never got excited enough about Jupyter to completely adopt it, so I develop python in a tmux session with one window running vim and the other executing code. I need to be able to play with ML models and data in a very informal way if I’m going to discover something new.
Collaboration: Two minds are better than one. N-many minds more so! Collaboration is the most enjoyable part of innovation, and for many researchers their main vehicle i.e. professors and high ranking industry folks. For me, it’s important to balance coding and collaboration because I learn different but important things about a research problem from my collaborators and the data.
Rigor: Talk is cheap. At some point, everything that we discuss in a collaboration has to be written down very precisely. This is doubly true for code, which never has to be production quality but certainly has to be 100% correct before publication.
Reflecting
As I mentioned at the beginning, a lot of the actual learning and innovating takes place absent any direct action toward those goals. The brain seems to do some kind of “brain magic” in its own time, on its own schedule (This “brain magic” might be what other people call “thinking” but I feel that word implies too much agency). I will be washing dishes or sailing the boat on a weekend, and meanwhile my brain is busy synthesizing what was learned and innovated in the last week all on its own. Regularly, I will be struck with a powerful new insight or direction for future research while in the middle of some non-work activity and must quickly jot it down in my Notes app for future consideration. In this regard, I am not really ever “off the clock,” which does negatively impact my work/life balance (but: do what you love, and you’ll never work a day in your life).
Since this arbitrary and uncontrollable “brain magic” is a significant driver of my research progress, I do make efforts to bring it about regularly. I’ve noticed that my best insights rarely come when I’m busily engaged in some activity; engaging in light or no activity is a fertile opportunity for them. This unstructured reflection time can be 15 minutes of zoning out on the couch, or a 30-minute walk about town. Some days, the most productive working hours are those spent forwarding through the woods, fighting blackberry brambles and thorns, all the while the brain doing its subconscious computation to make sense of what I read about over the last week.
In conclusion…
For the AI researcher, determining one’s success is rather difficult due to a lack of objective standards. Some researchers use their citation count; others their lab size; still others their personal income as a measure of success. I once asked a former and wonderful advisor, Prof. Noah A. Smith, how I would know if I was doing a good job as a researcher. Noah said bluntly, “You won’t. You’ll have to decide for yourself what ‘being a good researcher’ means and assess whether you’re at least meeting your own standard.”
My vision of success seems to change year by year. Recently I have been focused on identifying how the uniqueness of my background, knowledge, interests, and “brain magic” can be best leveraged to fill the gaps in current AI research. There’s research out there that only I can do because it fits my particular kind of weirdness and all the other weirdos in the field are chasing down their own unique shadows. Discovering and developing those Rik-specific ideas is my current definition of success. Ask me again in a year and — hopefully — I will have out-grown that definition as well.
Odds and ends
The famous vagabond and prolific mathematician Paul Erdos once said, “A mathematician is a device for turning coffee into theorems,” and this is as true for AI researchers as it is for other mathematicians. I would say that a proper coffee lifestyle is crucial for my performance as a researcher, with proper sleep being a helpful but not necessary aid as well. When in doubt about how to balance these, err on the side of coffee.
Variety is the spice of life because it forces the mind to work to adjust. While I’ve outlined the basic components of my daily work, I make explicit efforts to vary things as much as possible from day to day. This can involve working from different farms and islands in the Puget Sound, studying linguistics rather than math for a week, or simply wearing a really great outfit to make the day special. A small amount of chaos is your friend!
One can learn of new and interesting research on “Research Twitter,” where all the famous researchers talk about great research they’ve done or seen. But beware: even stoic and famous researchers, when given an audience like Twitter, can become keyed up, toxic, hyper-partisan fear mongers. Personally, I try to avoid social media so as to remain a somewhat healthy and well-adjusted person.
I should mention that I work for a very fantastic company called Kensho Technologies. We use machine learning to derive insights from unstructured data in the business and financial industry. We’re hiring!