What I learned from my MPhil

Here I listed the things I wish I had known before doing postgraduate studies. Note that I am a junior researcher and these realizations are only based on my experience (they are not meant to be a solid guide to anyone who wants to do research). Research experience varies per individual. But these are the things I would advise my former self. Feel free to email me your thoughts: “helbertpaat09” followed by “@gmail.com”

Importance of Supervisor.

I cannot underestimate the role of mentorship and guidance in research (in the same way that mentorship is valued in sports and other fields). I have heard successful stories of scientists who made great contributions through their life-changing encounter with a supervisor (some examples in mind are June Huh, Jinyoung Park, Kaiming He, Yang Song who was supervised by Stefano Ermon, Guibas Lab and other professors who were mentored by amazing labs that produce graduates who later on become professors and scientist at top AI labs). In addition, during a Fall mentorship program of the Learning Theory Alliance, I heard testimony from Prof. Christina Lee that students tend to acquire the taste of their supervisor and that her supervisor played a huge role in molding her to be an independent researcher. Great professors produce great students. The reality is that there are many supervisors who are very hands-off with their students and assume that the students are already good at being independent. Many students are surprised with this kind of setup. One should choose a supervisor who is not only passionate and expert on a field but passionate about mentorship and skills transfer as well. It is very easy to get lost and stuck in research so having an experienced supervisor who can give valuable insights and research direction is very important. It is also a must that your interests are aligned with your supervisor (or at least the supervisor has sufficiently deep knowledge about it); in other words, your advisor should be very good in the topic you want to be good at. In this way, the advice from the supervisor can help the student generate novel and impactful ideas. What I do to assess the quality of mentorship of a professor (or the culture of the research lab) is by reading the Acknowledgement portion of the PhD Thesis of some of his graduated students. For example, if you read the Acknowledgement section of the thesis of Professor Kevin Tian who did his PhD at Stanford (see this link), you will start to feel how amazing a supervisor Prof. Aaron Sidford is. Or if you read the Acknowledgement section of the PhD Thesis of Jiajin Li from CUHK regarding Prof. Anthony Man-Cho So. Sometimes, a professor would also post about his mentorship style on his website, such as the Advising Statement of Prof. Sharon Li from UW Madison, which would give potential students a hint on their supervision style.

Compare yourself not to other researchers but only to a past version of yourself.

In AI, ML, and computing, there are thousands of extremely talented students with multiple publications, amazing awards, and projects that are impactful to the community. It is very easy to get discouraged and frustrated when you put the focus of doing science to yourself and spend time comparing yourself with others. Why we do comparison, in a way, is rooted in the desire to find a way to make ourselves more reputable and esteemed. Doing science should be about the pure joy of learning and discovery and not about the learner’s honor or reputation. Richard Feynman has a related quote on this when he said the following:

I don’t like honors. I appreciate it for the work that I did, and for people who appreciate it, and I know there’s a lot of physicists who use my work, I don’t need anything else, I don’t think there’s any sense to anything else. I don’t see that it makes any point that someone in the Swedish Academy decides that this work is noble enough to receive a prize–I’ve already got the prize. The prize is the pleasure of finding the thing out, the kick in the discovery, the observation that other people use it [my work]–those are the real things, the honors are unreal to me. I don’t believe in honors, it bothers me, honors bother, honors is epaulettes, honors is uniforms.

In addition, Andrej Karpathy views comparison as detrimental. Honestly, overcoming the tendency to compare is a hard process. There are days when you feel that comparing yourself to other researchers could give you a view of where you stand. But in general, this is dangerous. Does honor really matter? Is it good to compare ourselves with the achievements of others? I must say you should only compare yourself with a previous version of yourself. Take delight in the fact that you have grown and you have become better at what you do. Find joy in knowing that you can contribute, in your small way, to the scientific community. Acknowledge that there are many students who are smarter than you. Learn to humble yourself and understand that you are just a small part of the scientific community.

Take complete ownership of your work.

I have entered the postgraduate program thinking that my supervisor or collaborators should always give me concrete steps in order to progress in my research topic. I realize that this is not the case. One should learn to be independent early on the postgraduate journey and learn to define the necessary next steps of the research on his own. The dependence on the mentor decreases over time and research independence is learned through the years. Of course your supervisor’s comments and suggestions matter in the overall direction of your research. But since you are far more familiar with the technical details, concepts, and intricacies of your ideas, experiments and results, learn to take the lead and be reasonable in your steps. In a way, doing research is similar to building your own startup company. As the founder, learn to fall in love with the problem. Find problems that have never been addressed before but are very common and widely applicable. Devise solutions to them from a first principles point of view. Your supervisor is there as a guide to give you insights and perspectives from a broader point of view, but you still are the builder of that startup.

Consistency and hard work can never be underestimated.

There are many days when it is very difficult to progress and move forward, especially in the 1st or 2nd year of research. This often happens when you are uncertain about the future outcome of your experiments or whether the idea will work out. Sometimes you feel the resistance to work and get discouraged. Always remember that even if good results are not produced, it is important to be consistent and believe that in the end, the hard work and consistency will pay off. It is important to enjoy the research process itself, the moments of successful implementation or failed results. Remember that even in a failed experiment or failed attempt to derive a better method, you always learn something new. And you can use this failure to generate new insights and design new experiments. I guess what really matters is to diligently keep going and believe that in the end you will get something meaningful. I like the quote from a CMU Professor I look up to (he usually produces professors in top universities) about hard work. Professor Thuomas Sandholm said this:

While an individual makes research progress in spurts with slower periods in between, in the long run (2-5 year horizon), success does come to those who work hardest.

Greatest writers are the greatest thinkers.

I can never underestimate the value of writing. Research made me love writing, not only for expression or communication of ideas but as a way to edit your own thoughts and clarify your own ideas. When you carefully craft a statement to describe a method or a problem, you begin to think and see it from a different point of view as well. My ideas in research usually come through writing. I write about my experimental steps and results all the time. It gives me a sense of clarity of what I am doing and whether it makes sense. I take confidence in the fact that my writing will lead me to good ideas. Writing makes the process of learning and doing research enjoyable. I like this statement from the computer scientist Paul Graham:

A good writer doesn't just think, and then write down what he thought, as a sort of transcript. A good writer will almost always discover new things in the process of writing. And there is, as far as I know, no substitute for this kind of discovery. Talking about your ideas with other people is a good way to develop them. But even after doing this, you'll find you still discover new things when you sit down to write. There is a kind of thinking that can only be done by writing.

I also like this quote from Professor Fei Fei Li:

Review process is highly random. But there is one golden rule that withstands the test of time and randomness — badly written papers get bad reviews. Period. It doesn’t matter if the idea is good, result is good, citations are good. Not at all. Writing is critical — and this is ironic because engineers are the worst trained writers among all disciplines in a university. You need to discipline yourself: leave time for writing, think deeply about writing, and write it over and over again till it’s as polished as you can think of.

Resistance hinders us from progressing.

We have the tendency to overthink things. And sometimes we resist doing the next task as we think we should be fully prepared before going to the next task. I can remember many times when I had trouble moving forward because I think I should do the current task perfectly. In the end, I only end up resisting doing the next tasks. A very relevant phrase for this is the phrase from the movie Top Gun: Maverick.

Don’t think, just do.

There are parts of research where we should stop overthinking and focus on finishing the execution. Once the execution is done, we can proceed with a retrospective assessment of what we just implemented.

Coursework is important, but only to a certain degree.

In doing postgraduate studies, you take the required courses. But one should note that it is enough to get a passing grade in these courses (without spending so much time to get an A+). What matters more is the research project you’re doing and the progress you make in that project. I agree with Andrej Karpathy when he said that

Undergrads tend to have tunnel vision about their classes. They want to get good grades, etc. The crucial fact to realize is that no one will care about your grades, unless they are bad.

Then he added that

Getting actual, real-world experience, working on real code base, projects or problems outside of silly course exercises is extremely important.

This is also applicable to postgraduate students.

Spend 20% time for breadth and 80% time for depth.

The percentage is not a hard rule. But I hope you get the point: you should spend far more time getting deeper on a specific topic than trying to be knowledgeable on a lot of related things. Let us be realistic. 4-6 years of PhD is not enough to be so good at many things. It is only enough to delve deep at a very specific topic during your PhD. While it is tempting to want to be all-knowing, this is counterproductive in research. What matters is to be good at one thing and be really great at it. This is related to what the book Zero to One defines as a definite person:

Instead of pursuing many-sided mediocrity and calling it “well-roundedness,” a definite person determines the one best thing to do and then does it. Instead of working tirelessly to make herself indistinguishable, she strives to be great at something substantive—to be a monopoly of one.

Spending 8 hours per day for research is enough.

But make sure that the time is undistracted, intensely focused time. Ofcourse if you can work longer, then that is better (most especially if you want to work in the academe and want to publish many papers). But make sure that you work without distraction and complete attention is given to the task at hand. See this guide from an MIT Professor Chara Podimata on her advising statement for her PhD students. If you work consistently giving 8 hours a day with intense focus, the progress will compile in weeks and months. Then you don’t realize you’ve finished the project by just being consistent and undistracted.

Breaking down a big problem into smaller ones.

This strategy will ensure that you work continuously on a subtask that is not so hard but at the same time not so easy. I am inspired by the quote from the mathematician George Polya who said that

If you can't solve a problem, then there is an easier problem you can solve: find it.

This is very useful to almost any task of a researcher or any problem he deals with. If you find understanding a 10-page purely theoretical paper very challenging, then start by focusing first on a single paragraph only, allot sufficient time to it and make sure to comprehend it well. If you find writing code for a project a bit daunting, start from writing code for a small aspect of it. If you find studying an entire codebase implementation of a research paper really overwhelming, start from the simplest thing you can do such as reading the README file of it or making sure you understand the first n lines of code. This is very useful in almost anything. If you can break down a big problem into smaller portions, you can ensure that you will work continuously on small steps at a time, thereby ensuring progress. After finishing some significant amount of small subtasks or subproblems, it always surprises me to know that I am already almost done with the overall big problem at hand. But isn’t this approach too slow? It can be slow. But note that it is okay to be slow and choose to focus on smaller problems. In research, being slow but deep is more important than being fast but shallow in your understanding of the task and problem. Regarding slowness, I am always inspired by how people called Fields Medalist June Huh slow and wouldn't pass a qualifying exam. Being slow can be a strength most especially if it means solving small subproblems leading to being deeper in a subject.

Avoid multitasking.

I am an easily distracted person. It is very easy for me to have a lot of random thoughts in mind and jump from one task to another, sometimes unconsciously. But I realize that there is more joy and productivity when we avoid multitasking and focus on a single task at a time. What helps me with this is journaling my experiments using the Obsidian app. During a project, I write the steps I am currently doing for an experiment. By being aware of the current step that I need to finish, my brain is reminded that at this point in time, this is what I need to do.

Embrace boredom.

In a modern age where we are surrounded by things that steal our attention, real strength is in those who can enjoy boredom. Doing research is not always fun. In fact, there may be so many elements of it that are boring for a normal person such as reading papers all the time, reading someone else’s code, debugging a software, or writing a research paper. If your mind is exposed to dopamine-intense activities such as watching Youtube videos or social media exposure, it may be difficult to enjoy the boredom of the basic tasks necessary to progress in research. See also this link of an article that connects boredom to the creativity of great thinkers and writers.