Choosing the Research Topic and Reading Its Literature

---

---

Choosing the Research Topic and Reading Its Literature

Are you starting with your first research project? Do you want to publish a research paper but don’t know where or how to start? A few years back, I was in a similar position: I was completely new to machine learning research and had no idea where and how to begin.

Fortunately, I had the opportunity to be a part of the machine learning and vision group (Lab1055). With the proper guidance, several setbacks, and learning along the way, I have published my research in an ICCV Workshop. The learnings and principles I developed as a member of Lab1055 have helped me pursue research and publish in vision conferences, workshops, and journals (e.g., ECCV, WACV, ECML, PRL, CVPR, etc.).

Through this mini-blog series, I want to share these learnings and principles. I hope that these will help you hack your way into research and get something fruitful from it.

In this series, you will learn how to publish novel research.

This lesson is the first in a 5-part series on How to Publish Novel Research:

1. Choosing the Research Topic and Reading Its Literature (this tutorial)
2. Ideating the Solution and Planning Experiments
3. Planning and Writing a Research Paper
4. Planning Next Steps When Things Don’t Work Out
5. Ensuring Your Research Stays Visible and General Tips

To learn how to choose your research topic and go about reading its literature (Figure 1), just keep reading.

---

Choosing Your Research Topic

Nowadays, it’s hard to think of any problem where machine learning has not been applied. Everything seems to be virtually solved. And that’s why choosing a research topic in this vast space can be a tricky and overwhelming task. Here are a few ideas to help you narrow down your search.

---

Find Something That Excites You and Is Relevant to the Community

Without a doubt, the topic should pique your interest. However, not every topic that excites you can benefit the community. As a result, one should look for something that interests them and is relevant to the community. For example, if image classification piques your interest, you should try few-shot, zero-shot, unsupervised, self-supervised, or domain-generalized classification.

These topics gain more attention in the community as they model real-world constraints on available training data, unlike traditional supervised classification, which assumes an abundance of data. One method for identifying these trending or relevant topics is to conduct a topic-by-topic analysis of recent AI conferences. For example, Figure 2 shows the top 50 keywords of ICLR 2021 submissions. It indicates that topics such as GNNs, meta-learning, few-shot, unsupervised, supervised, and robustness are more popular, relevant, and trending in the community than topics like classification, CNNs, and so on.

---

Try to Find a Domain Where the Literature Is Not Very Crowded

When the literature is crowded, trying to be novel in your research becomes challenging and competitive. Crowded literature is more than just too many papers to read; it reflects how actively the community is working or publishing on similar topics. And given the rate at which these new works are published, you are likely to be scooped by them in terms of results or novelty.

---

Work with Your Advisor or Mentors

I strongly advise you to collaborate with your mentors (PhDs, Research Scientists, and so on), as well as any advisors, to identify promising and relevant topics. Working on something that intersects with your advisor’s or mentor’s expertise can greatly increase your chances of producing novel research. With their expertise and research experience, you can receive appropriate guidance and feedback to help you improve and publish your work.

---

Avoid Compute-Extensive Projects

Note how much computation is typically required to solve the topic. Avoid those that necessitate intensive computation unavailable or unaffordable. This is to prevent any disappointment later in the project. For example, someone working on large-scale image generation should keep track of the computation required to generate higher-resolution images. This is a standard and fundamental experiment in this literature.

---

Identify Low-Hanging Fruits and Potential Gaps

Identifying low-hanging fruits in the literature is probably one of the simplest ways to choose your topic. Low-hanging fruits are research topics that are simple to work upon but go unnoticed. These could be as simple as

• Combining two research topics into one:
  • Mancini et al. (2020) are the first to combine Zero-shot learning and Domain Generalization and propose a simple curriculum-based class/domain mixup strategy to train models that generalize under both domains’ semantic shift.
  • Ganea et al. (2021) present the first incremental approach to few-shot instance segmentation: iMTFA, which learns discriminative embeddings for object instances merged into class representatives.
  • Chauhan et al. (2020) first propose studying the topic of few-shot graph classification in graph neural networks (GNNs) to recognize unseen classes, given limited labeled graph examples.
• Applying a new class of algorithms or architectures:
  • Deng et al. (2021) first present a simple and effective transformer-based framework for visual grounding. Their TransVG method outperforms state-of-the-arts that rely on a complex module with manually designed mechanisms to perform the query reasoning and multi-modal fusion.
• Proposing new benchmarks or evaluations:
  • Hendrycks et al. (2021) propose four new real-world distribution shift datasets consisting of changes in image style, image blurriness, geographic location, camera operation, etc.
  • Gulrajani and Lopez-Paz (2020) implement DomainBed, a testbed for domain generalization (DG), including seven multi-domain datasets, nine baseline algorithms, and three model selection criteria. In addition, they test existing DG methods under their settings to understand how practical these algorithms are in realistic settings.

However, keep in mind that, because these are typically easy to identify and work on, you must act quickly to be the first to propose them. Another good strategy can be to identify the drawbacks/gaps in the current pipelines of space and work upon eliminating them. This could include gaps like making pipelines compute/time efficient without sacrificing performance, robust to shifts, etc.

---

Analysis and Understanding as Research Topics

Comprehensive analysis to provide a holistic understanding of a specific space can be a research topic in and of itself. Analyzing what works best, any intriguing phenomena, trade-offs, limitations, or standardized benchmarking can help you, and the community better understand the space and identify potential gaps to address in the future. The best part is that they get a lot of attention from the community through citations and discussions.

For example,

• Naseer et al. (2021) show and analyze several intriguing properties of Vision Transformers (ViTs), like their robustness to severe occlusions, perturbations, and domain shifts; their less texture bias compared to CNNs, and superior transfer learning capabilities.
• Xian et al. (2017) discuss limitations in zero-shot learning formulations and algorithms by comparing and analyzing a significant number of the state-of-the-art methods in-depth, both in the classic zero-shot setting and the more realistic generalized zero-shot setting.
• Chen et al. (2019) perform a consistent comparative analysis of several few-shot classification algorithms. They show that deeper backbones significantly reduce the performance differences among various state-of-the-art methods. Furthermore, in a realistic cross-domain evaluation setting, baseline methods compare favorably against other state-of-the-art algorithms.

---

Identify Applications

Applying existing ideas to a relevant topic(e.g., medical images, editing, navigation, etc.) can also serve as a potential research topic. Following are examples of such papers.

• Papadopoulos et al. (2019) aim to teach a machine to make a pizza by building a generative model that mirrors an ordered set of instructions. They learn composable module operations to either add or remove a particular ingredient through GANs.
• Machiraju and Balasubramanian (2020) study the natural adversaries in the field of autonomous navigation wherein adverse weather conditions such as fog have a drastic effect on the predictions of these systems. These weather conditions can act like natural adversaries that can help test models.
• Richardson et al. (2021) propose a StyleGAN encoder able to directly encode real images into style space and show that solving translation tasks through StyleGAN significantly simplifies the training process and has better support for solving tasks without pixel-to-pixel correspondence.

---

Reading the Literature

With the exponential growth of deep learning-related publications (Figure 3), it has become necessary to devise effective strategies to deal with the pacing literature. So, now that you’ve decided which topic to work on, let’s look at some popular resources/tools for understanding the topic better, skimming through its literature, and keeping yourself updated with the ongoing research in the community.

---

Survey and Analysis Papers

If you want to understand the fundamentals, different classes of algorithms introduced, or how they compare, survey papers are probably the best place to start. They are usually simple to locate and follow. On the other hand, analytical papers can help you better understand the topic by explaining gaps, limitations, trade-offs, best strategies, intriguing results, etc.

---

GitHub Compilations

Explore GitHub to get compilations (e.g., Awesome Visual Transformers, Awesome Zero-shot Learning, Awesome Self-supervised Learning, Awesome Visual Grounding, etc.) of research papers specific to your topic. For a kickstart, read initial papers (to get fundamentals) and top papers (to see where the trend is going). They also include links to any implementations, blogs, videos, etc., and are updated regularly to reflect the most up-to-date content. You can find these compilations easily by using the search term “awesome <topic name>.”

---

Conference and Workshop Proceedings

Following the proceedings of top conferences (e.g., CVPR, NeurIPS, ICCV, ICML, ICLR, ECCV, ACL, EMNLP, KDD, etc.) is an excellent way to stay up to date on the latest research. As an example, Figure 4 ranks out various computer vision conferences by h-index. Another great way to stay up to date is to attend area-specific workshops where you can find research talks, presentations, and submissions that are more relevant to your topic. In addition, these workshops often consolidate a specific research space, allowing you to understand the current trends better.

For example, the Workshop on Meta-Learning has been a popular NeurIPS workshop focusing on advancing meta-learning methods. Another popular workshop at ICML, Uncertainty and Robustness in Deep Learning, aims to make deep neural networks more reliable. The Adversarial Machine Learning in Real-World Computer Vision Systems and Online Challenges at CVPR focuses on recent research and future directions for security in real-world machine learning and computer vision systems. Finally, 3DVR Workshop (CVPR 2021) discusses the unique challenges and opportunities in the 3D vision for robotics.

---

Online Tools and Platforms

Here are a few tools you can use to ease your search of papers relevant to your topic:

• With Connected Papers, you can build a graph of papers relevant to a particular field and discover prior or derivatives works in your field of interest. It further allows you to create a bibliography for any future use cases.
• Arxiv Sanity allows researchers to keep track of recent papers, search for papers, sort papers by similarity to any paper, see recent popular papers, add papers to a personal library, and get personalized recommendations of (new or old) Arxiv papers.
• Alpha Signal provides you with a weekly summary of research papers trending and worth reading.
• Using Google Scholar, you can add your personalized keywords, fields, researchers and get notifications whenever any paper relevant to them gets uploaded.
• Twitter, in my opinion, is by far the best place to stay up to date. If you follow the right people, research labs, and conferences, you will find a wealth of content, insights, and collaboration opportunities. The best part is that you can directly share your thoughts or ask questions with the community. Figures 5 and 6 recommend several researchers, academic, and industry labs that you can follow to create an excellent feed for yourself.

---

Reading Strategy

After you’ve read the fundamental and introductory papers, devise a reading strategy that will help you efficiently filter out the relevant content and save you several hours of reading irrelevant articles.

• Examine the abstract for a high-level summary of the work.
• Look for any introductory figures or illustrations (to get a sense of the approach).
• Skim through key results, and so on.

Papers are organized in a consistent format (Abstract, Introduction, Related Work, Methodology, Experiments, and Conclusion), making it simple to find anything specific. Aside from that, you can get a good overview by looking at their posters, spotlight videos, or blogs.

Keshav (2020) proposes a three-pass approach for reading papers.

• The first pass gives you a general idea of the paper. It involves reading the abstract, introduction, conclusions; glancing over references to decide whether the paper is relevant to you and needs any other passes. After the first pass, you should be able to answer questions about the category, context, contributions, correctness, and clarity of the work.
• The second pass lets you grasp the paper’s content by focusing on figures, illustrations, or diagrams, marking any unread references for future reading. This pass will help you see if the work can be relevant to your topic (can it be a potential baseline, related work, or even a solution to your topic). If yes, then go for a third pass to fully understand it.
• The third pass involves reading the paper carefully ​​to identify its strong and weak points. In particular, you should be able to pinpoint implicit assumptions, missing citations to relevant work, and potential issues with experimental or analytical techniques.

Consider yourself a reviewer and ask pertinent questions to ensure a thorough assessment. For example, are there any simpler methods/guidelines that the authors did not consider? Are the authors’ assumptions reasonable? Is their approach technically sound, or do they have any limitations (expensive computation, training/inference overheads, etc.)?

Think creatively to determine whether the presented idea can be extended, integrated, or have some applications. Highlight important content, thoughts, or criticism, or summarize it in a few paragraphs. This will greatly assist you when rereading the paper or writing about it in the related work section of your paper.

---

Summary

Selecting a research topic can be difficult in this vast machine/deep learning space. Not every topic that piques your interest can be turned into a successful research topic. As a result, one should look for domains relevant to the research community, align with their interests, and are not in a crowded space. Furthermore, take note of the compute typically required to solve these topics and avoid working on those where the compute is unavailable or unaffordable.

After you have narrowed down your list of topics, consult with your mentors and advisors to see if they’re particularly interested or experienced in any of them. Then, begin with them and look for any low-hanging fruits. But do not forget to act quickly if you want to be the first to claim it. Parallelly, look for any constraints, gaps, trade-offs, or intriguing phenomena that could lead to a good analysis or concerns to address.

Conducting a literature review can be a daunting task. Start with survey papers and GitHub compilations to understand the fundamentals and skim through the recent approaches. Next, follow the proceedings of top conferences and their area-specific workshops to stay updated with the ongoing research. Utilize online tools and platforms such as Twitter to obtain a curated set of content relevant to your issue. Look for blogs, posters, or spotlight videos to get a quick paper overview. Finally, but most importantly, devise a reading strategy for parsing a paper without wasting time.

I hope this lesson will assist you in narrowing your search for your research topic and efficiently dealing with its literature. Stay tuned for the next lesson on ideating for a solution planning your experiments.

---

Citation Information

Mangla, P. “Choosing the Research Topic and Reading Its Literature,” PyImageSearch, P. Chugh, R. Raha, K. Kudriavtseva, and S. Huot, 2022, https://pyimg.co/oravw

@incollection{Mangla_2022_Choosing,
  author = {Puneet Mangla},
  title = {Choosing the Research Topic and Reading Its Literature},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Ritwik Raha and Kseniia Kudriavtseva and Susan Huot},
   year = {2022},
  note = {https://pyimg.co/oravw},
}

---