John Sumpter (left) and Peter Crosthwaite (right) advocate for full disclosure of a study’s limitations at publication. Photo: COURTESY OF JOHN SUMPTER AND PETER CROSTHWAITE

---

“However, this study also had some limitations…”—this familiar phrase often appears not only in peer review comments or retrospective literature reviews, but also, increasingly, in the main body of research papers themselves. Rather than waiting for others to point out shortcomings, many authors now take the initiative to analyze the limitations of their own work. 

What kind of relationship exists between this self-disclosure of limitations and external oversight? What ripple effects might such openness have on the broader scientific enterprise? And how can researchers present their limitations more clearly and constructively? Recently, John Sumpter, honorary emeritus professor in the Department of Life Sciences at Brunel University London in the United Kingdom and author of the article “A ‘Limitations’ Section Should Be Mandatory in All Scientific Papers,” and Peter Crosthwaite, deputy head of the School of Languages and Cultures at the University of Queensland in Australia and editor-in-chief of Australian Review of Applied Linguistics, shared their insights on these questions in interviews with CSST. 

The power of interpretation 

CSST: In your view, what constitutes a well-written limitations section? How is it positioned within the overall structure of academic writing and interconnected with other sections? 

Sumpter: I consider that it needs to be honest. It needs to admit that the study had limitations and tell readers what those limitations were—all of them, not some of them—so that readers can assess the study appropriately and correctly and hence put it into context. There could be merit in also mentioning the main limitations of a study at the end of the abstract. This is because often readers read only the abstract of a paper. 

Crosthwaite: The limitations section is where authors highlight issues to appear self-critical and humble while showcasing scientific rigor and expertise. This critical part-genre requires addressing caveats about findings, methods, or claims, while justifying choices and minimizing the impact of limitations on the study’s validity. Crafting an effective limitations section demands skillful use of specialized knowledge and a balanced authorial stance to persuade the research community to view shortcomings favorably. A well-written limitations section addresses obvious methodological flaws (e.g., non-random sampling) and contextual constraints (e.g., cultural specificity), which peers are likely to scrutinize. 

In terms of structure, the introduction section frames the research problem and justifies the study’s approach. A strong limitations section then revisits these choices, explaining how methodological or scope constraints (e.g., sample size, context) align with the study’s aims but may limit generalizability. For example, if the introduction emphasizes a novel method, the limitations section might acknowledge its exploratory nature, preempting peer reviewers’ concerns about untested reliability. The discussion section interprets the study’s findings, and the limitations section then qualifies these interpretations by addressing potential weaknesses (e.g., confounding variables). By doing so, it prevents peers from overstating the study’s flaws, as authors can frame limitations as deliberate trade-offs rather than oversights. In the conclusion section, any added limitations bridge toward future research by identifying gaps that peers might otherwise highlight. For instance, acknowledging a small sample size can segue into a call for larger-scale studies, showing foresight and aligning with peer expectations for iterative science. 

Weight of worry 

CSST: If researchers are aware of the limitations, what may prevent them from addressing the issues of concern this time? As a scholar, how do you deal with the limitations section in your own studies? 

Sumpter: Researchers do not mention the limitations of their studies for two reasons. One is because they are unaware of them. In this case, reviewers should be asked to point them out, thus allowing authors to include those limitations in the final, published copy of their paper. But the main reason is that they do not want to undermine or negate their study by admitting that it has limitations. They want to present their study as providing robust, repeatable results. They do not want to suggest that the study may be limited in scope, or even have problems; definitely not major ones. They think that if they admit that their study is not as strong as it could have been, and possibly should have been, then editors may not want to publish it. 

If being honest about limitations means editors are less likely to publish a study, I do not consider that a significant risk. Instead, I consider it a way of helping researchers improve their future research. Transparent disclosure of limitations should be considered as a positive development. It enables future studies to address those limitations, for example, with better-designed follow-up studies. Addressing limitations might even open up entirely new, important, and exciting areas for subsequent research. 

As I have become older, more experienced as a researcher, and possibly wiser, I have progressively become more aware that essentially all studies have limitations. Thus, my more recent papers are likely to have been more open and honest about the limitations of my studies than my earlier papers were. But I now wish that I had added a specific limitations section to every one of my papers. Providing a limitations section would be extremely beneficial. It would add honesty and integrity to a publication. Hence, it would greatly increase the trust and confidence that readers and the public in general would have in the results and conclusions being presented—it would help put them in context. And, of course, if the public had greater trust and respect for the results and statements in publications, society in general would be much better informed; people would have much better understanding of issues, and hence be able to form more sensible, balanced opinions on those issues. 

Crosthwaite: It is well-known that most scholars early on in their career are under immense pressure to publish, creating ripe conditions for them to engage in questionable research practices (see Plonsky et al., 2024, for how prevalent this is in the applied linguistics discipline and all other disciplines). In one or two of my earlier works, there may have been a degree of “cat and mouse” in terms of the information I may have provided about my study’s limitations, and the information I may have left out in order to get the paper through the review process, although most of these will have been picked up at review. 

In not fully addressing the limitations of a study, authors are likely intentionally choosing to withhold this information to increase their chances of publication. While peer review may well reveal the limitations of reviewed studies, the reviewer has to do this work rather than the authors. Authors who fail to self-disclose risk appear less credible, as peers may perceive omissions as intentional. Conversely, thorough self-disclosure can better align with peer critiques, reinforcing the study’s transparency and reducing the likelihood of rejection.   

Promoting scientific integrity through self-criticism 

CSST: Do you deem the issue of providing a limitations section relevant to the wider challenges of the replication crisis and trust in science? 

Crosthwaite: The appropriate discussion of a study’s limitations serves to enhance transparency and reproducibility around the original work and leads to opportunities to resolve these limitations in future research. 

The answer to this is tied in with the larger issues behind the competitive world of academic publishing. According to Isbell et al. (2022) and their work on questionable research practices in applied linguistics research, these practices include inappropriate selection of statistical methods to provide more favorable outcomes, withholding methodological details, and choosing to exclude findings in a manuscript that contradict previous research or the research of others the authors align with. 

Sumpter: If a study finds inconclusive results but the topic is considered important, there are ways of addressing this problem. When I realize that a study is not as robust as it could be, I regularly put the word ‘preliminary’ in the title of the study, usually as the very first word, thus indicating that the study’s results might not stand up to subsequent scrutiny. However, whether preliminary or not, I think that all my published research, including the more innovative papers, turned out to be repeatable; that is, the conclusions were correct. The published results have been robust and hence independently repeatable. 

A limitations section should be mandatory in all published research papers. By that I mean in all disciplines, not just my own. Journals should insist on their inclusion. But, as I found out when I approached journals, they are quite reluctant to do so despite loving the idea. As with authors, editors may not want to admit that the research their journals publish has limitations because that could be considered to undermine the quality of the published research. Yet they know that all the studies their journals publish have limitations—some minor, some major. What gets published is hence what reaches society in general, often via the media. We would have a better society if there were more openness and honesty at all levels. 

Joint efforts for common goal 

CSST: What has been done and what needs to be done to empower academic scholars and institutions to optimize the presentation and communication of scientific limitations? 

Crosthwaite: The general increase in competence and knowledge of research methodologies in our field over the last decade (the methodological turn in applied linguistics) has led to a much better sense of awareness of the limitations of research studies that typically are picked up at review by good reviewers in good journals. In that sense, the system is ‘working’ to an extent. 

One area that can help is the idea of pre-registration of studies prior to the study being carried out. This involves a proposal being sent to a journal about a proposed study, which is then reviewed for limitations that can be addressed from the outset. However, these can be burdensome, so there is some debate as to whether this is the right way to go. Other options involve open peer review systems where reviewers’ comments on limitations can be seen publicly, preventing superficial reviews by biased or inattentive reviewers. 

It is hard for any field to police itself. Questionable research practices exist in every area of research, both within and outside of higher education. By recognizing this situation and taking active steps to resolve it, we can work to regain trust in the work that we do. But it will take all of us to do so. 

Edited by LIAN ZHIXIAN