A blog post about reviewing papers and preparing papers for publication.

We start with the following premise: all papers have limitations. There is not a single paper without limitations. A method may not be generally applicable, a result may not be completely justified by the data or a theory may make restrictive assumptions. To cover all limitations would make a paper infinitely long, so we must stop somewhere.

A lot of limitations fall into the following scenario. The results or methods are presented but they could have extended them in some way. Suppose, we obtain results on a particular cell type using an immortalized cell-line. Are the results still true, if we performed the experiments on primary or patient-derived cells? If the signal from the original cells was sufficiently robust then we would hope so. However, we can not be one hundred percent sure. A similar example is a method that can be applied to a certain type of data. It may be possible to extend the method to be applied to other data types. However, this may require some new methodology. I call this flavor of limitations vertical limitations. They are vertical in the sense that they build upon an already developed result in the manuscript. For certain journals, they will require that you tackle vertical limitations by adapting the original idea or method to demonstrate broad appeal or that idea could permeate multiple fields. Most of the time, however, the premise of an approach is not to keep extending it. It works. Leave it alone. Do not ask for more. An idea done well does not need more.

There is a perpendicular concept of horizontal limitations. These are limitations that affect the conclusions or results already presented in the study. How robust are the results to small perturbations of the data? Are there some parameters of the methods that are user chosen? The user should be aware of where they should focus their efforts in choosing parameters. Do the cells need to be genetically modified or prepared in some way? The concept is that the hypothesis explored with the study is fully justified. This includes applying the correct statistical modeling, for example. Identifying an horizontal limitation does not necessarily mean that more work should be done but it would be best to acknowledge them. If it is straightforward, tackling horizontal limitations can really solidify a paper.

I have noticed that many other reviewers and students worry about vertical limitations. I think this is misguided and makes papers less robust. The higher the stack, the wider its support needs to be. The reason that people focus on these is that they are often easier to spot. Instead, we would often be better placed making a foundation solid rather than build the leaning tower of Piza. So keep this in mind when you next review or write a paper.