Impact of research

By Aarnout van Delden

Three recent events inspired me to think about how we determine the impact of our work as scientists. First: the assessment of research activities in the past six years at the Departments of Physics and Chemistry at Utrecht 木瓜福利影视 (IMAU is part of the Department of Physics). Second: the third year bachelor course on Turbulence in Fluids, which I taught in February and March this year, together with my colleagues Henk Dijkstra and Anna von der Heydt. Third: the silver anniversary of Meteorologica, the Dutch language journal of meteorology.

Figure 1. Cover page of the 100th issue of Meteorologica.

Between the year of its foundation in 1992 and 1997 I was chief editor of Meteorologica. In those early years of my career as a university teacher and researcher, I spent a significant fraction of my working time on editing and writing articles for Meteorologica. I am happy and very proud that we managed to create a lively communication platform for the Dutch meteorological community, of which a significant fraction works as operational meteorologist. Nevertheless, I sometimes think that I should have devoted this time to doing 鈥渞eal research鈥, for which I would probably have received more credit than for the editing and writing of semi-popular articles in a language that is understood by only about 25 million Dutch speaking people, principally living in the Netherlands and Belgium, of which only about 1000 represent potential readers.

Faculty members receive credit for their research principally if they are author on many papers, which are cited many times within a few years after the publication, especially if these papers are published in well-known so-called high-impact journals, such as Nature and Science. To my surprise the list of journals with the highest impact in the domain of atmospheric sciences is at present headed by the Bulletin of the American Meteorological Society, the Journal of Climate and Climate Dynamics. I am not under the impression that these journals have published any articles yet with a long-lasting influence. In the case of the Journal of Climate and Climate Dynamics this is of course hardly possible, because these journals have existed for only 29 and 31 years, respectively.

What then determines this strong apparent reputation of the Journal of Climate and Climate Dynamics? This has to do both with current interest in climate change and with the definition of 鈥渏ournal impact factor鈥. According to the generally accepted definition, the 鈥渋mpact factor鈥 of a journal, in any given year, is the number of citations received in that year by articles published in that journal during the two preceding years, divided by the total number of articles published in that journal during the two preceding years. A period of two years to evaluate the impact of a scientific paper seems very short. In order to generate this type of short-term impact, a scientist is forced to apply the tricks of a journalist when choosing a topic of research and when writing a paper.

In the year 2014 the Thomson Reuter/Web of Science archive of scientific papers contained nearly 58 million papers [1]. Only 148 of these papers received more than 10 thousand citations, among them a paper written by Edward Norton Lorenz, published in the Journal of Atmospheric Sciences in 1963 [2]. This paper revealed for the first time that a numerical weather prediction model has 鈥渃haotic鈥 solutions, which depend sensitively on initial conditions. The remarkable implication of this finding is that the weather is not predictable beyond a certain time-horizon, which was estimated by Lorenz at about two weeks. In my lectures on Turbulence in Fluids I focused on the physics and mathematics of the model that Lorenz used to illustrate this message. Just out of interest, I performed a citation history analysis of Lorenz (1963), based on Google Scholar, which revealed that this article was cited only 6 times within a time-frame of two years after publication (figure 2). At the time, the atmospheric science research community, which was highly committed to building numerical models for weather- and climate-prediction, was not very receptive to Lorenz鈥檚 inconvenient message, which is why Lorenz鈥檚 article has hardly contributed to the 鈥渋mpact factor鈥 of the Journal of Atmospheric Sciences. However, when after 1975 the interest in chaos in nonlinear systems spread to all disciplines in the exact sciences and even further into the social sciences, the number of citations to Lorenz鈥檚 paper increased explosively (figure 2).

Next, I extracted the citation history of two other papers, which are early examples of the type of research that seems to dominate our field nowadays. The first paper, due to Wallace Broecker and published in Science in 1975 [2], gave a timely warning of upcoming global warming due to human induced increasing CO2-concentrations. Before 1970 the globe had experienced a few decades of global cooling, presumably due to atmospheric pollution with aerosols. Broecker鈥檚 climate prediction, which extended to the year 2010, turned out to be very accurate, a remarkable achievement, which was rewarded with a revival of references to his paper after 2010 (figure 2). The second paper, due to Syukuro Manabe and Jerry Mahlman and published in the Journal of the Atmospheric Sciences in 1976, represented a scientific breakthrough because it described the first successful climate model simulation of the seasonal cycle of the atmospheric general circulation including the seasonal zonal wind reversals in the stratosphere. Probably this paper was interpreted, not as a breakthrough, but more as a progress-report of an ongoing research project, concerned with developing a climate model, which is as faithful to reality as possible. Citations to this paper practically dropped off to zero after 1985. Nonetheless, both papers had significantly more short-term impact than the Lorenz (1963)-paper (figure 2).

Figure 2. Total number of citations (cumulative) to Lorenz (1963) [2], to Broecker (1975) [3] and to Manabe and Mahlman (1976) [4]. The vertical axis is logarithmic.

Finally, in an attempt to identify journals with a high long-term reputation, I constructed a list of 鈥渂reak-through鈥 papers that might form the backbone literature of a graduate course on the dynamics of the atmosphere, if I would not use a textbook, lecture notes or review papers. This admittedly incomplete list, which is my personal choice, but nonetheless is based on 25 years of experience in giving lectures in this field, consists of 42 鈥渃lassical鈥 papers, which were published between 1901 and 1991. From this list, which can be found on my website  [5], it appears that most 鈥渃lassical鈥 papers were published in the Journal of the Atmospheric Sciences (11) and in Quarterly Journal of the Royal Meteorological Society (8). Hence these two journals seem to be the journals with the highest long-term reputation in atmospheric sciences.

The accumulated number of 鈥渃lassical鈥 papers between 1900 and 1990 is plotted as a function of the year in figure 3. It is interesting to note that more than half of these 42 papers were published within the following three relatively short periods of time: 1947-1950 (7 papers in 4 years), 1961- 1964 (6 papers in 4 years) and 1978-1983 (9 papers in 6 years).

What happened after 1990? Climate and Global Warming came into the spotlight, while Atmospheric Dynamics went into the wings of the stage. Since 1990 great progress has been made in climate model development, remote sensing from satellites and data assimilation. All this has led to significant advances in deterministic weather prediction, which is now nibbling at Lorenz鈥檚 time horizon of 2 weeks. These advances have also produced an accurate and very valuable re-analysis of the state of the atmosphere over the past century, which is now under intense scrutiny. IMAU is participating actively in these developments.

Figure 3. Accumulation of knowledge in the domain of Atmospheric Dynamics in the twentieth century in terms of the number of 鈥渃lassical鈥 papers published after the year 1900, with an indication of the topic.