epiH Search

CLOSE



Epidemiol Health > Volume 33; 2011 > Article
Poorolajal, Cheraghi, Irani, and Rezaeian: Quality of Cohort Studies Reporting Post the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement

Abstract

The quality of reporting of cohort studies published in the most prestigious scientific medical journals was investigated to indicate to what extent the items in the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist are addressed. Six top scientific medical journals with high impact factor were selected including New England Journal of Medicine, Journal of the American Medical Association, Lancet, British Medical Journal, Archive of Internal Medicine, and Canadian Medical Association Journal. Ten cohort studies published in 2010 were selected randomly from each journal. The percentage of items in the STROBE checklist that were addressed in each study was investigated. The total percentage of items addressed by these studies was 69.3 (95% confidence interval: 59.6 to 79.0). We concluded that reporting of cohort studies published in the most prestigious scientific medical journals is not clear enough yet. The reporting of other types of observational studies such as case-control and cross-sectional studies particularly those being published in less prestigious journals expected to be much more imprecise.

INTRODUCTION

Randomized trials are generally considered the gold standard for evaluating both the efficacy and the potential side effects of new therapeutic or preventive interventions in both clinical medicine and public health [1]. However, much of the biomedical research into the cause of diseases comes from observational studies [2]. The results of these studies should be reported as transparently as possible "so that readers can follow what was planned, what was done, what was found, and what conclusions were drawn" [3]. Unfortunately, reporting of observational research is neither precise nor clear enough to assess the strengths and weaknesses of the evidence [4, 5].
To improve the reporting of observational research, a group of experts developed a checklist of items known as the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement [6]. The STROBE statement was developed in 2007 to assist authors in report writing of observational studies, including cohort, case-control, and cross-sectional studies, to support editors and reviewers in considering such manuscript for publication, and to help readers in appraising published articles critically [3].
Concerning valuable recommendations made in this statement, expected that reporting of observational studies published after 2007 being improved enough to assess the strengths and weaknesses of the evidence. The present study was set up to investigate the quality of reporting of cohort studies published in the most prestigious scientific medical journals and to indicate to what extent the items in STROBE checklist are noticed by both authors and publishers.

METHODS

We conducted a cross-sectional study and selected six top scientific medical journals with high impact factor (IF) among the most prestigious and important medical journals indexed in international databases. These included New England Journal of Medicine (N Engl J Med) (IF 50.017), Journal of the American Medical Association (JAMA) (IF 31.718), Lancet (IF 28.409), British Medical Journal (BMJ) (IF 12.827), Archives of Internal Medicine (Arch Intern Med) (IF 9.110), and Canadian Medical Association Journal (Can Med Assoc J) (IF 7.464). We were to include Nature and Annals of Internal Medicine in this survey, but they were not open access journals.
Among the observational studies, cohort studies are much more expensive and take longer follow-up time than case-control and cross-sectional studies. Therefore, the results obtained from cohort studies are of substantially superior quality to other observational studies. In fact, cohort studies are considered the gold standard in observational epidemiology. Accordingly, we selected cohort studies, either prospective or retrospective, for this evaluation.
We randomly selected 10 cohort studies published in each of the six medical journals from January to August 2010. To do so, we sorted the articles from newly published articles to the old ones. Then, looked for the cohort studies to find ten eligible articles. If there were not enough cohort studies, we searched second half of 2009 to obtain additional studies. Accordingly, we enrolled 60 cohort studies from six prestigious scientific medical journals. The studies were randomly assigned to four reviewers through drawing lots. The reviewers independently made the decisions on the number of items, from the STROBE checklist, which were addressed in the selected studies. The reviewers were not blinded to the names of the studies' authors and journals.
In order to check the reliability of the four reviewers' judgment on the quality of reporting of cohort studies, we conducted two consecutive pilot studies as follows. First, an article was randomly selected from JAMA. All four reviewers assessed the strengths and weaknesses of the evidence reported in the same article using the predetermined checklist of items. The disagreements were discussed to reach the same perception of the checklist in order to increase the between reviewers reliability. Then, the four reviewers evaluated the quality of reporting of another article that was randomly selected. There was no significant differences between the reviewers' judgment in the second pilot study (p=0.823).
In order to assess the validity of the reviewers' judgments, two reviewers made decision on the quality of reporting of each cohort study independently. Any disagreements were resolved by adjudication with a third author.
The STROBE statement included a checklist of 22 items. In order to estimate the quality of reporting of cohort studies more accurately, we divided the items into 47 sub-items (Table 1). We considered three choices for each sub-item (reported/not reported/not applicable). The percentage of each sub-item addressed in the selected articles was estimated. The total percentage for all sub-items was reported.
All statistical analysis was performed at 95% significant level using statistical software Stata version 10.0 (StataCorp, College Station, TX, USA).

RESULTS

In this survey, 60 cohort studies were selected for evaluation including 10 studies from the six prestigious scientific medical journals including: N Engl J Med, JAMA, Lancet, BMJ, Arch Intern Med, and Can Med Assoc J. From these, 56 studies were published in 2010 and four studies in 2009. The percentage of items and sub-items addressed by these studies are summarized in Table 1. The sub-items were not applicable in 7.1% (95% confidence interval [CI], 1.3% to 13.0%), were not reported in 23.6% (95% CI, 15.3% to 31.5%), and were reported in 69.3% (95% CI, 59.6% to 79.0%) of the cohort studies.
Of 47 sub-items investigated in this survey, nine sub-items were reported 100%, 22 sub-items were addressed in more than 90% of the studies, 28 sub-items were included in more than 75% of the studies, and 32 sub-items were addressed in more than 50% of the studies.

DISCUSSION

STROBE checklist of items provides valuable recommendations for both authors to report the results of observational studies clearly as well as for editors, reviewers, and readers to appraise such reports critically [3]. On overall, almost 69.3% of the items and sub-items in STROBE checklist were addressed by cohort studies published in six top scientific medical journals three year after dissemination of STROBE statement.
However, what has happened to the reports of other types of observational studies? The results of the present study represent the reporting of cohort studies published in six prestigious scientific medical journals that generally accept the well-done and well-written studies. However, there are numerous observational studies, the results of which are published in other less fastidious peer-reviewed medical journals. Thus, it is expected that the quality of reporting of such studies is much poorer than what reported in the present study, although the result of present study is not desirable enough. Furthermore, cohort studies are much more expensive and take longer follow-up time than other types of observational studies such as case-control and cross-sectional design. Hence, the reporting of cohort studies is generally expected to be of substantially superior quality to other observational studies. Accordingly, if this survey had been planned to assess reporting of case-control or cross-sectional studies, the estimated result would be much more undesirable.
We found no similar studies but one. Poorolajal et al. [7] conducted a similar study in 2007 to assess the reporting of cohort studies before STROBE statement being issued. However, the design, the results, and the number of sub-items, which were evaluated in that study, was not comparable with that of the present study.
The present study had a number of limitations. First, the limited number of studies evaluated in the present study may increase the possibility of random error. Second, randomly selection of cohort studies from a few prestigious medical journals may increase the possibility of selection bias. Third, the value of all sub-items was not really the same. Hence, adding up all percentages to estimate a summary measure might not be reasonable, although was done to help the readers make an overall judgment.
This result of the present survey represents the quality of the reporting of cohort studies in top scientific medical journals. Hence, we can generalize the results of this survey neither to other types of observational studies nor to the publication of less fastidious peer-reviewed medical journals.
We concluded that reporting of cohort studies published in the most prestigious scientific medical journals is not clear and desirable enough yet. The reporting of other types of observational studies such as case-control and cross-sectional studies particularly those being published in less fastidious peer-reviewed journals is expected to be much more imprecise. This issue should be the focus of the both authors' and editors' special attention when reporting and/or reviewing the reports of observational studies.

CONFLICT OF INTEREST

The authors have no conflicts of interest to declare for this study.

NOTES

This article is available from: http://e-epih.org/.

References

1. Gordis L. Epidemiology. 2008. 4th ed. Philadelphia: Saunders.

2. Glasziou P, Vandenbroucke JP, Chalmers I. Assessing the quality of research. BMJ 2004; 328: 39-41. 14703546. PMID: 14703546
crossref pmid pmc
3. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med 2007; 4: e296. 17941714. PMID: 17941714
crossref pmid pmc
4. Pocock SJ, Collier TJ, Dandreo KJ, de Stavola BL, Goldman MB, Kalish LA, et al. Issues in the reporting of epidemiological studies: a survey of recent practice. BMJ 2004; 329: 883. 15469946. PMID: 15469946
crossref pmid pmc
5. Tooth L, Ware R, Bain C, Purdie DM, Dobson A. Quality of reporting of observational longitudinal research. Am J Epidemiol 2005; 161: 280-288. 15671260. PMID: 15671260
crossref pmid
6. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med 2007; 4: e297. 17941715. PMID: 17941715
crossref pmid pmc
7. Poorolajal J, Tajik P, Yazdizadeh B, Sehat M, Salehi AR, Rezaei M, et al. Quality assessment of the reporting of cohort studies before STROBE statement. Iranian J Epidemiol 2009; 5: 17-26 (Persian). PMID: 2009
pmid
Table 1
Percentage of items in STROBE checklist which were addressed in reports of cohort studies published in six top scientific medical journals in 2010 (n (%))
Item Recommendation Not applicable Not reported Reported
1a Indicate the study’s design in the title/abstract 0 (0.0) 15 (25.0) 45 (75.0)
1b Provide in the abstract an informative and balanced summary 0 (0.0) 3 (5.0) 57 (95.0)
Introduction
2 Explain the scientific background and rationale for the investigation 0 (0.0) 0 (0.0) 60 (100.0)
3 State specific objectives, including any pre-specified hypotheses 0 (0.0) 1 (1.7) 59 (98.3)
Methods
4 Present key elements of study design early in the paper 0 (0.0) 21 (35.0) 39 (65.0)
5 Describe locations 0 (0.0) 2 (3.3) 58 (96.7)
5 Describe recruitment dates 0 (0.0) 2 (3.3) 58 (96.7)
5 Describe periods of follow-up 0 (0.0) 4 (6.7) 56 (93.3)
6a Give the eligibility criteria 0 (0.0) 4 (6.7) 56 (93.3)
6a Describe the methods of follow-up 0 (0.0) 2 (3.3) 58 (96.7)
6b Give matching criteria 54 (90.0) 0 (0.0) 6 (10.0)
6b Give number of exposed and unexposed in matched studies 52 (86.7) 5 (8.3) 3 (5.0)
7 Clearly define all outcomes 0 (0.0) 0 (0.0) 60 (100.0)
7 Clearly define all exposures 0 (0.0) 6 (10.0) 54 (90.0)
7 Clearly define all predictors 0 (0.0) 2 (3.3) 58 (96.7)
7 Clearly define all potential confounders 0 (0.0) 9 (15.0) 51 (85.0)
7 Clearly define all effect modifiers 0 (0.0) 41 (68.3) 19 (31.7)
8 Give sources of data 0 (0.0) 0 (0.0) 60 (100.0)
8 Method of measurement 0 (0.0) 1 (1.7) 59 (98.3)
9 Describe any efforts to address potential sources of bias 0 (0.0) 51 (85.0) 9 (15.0)
10 Explain how the study size was arrived at 33 (55.0) 15 (25.0) 12 (20.0)
11 Describe which groupings were chosen (if applicable) 10 (16.7) 8 (13.3) 42 (70.0)
12a Describe all statistical methods 0 (0.0) 0 (0.0) 60 (100.0)
12a Describe all statistical software 0 (0.0) 14 (23.3) 46 (76.7)
12b Describe any methods used to examine subgroups and interaction 0 (0.0) 42 (70.0) 18 (30.0)
12c Explain how missing data were addressed 0 (0.0) 33 (55.0) 27 (45.0)
12d explain how loss to follow-up was addressed 3 (5.0) 42 (70.0) 15 (25.0)
12e Describe any sensitivity analyses 3 (5.0) 44 (73.3) 0 (21.7)
Results
13a Report numbers of individuals at each stage of study 5 (8.3) 29 (48.4) 26 (43.3)
13b Give reasons for non-participation at each stage 5 (8.3) 42 (70.0) 13 (21.7)
13c Consider use of a flow diagram 0 (0.0) 51 (85.0) 9 (15.0)
14a Give characteristics of study participants 0 (0.0) 5 (8.3) 55 (81.7)
14b Indicate the number of participants with missing data 4 (6.7) 37 (61.6) 19 (31.7)
14c Summarize follow-up time 2 (3.3) 32 (53.4) 26 (43.3)
15 Report numbers of outcome events or summary measures 0 (0.0) 0 (0.0) 60 (100.0)
16a Give unadjusted estimates 0 (0.0) 23 (38.3) 37 (61.7)
16a Give confounder-adjusted estimates 1 (1.7) 6 (10.0) 53 (88.3)
16a Give estimates precision/confidence interval 0 (0.0) 2 (3.3) 58 (96.7)
16b Report category boundaries when continuous variables were categorized 8 (13.3) 7 (11.7) 45 (75.0)
16c Consider translating estimates of relative risk into absolute risk (If relevant) 21 (35.0) 32 (53.3) 7 (11.7)
17 Report other analyses done e.g., subgroups analysis and sensitivity analyses 0 (0.0) 19 (31.7) 41 (68.3)
Discussion
18 Summarize key results with reference to study objectives 0 (0.0) 0 (0.0) 60 (100.0)
19 Discuss limitations of the study 0 (0.0) 0 (0.0) 60 (100.0)
20 Give a cautious interpretation of results considering objectives 0 (0.0) 0 (0.0) 60 (100.0)
20 Explain results from similar studies 0 (0.0) 0 (0.0) 60 (100.0)
21 Discuss the generalizability of the study results 0 (0.0) 6 (10.0) 54 (90.0)
Other information
22 Give the source of funding and the role of the funders 0 (0.0) 1 (1.7) 59 (98.3)
1-22 Total (7.1) (23.6) (69.3)

STOBE, The Strengthening the Reporting of Observational Studies in Epidemiology.



ABOUT
ARTICLE CATEGORY

Browse all articles >

BROWSE ARTICLES
FOR AUTHORS AND REVIEWERS
Editorial Office
Graduate School of Cancer Science and Policy, National Cancer Center
323 Ilsan-ro, Ilsandong-gu, Goyang 10408, Korea
TEL: +82-2-745-0662   FAX: +82-2-764-8328    E-mail: enh0662@gmail.com

Copyright © 2019 by Korean Society of Epidemiology. All rights reserved.

Developed in M2community

Close layer
prev next