Skip to Main Content

Nursing: How to Read a Research Article

Evaluating the Evidence

Guidelines for Reading and Critiquing a Research Article from My American Nurse


Components of a research article (image for James Madison University)                

My 10-ish minute video on how to read a research article

 

 


1. Critiquing the research article

a. Title describes the article

b. Abstract summarizes the article

c. Introduction makes the purpose clear

d. Problem is properly introduced

e. Purpose of the study is explained

f. Research question(s) are clearly presented

g. Theoretical framework informs the research

h. Literature review is relevant, comprehensive, and includes recent research

i. Methods section details how the research questions were addressed or hypotheses were tested

j. Analysis is consistent with the study questions and research design

k. Results are clearly presented and statistics clearly explained

l. Discussion explains the results in relation to the theoretical framework, research questions, and significance to nursing

m. Limitations are presented and their implications discussed

n. Conclusion includes recommendations for nursing practice, future research, and policymakers


2. Is it an appropriate level of evidence?  (See the visual examples below)

3. Decide if the study is applicable to your practice.

Is it a Research Article?

 

Primary Research Examples

Primary sources are original materials.  It is authored by researchers, contains original research data, and is usually published in a peer-reviewed journal. Different methodologies can be used but the distinguishing characteristic is that the authors of the study designed and conducted the study first hand.  Examples include case studies and randomized control trials. 


Secondary Research Examples

Literature reviews are summaries of the literature on a particular topic. Reviews are generally considered "research", especially systematic and integrative reviews, but are not experimental in nature. There are several kinds of reviews: plain literature reviewssystematic reviews, and integrative reviews are the most common. Chapter 5 of Introduction to Nursing Research: Incorporating Evidence-based Practice (Cannon & Boswell, 2011, 2nd ed. Sudbury, Mass: Jones & Bartlett Learning) covers the purpose and process of a literature review in the context of writing a research article, thesis, or dissertation. How to undertake a literature search: a step-by-step guide (Watson, 2020, BJN, 29(7): 431-435) is a good overall guide.

Types of literature reviews:

  • Literature Reviews
    • summaries of relevant literature
    • generally descriptive
    • not necessarily any analysis of the literature
    • methodology of the literature search is not always given
    • good for gaining background knowledge of a subject without having to do all the searches and reading yourself.
    • good source for starting reading lists and literature searches.
    • not generally considered a good source for clinical decision making
    • Note: In the past, reviews were not differentiated by type, so older reviews may use systematic or integrative methodology but not be specified as such.
    • Reading: Ten simple rules for writing a literature review (Pautasso, M. (2013). PLoS Comput Biol9(7), e1003149.)
    • Reading: Conducting Your Literature Review (Hempel, S. (2020). Washington, DC : American Psychological Association.)
  • Systematic Reviews
    • specifically includes experimental research studies
    • search and selection methodology is very precise and should be explicitly described well enough for another researcher to duplicate the searches and the study selection. See Table 1 of this article (Hoojimans et al. (2012). PLoS One, 7(11): e48811) for a good example of describing the search methods.
    • the purpose of a systematic review is to reach some conclusion regarding the topic: for example, the selection of high quality studies to be used in a meta-analysis*, the gaps in current research, or the best clinical evidence for determining evidence based practice.
    • the first stage of meta-analysis studies--all meta-analyses should include a systematic review, but all systematic reviews do not lead to a meta-analysis
    • usually done in a group to reduce researcher bias in the selection and evaluation of individual studies
    • Reading: A practical guide to conducting a systematic review (Forward & Hobby, 2002, Nursing Times, 98(2), 36) provides some basic advice for conducting a systematic review.
      Reading:  PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. (Rethlefsen, M.L., et al. (2021). Syst Rev 1039. https://doi.org/10.1186/s13643-020-01542-z)
  • Integrative Reviews 
    • commonly include non-experimental research, such as case studies, observational studies, and meta-analyses, but may also include practice applications, theory, and guidelines
    • should have clear and precise search and selection criteria
    • search and selection methodology should be described well enough for another researcher to duplicate the process
    • selected literature should be analyzed, not just summarized--articles and groups of articles compared, themes identified, gaps noted, etc.
    • Reading: The integrative review: updated methodology (Whittemore & Knaf, 2005, Journal of Advanced Nursing, 52(5), 546–553) provides an overview of the purpose and practice of integrative reviews.
       
  • Scoping or Mapping Reviews
    • aims at determining the scope or extent of the research on a topic.

*A meta-analysis study is one where carefully selected data from previous studies is combined to bring more rigor to a statistical or other analysis. No additional experimental work is done (usually). A systematic review is necessary to be sure that the data from the selected studies is comparable and combinable

Levels of Evidence

In some journals, you will see a 'level of evidence' assigned to a research article. Levels of evidence are assigned to studies based on the methodological quality of their design, validity, and applicability to patient care. The combination of these attributes gives the level of evidence for a study.  Many systems for assigning levels of evidence exist.  A frequently used system in medicine is from the Oxford Center for Evidence-Based Medicine.  In nursing, the system for assigning levels of evidence is often from Melnyk & Fineout-Overholt's 2011 book, Evidence-based Practice in Nursing and Healthcare: A Guide to Best Practice.  The Levels of Evidence below are adapted from Melnyk & Fineout-Overholt's (2011) model. 

Graphic chart depicting Melnyk & Fineout-Overholt's Levels of Evidence model

Uses of Levels of Evidence: Levels of evidence from one or more studies provide the "grade (or strength) of recommendation" for a particular treatment, test, or practice. Levels of evidence are reported for studies published in some medical and nursing journals. Levels of Evidence are most visible in Practice Guidelines, where the level of evidence is used to indicate how strong a recommendation for a particular practice is. This allows health care professionals to quickly ascertain the weight or importance of the recommendation in any given guideline. In some cases, levels of evidence in guidelines are accompanied by a Strength of Recommendation.

About Levels of Evidence and the Hierarchy of Evidence: While Levels of Evidence correlate roughly with the hierarchy of evidence (discussed elsewhere on this page), levels of evidence don't always match the categories from the Hierarchy of Evidence, reflecting the fact that study design alone doesn't guarantee good evidence. For example, the systematic review or meta-analysis of randomized controlled trials (RCTs) are at the top of the evidence pyramid and are typically assigned the highest level of evidence, due to the fact that the study design reduces the probability of bias (Melnyk, 2011), whereas the weakest level of evidence is the opinion from authorities and/or reports of expert committees. However, a systematic review may report very weak evidence for a particular practice and therefore the level of evidence behind a recommendation may be lower than the position of the study type on the Pyramid/Hierarchy of Evidence.

About Levels of Evidence and Strength of Recommendation: The fact that a study is located lower on the Hierarchy of Evidence does not necessarily mean that the strength of recommendation made from that and other studies is low--if evidence is consistent across studies on a topic and/or very compelling, strong recommendations can be made from evidence found in studies with lower levels of evidence, and study types located at the bottom of the Hierarchy of Evidence. In other words, strong recommendations can be made from lower levels of evidence.

For example: a case series observed in 1961 in which two physicians who noted a high incidence (approximately 20%) of children born with birth defects to mothers taking thalidomide resulted in very strong recommendations against the prescription and eventually, manufacture and marketing of thalidomide. In other words, as a result of the case series, a strong recommendation was made from a study that was in one of the lowest positions on the hierarchy of evidence.

How to Read a Research Paper

Evaluating the evidence from medical studies can be a complex process, involving an understanding of study methodologies, reliability and validity, as well as how these apply to specific study types. While this can seem daunting, in a series of articles by Trisha Greenhalgh from BMJ, the author introduces the methods of evaluating the evidence from medical studies, in language that is understandable even for non-experts. Although these articles date from 1997, the methods the author describes remain relevant. Use the links below to access the articles.

References

Daly, J., Willis, K., Small, R., Green, J., Welch, N., Kealy, M., & Hughes, E. (2007). A hierarchy of evidence for assessing qualitative health research. Journal of Clinical Epidemiology60(1), 43–49. doi:10.1016/j.jclinepi.2006.03.014

McBride, W. G. ‘‘Thalidomide and Congenital Abnormalities.’’ Letter to the Editor. The Lancet 2
(December 16, 1961): 1358.

Melnyk, B. M. (2011). Evidence-based practice in nursing & healthcare: a guide to best practice (2nd ed.). Philadelphia: Wolters Kluwer/Lippincott Williams & Wilkins.

Slawson, D. C., & Shaughnessy, A. F. (1997). Obtaining useful information from expert based sources. BMJ (Clinical Research Ed.)314(7085), 947–949.

Hierarchy of Evidence for Quantitative Studies

The pyramid below represents the hierarchy of evidence, which illustrates the strength of study types; the higher the study type on the pyramid, the more likely it is that the research is valid. The pyramid is meant to assist researchers in prioritizing studies they have located to answer a clinical or practice question. 

evidence pyramid

For clinical questions, you should try to find articles with the highest quality of evidence. Systematic Reviews and Meta-Analyses are considered the highest quality of evidence for clinical decision-making and should be used above other study types, whenever available, provided the Systematic Review or Meta-Analysis is fairly recent. 

As you move up the pyramid, fewer studies are available, because the study designs become increasingly more expensive for researchers to perform. It is important to recognize that high levels of evidence may not exist for your clinical question, due to both costs of the research and the type of question you have.  If the highest levels of study design from the evidence pyramid are unavailable for your question, you'll need to move down the pyramid.

While the pyramid of evidence can be helpful, individual studies--no matter the study type--must be assessed to determine the validity.

Hierarch of Evidence for Qualitative Studies

Qualitative studies are not included in the Hierarchy of Evidence above. Since qualitative studies provide valuable evidence about patients' experiences and values, qualitative studies are important--even critically necessary--for Evidence-Based Nursing. Just like quantitative studies, qualitative studies are not all created equal. The pyramid below  shows a hierarchy of evidence for qualitative studies.

Adapted from Daly et al. (2007)

Primary/Secondary/Tertiary Sources

Types of Sources - Primary, Secondary and Tertiary

Types of Medical Literature

Medical literature is often classified based on how far removed the information is from the original source.

Primary Literature/Source
Primary sources are original materials.  It is authored by researchers, contains original research data, and is usually published in a peer-reviewed journal. Primary literature may also include conference papers, pre-prints, or preliminary reports.

Secondary Literature/Source 
Secondary literature consists of interpretations and evaluations that are derived from or refer to the primary source literature. Examples include review articles (e.g., meta-analysis and systematic reviews) and reference works. Professionals within each discipline take the primary literature and synthesize, generalize, and integrate new research.

Tertiary Literature/Source
Tertiary literature consists of a distillation and collection of primary and secondary sources such as textbooks, encyclopedia articles, and guidebooks or handbooks. The purpose of tertiary literature is to provide an overview of key research findings and an introduction to principles and practices within the discipline.

Primary Literature/Source

What is it?       

Original research results in journals, dissertations, conference proceedings, correspondence

Example of where you can  find it peer-reviewed journals usually found in CINAHL, or MEDLINE/PubMED
Secondary Literature/Source
What is it?   Abstracting and indexing services, review articles, systematic reviews, meta-analysis, practice guidelines
Example of where you can  find it Also found in Library Databases--be careful!
Tertiary Literature/Source 
What is it? Text books, encyclopedias,   handbooks, newspapers
Example of where you can find it AccessMedicine for Consumers
EBSCO's Consumer Health
   

Adapted from the Information Services Department of the Library of the Health Sciences-Chicago, University of Illinois at Chicago.

One more way to look at it:   Six Degrees of Separation (or Six Degrees of Kevin Bacon)

  • Primary Literature/Source:  someone talked to Kevin Bacon and wrote about it
  • Secondary Literature/Source:  someone else read the above article and summarized it for a radio show
  • Tertiary Literature/Source:  a magazine included a mention of the article in a summary of Kevin Bacon interviews