SuperTutor
$15/per page/Negotiable
Box 5.2
GUIDE TO AN OVERALL CRITIQUE OF A QUANTITATIVE RESEARCH
REPORT
Aspect of the Detailed Report Critiquing Questions Critiquing
Guidelines Title Is the title a good one, succinctly suggesting key variables and the Abstract study population? Does the abstract clearly and concisely summarize the main features of
the report (problem, methods, results, conclusions)? Introduction
Statement of Is the problem stated unambiguously, and is it easy to identify? Box 4.3, the problem Does the problem statement build a cogent, persuasive argument for the page 90 new study? Does the problem have significance for nursing? Is there a good match between the research problem and the paradigm
Hypotheses or and methods used? Is a quantitative approach appropriate? Are research questions and/or hypotheses explicitly stated? If not, is Box 4.3, research their absence justified? page 90 questions Are questions and hypotheses appropriately worded, with clear
specification of key variables and the study population? Are the questions/hypotheses consistent with the literature review and Literature the conceptual framework? Is the literature review up to date and based mainly on primary Box 5.4, review sources? page 122 Does the review provide a state-of-the-art synthesis of evidence on the 5-1 problem?
Conceptual/the Does the literature review provide a sound basis for the new study? Are key concepts adequately defined conceptually? Box 6.3, oretical Is there a conceptual/theoretical framework, rationale, and/or map, and page 145 framework
Method (if so) is it appropriate? If not, is the absence of one justified? Protection of Were appropriate procedures used to safeguard the rights of study Box 7.3, human rights participants? Was the study externally reviewed by an IRB/ethics review page 170 board? Was the study designed to minimize risks and maximize benefits to
Research participants? Was the most rigorous possible design used, given the study purpose? Box 9.1, design Were appropriate comparisons made to enhance interpretability of the page 230; findings? Box 10.1, Was the number of data collection points appropriate? page 254 Did the design minimize biases and threats to the internal, construct,
and external validity of the study (e.g., was blinding used, was attrition
Population and minimized)? Is the population described? Is the sample described in sufficient detail? Box 12.1, sample Was the best possible sampling design used to enhance the sample’s page 289 representativeness? Were sampling biases minimized? Was the sample size adequate? Was a power analysis used to estimate
Data collection sample size needs? Are the operational and conceptual definitions congruent? Box 13.1, and Were key variables operationalized using the best possible method page 309; measurement (e.g., interviews, observations, and so on) and with adequate justification? Box 14.1, 5-2 Are specific instruments adequately described and were they good page 347 choices, given the study purpose, variables being studied, and the study
population? Does the report provide evidence that the data collection methods
Procedures yielded data that were reliable and valid? If there was an intervention, is it adequately described, and was it Box 9.1, rigorously developed and implemented? Did most participants allocated to page 230;
the intervention group actually receive it? Is there evidence of Box 10.1, intervention fidelity? page 254 Were data collected in a manner that minimized bias? Were the staff
who collected data appropriately trained? Results
Data analysis Were analyses undertaken to address each research question or test each Box 16.1,
hypothesis? page 400; Were appropriate statistical methods used, given the level of Box 17.1, measurement of the variables, number of groups being compared, and page 429 assumptions of the tests? Was the most powerful analytic method used (e.g., did the analysis help
to control for confounding variables)? Were Type I and Type II errors avoided or minimized? In intervention studies, was an intention-to-treat analysis performed? Were problems of missing values evaluated and adequately addressed?
5-3 Findings Is information about statistical significance presented? Is information Box 17.1, about effect size and precision of estimates (confidence intervals) page 429; presented? Box 28.1, Are the findings adequately summarized, with good use of tables and page 687 figures?
Are findings reported in a manner that facilitates a meta-analysis, and
with sufficient information needed for EBP?
Discussion
Interpretation Are all major findings interpreted and discussed within the context of Box 19.1, of the findings prior research and/or the study’s conceptual framework? page 482 Are causal inferences, if any, justified? Are interpretations well-founded and consistent with the study’s
limitations?
Implications/ Does the report address the issue of the generalizability of the findings? Do the researchers discuss the implications of the study for clinical Box 19.1, recommendatio practice or further research—and are those implications reasonable and page 482 ns
Global Issues complete?
Box 28.2, Presentation Is the report well-written, organized, and sufficiently detailed for
critical analysis?
In intervention studies, is a CONSORT flow chart provided to show the
flow of participants in the study? Is the report written in a manner that makes the findings accessible to Researcher practicing nurses? Do the researchers’ clinical, substantive, or methodologic qualifications credibility
5-4 page 698 and experience enhance confidence in the findings and their
Summary interpretation? Despite any limitations, do the study findings appear to be valid—do assessment you have confidence in the truth value of the results? Does the study contribute any meaningful evidence that can be used in
nursing practice or that is useful to the nursing discipline? 5-5 BOX 5.3
GUIDE TO AN OVERALL CRITIQUE OF A QUALITATIVE RESEARCH
REPORT
Aspect of the Detailed Report Critiquing Questions Critiquing
Guidelines Title Is the title a good one, suggesting the key phenomenon and the Abstract group or community under study? Does the abstract clearly and concisely summarize the main
features of the report? Introduction
Statement of Is the problem stated unambiguously and is it easy to identify? Box 4.3, the problem Does the problem statement build a cogent and persuasive page 90 argument for the new study? Does the problem have significance for nursing? Is there a good match between the research problem on the one
Research hand and the paradigm, tradition, and methods on the other? Are research questions explicitly stated? If not, is their absence Box 4.3, questions justified? page 90 Are the questions consistent with the study’s philosophical basis,
Literature underlying tradition, or ideological orientation? Does the report adequately summarize the existing body of Box 5.4, review knowledge related to the problem or phenomenon of interest? page 122 Does the literature review provide a sound basis for the new
Conceptual study? Are key concepts adequately defined conceptually? 5-6 Box 6.3, underpinnings Is the philosophical basis, underlying tradition, conceptual page 145 framework, or ideological orientation made explicit and is it
appropriate for the problem?
Method
Protection of Were appropriate procedures used to safeguard the rights of study Box 7.3, participants’ participants? Was the study subject to external review by an page 170 rights IRB/ethics review board? Was the study designed to minimize risks and maximize benefits Research to participants? Is the identified research tradition (if any) congruent with the Box 20.1, design and methods used to collect and analyze data? page 510 research Was an adequate amount of time spent in the field or with study tradition participants? Did the design unfold in the field, giving researchers
opportunities to capitalize on early understandings? Was there an adequate number of contacts with study Sample and participants? Was the group or population of interest adequately described? Box 21.1, setting Were the setting and sample described in sufficient detail? page 528 Was the approach used to recruit participants or gain access to the
site productive and appropriate? Was the best possible method of sampling used to enhance
information richness and address the needs of the study?
Data collection Was the sample size adequate? Was saturation achieved? Were the methods of gathering data appropriate? Were data Box 22.1, gathered through two or more methods to achieve triangulation? page 548 5-7 Did the researcher ask the right questions or make the right
observations, and were they recorded in an appropriate fashion? Was a sufficient amount of data gathered? Were the data of
Procedures sufficient depth and richness? Are data collection and recording procedures adequately Box 22.1, described and do they appear appropriate? page 548 Were data collected in a manner that minimized bias? Were the
Enhancement staff who collected data appropriately trained? Did the researchers use effective strategies to enhance the Box 24.1, of trustworthiness/integrity of the study, and was the description of page 598; trustworthiness those strategies adequate? Table 24.1, Were the methods used to enhance trustworthiness appropriate page 587 and sufficient? Did the researcher document research procedures and decision
processes sufficiently that findings are auditable and confirmable? Is there evidence of researcher reflexivity? Is there “thick description” of the context, participants, and
findings, and was it at a sufficient level to support transferability?
Results
Data analysis Are the data management and data analysis methods sufficiently
described? Was the data analysis strategy compatible with the research
tradition and with the nature and type of data gathered? Did the analysis yield an appropriate “product” (e.g., a theory,
taxonomy, thematic pattern)? 5-8 Box 23.1,
page 559 Findings Do the analytic procedures suggest the possibility of biases? Are the findings effectively summarized, with good use of Box 23.1,
page 559 excerpts and supporting arguments? Do the themes adequately capture the meaning of the data? Does
it appear that the researcher satisfactorily conceptualized the themes
or patterns in the data? Does the analysis yield an insightful, provocative, authentic, and
Theoretical meaningful picture of the phenomenon under investigation? Are the themes or patterns logically connected to each other to integration form a convincing and integrated whole? Box 23.1
page 559;
Box 6.3,
page 145 Are figures, maps, or models used effectively to summarize
conceptualizations? If a conceptual framework or ideological orientation guided the
Discussion study, are the themes or patterns linked to it in a cogent manner? Are the findings interpreted within an appropriate social or
Box 23.1,
page 559 Interpretation cultural context? of the findings Are major findings interpreted and discussed within the context of
prior studies? Implications/ Are the interpretations consistent with the study’s limitations? Do the researchers discuss the implications of the study for recommendatio clinical practice or further inquiry—and are those implications
ns
Global Issues reasonable and complete? Presentation Is the report well written, organized, and sufficiently detailed for
critical analysis? Is the description of the methods, findings, and interpretations 5-9 Box 28.2,
page 698 Researcher sufficiently rich and vivid? Do the researchers’ clinical, substantive, or methodologic credibility qualifications and experience enhance confidence in the findings and Summary their interpretation? Do the study findings appear to be trustworthy—do you have assessment confidence in the truth value of the results? Does the study contribute any meaningful evidence that can be
used in nursing practice or that is useful to the nursing discipline? 5-10 Box 5.4
Guidelines for Critiquing Literature Reviews 1. Is the review thorough—does it include all of the major studies on the topic? Does
it include recent research? Are studies from other related disciplines included, if
appropriate? 2. Does the review rely on appropriate materials (e.g., mainly on primary source
research articles)? 3. Is the review merely a summary of existing work, or does it critically appraise and
compare key studies? Does the review identify important gaps in the literature? 4. Is the review well organized? Is the development of ideas clear? 5. Does the review use appropriate language, suggesting the tentativeness of prior
findings? Is the review objective? Does the author paraphrase, or is there an over
reliance on quotes from original sources? 6. If the review is part of a research report for a new study, does the review support
the need for the study? 7. If it is a review designed to summarize evidence for clinical practice, does the
review draw reasonable conclusions about practice implications? 5-11 Figure 5.5
Citation: Type of Study:
Location/setting: Literature Review Protocol Authors: __________________________________________________________
Title:
__________________________________________________________
Journal:
__________________________________________________________
Year:
______
Volume: ________ Issue: _____ Pages: ______ Quantitative Qualitative Mixed Method
______________________________________________________________ Key Concepts/
Variables: Concepts: ____________________________________________________________
Intervention/Independent Variable: ________________________________________
Dependent Variable: ___________________________________________________
Controlled Variables: ______________________________________________
Framework/Theory: ____________________________________________________________________
Design Type: Experimental Quasiexperimental Nonexperimental
Specific Design: ______________________________________________________
Blinding? None Single: ________________ Double________________
Intervention Description: ___________________________________________
_____________________________________________________________
Comparison group(s):___________________________________________________ Cross-sectional Longitudinal/Prospective No. of data collection points: ____
Qual. Tradition: Grounded theory Phenomenology Ethnography Descriptive Other:__________ Sample: Size: ___________
Sampling method:_______________________________
Sample characteristics:__________________________________________________
_______________________________________________________________
Type: Self-report Observational Biophysiologic Other____
Description of measures:________________________________________________
____________________________________________________________________
____________________________________________________________________
Data Quality: ___________________________________________________
_______________________________________________________________
Bivariate: t-test ANOVA Chi-square Pearson’s r Other:_______
Multivar: Multiple Regression Logistic Regression Other: _____________ Data Sources: Statistical Tests: Findings/
Effect Sizes/
Themes: ____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Recommendations: _____________________________________________________________________
____________________________________________________________________
Strengths:
_______________________________________________________________ Weaknesses: _______________________________________________________________
_______________________________________________________________
_______________________________________________________________
_______________________________________________________________
_______________________________________________________________
5-12 Figure 5.6
Methodologic Matrix for Recording Key Methodologic Features of Studies for a Literature
Review Authors Pub
Yr Country Dependent Variables Independent
Variables Study
Design Sample
Size Griffin et al. 2008 U.S.A. Nurses’ age, clinical
experience,
education, nurse
practitioner status Crosssectional,
correlation
al 332
nurses,
national
sample Twycross 2007 U.K. Perception of child’s
pain,
Use of analgesics,
Use of
nonpharmacologic
methods
Pain management
practices Knowledge of pain
management Vincent &
Denyes 2004 U.S.A. Polkki et al. 2001 Finlan
d Use of analgesics,
Perceived barriers to
optimal pain
management
Nurses’ use of
nonpharmacologic
methods Hamers et
al. 1997 Nether
- lands Nurses’ age, race,
clinical experience,
education, pain
experience
Nurses’ age,
education, clinical
experience, # own
kids
Level of experience
in pediatric nursing Crosssectional,
correlation
al
Crosssectional,
correlation
al
Crosssectional,
correlation
al
Crosssectional,
correlation
al 13
nurses, 1
surgical
ward
67 nurses
from 7
hospital
units
162
nurses
from 5
hospitals
695
nurses Assessments of child’s
pain, Confidence in
assessment, Use of
analgesics 5-13 Samplin Data
g
Collection
Method
Random Self-report
questionnair
e Age of
Childre
n
8–10 Convenience Observation
, self-report 0–16 Convenience Observation
, self-report
questionnair
e
Self-report
questionnair
e 3–17 Video,
vignette,
self-reports 5–10 Convenience
Convenience 8–12 Margolius
et al. 1995 U.S.A. Perceptions of child’s
pain, Perceived
adequacy of pain
management Nurses’ education,
age, years of nursing
experience 5-14 Crosssectional,
correlation
al 228
nurses, 1
pediatric
setting Convenience Self-report
questionnair
e NA Figure 5.7
Two Results Matrices for Recording Key Findings for a Literature Review
Independent Variable: ________________ (Code 1)
Authors Pub
Year DV.a DV.b DV.c DV.d Independent Variable: ________________ (Code 2)
Authors Pub
Year DV.a DV.b DV.c 5-15 DV.d Figure 5.8
Evaluation Matrix for Recording Strengths and Weaknesses of Studies for a Literature Review Authors Year of
Publication Major Strengths Major Weaknesses 5-16 Quality
Score Log of Literature Search Activities in Bibliographic Databases
Date Database Keywords
Used Subject
Heading
Used Author
Search Name 5-17 Restrictions
to Search Other
Information
on Strategy Yield Useful Websites for Chapter 5: Resources for Literature Reviews
Note: The following websites were functional as of the date of going to press and may no
longer be active.
Aries Knowledge Finder http://www.kfinder.com CINAHL http://www.ebscohost.com/cinahl/ CINAHL tutorial, University of
Florida
EBSCOhost http://www.hscl.ufl.edu/help/CINAHL/tutorials.htm ISI Web of Knowledge http://www.thomsonisi.com/ National Library of Medicine http://www.nlm.nih.gov/ Ovid http://www.ovid.com/site/index.jsp ProQuest http://www.proquest.com/ PubMed http://www.ncbi.nlm.nih.gov/PubMed PubMed tutorial http://www.nlm.nih.gov/bsd/disted/pubmedtutorial/ PubMed tutorial, Mount Sinai
School of Medicine http://www.mssm.edu/library/tutorials/pubmed.html http://www.ebscohost.com/ 5-18
Attachments: