|
February 1998
1. Peer Review of the Quality of Care: Reliability and Sources of Variability for Outcome and Process Assessments.
Smith MA, Atherly AJ, Kane RL, Pacala JT; JAMA, November 19, 1997; 278:1573-1578.
[ see abstract below ]
2. Consumer Reports in Health Care: Do They Make a Difference in Patient Care?
Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V; JAMA, November 19, 1997; 278:1579-1584.
[ see abstract below ]
3. Health Care Quality: Incorporating Consumer Perspectives.
Cleary PD, Edgman-Levitan S; JAMA, November 19, 1997; 278: 1608-1612.
[ see abstract below ]
4. Managed Care Is Not the Problem, Quality Is. (Editorial)
Brook RH; JAMA, November 19, 1997; 278:1612-1614.
No abstract available
Since 1990 the Journal of the American Medical Association has presented an annual coordinated theme issue event of the ten scientific journals of the American Medical Association. This year's theme described in this review relates to "quality of care." This topic is clearly of interest to both experts and readers of the medical literature. The topic "quality of care" range in the top 20 of 73 topics studied by 1997 surveys of JAMA experts and JAMA readers. This particular edition of JAMA, as defined by its editors, is to develop a new proposal and a call to action related to driving the American healthcare system, and warrants consideration by thoughtful physicians and other healthcare professionals.
This edition of JAMA contains several articles and editorial commentary dedicated to the pursuit of quality. All articles and perspectives are important considerations for the contemporary healthcare provider, and will be briefly summarized in this review.
The article by Smith et al acknowledges the importance of peer assessments in judging the quality of care, but also highlights the major drawback of poor interrater reliability in assessing quality of care. These authors suggest strategies to achieve adequate reliability for assessments of quality as well as outlining future research needed in order to identify additional sources of systematic bias for both process and outcome measures.
Longo et al address the matter of consumer reports, which were originally designed to assist consumers in making more informed decisions about their personal health care, but have now led to positive changes in the behavior of clinicians in health care delivery organizations. These investigators examined the impact of an obstetrics consumer report developed and issued by the Missouri Department of Health on hospital behavior. As a result of their formal study, they conclude that the public release of consumer reports may actually facilitate the improvement in the quality of health care services, in that these changes usually occur in competitive markets. The primary point in this presentation appears to be that feedback, public visibility and competition all work to improve policy.
Written from the perspective of users of the health care system, Cleary and Edgman-Levitan ask the following questions in their article:
- What are the most important health care quality gaps and/or challenges?
- What major challenges should we anticipate in this area in the near future?
- What should be the role of federal and state agencies, accreditation organizations, and philanthropic foundations in addressing these challenges?
The authors then proceed to discuss the needs, challenges, and potential action steps for increasing the prominence of the users perspective in: 1) the conceptualization and definition of quality; 2) the measurement of quality; and 3) routine quality assessments and improvement.
Another separate section of this edition of JAMA reviews seven high-profile organizations dedicated to quality in health care. With the increased dialog and press coverage related to various organizations who claim to be focused on the evaluation of quality, it will be of interest to the reader to review the mission and process described in each of the following organizations presented in this summary article: National Committee for Quality Assurance (NCQA), Agency for Health Care Policy and Research (AHCPR), Foundation for Accountability (FACCT), Health Care Financing Administration (HCFA), Institute for Health Care Improvement (IHI), The National Patient Safety Foundation (NPSF), and the Joint Commission on Accreditation of Health Care Organizations (JCAHO). A review of each of these organizations reveals considerable overlap in mission but given the vast need for improvement in quality and process throughout health care, such overlap is probably not necessarily problematic.
Finally, the reader of this focused edition should review the editorial by R.H. Brook. In this article, Dr. Brook describes what is happening in the provision of health care throughout the western world and how incentives can be designed which may better change the health system such that quality is rewarded rather than simply the organizational skills or degree of reimbursement. Although not stated explicitly, one might interpret Dr. Brook's comments as emphasizing more accountability in health care throughout the system, and thereby tracking quality at least as much as cost as we engaged in further change.
Return to the Current Literature Review Front Page, or read the abstract:
ABSTRACT
1. Peer Review of the Quality of Care: Reliability and Sources of Variability for Outcome and Process Assessments. (Smith et al)
Context: Peer assessments have traditionally been used to judge the quality of care, but a
major drawback has been poor interrater reliability.
Objectives: To compare the interrater reliability for outcome and process assessments in a
population of frail older adults and to identify systematic sources of variability that contribute to
poor reliability.
Setting: Eight sites participating in a managed care program that integrates acute and
long-term care for frail older adults.
Patients: A total of 313 frail older adults.
Design: Retrospective review of the medical record with 180 charts randomly assigned to 2
geriatricians, 2 geriatric nurse practitioners, or 1 geriatrician and 1 geriatric nurse practitioner and 133 charts randomly assigned to either a geriatrician or a geriatric nurse practitioner.
Main Outcome Measures: Interrater reliabilities for structured implicit judgments about
process and outcomes for overall care and care for each of 8 tracer conditions (eg, arthritis).
Results: Outcome measures had higher interrater reliability than process measures. Five
outcome measures achieved fair to good reliability (more than 0.40), while none of the process
measures achieved reliabilities more than 0.40. Three factors contributed to poorer reliabilities for
process measures: (1) an inability of reviewers to differentiate among cases with respect to the
quality of management, (2) systematic bias from individual reviewers, and (3) systematic bias
related to the professional training of the reviewer (ie, physician or nurse practitioner).
Conclusions: Peer assessments can play an important role in characterizing the quality of
care for complex patients with multiple interrelated chronic conditions, but reliability can be poor.
Strategies to achieve adequate reliability for these assessments should be applied. These strategies
include emphasizing outcomes measurement, providing more structured assessments to identify
true differences in patient management, adjusting systematic bias resulting from the individual
reviewer and their professional background, and averaging scores from multiple reviewers.
Future research on the reliability of peer assessments should focus on improving the ability of
process measures to differentiate among cases with respect to the quality of management and on
identifying additional sources of systematic bias for both process and outcome measures. Explicit
recognition of factors influencing reliability will strengthen efforts to develop sound measures for
quality assurance.
2. Consumer Reports in Health Care: Do They Make a Difference in Patient Care? (Longo et al)
Context: Consumer reports in health care are a relatively recent phenomenon. Primarily
designed to assist consumers in making more informed decisions about their personal health care,
they appear to have an important by-product--they led to positive changes in the behavior of
clinicians and health care delivery organizations. While there has been much speculation on their
impact on health care consumer behavior, consumer reports offer an effective strategy in
improving the quality of patient care.
Objective: To examine the impact of an obstetrics consumer report developed and issued by
the Missouri Department of Health on hospital behavior.
Design and Setting: A retrospective study of hospital behavior using both primary survey
and secondary clinical data.
Participants: Consumer reports were issued in 1993 to all Missouri hospitals providing
obstetrical services (n=90). A survey was conducted a year later, and the results were analyzed
with other available data to determine the effect of the report. Two hospitals discontinued
obstetrical services by the time of the survey; of the remaining 88 hospitals, 82 (93%) responded
to the survey.
Main Outcome Measures: The following outcomes were examined: (1) number and
percentage of hospitals that previously did not have services at the time report was issued, but
had, or planned to have, services after a guide was published; (2) the percentage of obstetrical
policies that were changed, planned to change, or are under discussion for change (car seat
program, obstetrical follow-up services, formal transfer agreement, nurse educator for
breast-feeding, and availability of tubal ligations); and (3) clinical outcomes, including
satisfaction, appropriateness of charges, and the rates of cesarean delivery, high-risk infant
transfer, ultrasound, vaginal birth after cesarean, very low birth weight, and newborn death.
Results:Within 1 year of the report, approximately 50% of hospitals that did not have car
seat programs, formal transfer agreements, or nurse educators for breast-feeding prior to the
report either instituted or planned to institute these services. Hospitals in competitive markets that did not offer one of these services at the time of the report were more likely to institute a service and/or were about twice as likely to consider improving several indicators. Clinical outcome
indicators all improved in the expected direction.
Conclusion: Public release of consumer reports may be useful not only in assisting
consumers to make informed health care choices, but also in facilitating improvement in the
quality of hospital services offered and care provided. Changes occur especially in competitive
markets.
3. Health Care Quality: Incorporating Consumer Perspectives. (Cleary et al)
The goal of this article is to address, from the perspective of users of the health care system
(consumers), the following questions: What are the most important health care quality gaps
and/or challenges; what major changes should we anticipate in this area in the near future; and
what should be the role of federal and state agencies, accreditation organizations, and
philanthropic foundations in addressing these challenges? We discuss the needs, challenges, and
potential action steps for increasing the prominence of the user's perspective in 3 areas: (1) the
conceptualization and definition of quality; (2) the measurement of quality; and (3) routine quality
assessment and improvement. The article concludes by making recommendations about the role
that different agencies and organizations can and should play in meeting these challenges.
|
