Quality improvement is a formal approach to the analysis of performance and systematic efforts to improve it. The requirements for documenting quality improvement (QI) projects for MOC include preparing written descriptions of purpose, methods, results, and next steps. Individuals who lead quality improvement (QI) projects should consider how this information would be useful to others if shared locally, shared with others elsewhere, or might become a scholarly contribution to improving care.
The information below will help leaders of QI projects develop project reports, presentations (including posters), and manuscripts for publication. The information is organized into the following sections:
A wide range of general considerations are involved in deciding what to report about a QI project. Some important ones are:
a. General expectations for health science presentations and publications. The development of reports, presentations, and scholarly manuscripts involves a wide range of factual knowledge and technical skills. Most leaders of QI projects at an academic institution are aware of much of that knowledge and have many of those skills, but they "don't know what they don't know."
For a broad overview of considerations, review the table of contents of a general reference. For example, SEE THE DETAILED TABLE OF CONTENTS (8 pages!) for Lang, TA. How to Write, Publish, & Present in the Health Sciences. Philadelphia, PA: American College of Physicians, 2010.
b. Specific expectations for QI publications (SQUIRE 2.0 guidelines). The Standards for Quality Improvement Reporting Excellence (“SQUIRE 2.0 guidelines)” outline the recommended content for publications describing quality improvement efforts (see http://www.squire-statement.org/guidelines). Several journals use these guidelines in assessing QI manuscripts. The guidelines outline and explain 18 elements of information to include in a scholarly manuscript about a QI initiative:
Title and Abstract
1. Title
Introduction 3. Problem description Methods 7. Context |
Results
13. Results Discussion 14. Summary Other Information 18. Funding |
c. Consider your audience and their interests. Who is your audience and what interests them? Most audiences are not fundamentally interested in what you did. They are primarily interested in implications for their specific issues/concerns. Understand your audience’s interests and focus on information that addresses their interests. Accept that many of the things that were interesting to you may not be included in your communications.
To whom do you want to describe the QI project and what are their interests? The typical audiences are summarized in the table below along with a communication’s usual relevant purpose, format, and content.
Audience |
Purpose |
Format |
Content |
QI MOC Program |
Approval for MOC
|
UM QI report form outlining information to be described |
Written stepwise description of one or more cycles of improvement efforts and of activities of participating physicians |
Local clinical leaders |
Communicate activities and impact in context of local clinical priorities |
Administrative report (typically 1-2 pages) |
Written summary highlighting purpose, rationale, interventions, results, and next steps |
Colleagues with similar problem at other institutions |
Share what was done so others can identify aspects they want to implement |
Poster or verbal presentation (limited space/time to communicate)
|
Highlights of purpose, rationale, methods including interventions, results, limitations, and recommendations |
Others with similar problem interested in what is unique about the project, how the methods and results can be adapted to improve care elsewhere and used to develop future innovations. |
Scholarly sharing with others: what was done and how it advances the field |
Manuscript for journal article |
Description of current evidence concerning the problem and how it has been previously addressed, rationale for study, methods including interventions, results, limitations, implications for practice and for research. |
Project leads will have already documented a QI project on the final report form for MOC. More detail is provided below concerning how that information can be incorporated into reports for local leaders, presentations to colleagues outside the institution, and publication/scholarly contributions for external audience.
The documentation for QI MOC can be easily revised into a report on local clinical impact. Points to consider in making the revision include:
Focus and brevity. A one or two-page summary plus a data table or figure often provides sufficient information for local clinical leaders to understand key activities and develop broader plans and priorities for future local QI initiatives. More detailed information may be placed in appendices.
Content. In general, the report will follow the organization of the documentation for MOC, describing the problem and the sequence of a data guided cycle of improvement. The underlying cause may be summarized on one section and the intervention may be summarized in one section, with the results for both points in time presented in one or two tables or figures.
Statistical tests. Formal statistical tests of the probability that results occurred by chance are helpful, but they likely will not be needed if results demonstrate practically meaningful change sufficient for administrative decisions. (Leaders do not have evidence of > 95% probability for most decisions.)
Recommendations. Recommendations for action should be clear, including how the change will be sustained, remaining causes that should be addressed next, and where similar problems exist in the organization that could benefit through “spreading” the activities in this project.
Click here to see one example of a report of a QI project that was designed for clinical leaders. It is a one-page executive summary plus a figure showing the main results. The full report that presents step-by-step activities and results in more detail is appended in case a leader is interested in more information. The executive summary is organized by several headings commonly used in reporting research results, an organization that clinical leaders are likely to easily recognize and follow.
The documentation for MOC can be fairly easily revised into a poster or short verbal presentation that highlights important aspects of the project. Points to consider in making the revision and expansion include:
Brevity and highlights. The constraints of poster space and of time for verbal presentations limit the content to highlights of important information and activities.
Audience and their practice context. The audience of colleagues will typically include people practicing in other settings and institutions, so throughout the presentation consider what should be addressed regarding differences between the local practice setting and typical practice settings of others.
Content. The main content will follow the organization of the documentation for MOC, describing the problem and the sequence of a data guided cycle of improvement.
The underlying cause may be summarized on one section and the intervention may be summarized in one section, with the results for both points in time presented in one or two tables or figures.A possible addition is a few key references to previous work relevant to the problem, the methods, and the results.Statistical tests. Formal statistical tests are often performed to demonstrate that results had a very low probability of occurring by chance, but may not be needed. (Pilot studies may not have enough subjects – “power” – to detect change of the expected magnitude.)
Limitations. A few key limitations of the project should be noted, including important methodological limitations and factors in the local setting (providers, patients, operational context) that may enhance or limit the generalizability of the findings to other settings.
Implications/recommendations. Based on the study results, include recommendations for changes to practice in settings in which the audience provides care. Implications for future studies may also be relevant.
The documentation for MOC can be the core of a scholarly publication, but meaningful additional effort is often needed for the:
Introduction – adding the current status of the field (illustrative previous work of others) and the potential contribution of this study. (Reference librarians can help design and carry out literature searches.)
Discussion – adding how the results relate to findings of others and implications for practice and future studies
Points to consider in adapting documentation for MOC into a manuscript for publication include:
Journal specifications. Review the information for authors, including types of manuscripts and manuscript format and length.
Length. What ever length is allowed, you know enough about your project to write more. Keep in mind what is most important to individuals in the audience and how much information is adequate.
Audience and their practice context. The audience is people practicing in other settings and institutions, so throughout the presentation consider what should be addressed regarding differences between the local practice setting and typical practice settings of others.
Content. The main content will follow the organization of the documentation for MOC, describing the problem and the sequence of a data guided cycle of improvement. To the introduction add illustrative references to important previous work and highlight the expected contribution of this study.
The underlying cause may be summarized on one section and the intervention may be summarized in one section, with the results for both points in time presented in one or two tables or figures.Statistical tests. Formal statistical tests are usually performed to demonstrate that results had a very low probability of occurring by chance.
Limitations. Address strengths and limitations of the project, including important methodological strengths and limitations and factors in the local setting (providers, patients, operational context) that may enhance or limit the generalizability of the findings to other settings.
Implications/recommendations. Based on the results of the study and the discussion of their additions to the field, include recommendations for changes to practice in settings in which the audience provides care and implications of the results for future studies.
Individuals performing innovations and improvements in clinical care should understand how Institutional Review Boards (IRBs) differentiate non-regulated quality improvement (QI) and regulated research. Three key points are:
“Research” or “quality improvement?” IRB approval for human subjects involvement is not needed to perform QI activities or to present or publish QI results internally or externally. However, federal regulations define “QI activities” more narrowly than the term QI is sometimes used. You should understand those technical definitions and be sure your project does not include sufficient “research” components that IRB approval is needed before you initiate the project.
Formal documentation that a QI activity is not regulated. Journals sometimes ask for formal documentation from an IRB that a QI activity described in a manuscript did not require IRB review. If you expect to publish, consider likely journals and their requirements regarding documentation that a QI project is not regulated. If documentation of “not regulated” status may eventually be needed, request a determination letter from the IRB prior to initiating the project. The early request assures “no surprises,” which may occur if you conduct the project without confirmation from the IRB.
HIPAA. HIPAA requirements regarding patient privacy apply whether an activity is research or quality improvement.
Each of these topics is addressed below.
a. “Research” or “quality improvement?” “Research” must meet several federal regulations for human subjects involvement, which IRBs assure. “QI” activities are “non-regulated” and do not require IRB oversight. However, federal regulations make technical distinctions between “research” and “quality improvement” that are not always reflected in common uses of these terms. In summary, federal definitions are:
Research is designed “to test a hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge (expressed, for example, in theories, principles, and statements of relationships).” Some features likely to characterize research are: the purpose of testing issues beyond current science and experience, randomization of subjects, fixed protocols, and researchers with no ongoing commitment to improvement of the local care situation.
Quality improvement involves interventions designed solely to enhance the well-being of patients and have a reasonable expectation of success. Some features likely to characterize QI are: applying known science to improve care, no unusual grouping of patients, ongoing adjustments/improvements to local activities, and personnel involved in ongoing improvement in the local situation.
The intent to publish (share descriptions/results with interested others) is not a criterion for determining whether an activity is “research” or “QI.” Descriptions of local QI activities can be published as examples of what worked in one local context. Publications of research share generalized knowledge and recommendations regarding what should occur more broadly.
Two ways in which something referred to as “QI” in common language may better fit the definition of research are:
In both instances, the efforts go beyond the narrow focus on doing “local QI” to performing “research” about how to go about doing QI.
If you are unsure whether your QI activity involves elements of research, our local IRB (IRBMED) suggests using the following resources to guide your assessment of your activity:
You can also contact IRBMED for guidance. Read their information in advance so that you can explain your situation and questions in their terms.
b. Formal documentation that a QI activity is not regulated. Some journals may accept an author’s statement that the described QI activity required no IRB review, while other journals may want more formal documentation. The following three levels exist for documenting that a QI activity requires no IRB review. Check journals to which you are likely to submit a manuscript to determine the level of documentation you may eventually need.
Self-determination - informal. Review the above information and links to confirm your understanding that it is “QI” and do the project without any eResearch application to IRBMED.
Self-determination - semi-formal. Fill out an abbreviated eResearch application, “Activities Not Regulated...” application type, and get a “Self-determination” letter.
IRB determination. Fill out an abbreviated eResearch application, “Activities Not Regulated...” application type, submit it to IRBMED, and receive a formal “Not Regulated” determination. This often involves a clarification request from an IRB representative.
To obtain a "Self-determination" letter of non-regulation or a formal letter from IRBMED of non-regulation:
For further general information on filling out an eResearch application, see https://research.medicine.umich.edu/our-units/institutional-review-boards-irbmed
c. Health Insurance Portability and Accountability Act (HIPAA). HIPAA requirements concerning patient privacy are separate from the regulation of research. QI activities involving patient information must follow HIPAA requirements. External presentations and publications need to maintain the privacy of individual patients involved in the project.
Suggestions for the following common problems are provided below
a. What is an “advance in the field”? The maturity of a field affects what information is an advance. An area of study typically evolves through publications focusing on:
Review current literature regarding the content and process you are addressing. Determine what contribution your manuscript will make to what is currently known and organize your manuscript around its significance to advancing what is known. Early in the evolution of a field fairly straightforward case reports are adequate. After several case studies have been published, another “me too” case report makes little contribution to the field (although doing the replication locally may be relevant for local purposes and reports). As a field matures, increasing effort is involved in highlighting relevant past work and putting the results in context.
b. Identify your aspirations early. When developing your QI project, consider the audience(s) likely to be interested in the results and your personal interests and resources. What reports, presentations, or publications do you expect to prepare? Considering this early is important because it affects some aspects of how the project is carried out. For example, will formal statistical tests of probability be needed and are sample sizes appropriate to detect the expected amount of change? Deciding early is also important for determining the extent and type of records and documentation that will be relevant for the audience and type of communication.
Your plans regarding reports, presentations, and publications may change as the project evolves. However, adapting a plan is usually easier than having no plan. If you have no initial plan, at the end of the project you may have overlooked a crucial aspect or you may have to spend appreciable time and resources recovering information.
c. Identify the needed skills. Identifying your aspirations early will also help you identify relevant skills that may be needed to carry out the project or to develop appropriate communications. The project lead may have ready access to skills needed to prepare a simple local report. More complex communications may involve assistance with literature searches, developing measures, statistical analysis, graphic design, or technical writing. Plan ahead to assure that needed skills will be available.
d. Selecting venues to submit presentation proposals and manuscripts. Project leaders are already likely to be aware of local institutional leaders who would be interested in a report on the project and its results.
Selecting venues to submit proposals for presentations (verbal or poster) and manuscripts for publication is often more difficult. The large number of regional and national meetings and the variety of journals offer many options.
One approach to narrowing the options is deciding whether your project's strength is in:
Innovations frequently apply existing improvement processes to new clinical contexts. This type of advance increases the likelihood of acceptance in meetings and journals related to the relevant clinical area. However, improvements addressing clinical areas that are national priorities (e.g., value-based performance measures, Joint Commission core measures) may be of interest both to clinical specialists engaged those areas and to QI professionals. Meetings and journals of either group may be considered.
Project leaders are likely to know from their own clinical backgrounds the meetings and journals most relevant to the clinical area addressed by the project. Project leaders are less likely to be aware of journals focusing on improvement in clinical care and patient safety. For examples of these journals, see:
Although the lists overlap in including journals that publish on QI in general, they focus on different detailed content and vary in the level of information presented about QI.
If a journal rejects a manuscript, reconsider the selection of journals. Does a rejection letter indicate the manuscript is fatally flawed or is the emphasis on it not fitting the interests/priorities of the journal? If the issue is interest/priority, consider other journals for which the content is relevant and edit the manuscript to reflect their priorities. Published manuscripts may have been through 2 to 4 previous rejections, with the review process helping shape and match the content a relevant journal.
e. Communication skills: writing and presenting. No matter how important the project, if information about it is not communicated in ways that are clear and easy for the audience to understand, your communications will be ignored.
A few illustrative recommendations include:
The TABLE OF CONTENTS for How to Write, Publish, & Present in the Health Sciences details many more points. Recommendations range from using effective words, sentences, and paragraphs to choosing the layout, fonts, and color for posters and for slides.
f. Formative feedback from representative audience members. In creating communications authors are often unaware that they both know too much and know too little.
These problems can be identified only if project leads share drafts of communications with representative members of the intended audience for feedback.
g. Peer review and common errors by authors. You can greatly improve your likelihood of acceptance by understanding how the review process works, the “audience” of reviewers, and common errors in manuscripts.
A GOOD OVERVIEW OF THE PEER-REVIEW PROCESS is presented in Shea JA, Caelleigh AS, Pangaro L, Steinecke A. Review process and publication decision. Academic Medicine, 2001; 76:911-921.
Common REASONS THAT REVIEWERS REJECT AND ACCEPT MANUSCRIPTS are summarized in Bordage G. Reasons reviewers reject and accept manuscripts: The strengths and weaknesses in medical education reports. Academic Medicine, 2001; 76:889-896. The reasons apply to all types of manuscripts and include:
For manuscripts reporting QI projects, failure to include information relevant to elements of the SQUIRE guidelines increases the likelihood of rejection.
When considering comments from reviewers, if they misunderstood or overlooked information in your manuscript, the underlying cause is that you did not present the information clearly for the reader. For an elaboration of this point, see Eva KW. THE REVIEWER IS ALWAYS RIGHT: peer review of research in medical education. Medical Education, 2009; 43:2-4.
h. Identify resources for guidance. Suggestions from individuals who have developed successful communications often provide excellent advice and mentoring. In seeking guidance:
For reports to leadership, who understands the priorities of leaders and can provide models of formats for reports that the leaders are used to seeing?
For presentations (poster or verbal), who has had previous proposals accepted at the relevant meeting and what advice can they provide regarding increasing the likelihood of acceptance
For publications, who has had manuscripts accepted by the targeted journal and what advice can they provide regarding the focus and presentation of your planned manuscript?
UMHS QI MOC Program personnel can provide lists of possibly relevant individuals:
Watch for more general resources for guidance, including:
Live sessions offered locally or at national meetings on developing more effective presentations and on how to write for publication.
Online sites providing guidance for publishing QI studies
Technical writers retained by some departments to assist with manuscript preparation.