Contributors to the guideline

Identification of priority questions and critical outcomes

In consultation with members of the GDG, the systematic review team and guideline methodologists, the WHO Steering Group first drafted a list of recommendation questions and potential critical and important outcomes related to interventions to prevent and treat maternal peripartum infections. The potential outcomes were identified through a search of key sources of existing clinical guidelines and relevant systematic reviews. This exercise generated a total of 34 potential outcomes relating to various interventions for preventing and treating maternal peripartum infections. WHO then consulted a larger group of international experts and stakeholders (including midwives, obstetricians, neonatologists, researchers, experts in health care programmes and representatives of user groups) to rank the outcomes for each guideline question through an electronic survey. Survey participants were asked to rank the relative importance of outcomes on a nine-point scale ranging from 1 (not important) to 9 (critical). The median score was calculated for each outcome based on the participants' responses, to determine outcomes that are “critical” (median score ≥7) and “important but not critical” (median score 4–6) for making decisions about the recommendations. To ensure consistency, the Steering Group reviewed the final list of critical and important outcomes for each guideline question. The prioritized outcomes rated critical and important were included in the scope of this document for evidence searching, retrieval, grading and formulation of recommendations. The list of critical and important outcomes according to the prioritized questions is provided in Annex 2.

Identification and retrieval of evidence

The systematic review team and guideline methodologists, in collaboration with the WHO Steering Group, retrieved evidence for each recommendation question from Cochrane reviews of randomized controlled trials and systematic reviews of non-randomized studies, as needed. The Steering Group provided the methodologists with standard operating procedures and a briefing on the desired output of the systematic reviews, and together agreed on the format and timelines for reporting. Using the assembled list of recommendation questions and prioritized outcomes from the scoping exercise, the WHO Steering Group, along with the systematic review team, identified systematic reviews that were either relevant or potentially relevant and assessed whether they needed to be updated. A systematic review was considered out of date if the last search date was two years or more prior to the date of assessment. For reviews found to be out of date, their authors were requested to update them within a specified time period. In instances where the authors were unable to do so, the updates were undertaken by the external team of systematic reviewers, in consultation with the WHO Steering Group.

Cochrane systematic reviews were the primary source of evidence for the recommendations included in this guideline.¹ The Cochrane reviews relating to interventions to prevent and treat maternal peripartum infections were based on studies identified from searches of the Cochrane Pregnancy and Childbirth Group's Trials Register. The Cochrane Pregnancy and Childbirth Group's Trials Register is maintained by the Trials Search Coordinator and contains trials identified from: monthly searches of the Cochrane Central Register of Controlled Trials (CENTRAL); weekly searches of MEDLINE; weekly searches of EMBASE; hand searches of 30 journals and the proceedings of major conferences; and weekly current awareness alerts for a further 44 journals plus monthly BioMed Central email alerts. The details of the search strategies for key databases such as CENTRAL, MEDLINE and EMBASE, the list of hand-searched journals and conference proceedings and the list of journals reviewed via the current awareness service can be found in the “Specialized Register” section within the editorial information about the Cochrane Pregnancy and Childbirth Group. Trials identified through the searching activities described above are each assigned to a review topic (or topics). The Trials Search Coordinator at the Cochrane Centre searches the register for each review using the topic list rather than keywords.

The assessment of the quality of individual studies included in Cochrane reviews of intervention studies follows a specific and explicit method of assessing the risk of bias. Briefly, two review authors independently assess the risk of bias using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions along six domains (14). Each included study is assessed and rated as being at low, high or unclear risk of bias for random sequence generation, allocation concealment, blinding of study personnel, participants and outcome assessors, attrition, selective reporting and other sources of bias such as publication bias. The assessment along these domains provides an overall risk of bias that indicates the likely magnitude and direction of the bias and how it is likely to impact on the review findings. All Cochrane reviews are preceded by the publication of a peer-reviewed protocol describing the review's proposed methods and search strategy.

The WHO Steering Group and methodologists worked together to determine the appropriateness and suitability of each systematic review in providing evidence for the priority questions by assessing its relevance, timeliness and quality. Relevance was ascertained by examining whether the population, intervention, comparisons and outcomes (PICO) considered in the full text of the review were compatible with those in the guideline question. The quality of each review was determined by assessing the clarity of its primary question with respect to the PICO; comprehensiveness of the search strategies and databases; potential for bias in the study selection and data extraction processes; methods of risk-of-bias assessment; and methods of data synthesis and reporting.

In situations where there were no suitable systematic reviews (Cochrane and non-Cochrane) or where the reviews lacked relevant data for specific guideline questions, new systematic reviews were conducted to inform the development of the recommendations. In such cases, the systematic review team was asked to prepare a standard protocol with a clear PICO question, and criteria for identification of studies, including search strategies for different bibliographic databases, methods for assessing risk of bias and a data analysis plan before embarking on the review. Then, the Steering Group reviewed and endorsed the protocol. To identify relevant studies, systematic searches of various electronic sources were conducted, including MEDLINE, EMBASE, CENTRAL, CINAHL, Popline, NLM Gateway and WHO regional databases. The search strategies employed to identify the studies and the specific criteria for inclusion and exclusion of studies were described in the individual systematic reviews. Studies from low-, middle- and high-income countries were considered, and there were no language restrictions. The entire systematic review development process was iterative, with the systematic reviewers and guideline methodologists constantly communicating with the WHO Steering Group to discuss challenges and agree on solutions.

Quality assessment and grading of the evidence

Quality assessment of the body of evidence for each outcome was performed using the GRADE approach (15). Using this approach, the quality of evidence for each outcome was rated as “high”, “moderate”, “low” or “very low” based on a set of criteria. The final rating of quality of evidence was dependent on the further consideration of the factors briefly described below:

Study design limitations: The risk of bias was first examined at the level of individual study and then across studies contributing to the outcome. For randomized trials, quality was rated as is (i.e. “high”) or downgraded by one (“moderate”) or two (“low”) levels depending on the minimum quality criteria met by the majority of the studies contributing to the outcome.

Inconsistency of the results: The similarity in the results for a given outcome was assessed by exploring the magnitude of differences in the direction and size of effects observed from different studies. The quality of evidence was not downgraded when the directions of the findings were similar and confidence limits overlapped, whereas the quality was downgraded when the results were in different directions and confidence limits showed minimal or no overlap.

Indirectness: The quality of evidence was downgraded where there were serious or very serious concerns regarding the directness of the evidence – i.e. whether there were important differences between the research reported and the context for which the recommendations are being prepared. Such differences were related, for instance, to populations, interventions, comparisons or outcomes of interest.

Imprecision: This assessed the degree of uncertainty around the estimate of effect. As this is often a function of sample size and number of events, studies with relatively few participants or events and thus wide confidence intervals around effect estimates were downgraded for imprecision.

Publication bias: Quality rating could also be affected by perceived or statistical evidence of bias to underestimate or overestimate the effect of an intervention as a result of selective publication based on study results. We considered downgrading evidence by one level if we had a strong suspicion of publication bias.

GRADE profiler software was used to construct summary of findings tables for each question: these tables include the assessment and judgements on the elements described above for each outcome and the estimated risks. Relevant information and data were extracted in a consistent manner from the systematic reviews relating to each key question by applying the following procedures: first, up-to-date review documents and/or data (e.g. RevMan file) were obtained from the review authors or the Cochrane Pregnancy and Childbirth Group. Second, analyses relevant to the critical and important outcomes were identified and selected. The data were then imported from the RevMan file (for Cochrane reviews) or manually entered into the GRADE profiler software (for non-Cochrane systematic reviews). For each outcome, GRADE assessment criteria (as described above) were applied to evaluate the quality of evidence. In the final step of the assessment process, GRADE evidence profiles were generated for each guideline question.

Formulation of recommendations

The GRADE framework was applied to formulate each recommendation based on the synthesized evidence. For each guideline question, the WHO Steering Group used the corresponding summaries of evidence for the critical outcomes, overall quality of the evidence, balance between benefits and risks, values and preferences, and resource implications to draft the recommendations. The draft recommendations, evidence summaries, the corresponding GRADE tables and other related documents were provided in advance to members of the GDG, who were then asked to comment on the document in tracked mode. The GDG members and other participants were then invited to attend a technical consultation (see Annex 1 for the full list of participants) organized at WHO headquarters in Geneva, Switzerland, in April 2015. At the technical consultation, the GDG members systematically reviewed and discussed these documents to finalize the recommendations and determine their direction and strengths.

Declaration of interests by external contributors

According to WHO regulations, all experts must declare their relevant interests prior to participation in WHO guideline development process and meetings. All GDG members and external contributors were, therefore, required to complete a standard WHO Declaration of Interest (DOI) form before engaging in the guideline development process and before participating in guideline-related meetings. The WHO Steering Group reviewed all declarations before finalizing experts' invitations to participate in the guideline development. Where any conflict of interest was declared, the Steering Group determined whether such conflicts were serious enough to affect the expert's objective judgement on the guideline development process and recommendations. To ensure consistency, the Steering Group applied the criteria for assessing the severity of conflict of interests in the WHO handbook for guideline development (13) for all experts. All findings from the DOI statements received were managed in accordance with the WHO DOI guidelines on a case-by-case basis and communicated to the experts.

The procedures for the management of declared conflicts of interests were undertaken in accordance with the WHO guidelines for declaration of interests (WHO experts). Where a conflict of interest was not considered significant enough to pose any risk to the guideline development process or reduce its credibility, the experts were only required to openly declare such conflict at the beginning of the GDG meeting, and no further actions were taken. Conflict of interest that warranted actions by the WHO staff arose where experts had obtained funding from a body or an institution to perform primary research or had performed a systematic review directly related to any of the guideline recommendations. At the GDG meeting, the concerned experts were restricted from participating in discussions and/or formulating recommendations pertaining to their academic conflicts of interest. Annex 3 shows a summary of the DOI statements and how the Steering Group managed declared conflicts of interest.

Decision-making during the technical consultation

The technical consultation process was guided by the following protocol: the meeting was designed to allow participants to discuss each of the recommendations drafted by the WHO Steering Group. Where necessary, each of these recommendations was revised, as needed, through group discussion. The final adoption of each recommendation was made by consensus – defined as the agreement by three quarters or more of the participants – provided that those who disagreed did not feel strongly about their position. Strong disagreements would have been recorded as such in the guideline (there was no record of such disagreement throughout the technical consultation). If the participants were unable to reach a consensus, the disputed recommendation, or any other decision, would be put to a vote. Voting is by a show of hands by members of the GDG. A recommendation or decision stands if a simple majority (more than half of the participants) vote in support of it, unless the disagreement is related to a safety concern, in which case the WHO Secretariat would choose not to issue a recommendation at all. WHO staff at the meeting, external technical experts involved in the collection and grading of the evidence, and observers were not eligible to vote. If the issue to be voted on involves primary research or systematic reviews conducted by any of the participants who have declared an academic conflict of interest, the participants in question would be allowed to participate in the discussion but would not be allowed to vote on the subject of discussion. In addition to evaluating the scientific evidence and its quality, values and preferences, relevant applicability issues and costs were also taken into consideration when formulating the final recommendations.

The technical consultation also determined the strength of each recommendation. By default, the strength of the recommendations discussed was aligned initially with the quality of the evidence (i.e. at the start of the discussion, “strong recommendations” were based on evidence of “moderate” and “high” quality, while “conditional recommendations” were based on evidence of “low” and “very low” quality). In addition to the quality of the evidence, the following factors were considered when determining the strength and direction of the final recommendations: the balance of benefits versus harms, values and preferences, and resource implications. The consideration of values and preferences was based on the experience and opinions of members of the GDG. Cost evaluation relied on reported estimates obtained during the evidence retrieval process as well as experiences and opinions of members of the GDG. Evidence-to-decision tables were used to note and synthesize these considerations and record the reasons for changes made to the default strength of the recommendations.

Document preparation and peer review

Prior to the technical consultation, the WHO Steering Group prepared a draft version of evidence summaries and corresponding recommendations. The draft document was made available to the participants in the technical consultation two weeks before the meeting for their comments. During the meeting, the draft recommendations were modified in line with participants' deliberation and remarks. Following the meeting, members of the Steering Group drafted a full guideline document with revisions to accurately reflect the participants' deliberations and decisions. The draft guideline document was sent electronically to GDG members for further comments before it was sent to the External Review Group for peer review. The Steering Group carefully evaluated the inputs of the peer reviewers for inclusion in the guideline document accordingly. After the technical consultation and peer review, the modifications made by the Steering Group to the guideline were limited to correcting factual errors and improving language to address any lack of clarity. The revised final version was returned electronically to participants in the technical consultation for their final approval.

Footnotes

1

As part of the Cochrane pre-publication editorial process, reviews are commented on by three peers (one editor and two referees external to the editorial team) and the Group's Statistical Adviser (see http://www​.cochrane.org/cochrane-reviews). The Cochrane Handbook for Systematic Reviews of Interventions describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of health care interventions.