Guideline development process

2Guideline development process

2.1. Scope of the guidelines

Development of these guidelines began with the WHO global update 2005: particulate matter, ozone, nitrogen dioxide and sulfur dioxide, and a subsequent planning meeting on WHO guidelines for indoor air quality held in Bonn in 2006. A proposal for indoor air quality guidelines for household fuel combustion was developed based on the outline plan drawn up in Bonn in 2006, relevant existing guidelines and consultation with partners.

Following planning approval and establishment of the WHO Steering Group (SG) and Guidelines Development Group (GDG), a planning meeting was held in Geneva in January 2011 attended by members of the GDG and WHO staff. This set out the scope, topic areas and priorities, and defined the approach for conducting systematic reviews and obtaining other evidence required for the recommendations.

At this meeting, the group decided it was not necessary to review the evidence informing the published WHO guidelines for air quality(AQG) (13, 14) and that these AQGs would provide the air pollutant standards for the current guidelines. For convenience, the guidelines for particulate matter (PM10 and PM_2.5) published in 2006, and all other combustion-derived indoor pollutants reviewed in 2010, are reproduced here (Table 2.1).

Table 2.1

Summary of published WHO air quality guideline values.

Evidence reviews were then commissioned, and drafts which had undergone a first round of external peer review were discussed at the main GDG and SG meeting, held in April 2012, in Delhi, in collaboration with the Indian Council of Medical Research (ICMR). At this meeting the scope was finalized, recommendations were drafted and decision tables used to set the strength of the recommendations.

2.2. Evidence review

2.2.1. Evidence required to address scoping questions

The first step in the evidence search and retrieval procedure was to identify and define the evidence required to address the scoping questions. Due to the nature of the policy challenges being addressed and the scarcity of experimental studies directly assessing the impact of household energy interventions on health, several distinct areas of evidence were required for each scoping question. These areas of evidence are summarized in Table 2.2. Those amenable to PICO (population, intervention, comparator, outcome) framing are indicated, and elaborated further below.

Table 2.2

Areas of evidence sought for each scoping question.

It was determined that a range of different types of reviews would be required to capture the varied and broad nature of the evidence required; accordingly, the following types of review have been conducted:

a systematic review (with meta-analysis if included);
a summary of a systematic review (with meta-analysis if included), where the review summarized is a recently conducted or published systematic review on a relevant topic;
a summary and synthesis of systematic reviews and other evidence, where the review brings together summaries of completed systematic reviews (with meta-analyses if included), and other evidence, and includes some synthesis of this evidence;
a model, which is used here to describe the emissions rate model in Review 3, and the IERs in Review 4;
a narrative review, where an overview of a set of issues that have not been the subject of asystematic, defined literature search is provided.

Details of the type of review used for each area of evidence, and the nature of evidence included, are provided in Table 2.3, Section 2.2.3.

Table 2.3

Summary of systematic and other evidence reviews, methods for individual study quality assessment, and overall evidence quality assessment. Further details of the assessment of quality, including upgrading and downgrading for specific outcomes, are available (more...)

2.2.2. Framing of questions

As noted in Table 2.2, it was possible to frame questions on two topics using the PICO format. These addressed (i) impacts of interventions on health outcomes (PICO-1, 2(a) in Table 2.2), and (ii) impacts of interventions on household levels of PM_2.5 and CO (PICO-2, 2(d) in Table 2.2). These are presented below, with additional explanation of the rationale for the outcomes selected.

PICO-1

Impact of interventions reducing HAP exposure on health outcomes: outcomes identified with an asterisk (*) are those assessed by the GDG as most important.

PICO-2

Impact of interventions on average levels of household air pollution.

Impacts of interventions on health outcomes (PICO-1)

Although it was judged important to review evidence for all child and adult health outcomes linked to HAP exposure, the GDG determined that the focus should be on specific important outcomes, that is, those that have an impact on child survival and development (e.g. acute lower respiratory infections (ALRI), low birth weight, stillbirth) and those responsible for a large burden of disease in the 2010 Global burden of disease (GBD) study (i.e. Global burden of disease chronic obstructive pulmonary disease (COPD), cardiovascular disease (CVD) and lung cancer) (3). These outcomes are indicated in the PICO-1 table below with an asterisk (*).

The epidemiological studies of these important health outcomes provide the largest and most robust source of evidence on the expected impacts of interventions on the risk of disease. The lack of HAP and/or exposure measurement in most, however, means that the exposure levels associated with these impact effect findings can only be estimated.

This leaves the question of risk levels with intermediate exposure reductions essentially unanswered. This latter (and critical) area evidence is provided by the exposure-response evidence, in particular the IER functions covered by topic 2(b) in Table 2.2, although not available for all of the important disease outcomes listed in the PICO-1 table. Where such evidence is available it is indicated in the PICO-1 table below by inclusion of [IER].

Impact of interventions in everyday use on household levels of PM_2.5 and CO (PICO-2)

The second area of evidence amenable to the PICO format examines the impacts of solid and clean fuel interventions on kitchen levels of PM_2.5 and CO, when these devices and fuels are in everyday use. Eligible studies were not found for all of the interventions listed. The important outcomes considered were average 24- or 48-hour kitchen levels of the above pollutants. Evidence of effect on personal PM_2.5 and CO exposure was also sought (and reported in the systematic review), but as this evidence was very limited and only available for some interventions, these are not included in the PICO table.

2.2.3. Evidence reviews and other information supporting recommendations

Evidence reviews

A series of reviews were conducted to obtain the evidence set out in Table 2.2, with the exception of topic 1(c) which used a modelling approach (see below), and topic 2(b) for which recently developed models combining risk data for multiple combustion sources were the primary source. Table 2.3 below shows how evidence has been reviewed or generated through models and how the evidence quality was assessed. Assessment of the overall quality of evidence for each topic is provided in Annexes 4–7, and full details of the rationale and methods used are available in Methods used for evidence assessment available at: http://www.who.int/indoorair/guidelines/hhfc.

As noted in Section 2.2.1, the evidence was summarized in different ways, depending on whether a new systematic review was conducted and reported in full, or whether existing (mostly published) systematic reviews were summarized.

Summaries nitrogen dioxide (of reviews were used where evidence on a range of outcomes was required and space would not have allowed full reporting of all systematic reviews, and/or high quality systematic reviews on the topic had recently been published. Where it was judged important to combine systematic review findings with other evidence, a synthesis was included. In one case (climate impacts and finance) a narrative review was judged to be the best approach, given the complex, multidisciplinary nature of this issue and the fact that this evidence served as context for implementation of the recommendations.

As noted, new systematic reviews were conducted for the purposes of these guidelines, unless a recent completed review meeting content, quality and peer-review criteria was available. In practice, this applied to recently published systematic reviews of (i) risks of asthma and wheeze in children with gas cooking and NO2 exposure (16) and (ii) health risks of kerosene use (17). In each case, the methods (key questions, search terms and strategy) for the published review were assessed, and a summary prepared (see Review 5, Section 4: Health risks of gas; Review 9: Health risks from kerosene).

For all new systematic reviews, search, data extraction and study quality assessment methods (described in Table 2.3 and in detail in the full texts of the systematic reviews, see: http://www.who.int/indoorair/guidelines/hhfc) were broadly similar. The details varied according to the type of evidence incorporated, e.g. laboratory testing, epidemiological studies, policy and case studies of adoption. There are also some variations in databases searched (in part as appropriate to the topic) and in languages included. For the systematic reviews of coal use (Review 8), Chinese language studies were included because a high proportion of the world's coal using households are located in China and important research had been conducted there. For other topics, where non-English languages were included (e.g. Chinese) it was found that searching databases in other languages made little, if any, difference to the included set of studies.

The quality of individual studies contributing to these reviews was assessed using standard methods applicable to the type of study. This varied considerably, ranging from laboratory emission studies to epidemiological studies, and case studies of implementation programmes. A summary of the methods used is provided in column 3 of Table 2.3, with further details in the full texts of the reviews (available at: http://www.who.int/indoorair/guidelines/hhfc).

Methods used for assessing the quality of the overall evidence provided by these reviews are described in section 2.3 below, and summarized in column 4 of Table 2.3.

Emissions model

In order to select a model suitable for the purposes of these guidelines, three commonly employed methods were reviewed (see full description in Review 3). Each of these combines the rate of pollutant emission (in terms of mass) within a room (e.g. kitchen) with mathematical models of pollutant transport and fate to provide estimates of indoor pollutant concentrations. These three types of model range from simple constructs to complex computer-based simulations and all have the capacity to provide indoor concentration estimates indicative of those observed in homes due to the device and fuel in question. These are:

The single zone model, which assumes that the pollutant emitted into room air is uniformly mixed throughout the space. Concentration is determined by emission rate and a number of other factors that can be incorporated into the model, including duration of combustion, room volume and air exchange rate.
The three zone model, which divides the room into three zones – a plume rising above the combustion device; warm air within a given distance from the ceiling; and the rest of the room. It is assumed that uniform mixing occurs in each zone. In other respects, this approach is similar to the single zone model.
The computational fluid dynamics model, which considers the forces involved in determining transport of air and pollutants within a room, by dividing the space into a large (or very large) number of small units, and developing equations incorporating momentum, thermal energy and conservation of mass for determining the resulting air pollutant concentrations.

Single zone models have been applied in work on household energy and air pollution for around 30 years, and this approach was adopted for the current purposes. The single zone model has the merit of simplicity in respect of the assumptions used. This is important when developing an approach that can be applied to populations. Such a model needs to account for a wide variation in factors (i.e. room size, air exchange rate, and duration of device use) which determine area concentration for any given emission rate. These have been incorporated by using a range of empirically-derived values for each factor combined in a Monte Carlo simulation. The input data used for the model were obtained from measurements made in India, and are summarized in Table 2.4.

Table 2.4

Input distributions for air exchange rates and kitchen volumes.

The output of the model for any given emission rate is therefore a distribution of air pollution concentrations, which can be used to describe the percentage of homes that achieve a specific air pollution goal, such as those in the air AQG. Examples of these distributions are provided in the full description of the model (Review 3). This modelling approach can be applied to any of the pollutants for which AQGs have been determined by WHO, if emissions data are available. For practical reasons, the model has been used to provide guidance just for PM_2.5 and CO, as these pollutants together serve as sufficient indicators of the health damaging potential of household fuel combustion in most situations.

Assessment of the quality of this evidence for the purpose of these guidelines (i.e. providing guidance on emission rates that will allow the AQGs to be met), is based on validation studies. The approach to this assessment is summarized in column 4 of Table 2.3, described in more detail in Annex 4 (Assessment of evidence for Recommendation 1), and in full in Review 3.

2.3. From evidence to recommendations

2.3.1. Overview

Potential interventions for addressing the health consequences of current global patterns of household fuel combustion tend to be complex. That is, they are actually a combination of interventions, not only use of effective technologies and cleaner fuels, but also action by multiple stakeholders to ensure equitable and lasting adoption.

The development of recommendations to address these issues therefore needs to draw on a wide range of evidence. This includes population studies of fuel use and exposure, laboratory emission data, epidemiological studies of exposure and health outcomes risk, intervention impact studies, qualitative evidence on user perceptions about change, and policy analysis. These sources of evidence use very disparate methods and research paradigms. In this field, randomized controlled trials – the gold standard of evidence of effectiveness – are rare. This is partly due to the practical difficulties of conducting these, but also because their relevance for evaluating the effect of complex interventions can be limited.

Grading of recommendations assessment, development and evaluation (GRADE), the standard WHO method for assessing certainty of effect (‘quality of evidence’) and setting the strength of a recommendation, provides a valuable framework for moving from evidence to recommendations. However, this system does not allow for comprehensive assessment of all evidence sources relevant to this topic. It also categorizes much of the evidence available as low or very low quality, an assessment that may undervalue the contribution such evidence can make.

In order to apply GRADE principles to the development of recommendations in this field, modifications have been made to the standard method. This revised approach, termed grading of evidence for public health interventions (GEPHI), is outlined in Section 2.3.2 below. A more detailed explanation of how each stage in the process has been applied for all types of evidence contributing to the guidelines is described in ‘Methods used for evidence assessment’, available at: http://www.who.int/indoorair/guidelines/hhfc. This includes an explanation of how results from this revised approach can be compared with standard GRADE methods.

The final stage of the process, that is, using GRADE decision tables to determine the strength of each recommendation, is relevant and applicable to this topic and was carried out in the standard manner. Since one component of the table (quality of evidence) has been derived from the modified GEPHI approach, for the purposes of clarity these tables have been renamed ‘decision tables for strength of recommendations.

2.3.2. The causal chain

A causal chain approach has been adopted to provide a framework for assessing the relationships between the varied types of evidence and complex interventions, (Figure 2.1). Using this approach, evidence that informs sequential and multiple links in the chain can be evaluated, and the overall coherence of evidence relating interventions to health outcomes can be assessed. For further explanation of the causal chain diagram and its application to the process of evaluating evidence, please refer to Methods used for evidence assessment, available at: http://www.who.int/indoorair/guidelines/hhfc.

Figure 2.1

Causal chain relating household energy technology, fuel and other interventions to health and safety outcomes via intermediate links.

The focus of this causal pathway is the source of the combustion emissions (for cooking, heating, lighting and other purposes in the home), since reduction of emissions is the most critical underlying factor for measures aimed at achieving the AQGs.

It is recognized, however, that other aspects of the home environment (for example, ventilation through chimneys, windows and eaves) and behaviour (how the stove is used, time spent by individuals in various micro-environments in and around the home) also play a part in the total dose of air pollutants and hence health effects. These have an impact on the causal chain at varying points, and insofar as available evidence allows, these other aspects are considered. Examples of the factors that can be assessed at each stage of the causal chain are shown in Table 2.5. The ways in which the different types of evidence described above provide information on different components of the causal chain are illustrated by the ‘pathways’ shown in Figure 2.1, and elaborated in Table 2.6.

Table 2.5

Examples of factors that may be assessed at each link.

Table 2.6

Examples of the types of evidence providing information on pathways in the causal chain.

Not included in the illustration of pathways in Figure 2.1 is evidence on factors influencing effective and equitable adoption of improved technologies, cleaner fuels and other interventions, as well as maintenance and replacement. These are indicated in the box in Figure 2.1, and reported in full in Review 7, Factors influencing adoption, available at: http://www.who.int/indoorair/guidelines/hhfc.

2.3.3. Assessment of the quality of the evidence

Assessing the strength of evidence for causal inference

When assessing the strength of evidence, a distinction was drawn between:

assessment of strength of evidence for causal inference, and
assessment of confidence in effect sizes.

The reason for taking this approach was that evidence may support causation and therefore removal of HAP exposure would result in a health benefit. The same evidence may not, however, support a high level of confidence in the estimate of health impact (i.e. risk reduction). The Hill viewpoints were used (as relevant) for the former, and GEPHI, the revised version of GRADE (described below) was used for the latter, recognizing that some aspects of the evidence could be assessed by both methods. This approach of distinguishing causal inference from confidence in effect size was used mainly for estimating the impacts of interventions on specific health outcomes.

2.3.4. Adaptation of the GRADE methodology

Modifications of the GRADE method for evidence evaluation, called in this guidelines project GEPHI include:

Entering non-randomized experimental evidence as ‘moderate quality’ (but with non-controlled before and after studies more likely to be downgraded), see rationale below;
Allowing upgrading by one level for each of the following where present:
1. If studies carried out using different designs and in widely different settings reported a similar direction of effects (note: statistical heterogeneity led to downgrading), see rationale below.
2. If there was supportive analogous evidence from other combustion sources, namely AAP, second-hand and active smoking.
Assessing the coherence of evidence contributing to different parts of the causal chain.

Details of the methods used for each review are provided in column 4 of Table 2.3, and in the Assessments of quality of evidence for each recommendation (Annexes 4–7). The GEPHI assessment tables for the two evidence reviews using the PICO format (impacts of intervention on health outcomes; impacts of interventions on kitchen HAP) are presented in tabular format in Annex 5. In the interests of transparency, an explanation is provided in Methods used for evidence assessment available at http://www.who.int/indoorair/guidelines/hhfc of how comparison between the standard GRADE and the GEPHI assessments can be made.

Rationale for assessment of strength of non-randomized experimental studies

Experimental studies, even if non-randomized, were judged to provide stronger evidence for these guidelines, for the following reasons.

Their main application was provision of evidence on the impact of improved stoves and cleaner fuels on household air pollution and personal exposure levels. The available studies fall into two main groups:

Those using cross-sectional or other designs in which comparison is made between homes that have started using the intervention fuel/technology mostly of their own volition and at their own expense with those continuing to use traditional methods.
Those using experimental designs where the new fuel/technology has been introduced into the home as part of a project or study, and comparison can be made using the home as its own control, and/or with parallel groups of control homes, where available.

The GDG judged that the first approach provides weaker evidence because the decision to adopt improved fuels/technologies is very strongly associated with socioeconomic and other development-related factors. These, in turn, may influence the way the new technology is used. These factors may differ markedly between groups of homes in the first study design and can (at best) be only partly controlled for. In the second study design these factors are controlled for by virtue of the house being its own control.

Nonetheless, factors such as seasonal practices and numbers of family members being cooked for will change over time. In the better quality experimental studies, these factors have been recorded and controlled for. Non-randomized experimental studies that had not examined or addressed these issues and were therefore subject to bias were downgraded in the GEPHI assessment.

Rationale for upgrading with consistency of effect across differing settings and study designs

This draws on the Hill viewpoint which refers to the importance of broadly the same answer being found in quite a wide variety of situations and techniques (20). For the diverse sets of epidemiological studies reviewed for the current guidelines, techniques has been interpreted as study designs (21).

2.3.5. Determining the strength of recommendations

The GDG used the decision tables for strength of recommendations to agree on the quality of evidence and certainty about harms and benefits, values and preferences, feasibility and resource implications and drew on these domains to set the strength of the recommendations.

These tables are further described in Methods used for evidence assessment available at: http://www.who.int/indoorair/guidelines/hhfc, and the complete versions are presented in Annexes 4–7.

The strength of the recommendation was set as either:

strong: the guideline development group agrees that the quality of the evidence combined with certainty about the values, preferences, benefits and feasibility of this recommendation means it should be implemented in most circumstances; or
conditional: there was less certainty about the combined quality of evidence and values, preferences, benefits and feasibility of this type of recommendation meaning there may be circumstances or settings in which it will not apply.

2.3.6. Procedure for group decisions

All decisions were reached by consensus, either at the GDG meetings or through the WHO-hosted EZCollab web facility which was used for finalizing wording of the recommendations, and responding to the external peer review comments on the recommendations. It was agreed at the beginning of the GDG meeting that, should there be disagreement a vote would be taken and a two thirds majority would be required for a decision to be carried.

Publication Details

Copyright

All rights reserved. Publications of the World Health Organization are available on the WHO website (www.who.int) or can be purchased from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857; e-mail: tni.ohw@sredrokoob).

Requests for permission to reproduce or translate WHO publications –whether for sale or for non-commercial distribution– should be addressed to WHO Press through the WHO website (www.who.int/about/licensing/copyright_form/en/index.html).

Publisher

World Health Organization, Geneva

NLM Citation

WHO Indoor Air Quality Guidelines: Household Fuel Combustion. Geneva: World Health Organization; 2014. 2, Guideline development process.

Table 2.1Summary of published WHO air quality guideline values

Pollutant (unit for guideline)	Mean concentration over averaging time							Unit risk	Comments	Ref.

	10 min	15 min	30 min	1 hour	8 hours	24 hours	1 year
PM_2.5 (μg/m³)	–	–	–	–	–	25^a	10	–	24-hour guideline max 3 days/year	WHO 2006 (14)

PM₁₀ (μg/m³)	–	–	–	–	–	50^b	20	–	24-hour guideline max 3 days/year	WHO 2006 (14)

Benzene (risk of leukaemia per 1 μg/m³)	–	–	–	–	–	–	–	6.0 × 10⁻⁶	No safe level	WHO 2010 (13)

CO (mg/m³)	–	100		35	10	7	–	–	–	WHO 2010 (13)

Formaldehyde (mg/m³)	–	–	0.1	–	–	–	–	–	–	WHO 2010 (13)

Naphthalene (mg/m³)	–	–	–	–	–	–	0.01	–	–	WHO 2010 (13)

Nitrogen dioxide (μg/m³)	–	–	–	200	–	–	40	–	–	WHO 2010 (13)

Polycyclic-aromatic hydrocarbons^a (risk of lung cancer per 1 ng/m³ B[a]P	–	–	–	–	–	–	–	8.7 × 10⁻⁵	No safe level	WHO 2010 (13)

a: In view of the difficulties in developing guidelines for PAH mixtures, benzo[a]pyrene was considered to represent the best single indicator compound (see WHO 2010) (13).

b: The 24-hour average values for PM₁₀ and PM_2.5 refer to the 99th percentile of the distribution of daily values, i.e. the fourth next highest value of the year.

Table 2.2Areas of evidence sought for each scoping question

Scoping question	Evidence required for scoping question	Framing of evidence requirement (topic)
1. Emission rates to meet AQGs	a. Published WHO AQGs	a. Reference to AQGs (PM_2.5, CO).
	b. Emission rates of key pollutants from traditional devices/fuels and intervention options	b. Summary of laboratory and field test results for PM_2.5, CO (and other important pollutants).
	c. Relationships between emission rates and indoor air quality	c. Model relating emission rates to predicted kitchen concentrations for PM_2.5 and CO.
2. Policy during transition	a. Disease risks from household air pollution (HAP) and estimated effect sizes for impacts of interventions	a. Summary of evidence relating HAP to specific disease outcomes, strength of evidence for causal inference and intervention effect sizes: defined by PICO-1 (see Section 2.2.2).
	b. Relationships between level of exposure and level of risk for important¹ disease outcomes across the full range of exposure seen with intervention options	b. Summary of evidence on exposure-response relationships for important disease outcomes.
	c. Levels of HAP and exposure experienced by populations using traditional stoves/fuels, and intervention options	c. Summary of observational population-based studies with measured average PM_2.5 and CO.
	d. Impacts of interventions on HAP levels achieved with stoves/fuels in everyday use	d. Summary of observational (where relevant) and experimental studies (randomized and non-randomized) with measured average PM_2.5 and CO: defined by PICO-2 (see Section 2.2.2)
	e. Nature and extent of barriers to transition to improved solid fuel stoves and clean fuels	e. Summary of evidence on factors influencing the adoption and sustained use of interventions.
3. Coal use	a. Health impacts of solid fuel use	a. As for 2(a) and (b) above.
3. Coal use	b. Health risks specific to household use of coal	b. Summary of evidence on carcinogenicity, toxic contaminants, and constraints on clean combustion of coal.
4. Kerosene use	a. Health risks specific to household use of kerosene	a. Summary of evidence on kerosene use, levels of pollutants, and health impacts.

1: See PICO-1 table in Section 2.2.2 (below) for details of important health outcomes.

PICO-1Impact of interventions reducing HAP exposure on health outcomes: outcomes identified with an asterisk (*) are those assessed by the GDG as most important

PICO-1	Description
Population	The 2.8 billion people (15) using solid fuels, that is biomass (wood, animal dung, crop wastes, charcoal) or coal as their primary cooking and heating fuel, with open fires or traditional stoves.
Intervention	Clean fuels (LPG, electricity) and/or a range of ‘improved’ solid fuel stoves, delivering substantial reductions in HAP exposure. Exposure levels have mostly not been measured in the relevant studies, but have been estimated to lie between the WHO annual IT-1 of 35 μg/m³ PM_2.5 at the lower end of the range and 75 μg/m³ PM_2.5 at the upper end.
Comparator	Households using solid fuels with traditional stoves
Outcome	Young child (under 5 years)	Adult
	Acute lower respiratory infections (ALRI)* [IER]	Chronic obstructive pulmonary disease (COPD)* [IER]
	Low birthweight*	Lung cancer with coal exposure* [IER]
	Stillbirth*	Lung cancer with biomass exposure* [IER]
	Stunting*	Cardiovascular disease*^a [IER]
	All-cause mortality under 5 years*	Cataract
	Cognitive development	Other cancers
		Asthma (adult and child)

a: Very few primary studies are available on the risk of cardiovascular outcomes with exposure to solid fuels or associated HAP levels, and health risk assessment has relied on interpolation from risk functions for other combustion sources; as a consequence, GEPHI assessment of evidence quality (explained below) has not been applied to this outcome.

PICO-2Impact of interventions on average levels of household air pollution

PICO-2	Description
Population	The 2.8 billion people using solid fuels, that is biomass (wood, animal dung, crop wastes, charcoal) or coal as their primary cooking and heating fuel, with open fires or traditional stoves.
Intervention	Improved solid fuel stoves	Clean fuels
	With chimney	Liquefied petroleum gas (LPG)/natural gas
	Without chimney	Ethanol
	Mixed (stove plus other improvements to kitchen and cooking arrangements)	Biogas
		Solar cookers
		Electricity
Comparator	Households using solid fuels with traditional stoves
Outcome	Average 24-hour (or 48-hour) concentrations of: Kitchen PM_2.5 Kitchen carbon monoxide (CO)

Table 2.3Summary of systematic and other evidence reviews, methods for individual study quality assessment, and overall evidence quality assessment

Further details of the assessment of quality, including upgrading and downgrading for specific outcomes, are available in the assessments of evidence quality for each of the specific recommendations (Annexes 4–7)

Review number, (short) title and area of evidence (topic) addressed	Type of review and evidence included	Methods for assessment of individual study quality	Methods for assessment of quality of the set of evidence compiled in the review
Review 1: Fuel use Global and regional summary of household fuel and technology use for cooking, heating and lighting	Synthesis and analysis: Includes a synthesis and analysis of survey data and national statistics reports from low, middle and high income countries on the fuels and technologies used by households for cooking, heating and lighting, as well as a summary of trends in solid cooking fuel use based on modelled estimates.	All surveys or reports included in the analysis had to meet minimum criteria for date, population representation, and methods of data collection (e.g. survey questions used).	A global dataset was available for cooking fuel use. Data related to heating, lighting and cooking devices were less comprehensive and in some cases no aggregated figures by region could be calculated and only national level statistics are presented. Inconsistencies in data collection or reporting methods were reconciled through grouping of response data or exclusion from analysis.

Review 2: Pollutant emissions Summary of laboratory and field test results for PM_2.5, CO (and other important pollutants).	Systematic review: Includes mainly laboratory and a smaller number of field-based measurements of emissions of major pollutants from a representative range of stove and fuel types.	Suitability of the emission test protocol used, adherence to protocol and quality assurance of methods used.	Suitability of test protocols used for emissions testing, and extent of adherence to protocols and quality control/assurance by laboratories. Identification of issues for interpretation of results from laboratory and field testing.

Review 3: Emissions model Model relating emission rates to predicted concentrations for PM_2.5 and CO.	Model: Includes a review of alternative modelling approaches, and a description of the single zone box model using Monte Carlo simulation to link emission rates (inputs) with distributions of average concentrations PM_2.5 and CO in the home (outputs).	Not applicable, although the limitations of the model and currently available data inputs are discussed.	Validation of predicted kitchen concentrations of PM_2.5 and CO against empirical data from homes in India and countries from other regions; interpretation of findings in light of evidence of emissions from multiple sources in real-life settings, including from neighbouring homes and other sources of combustion-derived air pollution.

Review 4: Health impacts of household air pollution (HAP)	Summary and synthesis of systematic reviews, modelling and other evidence:
Summaries of evidence relating HAP to specific disease outcomes, causal evidence and effect sizes: defined by PICO-1, exposure-response evidence, risk from use of gas, and impacts of smoke reduction on vector-borne disease.	1. Summary and synthesis of completed systematic reviews and meta-analyses (SRMA) of epidemiological studies linking HAP exposure (from solid fuels and gas) to a range of health outcomes.	1. Intervention impact estimates: Evaluation of study quality was carried out using versions of Newcastle-Ottawa scale, adapted to each type of study design.	1. Intervention impact estimates: Bradford Hill viewpoints to assess strength of causal evidence; assessment using GEPHI^a for quality and precision of intervention effect estimates (details of grading by disease outcome are provided in Annex 5).

	2. Summary of all available exposure-response evidence including newly developed integrated exposure-response models for several disease outcomes.	2. Integrated exposure-response functions: all studies of risk associated with household air pollution which contributed to these integrated models were assessed individually as described above.	2. IER functions: GRADE^b domains (number of studies, study design, risk of bias, indirectness, imprecision, inconsistency, and publication bias) were used as a guide, and applied in the most appropriate way given the nature of the available evidence. Assessment was made (i) generically for the IER approach and model assumptions, and (ii) in respect of specific issues for each disease outcome.

	3. Summary of published SRMA of health risks from household use of gas.	3. Individual study quality assessment was not available, but general quality issues for studies (especially for exposure assessment) were identified and discussed.	3. Assessment focused on the inconsistency across findings of the SRMA between the two measures of exposure (gas and NO2) and the two outcomes (wheeze and asthma), and potential explanations for this.

	4. Summary of published systematic review of risk of vector-borne disease (VBD) from potential interventions to reduce HAP exposure.	4. Individual study quality assessment was not available, but general quality issues including confounding are discussed.	4. Meta-analysis was not attempted, so formal assessment with GEPHI not conducted. Assessment focuses on lack of experimental studies, and potential for confounding.

Review 5: Population levels of HAP and exposure Summary of observational population-based studies with measured average PM_2.5 and CO.	Systematic review: Includes studies which have measured 24-hour or 48-hour concentrations of PM and CO in kitchens, other rooms within homes, in the local ambient air, and personal exposure to these same pollutants for men, women and children.	Methods used for selecting samples of homes and individuals, protocols for measurements of PM2.5 and CO, and evidence of quality assurance procedures.	GRADE domains (number of studies, study design, risk of bias, indirectness, im precision, inconsistency, and publication bias) were used as a guide. All eligible studies provided measures of long-term average (e.g. 24 hour or 48 hour) levels of pollutants, which increased consistency of the findings.

Review 6: Intervention impacts Summary of observational (where relevant) and experimental (randomized and non-ranomized) studies with measured average PM_2.5 and CO defined by PICO-2.	Systematic review and meta-analysis: Includes studies which provide data on average home-based 24-hour or 48-hour kitchen and/or personal PM_2.5, PM₄ or CO using either experimental designs, or observational studies of intervention programmes.	Evaluation of study quality using versions of Newcastle-Ottawa scale, adapted to each type of study design, including adequacy of description and application of standardized HAP measurement.	Assessment using GEPHI to assess the quality and precision of estimates for each pollutant (PM_2.5, CO), and for each group of stove or clean fuel intervention (details of grading by intervention type and pollutant are provided in Annex 5).

Review 7: Factors influencing adoption Summary of evidence on factors influencing the adoption and sustained use of interventions.	Systematic review: Includes quantitative, qualitative and policy/case studies in LMICs reporting on factors influencing adoption and/or sustained use of improved solid fuel stoves, and four types of clean fuel (LPG, biogas, alcohol, solar cookers)	Quantitative studies: Used versions of Newcastle-Ottawa scale, adapted to each type of study design. Qualitative studies: Used methods described by Harden et al. 2009 (18). Policy and case studies: Used methods for case studies described by Atkins & Sampson 2002 (19).	GRADE domains (number of studies, study design, risk of bias, indirectness, inconsistency, imprecision for quantitative evidence, publication bias) were used as a guide, and applied in the most appropriate way given the nature of the available evidence; consistency of findings across different study designs and settings was important.

Review 8: Coal Summary of evidence on carcinogenicity, toxic contaminants, and interventions to reduce adverse health effects including bans and other restrictions on household use of coal.	Summary and synthesis of systematic reviews and other evidence: 1. Synthesis of studies on health risks from coal arising from products of incomplete combustion, drawing on published systematic review, and other studies identi-fied through systematic search.	For carcinogenicity, methods used are those described by the International Agency for Research on Cancer (IARC).	Three distinct components contributed to the overall evidence available for coal: carcinogenicity, health effects from products of incomplete combustion (PICs), and toxic contaminants. Carcinogenicity: IARC methods are based on assessment of human epidemiology, animal evidence and mechanistic evidence.

	2. Summary of evidence from IARC monograph on carcinogenicity of emissions from household coal use. 3. Systematic review of studies (intervention and observational) relating to health risks from toxins in coal.	Quality assessment of studies of health risks from coal use (from products of incomplete combustion, and from toxic contaminants) was based on evaluation of methods used for sampling, exposure and outcome assessment, and analysis including adjustment for confounding. This information was used to provide an overall quality assessment.	PIC effects: For lung cancer, the GEPHI assessment from Review 4 (Health impacts of household air pollution) was used. For other (non-cancer) outcomes, due to small numbers of studies and heterogeneity of outcome definitions, meta-analysis was not conducted and GEPHI was not applied. GRADE domains (number of studies, study design, risk of bias, indirectness, inconsistency, imprecision, publication bias) were used as a guide. Toxic contaminants: Assessment of quality was based on the combination of studies (including experimental studies) reporting coal toxin content, emission and area concentrations of toxins, and population studies of specific outcomes (e.g. fluorosis, arsenicosis) in areas where households burn coal.

Review 9: Kerosene Summary of evidence on kerosene use, levels of pollutants, and health impacts.	Summary of systematic review: Includes studies reporting on kerosene use (fuel type/grade and devices) for cooking, heating and lighting; emissions of major pollutants and area concentrations; epidemiological studies on health risks with kerosene use in the home.	Evaluation of study quality was based on methods used for exposure and outcome assessment, and analysis. A formal quality scoring tool was not used.	Due to the substantial heterogeneity in study methods, quality and findings, meta-analysis was not attempted for any of the health outcomes, and GEPHI assessment was not applied. GRADE domains (number of studies, study design, risk of bias, indirectness, inconsistency, imprecision for quantitative evidence, publication bias) were used as a guide.

Review 10: Safety Summary of evidence on burns, scalds and poisoning.	Systematic review: Includes studies reporting on rates of burns and poisoning from household energy use, risk factors, impact of interventions. Also includes a description of a newly developed safety testing protocol for solid fuel stoves.	Evaluation of intervention study quality using versions of the Newcastle-Ottawa scale, adapted to each type of study design.	For studies of risk factors, methodological issues such as case selection – most were drawn from facilities with few population studies – were a quality concern. Experimental studies (both randomized and non-randomized designs available) were generally of good quality, but too variable in terms of interventions and outcomes to carry out meta-analysis.

Review 11: Climate impacts Summary of evidence on climate impacts (warming) from inefficient use of non-sustainable bio-mass, and emissions from incomplete fuel combustion.	Narrative review: In view of the complexity of climate science (and this not being the main focus of the guidelines), a narrative review providing an overview of the impacts of household fuel combustion on climate was provided. This draws on a recent comprehensive United Nations Environment Programme (UNEP) report on the effects on climate of short-acting pollutants and other published studies.	Individual climate science studies on the impacts of household fuel combustion pollutants on climate warming were not assessed separately.	The overall evidence provided by the climate science studies on the impacts of household fuel combustion pollutants on climate warming was not assessed. The consistency of evidence indicating substantive net warming effects draws strongly on the conclusions of the UNEP report.

a: GEPHI: Grading of evidence for public health intervention (see Section 2.3.3)

b: Grading of recommendations assessment, development and evaluation.

Table 2.4Input distributions for air exchange rates and kitchen volumes

Parameter	Unit	Geometric mean	Range		SD¹
Parameter	Unit	Geometric mean	minimum	maximum	SD¹
Air exchange rate (α)	per hour	15	5	45	7.5
Kitchen volume (V)	m³	30	5	100	15
Device burn time	hours per day	4	0.75	8	2

1: SD: Standard Deviation

Figure 2.1Causal chain relating household energy technology, fuel and other interventions to health and safety outcomes via intermediate links

Table 2.5Examples of factors that may be assessed at each link

Interventions	Emissions and safety features	Environmental level	Personal exposure	Health and safety outcomes
Improved solid fuel stoves Clean fuels and associated technologies for cooking, heating and lighting	Emission rates of toxic pollutants directly into homes Emissions (e.g. via flue) to ambient air Inherent safety (e.g. stability, enclosed flame, raised surface)	Concentrations of pollutants in kitchen and other areas of the home Concentrations of pollutants in ambient air that can enter the home	Exposure of children, women and men to pollutants Function of time spent in various micro-environments Exposure to open flame or risk of falling pots with hot liquids	Range of child and adult health outcomes from exposure to pollutants (ALRI, COPD, lung cancer, CVD, etc.) Safety outcomes (e.g. burns, scalds) Other health and socio-economic impacts

Table 2.6Examples of the types of evidence providing information on pathways in the causal chain

Pathway	Type of evidence	Explanation
A	Laboratory emissions testing	Provides information on the rates of emissions of toxic pollutants, for example in relation to a unit of energy delivered
B	Emissions model	Relates emission rates to predicted concentrations in the home, based on assumptions about duration of use, air exchange rates and kitchen volume
C	Epidemiological studies	Investigate the risk of a range of disease outcomes among those using more polluting fuels compared to groups with lower exposure, for example using clean fuels; such studies may or may not include measurement of HAP and/or exposure
D (a, b)	Experimental studies	Randomized and non-randomized experimental studies that measure the impact of introducing an improved stove or clean fuel on pollutant concentrations or personal exposure
E	Epidemiological studies	Studies which include exposure assessment may allow investigation of the relationship between exposure and disease risk
F	Experimental and observational studies	Randomized and non-randomized experimental, and some observational, studies that investigate the impact of an improved stove or clean fuel directly on risk of health outcomes