Summary

Direct, quantified evidence on exposure-response relationships is limited to two studies for one outcome (child ALRI), and these two sets of data could not easily be combined. Some exposure-response evidence is available for other outcomes, provided in Review 4. While this is useful for establishing causation, it is not quantified in terms of exposure. The IERs are a relatively new approach and have some important assumptions, but are mostly based on an extensive and broad evidence base. That developed for child ALRI is based on the only directly assessed individual exposure data, and shows consistency between the IER and pooled estimate for the predominantly observational epidemiological studies available.

Overall, the IER evidence was assessed as being of moderate quality.

Studies

All studies included were cross-sectional. The inclusion criteria covered: provision of adequate detail on sampling criteria, sampling methods (including specification of sampling devices, flow rates, calibration procedures etc.), analytical methods (including specification of analytical instrumentation, sensitivity and, wherever applicable, specificity of method), calibration standards and corrections for measurement errors (such as co-locating or calibrating against gravimetric samplers for light-scattering devices used for measuring PM).

Risk of bias

As the inclusion criteria for this review covered those aspects of study design and conduct that may bias results, the risk of bias was judged to be low. In order to provide comparable average levels of PM and CO, only those studies reporting 24-hour or 48-hour measurements were included. As studies have variously measured PM_2.5, PM₄ and PM₁₀, results for these different particle size cut-offs were reported (and averaged) separately.

Heterogeneity

No formal assessment of statistical heterogeneity was made. It was expected that household and personal exposure levels would vary greatly due to the variability in household energy use, housing type, seasonal factors, etc. The pattern of variability was, however, generally not suggestive of unreliable results, as values for homes using traditional stoves and solid fuels reported high (albeit variable) levels of PM and CO, and did not have unusually low values. Some of the studies of homes using clean fuels found levels higher than might be expected on the basis of emission rates, but these could be explained by multiple stove/fuel use in the study homes, and emissions from neighbours and other external sources of combustion.

Precision

The precision of estimates varied considerably by type of stove and fuel, by outcome (pollutant) measure, and level of aggregation. Precision of global results is considered here, but are lower for the regionally-stratified results also reported by the systematic review. For area kitchen levels with traditional solid fuel stoves, there were almost 20 studies with more than 600 subjects for PM_2.5 and larger numbers for CO. Adequate precision was also available for area kitchen levels with improved solid fuel stoves, and for personal exposure measurements of both PM and CO with solid fuel use among both women and children.

There was less precision (due to lower subject numbers) in studies measuring kitchen area concentrations when clean fuels were used. Three studies (56 subjects) measured PM and only one study (9 subjects) measured CO. No personal PM or CO exposure data were available for clean fuels. All of the weighted average pooled estimates are provided with standard deviations.

Publication bias

This was not formally assessed due to the large variability expected between studies from different regions. It is possible that some unpublished studies have not been included.

Summary

This assessment found that the evidence for the majority of area and personal exposure for PM and CO was of moderate quality, but precision was limited (principally by small numbers of studies and subjects) for homes using clean fuels.

Study design

The available evidence was drawn from a wide range of study designs, namely randomized trials, before-and-after studies, cross-sectional surveys, economic and survival analyses, in-depth and semi-structured interviews, focus group discussions and several mixed-methods studies. While the studies were not always designed primarily to answer questions about adoption (e.g. some of the health studies), the majority were, and the designs used were generally appropriate for the purpose, with the caveat of a lack of longer-term prospective studies of sustained adoption. The extent to which findings from these different study designs are consistent is considered below.

Risk of bias

Quality assessments were conducted according to established criteria for each study design (described in full in Review 7). Some quantitative studies used sampling that may not have been representative, and some used only simple descriptive (unadjusted) analyses, but were considered of high internal validity. It was not possible to perform a formal assessment of risk of bias for the qualitative or case studies, but there was reasonable coherence of findings across study designs, discussed further below. The sensitivity analyses carried out with, and without, the studies assessed to be of low quality found this did not affect the findings. This assessment supported the conclusion that there was no serious risk of bias.

Indirectness

All the studies included were required to provide direct evidence on adoption and/or sustained use of improved solid fuel stoves or use of one of the four clean fuels included in the review (LPG, alcohol fuels, biogas, solar cookers).

Inconsistency

It was not possible to use measures of statistical heterogeneity to assess this body of evidence. However, an indirect assessment was made, based on evidence reported during the data extraction and synthesis process. Studies which were out of line with the majority of studies were noted in the initial synthesis stage. An assessment of the records and the overall synthesis based on combined study designs showed that inconsistency among studies of similar interventions in comparable settings was uncommon. It was concluded that inconsistency was not a major issue.

Imprecision

Imprecision reflects subject and event numbers. For quantitative studies, the individual sample sizes and their representativeness were summarized, with 19 out of 22 studies having a sample size of 200 or more individuals (including surveys of more than 1000 individuals). Assessment of an overall pooled effect was not appropriate due to the different interventions and outcomes. For qualitative evidence, sample size is less relevant, as meaningful information can be obtained from a smaller number of study participants. For case and policy studies, it was possible to assess precision of some elements of the data used (e.g. quantitative components of studies) when this information was reported. Some of the case studies made use of cross-sectional or longitudinal surveys; 17 such studies had sample sizes of 200 or more individuals.

Publication bias

This cannot be assessed in the formal way used for quantitative systematic reviews-using funnel plots and statistical tests – but publication bias may never theless be present. While it was difficult determine whether there was a bias towards not publishing unsuccessful programmes, there were multiple examples of projects and programmes with mixed experience of adoption and sustained use. A related form of publication bias may arise from non-peer-reviewed (and to a lesser extent also from peer-reviewed) reports published by authors who have managed or were otherwise very close to the implementation process. Only one fifth of the studies included were peer-reviewed, with the rest being research reports, dissertations, conference proceedings and book chapters. About 12 of the 101 studies included seem likely to have been written and/or published by authors closely associated with the implementing programme or agency. It was reassuring to find no marked differences in findings between these two groups of publications, other than case studies (which were less likely to be peer-reviewed) usually focusing on a wider spectrum of factors influencing uptake and including the domains concerned with regulation, certification and institutional arrangements.

Consistency of evidence

One of the additional criteria proposed with GEPHI was recognition of similar findings among studies conducted using different designs, and across multiple settings. This appears to be a feature of this set of studies. Different forms of consistency were considered as follows:

• Consistency of evidence across different study designs: findings supported by more than one study type are likely to be more valid or of greater relevance than findings supported by a single design or paradigm. This is one of the strengths of this evidence base, although in terms of consistency it is weaker for some of the clean fuels (in particular alcohol fuels) due to the limited available evidence.
• Consistency of evidence across different settings: findings supported by studies using very distinct interventions, settings, socioeconomic and cultural contexts are likely to be more valid or of greater relevance than findings supported by studies from one or a few settings. Studies were identified across Africa, Asia and Latin America which is an additional point of strength of this evidence base.

Other GRADE criteria

There are other GRADE criteria which could not be considered in the assessment of this set of studies. While a large effect can lead to upgrading, this was not relevant as too few (quantitative) studies have provided comparable effect estimates and no attempt to pool such effects was made. Plausible confounding can weaken observed effects and, if present, potentially lead to upgrading, but this was not consistently assessed across studies and could not be evaluated. Finally, investigation of an exposure-response relationship was not supported by data, and could not be considered.

Summary

Assessment of this set of evidence suggests that it provides a consistent and moderately strong basis for drawing conclusions about the design and delivery of programmes to ensure more effective adoption and use of improved solid fuel stoves and cleaner fuels. Among clean fuels, the evidence for alcohol fuels and to a lesser degree that for solar cooking, is weaker. It is also notable that no studies of adoption of newer improved solid fuel stove technologies (i.e. advanced combustion fan-stoves) were available.

Overall, the evidence on factors influencing adoption was assessed as being of moderate quality.

Traditional chulha

For the traditional chulha over 60% of the distribution for the modelled 24-hour PM_2.5 concentrations was between 500 and 1800 μg/m³ with a mode of 800 μg/m³. This is very similar to the kitchen concentrations reported by studies in the WHO South-East Asia region (826 ± 1038 μg/m³) and globally for solid fuel users (972 ± 876 μg/m³) (see Review 5).

Rocket stove

Modelled 24-hour PM2.5 concentrations derived from field-based emissions rates of the rocket stove (which should be more comparable to observed levels of air pollution than laboratory-based rates), had a mode of around 500 μg/m³, a reduction of around 300 μg/m³, or nearly 40% compared to the traditional chulha. CO emissions were reduced from a mode of around 11 mg/m3 to 5 mg/m3, or by around 55%; this concentration of CO for the rocket stove (5 mg/m³) lies below the WHO 24-hour AQG.

The review of intervention impacts (see Review 6) found that this type of stove reduced PM_2.5 by an average of 260 μg/m³ and CO by 3.41 ppm (3.9 mg/m³), with weighted mean percentage reductions of 48% and 39% respectively, and post-intervention means of 410 μg/m³ and 6.6 ppm (7.6 mg/m³) respectively. Given the variability in data, devices, fuel used and other factors, these results can be considered consistent. Although the post-intervention CO mean is slightly above the WHO 24-hour AQG, this distribution is positively skewed, and the majority of studies found values below this guideline value.

LP Gas

The model predicts that almost all LPG-using homes would have concentrations of PM_2.5 below the WHO interim target (IT-1) value of 35 μg/m³, and CO levels below 1 mg/m³. The review of population-based studies (see Review 5) found levels of PM_2.5 in gas (and other clean fuel-using homes) to be mainly in the range 35–80 μg/m³, with some above 100 μg/m³. Equivalent CO data were not reported. The review of intervention impacts (see Review 6) found few studies of clean fuels (one for LPG, four for ethanol): reductions for PM_2.5 were between 60% and 80% with post-intervention mean PM_2.5 levels between 120 and 280 μg/m3. For CO, reductions were very similar in percentage terms, with post-intervention means between 2.7 and 5.9 ppm (3.1 and 6.6 mg/m³).

Even allowing for variability and differing circumstances, it is clear that the measured levels of PM and CO in homes using clean fuels are much higher than predicted. This does not undermine the model, but points towards other explanations. These include continued use of the traditional stove (even in stove/fuel evaluation studies), along with the new one (known as stacking), other emission sources in and around the home (kerosene lamps, waste burning), and external sources such as fuel combustion from other homes and other sources of combustion contributing to outdoor air pollution entering all homes. The review of population studies (see Review 5) found that average outdoor PM_2.5 concentration in the vicinity of solid-fuel using homes was 106 (SD = 79) μg/m³. Most of these studies were conducted in rural areas. Clearly, if ambient pollution levels are this high, it will not be possible for homes exclusively using LPG or other clean fuels to reach the levels below 35 μg/m³ as predicted by the emissions model. Any concurrent use of the traditional stove, polluting lamps and other combustion sources will further increase the pollution in and around the home.

Vented stoves

The emissions model allows for ventilation (with a flue or chimney) by assuming (based on empirical data from several studies and countries) that the fraction of total emissions entering the room lies between 1% and 50% with a mean of 25% and standard deviation of 10%. On average, therefore, it is expected that emissions entering the room from vented stoves are 75% lower than with unvented stoves. The review of intervention impacts (see Review 6) found that solid fuel stoves with chimneys (for which there were 23 and 22 estimates for PM_2.5 and CO respectively) did indeed achieve a greater reduction of PM_2.5 and CO than unvented stoves. This reduction was 63% for both pollutants, with post-intervention means of 370 μg/m³ and 4.2 ppm (4.8 mg/m³) for PM_2.5 and CO respectively.

Although vented stoves achieved larger reductions in emissions as predicted, the improvements were nowhere near as large as might be expected and post-intervention levels remained high. It should be noted, however, that several of the chimney-stove studies reported the largest reductions in emission levels of all stoves and fuels studied in the review (see Review 6). Three such studies reported PM_2.5 levels of between 50 and 80 μg/m³ post-intervention, which are more consistent with the larger reductions predicted by the model. These findings do not undermine the model but point towards reasons why this much better performance is not being achieved more widely. As discussed above, other sources in the home and AAP are likely to be responsible. In the case of vented stoves, the fraction not entering the room is simply moved to the outside of the home, contributing directly to the often high levels of outdoor pollution, as reported in Review 5.

Systematic review of health risks of HAP

A substantial range of health outcomes have been studied, including most of those which have been found to be causally related to tobacco smoking. Risk estimates for those important outcomes, for which GEPHI assessment found the evidence to be mostly of moderate quality (in a few cases low), suggest that reduction of exposure to the estimated 35–80 μg/m³ PM_2.5 range would result in risk being reduced by between 20% and 50%, depending on the outcome, and possibly more for some outcomes including severe child ALRI, lung cancer with coal use, and COPD in women.

Studies reporting on health impacts of interventions

A summary of studies reporting on the health impacts of interventions (experimental and observational) is provided in Table A5.1 (reproduced from Review 4), and discussion of the most important of these follows.

Table A5.1

Summary of studies reporting on health impacts of household fuel combustion interventions.

The single RCT (RESPIRE) with a well-accepted improved stove was carried out in rural Guatemala, and studied the impact of a chimney stove on child ALRI up to 18 months of age. Detailed, repeated measurement of kitchen pollution and child exposure was included; this allowed both intention-to-treat (ITT – improved stove versus open fire) and exposure-response analyses to be conducted. The ITT analysis found that the intervention group had a 90% reduction in kitchen air pollution, and a 50% reduction in child exposure (to an equivalent of around 125 μg/m³ PM_2.5) which was associated with a relative risk (RR) of 0.78 (95% CI: 0.59, 1.06) for all physician diagnosed pneumonia and an RR of 0.67 (95% CI: 0.45, 0.98) for severe pneumonia. While these risk reductions (22% and 33% respectively) are less than seen in the observational study pooled estimates (37% for all pneumonia and 60% for severe pneumonia), these findings are consistent if the intervention group mean child exposure level is taken into account as this is considerably higher than the HAP concentrations estimated for the observational studies. The exposure-response evidence adds further detail to this question, as it provides an estimate of how much this difference in risk would be expected to be.

The other important studies with health outcome definitions that can be compared with the epidemiological study systematic reviews and the IER functions are a set of three cohort studies investigating the impact of long-term (10 years or more) use of an improved chimney stove as part of the Chinese national improved stove programme. The studies were all conducted in the coal-using area of Xuanwei, and examined impacts on COPD, lung cancer and adult mortality from ALRI. The findings showed reductions in risk of between 40% and 50% for all three outcomes (other than for COPD for women where the risk reduction was 25%). These studies did not include HAP or exposure measurements (apart from a very small, separate investigation in the same area), so the reductions and intervention-group levels can best be inferred from the results of the intervention impacts review (see Review 6). This found large reductions (63%) but post-intervention levels of more than 300 μg/m³ PM_2.5. The one study of a chimney stove evaluation in China included in that review found baseline level of 270 μg/m³ PM₄ and a 43.3% reduction in kitchen concentration to 150 μg/m³ with the intervention, but this was not conducted in Xuanwei, and was for biomass rather than coal. Nevertheless, these findings imply that substantial risk reductions were seen in association with 40–60% reductions in kitchen PM_2.5 and that levels in the intervention groups were probably at least 150 μg/m³. Interpretation of should be cautious, however, as no personal exposure data are available from these settings.

Integrated exposure-response functions

The IER functions, described in Review 4, provide estimates of how risk varies with exposure for five important health outcomes, child ALRI, IHD, stroke, COPD and lung cancer. The IER curves for IHD and stroke do not include any empirical risk estimates for HAP, as no studies were available at the time the curves were developed (and only one study with adjusted risk estimates is available now). The curve for child ALRI is reproduced here to assist with discussion (Figure A5.3); for other outcomes, please refer to Review 4.

Figure A5.3

The relationship between level of PM_2.5, exposure (μg/m³) and relative risk (95% CI) of child ALRI based on the integrated exposure-response (IER) function for exposure over the range 0–600 μg/m³.

Child ALRI: The most notable feature of the curve for ALRI is that it is relatively steep at low exposure levels (even below the IT-1 annual average value of 35 μg/m³ PM_2.5), and flattens off at around 200 μg/m3, continuing to rise steadily across the rest of the exposure range and, unlike for other outcomes, is not bounded at the upper end by active smoking risk data. With this curve, it can be seen why the risk reduction in ITT analysis of the RESPIRE study, with intervention group exposure in excess of 100 μg/m³ PM_2.5, would be less than that seen in the meta-analysis of epidemiological studies which compared traditional solid fuel use to (estimated) levels in the range 35–80 μg/m³ PM_2.5.

Lung cancer: The IER curve for lung cancer is much closer to being linear, rising to very high levels of risk with active smoking. This linearity across the HAP exposure range would explain why the 40–60% reductions in exposure (estimated) for the Chinese NISP stoves in Xuanwei would result in 40–50% reductions in lung cancer risk.

COPD: The IER curve for COPD is less clearly defined as the HAP risk estimates do not fit well with the function derived for the other sources of exposure. The reason for this is unclear, but could be related to lifelong exposure (during pregnancy and from birth) to HAP having a greater effect (dose for dose) on the airways and lungs than, for example, smoking which usually does not start until the late teenage years at least. The curve suggests a function which is more linear than for ALRI, but perhaps not as linear as for lung cancer. If confirmed, this would also be consistent with the 25–50% reductions in risk seen in the Xuanwei cohort study.

Discussion

This review of the consistency of key evidence contributing to the causal chain model has shown a good degree of consistency for the health risk and intervention impact findings, but more mixed results for the findings on consistency between HAP levels based on predictions from the model and measured levels. Specifically in respect of the latter, it is the findings for the lower emission devices and fuels that differ from what is observed, but the reasons are likely to lie with contributions from other sources, both inside and outside the home. These issues have very important implications for implementation, and these are captured in the general considerations accompanying the recommendations.

While the evidence on health risks and intervention impacts appears to be quite coherent, there are gaps and weaknesses in the evidence, including the lack of intervention trials, the virtual lack of empirical data on HAP exposure and IHD/stroke and the uncertainty concerning the COPD IER, which require attention from the research community.

Three randomized clinical trials are currently under way, all of which are studying the impacts of improved stoves (both standard natural draught rocket-type as well as fan-assisted) on birth outcomes (pre-term birth, birth weight) and ALRI. The trial locations together with the lead investigating institution, principal investigator and trial registration numbers are provided in Table A5.3. Two of these (Ghana, Nepal) also plan to include a clean fuel option in the trial.

Table A5.3

Randomized clinical trials testing the impacts of reducing HAP exposure currently in progress.