Policy-makers must have an in-depth understanding of how their health system is working if they are to improve it. Health system performance assessment (HSPA) gives them the details they need. It is a process of monitoring, evaluating, communicating, and reviewing how far a health system meets key objectives and supports overarching health system goals. However, HSPA exercises often rely on understanding a multitude of metrics. The sheer number of indicators can be overwhelming and many do little to explain the policy challenges. Policy-makers need to focus on what is policy relevant, which makes a subset of indicators selected to support decision-making a practical asset.

This policy brief tests the concept of ‘tracer’ indicators that respond to policy questions. It operationalizes the WHO-Observatory global HSPA framework and the renewed OECD HSPA framework and demonstrates the need for (and value of) prioritizing particular information. It builds on the existing health system assessment metrics and methodologies, such as the OECD’s Health Statistics database and Health at a Glance reports, the WHO Regional Office for Europe Health for All database and the European Observatory’s body of work on HSPA and in the Health Systems in Transition (HiT) reviews.

The two frameworks (WHO-Observatory and OECD) serve as blueprints – linking either to the performance of each discrete health system function or to the performance of the health system as a whole. They are aligned and complementary in that they build on common elements and can both be used to assess health systems performance and set it in the context of broader societal challenges. The WHO-Observatory framework sets out detailed assessment areas and highlights the dynamics of performance pathways between health system functions and goals, while the OECD renewed framework emphasizes key policy elements, the broad categories and main relationships – from which users can “zoom in” as needed. There are many common elements in both frameworks, albeit with different levels of emphasis.

This policy brief uses health workforce, digital health, people-centredness and access and quality as “test” areas. It explores how policy questions for each (Box 1) can be addressed through careful selection of a targeted (sub)set of “tracer” indicators. The indicators are intended to allow policy-makers to navigate performance much more easily. They are therefore selected at a high level and as a starting point for assessing performance. They are not chosen to detail specific processes.

Box 1

Policy questions as entry points for assessing performance.

The brief recognizes the progress that has already been made in moving beyond measuring inputs (financial, physical and human resources) to assessing health objectives and goals and takes a further step in linking indicators to health system functioning and policy options.

Each of the four sections addresses a specific policy area. The workforce section, for example, considers policies to ensure the delivery of high-quality and accessible health services and emphasizes health workforce shortages, skill-mix and retention. Suggested indicators include density and distribution of health workers, migration inflows and outflows, ratio of doctors to nurses, share of GPs in physician workforce, job satisfaction and burnout rates by occupation.

The digital health section explores readiness, utilization, and digital health literacy underscoring the importance of a focused assessment of the role of digital health in the health system. The indicators proposed for this section include existence of digital health governance at the national level, share of health facilities with ICT equipment and access to the internet, use of electronic records, telehealth and e-prescribing, as well as health literacy.

The people-centredness section addresses trust, participation and enabling the co-production of health care because of the significance of health systems in meeting people’s diverse needs. This can be measured via share of patients involved in decisions about their care, ability of people to have a formal role in health policy decision-making, as well as share of people who have trust in the health system.

The access and quality section tackles these outcomes of health services through the high-level tracer indicators of avoidable mortality and unmet need. These are “initial explorations”: they point the way to policy priorities and options but cannot replace the further investigation needed to understand all the causes of sub-optimal performance.

The selection of tracer indicators is very much a work in progress. The choices made draw on various international and regional reports and datasets, including OECD’s Health at a Glance and Health Statistics database and the WHO Regional Office for Europe’s Health for All database, and pick those indicators with the best potential to both reflect performance issues and provide a plausible narrative for the interpretation of the indicators. The brief is the beginning of discussions on how to select and use the “best” tracers to assess specific health system functions and outcomes. It does not attempt to offer a definitive list of indicators. Instead, the examples, and the way their use is illustrated, advance the understanding of how indicators for specific areas (workforce, service delivery) can be mapped onto the WHO-Observatory and OECD frameworks and of how they can be mobilized to address key policy issues. It demonstrates that tracer indicators, in combination with the HSPA frameworks, can help identify causes of performance problems, identify policy options and monitor change, as well as highlighting data gaps and limitations.

Selected (“tracer”) indicators have a valuable contribution to make. They offer policy-makers the chance to use HSPA as a practical, not just a conceptual, tool. Nonetheless, there are significant limitations around data availability, level of disaggregation, timeliness, comparability and quality, particularly at the international level. A caveat stems from the intrinsic nature of tracer indicators. They are reflections of specific health system areas and flag where there may be a systemic issue in a given function, but systems are complex (as captured in the frameworks) so all indicators must be understood within the broader context. Similarly, indicators taken alone are static, while performance issues need to be followed over time. Insights gained from tracer indicators need to be supplemented with more in-depth investigations into root causes of the problems identified, and policy-makers need to grasp these limitations to avoid any inadvertent misinterpretation.

That said, synthesizing the multitude of HSPA indicators into a smaller set with critical policy relevance will help decision-makers. The groundwork in this brief establishes an approach that has four complementary elements:

• Focusing on a targeted set of tracer indicators that reflect performance issues in different parts of the health system (supported by appropriate tracers for health system functions and outcomes).
• Mapping indicators onto the WHO/Observatory and OECD HSPA frameworks, tracing links and causal pathways between functions, subfunctions, assessment areas, intermediate objectives and final goals, making clear health system performance dynamics.
• Using policy-relevant questions as the way into assessment – framing performance and the metrics around practical concerns so that the focus is on the needs of policy-makers and can inform concrete policy actions.
• Providing a broader narrative for the meaning and scope of each indicator to prevent misinterpretation and to highlight gaps, limitations and pathways for improved data collection and the development of new metrics.

This brief serves as a proof of concept and represents the first step towards policy-oriented HSPA that will facilitate health systems transformation and the achievement of health and societal goals. There are four concrete lessons for health system leaders for future development in this field:

• Invest in data: resources are needed in national systems and at the international level to facilitate efficient, timely, safe, and reliable data collection and information flows. Digitalization can enable faster, better data collection but investment is crucial if systems are to develop capacity to validate and deploy appropriate tools.
• Apply HSPA to practical policy questions: framing HSPA exercises through policy questions, employing tracer indicators and mapping them onto HSPA frameworks can make HSPA a practical tool that identifies system weaknesses and policy options. Expanding the methodology to other health system areas is a logical next step.
• Revise the existing health data body: a comprehensive review of existing health indicators would give a deeper understanding of fitness for purpose and identify data gaps. The development of new indicators could address the crucial data gaps identified in this brief and have huge policy relevance. Going beyond the selection of available indicators to explore innovative uses of existing indicators and to create or expand novel indicators would make tracer indicators more meaningful.
• Collaborate across organizations: close collaboration among key international organizations and bodies working in this field, notably the WHO, the OECD, the EU and the European Observatory on Health Systems and Policies, to align methodologies and develop a shared dashboard of tracer indicators would support policy decision-making.

This brief reflects the agendas of both the Tallinn Charter 15^th Anniversary Health Systems Conference: Trust and Transformation – Resilient and Sustainable Health Systems for the Future and the OECD’s Health Ministerial Meeting: Better Policies for More Resilient Health Systems.