Strategies for Successful Implementation: Guidance for Addressing Technical Climate-SRHR MEL Challenges
Technical Guidance Monitoring, Evaluating, and Learning from Climate-SRHR Action
Climate-SRHR portfolios and projects pose unique MEL challenges. In this section, we provide technical guidance on how to address some of the biggest challenges that these projects face. You’ll find tools, examples, and other resources embedded throughout to support you as you learn.
What if my organization’s MEL resources are limited?
In general, resource limitations can make MEL more challenging, including limitations around budget, time, and technical expertise. This is even more evident in climate-SRHR projects, which in some cases require different MEL approaches than traditional SRHR-focused programming.
Each project’s MEL framework should be tailored to the project’s context, learning goals, budget, expertise, and donor requirements. Smaller, pilot projects will have different MEL needs than large, scaled-up programs.
If your aim is to generate evidence that can be shared in peer-reviewed, academic journals, you may want to explore developing a partnership with researchers or an academic institution. External endline evaluation can also be a valuable part of gathering unbiased evidence about a project’s impact.
What if the Climate-SRHR indicator set doesn’t meet my needs?
Climate-SRHR projects are incredibly diverse. The indicators included in this indicator set are high-level and designed to provide coverage of common activities in a way that can readily be aggregated across projects. They cover many, but not all, potential indicators that your project may contribute to. You may need to customize or adapt these indicators for your specific needs. When doing so, remember that all indicators should be SMART: specific, measurable, achievable, relevant, and time-bound.
Customizing and adapting output indicators
You may need to include indicators specific to different SRHR outcomes, supply chains, contraceptive products, data and information systems, research, or other programmatic areas. If your project also includes activities that are focused on a sector beyond SRH, you will need to develop output indicators specific to those activities following best practices for those sectors. As you do so, try to consider if your indicator can be developed to align with or nest into an existing indicator in the dataset.
You may also wish to track multiple related output indicators. For example, in addition to tracking the number of retrofitted health facilities, you may also wish to track the number of individuals receiving SRH services at retrofitted health facilities. Similarly, in addition to tracking the number of preparedness plans developed, you may also wish to track the number of preparedness planning sessions and the number of individuals participating in development of the preparedness plans (disaggregated by age and gender). Do not feel that you should exclusively draw from the Climate-SRHR indicator set or limit yourself to only one indicator per domain; include all of the indicators that you think are relevant to monitoring your project’s activities.
Finally, in some cases, it may be more feasible and accurate to track higher-level output indicators, such as number of advocacy campaigns or number of learning events, rather than number of people reached. We encourage implementers to track both wherever possible.
Customizing and adapting outcome and impact indicators
Clear documentation of a project’s outcomes - that is, the changes that occurred as a result of project activities - is essential for understanding to what extent a project achieved its aims. These data are vital for validating a project’s theory of change and identifying effective and high-impact practices to replicate and scale.
Climate-SRHR projects may have outcomes that are related to SRHR and to climate resilience, both separately and together. Key outcomes for climate-SRHR projects may include changes to:
Climate-sensitive SRH outcomes (e.g., rate of severe maternal morbidity)
Health system and health service resilience (e.g., percent of health facilities upgraded to include climate-resilient infrastructure)
Measures of vulnerability, exposure, and individual and community resilience (e.g., composite resilience measures, perceived readiness, or targeted domains of resilience) Knowledge, adaptation processes, and behaviors (e.g., level knowledge about climate-related risks to SRH and adoption of strategies to respond to these risks)
To measure the change that takes place across a project, you will need to conduct a baseline assessment.
Impact - the broad societal change a particular project aims to contribute to - may not be visible over a project lifecycle. It may be more readily tracked at the portfolio level. For funders, it’s important to track how SRHR outcomes and climate resilience outcomes shift overall - beyond the scope of any one project. This allows funders to see the aggregate results of multiple investments across a region, as well as see changes that occur beyond a single project’s timespan. Existing global indicator sets are a valuable resource for identifying impact indicators and tracking progress towards portfolio goals over time. For more information on this, including examples of how project indicators can “ladder up” into global indicators, see How can global indicators can be used to track change in SRHR and climate resilience outcomes?
How can global indicators can be used to track changes in SRHR and climate resilience outcomes?
Many of the project-level indicators in this dataset align with and can be linked to global indicator sets.
For example, a project might be working with health facilities to improve maternal health outcomes in a region where extreme heat and flooding contribute to maternal mortality. A core outcome indicator for this project would be indicator 1.8: maternal mortality rate in facilities benefiting from climate-responsive SRHR activities. A portfolio could track this across all projects targeting maternal health, or could track maternal mortality rate at the national level. This aligns with SDG indicator 3.1.1: maternal mortality ratio, which tracks progress towards sub-goal 3.1: By 2030, reduce the global maternal mortality ratio to less than 70 per 100,000 live births.
Another project might be working with climate-affected communities to increase women’s access to voluntary family planning methods. A core outcome indicator for this project would be indicator 1.13: proportion of women of reproductive age (15-49) in communities benefiting from climate-responsive SRHR activities who have their need for family planning satisfied with modern methods. A portfolio could track this across all projects targeting access to modern contraception, or could track access to modern contraception at the national level. This aligns with SDG indicator 3.7.1: proportion of women of reproductive age (aged 15–49 years) who have their need for family planning satisfied with modern methods, which tracks progress towards sub-goal 3.7: By 2030, ensure universal access to sexual and reproductive health-care services, including for family planning, information and education, and the integration of reproductive health into national strategies and programmes.
Tool: Global Indicator Sets for Tracking Changes in SRHR and Climate Resilience Outcomes
We recommend selecting national-level indicators that are part of global measurement frameworks for ease of comparison across countries. For SRHR indicators, we recommend drawing on Guttmacher’s SDG-aligned SRHR indicators and the FP2030 Indicators. For climate resilience indicators, we recommend drawing on the ND-GAIN Index and the World Bank indicator set, which includes both a subset of climate-related indicators as well as a broad range of socioeconomic indicators linked to climate resilience. Focus on the domains of SRH and climate resilience of greatest relevance to your portfolio. For additional indicator sets, visit our Climate-SRHR MEL Toolbox.
What if a climate hazard or emergency affects my implementation site?
Climate change is having significant impacts on SRH outcomes and service delivery. When a project implementation region is affected by a climate hazard during implementation, this may influence the project’s results. In some cases, no change (or even a small negative trend) may be considered successful; for example, if a project’s activities are able to maintain the level of access to services in the face of climate change, or if a project is able reduce the degree of negative impact on SRH from a particular climate hazard, that could indicate that is effective. Unfortunately, this is a situation in which conventional pre-/post-assessments (baseline/endline assessments) may be more limited in their ability to demonstrate effectiveness. In these cases, it is helpful to use a case-control approach in evaluation, enabling comparison between the region(s) that benefited from project activities and those that did not.
Ultimately, most climate resilience measures are proxy measures; it is not always possible to collect data that can indicate whether a project’s activities made a meaningful difference in individual or community level capacity to survive and recover from climate and weather emergencies. Therefore, if an extreme climate or weather event occurs during implementation, this can serve as a valuable opportunity for documenting a project’s activities to climate resilience.
In the immediate aftermath of a weather emergency, the focus is often on providing life-saving care. As communities move from the emergency response stage to the recovery stage, it may be possible to assess both shorter and longer term impacts on:
Whether health facilities were able to ensure that SRH services remained available, disaggregated by facility type, SRH service type, and geography
Changes in ability to access SRH services, disaggregated by age, gender, and geography
Changes in SRH outcomes, disaggregated by age, gender, and geography
Changes in SRH behaviors and preferences, disaggregated by age, gender, and geography
These moments are vital opportunities for documenting how climate hazards impact SRH outcomes and service delivery, as well as evaluating the effectiveness of climate-adaptive programming activities that are intended to strengthen overall climate resilience. In order for projects to effectively evaluate whether a particular adaptation was effective, they need to 1) have a clear understanding of what their adaptation or innovative approach is (e.g., what are they doing that is different from before?) and how it will positively impact resilience to a particular climate hazard and 2) have a baseline to compare against.
Given the shortage of data on connections between climate change and SRHR, if a project tracks both 1) the broad suite of SRH outcomes that they are tracking progress towards and 2) general exposure to climate hazards, then it is possible for those projects to retroactively examine connections between experiencing climate change and SRH outcomes.
Ultimately, additional measurement requires additional resourcing, both in terms of funding and technical research capacity. Funders should consider allotting specific funding for such rapid post-disaster assessments.
Tool: Monitoring SRH in Emergency Settings
Climate-related extreme weather can produce and exacerbate humanitarian crises and emergencies. The Inter-Agency Field Manual on Reproductive Health in Humanitarian Settings is a rich resource for MEL of SRH activities in climate- and extreme weather-affected settings. The Field Manual provides guidance specific to monitoring Minimum Initial Service Package (MISP) implementation, as well as broad MEL guidance for humanitarian settings.
How can a project forward a rights-based approach when working on contraception access and climate change?
Across all activities - whether you are working on service delivery, policy change, capacity building, etc. - make sure to forward a rights-based approach and a range of SRH services.
Use of modern contraception is not a strategy for mitigating climate change, and it does not inherently signal increased resilience to climate change. Rather, access to health services and improved individual SRH outcomes are signals of overall increased health and thus, increased climate resilience. Avoid monitoring only the number of modern contraception users, and consider adding in other metrics of SRH service access and quality where feasible.
For example, Indicator 2: Climate-responsive SRH services are accessible in climate-vulnerable regions tracks the number of individuals in climate-affected areas receiving SRH services. This includes the full breadth of SRH services, and not just access to and uptake of modern contraception. By embedding all SRH services within this indicator, the indicator serves to reinforce activities, measurement, and policy development around all domains of SRH, thus forwarding a more holistic and rights-based understanding of SRH and its contribution to climate resilience.
What SRH medications and commodities should a project track?
SRH medications and supplies include a range of products required to ensure the standard of care is met for SRH services, including pregnancy and delivery, experiences of GBV, HIV/STIs, contraception, etc.; for comprehensive lists of products needed for common and essential procedures, see the IAWG/UNFPA Reproductive Health Kits Manual (6th edition) and the Reproductive Health Access Project’s Reproductive Health Procedures Equipment and Supplies Lists. In resource limited settings, or when drawing data from national-level datasets, note that it may only be possible to monitor availability of contraceptive products and not other SRH products.
Tool: Calculating what SRH supplies are needed
IAWG’s Reproductive Health Kit Calculator can be used to determine the quantity of supplies needed based on facility type and catchment area size.
What are extreme climate and weather events, and how should we measure and define them?
Extreme weather has many different definitions and can be measured in many different ways. It can include:
Relative extremes: These measures compare present-day weather observations (such as daily temperature) with the historic climate in that geography (typically over a 30 year reference period). Extremes - also called anomalies - are often defined as those in the 90th or 95th percentile of observations. While they are more specific to each place and can provide insights into deviations from the historic local climate, they typically involve greater familiarity with climate data to calculate.
Absolute extremes: These measures determine weather extremes based on whether or not present-day weather observations cross a set, pre-determined threshold. For example, an absolute extreme heat day may be defined as a day when the daily temperature exceeds 35 degrees Celsius. They may be time bound, such as the number of centimeters per rain during a 24 hour period.
Emergencies: Emergencies are often defined by not only the weather event itself but also its human impact. Emergencies may be defined by national disaster management authorities. The EM-DAT database defines emergencies based on number of fatalities, number of people affected, whether a state of emergency was declared, and whether there was an international call for assistance.
Community-defined extremes: Community members may have their own criteria for recognizing extreme weather and weather-related emergencies. This is especially true for those who are highly dependent on the land, such as pastoralist and other traditional communities.
Definitions for each of these terms may vary by country. These extremes can be tracked on various timescales, including daily, multi-day (e.g., heat waves or flooding events), or longer term (e.g., drought). The decision about how to define extremes and what types of extremes to monitor will depend on what data are available, how other stakeholders in the implementation region define these terms, and the organization's technical capacity.
When it comes to measurement, include all climate hazards that the project is explicitly trying to address as well as all other types of climate-related emergencies and weather extremes that may affect project functioning and the lives of beneficiaries. This can be done via:
Installation of a weather station or temperature datalogger at each implementation site to enable collection of real-time weather data
Extracting weather-related emergency data from the EM-DAT database
Coordination with the national meteorological agency and national disaster management authority to ensure timely warning in advance of extreme weather and documentation of any national emergencies
Global temperature and precipitation datasets, such as CHIRPS and CHIRTS
Choose a data source and hazard definition that is feasible for your organization’s expertise. Measuring relative extremes using global climate datasets is the most technically complex option, while measuring absolute extremes using weather stations or recording EM-DAT or nationally recognized emergencies are less technically complex options.
What indicators and tools can be used for measuring climate adaptation, vulnerability, and resilience?
Measuring Climate Vulnerability
The ND-GAIN index offers globally comparable measures of climate vulnerability and readiness. However, this is a national-level measure. For some projects, it may be more valuable to focus on a localized quantitative measure of climate vulnerability. Some countries provide climate vulnerability ratings for specific districts or regions within the policy documents (e.g., NDC, NAP, HNAP, etc.). In other cases, there may be existing research from the region that provides this information. In cases where these data do not exist, project MEL teams can implement local measurement of climate vulnerability tailored to the specifics of the project. Visit Tools for Measuring Climate Vulnerability on our tools and resources page to learn more.
If you’re planning an endline evaluation that measures change in local climate vulnerability, changes in climate resilience, or changes in policy, you should also include contextual indicators that will be captured within your baseline and endline evaluation.
Indicators for Climate-Adaptive Behaviors
The Clim-Eval Community of Practice’s Good Practice study provides the following examples of indicators for behavioral adaptation:
General adaptive capacity: Percentage change in stakeholder behaviors utilizing adjusted processes, practices, or methods for managing climate change risks (UNDP 2007)
Agricultural adaptive capacity: Change in farmers adopting environmentally sustainable agricultural technologies and practices (supporting indicators: farmers practice compositing, multiple cropping, intercropping, rotations, biological control, integrated pest management) (World Bank 2005)
Adaptive capacity related to awareness: Modification in behavior of targeted population (Adaptation Fund 2011, 2014)
Technological adaptive capacity: Percent of targeted groups adopting adaptation technologies by technology type and gender (GEF 2012)
Note that these indicators could also be adapted to reflect adaptation behaviors both within and beyond the health sector.
Tool: Measuring Climate Vulnerability, Climate Adaptation, and Climate Resilience
Climate vulnerability, climate adaptation, and climate resilience are distinct, but related, topics. For more on these definitions, visit Key Terms under our SRHR, Climate Change, and MEL 101 page. Quantitative tools for measuring each of these are available in our Climate-SRHR MEL Toolbox.
Tool: Measuring Climate Adaptation
Many SRHR projects in climate-affected regions will also integrate multi-sectoral climate adaptation efforts. GIZ maintains a repository of climate adaptation indicators which includes a sub-set of indicators focused specifically on climate adaptation actions. The Good Practice Study from the Clim-Eval Community of Practice also outlines a range of climate adaptation indicator frameworks and example indicators for climate adaptation.
What’s the difference between different types of climate policies?
When monitoring the policy context, it’s important to be familiar with the range of policy documents that shape and inform climate-SRHR action at the national level. There are three main types of climate policies to consider: Nationally Determined Contributions (NDCs), National Adaptation Plans (NAPs), and Health National Adaptation Plans (HNAPs).
NDCs establish a country’s climate mitigation commitments under the 2016 Paris Agreement. While some NDCs include climate adaptation, this is not mandatory. Signatories to the Paris Agreement update these policies every five years. The UNFCCC secretariat oversees the NDC process.
NAPs outline a country’s climate adaptation strategies. Also referred to as the “NAP Process,” this country-driven process identifies climate risks and engages with diverse stakeholders to develop medium- and long-term options for addressing those risks. NAPs were introduced in the 2010 Cancun Agreement and initially focused on Least Developed Countries (LDCs), though were expanded to all countries under the Paris Agreement. Each country oversees their own NAP process, with support from the NAP Global Network, UNEP, UNDP, and others under the UNFCCC.
HNAPs - like NAPs - consist of both a process and plan. HNAPs outline a country’s health sector climate adaptation strategies. They also serve as a mechanism for ensuring that the health sector is represented in the NAP process. Each country’s Ministry of Health leads their own HNAP process, with technical support from WHO and other stakeholders.
Ideally, a country’s NDC, NAP, and HNAP will all be aligned. In practice, each policy process may have slightly different priorities and timelines depending on which stakeholders are involved and how engagement is coordinated across sectors and policy processes.
There are also other important national and sector-specific policies and strategies which may include (or impact) different domains of climate and SRHR, including those related to health, environment, water, gender, disaster management and emergency response, and economic development.
Tool: Understanding NDC, NAP, and HNAP Processes
The NDC Partnership includes a range of resources on NDC and NAP development, including this overview of how NDCs and NAPs align and interact. NAP Global Network provides a list of frequently asked questions about NAP processes. WHO also provides guidance for HNAPs in Quality Criteria for Health National Adaptation Plans.
Tool: Gender and SRHR in Climate Policies
The Gender Climate Tracker analyses the robustness of gender integration in climate policies like NDCs. A database of country-level data on SRH integration in climate policies is available as a companion to our report Sexual and Reproductive Health and Rights in National Adaptation Plans & Health National Adaptation Plans: A Global Review
Should climate-responsive projects take steps to reduce their own emissions?
This framework does not focus on climate mitigation; It focuses on building gender-responsive climate adaptation and resilience to climate change.
However, climate adaptation will be less costly and less difficult if climate mitigation measures are successfully taken at a global scale. Given the importance of collective action to reduce carbon emissions as part of mitigating climate change, we consider internal efforts to reduce a funder or implementer’s carbon emissions part of the “gold standard” for climate-responsive design. Doing so also demonstrates credibility, commitment, and leadership.
Organizations delivering or funding health programs and services can take steps to reduce their own carbon emissions and environmental footprint, in addition to making efforts to adapt to climate change. This can include avoiding and reducing waste; transitioning to renewable energy; reducing travel via telework and remote meetings; purchasing carbon offsets for transportation; choosing lower emissions transportation routes; prioritizing locally procured goods; or prioritizing recyclable, compostable, and/or biodegradable goods during procurement. For example, after Doctors Without Borders/Médecins Sans Frontières (MSF) committed to reducing their net emissions by 50% by 2030, teams in Kyrgyzstan and Mozambique piloted an initiative to replace single-use face masks with reusable face masks. They’re also signatories to the Climate and Environment Charter for Humanitarian Organizations and issued an Environmental Pact outlining their organizational climate and environment commitments.
Ultimately, it is critical to center accountability and justice in discussions about climate change mitigation. Healthcare programs in low- and middle-income countries are not the major contributors to climate change. Moreover, these organizations are already overburdened with responding to the effects of a climate crisis that they did not create, often with insufficient financial resources.
Climate-responsive projects or programs should never aim to increase uptake of family planning as a mechanism for emissions reduction. Doing so undermines a rights-based and justice-centered approach, treating women’s bodies as tools for climate mitigation. This risks incentivizing coercive practices. It misplaces responsibility for climate change onto women and girls - particularly women and girls of color in low- and middle-income countries - instead of on corporations, governments, and individuals in high-income regions that are the greatest contributors to climate change. For more on this topic, see How can a project forward a rights-based approach when working on contraception access and climate change?
