Socialize

FacebookYoutube

A RESPONSE TO “MAKING A CASE WHY THE 2010 EPI IS A BAD RANKING FOR SIERRA LEONE”

By Abu-Bakar S. Massaquoi

Many thanks to the writer for the rebuttal. The rebuttal, however, does not question my position that the 2010 EPI (not the 2018 EPI) is a more locally provable measure of environmental performance for the purposes highlighted in my commentary. I expected a reference to recent national assessments to challenge or confirm claims made by the 2018 EPI, so that we can locate its usefulness in the current situation. For the lack of this, I will restate my position in this debate more clearly.

My position is that you cannot give legitimacy to recent evidence (based largely on global default data) without cross-validating the data with a national assessment. This is problematic for the measurement of environmental performance because default data and the substitute indicators through which they are sought, are mostly lower in national relevance and utility. Cross-validation is vital to analyse progress made, because it helps to fully establish the opportunities and risks inherent in global default data. Data that you cannot cross-check (such as values in the 2018 EPI) have limitations and characteristics that can affect the definition of progress itself.

A jump in the ranking can become irrelevant, or politically obstructive, when EPI values are based on incomplete data (as accepted by authors of the 2018 EPI). Besides, the lack of national data to support a so-called jump in a certain ranking is problematic for understanding the assumptions embedded in the nature and purposes of the issue categories included in the index. To further shed light on the inarticulacy of the rebuttal, I will consider the key arguments made in the article (as follows):

  1. A) “Indicators are a relative measure, not an absolute measure…”

This is not true because environmental performance indicators can both be relative and absolute. The author should consider the following reasons for the EPI’s preference for relative indicators:

1)            An absolute EPI would describe environmental performance in the most definite terms (such as an increase or decrease in a certain measure) while a relative EPI recognizes the difficulty in trying to quantify certain qualitative parameters.

2)            An absolute EPI presents the disadvantage that an increase in a certain measure indicates “high environmental performance” while a relative EPI increases the sharpness with which we can distinguish between settings and measures on the basis of science and practice.

3)            An absolute EPI would lead researchers to nearly always find that environmental performance in developed nations, however small in quantitative terms, would be greater than that in developing countries, however huge the measurement.

4)            Relative EPIs do a better job of complementing analyses of environmental performance because they provide a better framework by which to gauge distinct qualities and quantities of progress.

  1. B) EPI cannot be the best environmental performance index in the world

While I agree that the EPI is not the only measure of environmental performance in the world, it wouldn’t be hazardous to demur to the proposition that the EPI is not the best (most referenced) tool and methodology in the world. Let’s examine the facts. Firstly, the EPI works as a benchmark index that can be more easily used by policy-makers, researchers, and practitioners. It is more comprehensive to the point that even the Global Green Economy Index (hailed by the writer as a better index) derives data DIRECTLY from the EPI to compute the six sub-categories that define its environmental dimension (agriculture, air quality, water, biodiversity and habitat, fisheries, and forests). Likewise, the EPI is by far the most relevant environmental performance index in the world because: 1) data mostly measures something about the environment that applies to most countries in most circumstances; 2) data is generated through an established methodology, peer reviewed by the scientific community, and endorsed by various international organisations (unlike the GGEI that is a tool run by a business entity); and 3) datasets cover more countries (data is more spatially and temporally complete) and there is a commitment to continued generation of such data into the future. Furthermore, the EPI has the broadest implementation, and thus serves as a good example. Besides, there are more publically available EPI data that can be compared between countries with confidence. The composite nature of the EPI also gives it the advantage to present broader pictures of progress, with benefits for communicating to a much broader audience. Clearly, the EPI is more widely accepted because it offers better measures of progress and a broader measurement of environment-development interactions.

  1. C) The Global Green Economy Index (GGEI) is a better measurement

This is also not true. The GGEI cannot be better than the EPI because of its narrow focus on green growth and green economy issues (I will return to this point later). Moreover, the GGEI was launched in 2010, 10 years after the first environmental performance index (then named environmental sustainability index) was published. Also, the GGEI is published by a PRIVATE U.S-BASED CONSULTANCY (Dual Citizen LLC), while the EPI is done by first-rate Universities like Yale and Columbia and global organisations like the World Economic Forum and the Joint Research Centre of the European Commission. Furthermore, the GGEI has limited data quality and coverage. GGEI covers fewer countries and follows a top-down methodology that is widely critiqued in the literature. Top-down approaches cannot be used to measure certain elements of performance, which limits both the scope of measurement and the overall quality of datasets. Additionally, as mentioned before, the GGEI is primarily applicable to the measurement of performance in relation to green economy issues. Even so, concerns about a scope dilemma in terms of definitions and concepts to include are increasing. For instance, previous assessments have been widely criticized because they appear not to fully engage with the problems of economic measurements of the green economy. Economic aspects known to have been excluded from GGEIs include the environmental impacts of economic activities, the scale of economic response to environmental challenges, and the nature of economic change due to shifting environmental performance. So, the GGEI is not as integrated and comprehensive as the author would want us to believe. The index would match the EPI’s broad purpose if the authors put greater emphasis on alternative, more inclusive measures of social, environmental, and economic progress.

  1. D) A jump in the 2018 EPI should be appreciated

Certainly. I agree 100%. I disagree, however, that a jump in the 2018 EPI implies improved environmental performance for the following reasons. Firstly, not every indicator is applicable to every country in the 2018 EPI because countries differ in geography, natural resource endowments, and physical attributes. So, the materiality of the 2018 index cannot be considered in view of a jump or a slump in current and previous indices respectively. What also supports this position (as noted in my commentary) is the fact that every iteration of the EPI requires changes to the methodology and an expanded country coverage. How then can you boldly claim that environmental performance improved in 2018 when the country is compared to MORE countries using a method that is different from the procedure used for previous EPIs? The simple answer is that changes in methodology between versions of the EPI mean that historical scores are not comparable. A jump or slump is largely due to additions and subtractions of indicators, new weighting procedures etc, not necessarily from better or worse environmental performance. Improvement can only be ascertained through the comparison of EPI values to true-within country changes in performance (documented over time). So, I urge the writer to provide peer reviewed data to allow for cross-version comparisons of both 2010 and 2018 EPI scores. That way, we can technically qualify claims about environmental performance (including the claim that the EPA is on track) and monitor and control progress moving forward. Anything else, including to prefer the 2018 EPI to the 2010 EPI without reason, like the bold claims about progress made throughout the piece, will be strictly unfitting to the contexts of this debate.

  1. E) EPI methodology has deficiencies

I agree 100%, but that does not diminish its local utility at all. Using methodological issues to discredit the broad relevance of the EPI makes no point at all. The argument should be that EPIs (whether recent or not) can generate remarkable political and media attention, and have tremendous positive impact on country-level environmental decision-making. Here is a direct quote from my commentary on important deficiencies of the EPI methodology:

 

“…. while significantly refined to serve the current purpose, the EPI is still considered (by most scholars) a “work-in-progress” due to its limited “explanatory power”. Recent publications of the index make certain that the authors do not claim to have delivered a scientifically sound, comprehensive piece of work”.

  1. F) Environmental health depends on ecosystem vitality

This is clearly a misreading of the EPI methodology.  The selection of weights in environmental health has an empirical basis while the selection of weights for ecosystem vitality is subjective. This shows that environmental health (as in improving human environments) is treated as a precondition for ecosystem vitality. The point is that countries that perform well on environmental health normally score well on the EPI, whether or not the input from their performance on ecosystem vitality is significant. Besides, the relative weight given to each policy objective is informed by the variance of each. A simple 50-50 weighting would give more influence to Environmental Health because it has a much wider spread (wider coverage of issue categories). To account for this potential imbalance, the 2018 EPI gives a weight of 40% to Environmental Health and 60% to Ecosystem Vitality. As noted already, these weights do not reflect a prioritization of nature over humans (or suggest that environmental health depends on ecosystem vitality as the writer wrongly maintains). Rather, weighting in the context of EPIs is done to produce a more balanced and useful final measurement of environmental performance.

  1. G) Arguments solely focused on rankings can cause a distraction

This is a further misreading of my commentary, which was a REBUTTAL to an article published by Standard Times on whether the President’s reference to the 2010 EPI was out of place. Certainly, I could not have engaged the writer on issues he did not raise, or provide policy prescriptions when the focus of my commentary was clear. To show my enthusiasm for contributing to our practice, I did promise to examine the real value of President Bio’s manifesto on environmental governance in a separate article. The writer’s final point is also misleading because it argues that while “rankings” indicate progress or show where improvements are needed, they should not be discussed when the scores are low. Attempting to make sense of any environmental performance ranking would, in the writer’s view, amount to “canonizing or overstretching” the index. Interesting! Let me emphasize that a conversation on “rankings” can have both intended and unintended consequences on political decision-making. Environmental management approaches that put heavy emphasis on performance measurement can influence the types of policy goals institutions set and the overall quality of policy change that they engender. They say, “what gets measured, gets managed”- even when it is pointless to measure, manage (and discuss it), and even if it harms the purpose of the organisation to do so. Yes, “it is the improvement of domestic policies that drive performance that will lead to better rankings…”, but such improvement only comes about through goal setting processes that are shaped by conversations about data availability and quality.  Goal setting and performance measurement processes (which lend primarily from global rankings) are just as important for the outcome of public policy as the political process. This demonstrates how rankings can affect the measurement of progress towards different environmental policy visions and entrench path dependency towards stronger environmental governance paradigms.

To end, I would like to caution against an attempt to reduce the debate to a situation of “my interest” against those of national environmental management institutions, especially the EPA (as has been done in this first instance). I would write a direct criticism of the EPA and other agencies if I fancy doing that. I am more interested in having a professional debate that could position these agencies (the EPA especially) better and drive fair, equitable, and lasting policy change.

© Abu-Bakar S. Massaquoi

Posted by on 2:02 am. Filed under Breaking News, ENVIRONMENT, NEWS, OTHER NEWS, THE ENVIRONMENT. You can follow any responses to this entry through the RSS 2.0. Both comments and pings are currently closed.

You must be logged in to post a comment Login