Share this Article & Support our Mission Alpha for Impact
This is some text inside of a div block.
This is some text inside of a div block.
This is some text inside of a div block.

Reliance on ESG data providers: Better as "opinions" than "ratings"?

Post by 
Text Link
The ESG data market is booming. Depending on which source you look at, there are currently between 125 and 600 providers of ESG data, ratings, and rankings. This is not surprising given that investors have been spending and are expected to keep spending millions on ESG data. In 2019, users spent over $600 million and according to some sources, this should reach $1 billion for 2021. [1] [2] [3]

ESG data providers are not all created equal, and as opposed to credit rating agencies, ESG data providers do not have methodologies that are in the least regulated. They tend to have their own proprietary rating and ranking methodologies, and while there is overlap between providers, they can use different sources to gather the underlying raw data for the same ESG issues. Some providers make their methodologies transparent, but most do not.

Differences in methodologies and data sourcing have caused the widely discussed differences in ratings between the providers and as a result, the correlations between ESG data providers can be very low. In an often-cited paper [4], these correlations can average around 0.60 which is significantly lower than the correlations between credit rating agencies, such as Moody’s and S&P, which tend to remain close to 0.995.

Given the differences in ESG ratings, investors would be better-off thinking of them as ESG data “opinions”.

Similar to sell-side research opinions which can vary from firm to firm, ESG data providers should serve as an input to the investment process rather than a determination of the underlying issuers’ ESG quality. In an upcoming academic paper, “Inconsistencies in methodologies of ESG data providers and green bond standards” authors Kim Schumacher and Tamara Close identify 11 major inconsistencies between different ESG data providers based both on the type of data provider and their underlying methodologies. These include differences in the definitions of materiality, normalisation techniques, aggregation and weighting, survivorship bias and missing data, use of standards and metrics, creation of benchmarking and peer groups, sources and timing of data collection, and conduct vs product-based scoring methodologies. The authors then also point out how these inconsistencies can have a material impact on an investment portfolio. This includes potential cherry picking of ESG ratings, as well as factor, geography, and size biases.

While ESG data can have a significant and necessary place in the investment process, a more holistic and in-depth analysis is required.

Using ESG data as an input to an investment process can strengthen the process and provide a wider lens for the investment process as it encompasses the assessment of the intangibles of an investment. However, to ensure credibility and integrity, ESG data needs to follow the same data management processes as other material investment data. This includes validation and data quality checks.

The quality of an ESG rating will also depend on the comparability of its source data and the methods used to analyse this data. For instance, data from developing countries, or countries with diverging regulatory standards, can be tainted by gaps or bias. Therefore, companies from the same sector may be assessed in “very different ways as the context in which the underlying data was produced was highly divergent.” [6]

In addition, simply taking these ratings/data at face value without a holistic view will potentially create unintended sustainability exposures in a portfolio and can actually create the opposite effects of what an investor originally intended.

For instance, if a company has a best practice diversity & inclusion, employee health and safety, or other type of ESG policy, but does not make the policy publicly available, certain ESG ratings providers will give that company a low rating (or the industry average) for that ESG issue. At the other extreme, companies that publish well written, comprehensive policies will see themselves receive higher ratings even though these policies may not be followed at the individual companies. Simply having a policy for an ESG issue does not mean the issue is being properly managed at a company [7].

Some ESG data providers can also be sector-neutral, meaning that companies even in sectors with significant ESG risks (such as the oil and gas sector) can still score high on ESG metrics. A high ESG rating therefore does not necessarily mean a company is more sustainable or takes “better care of the environment or society” [8]. Hence the peer group or benchmark that is used to determine the ESG score or metric becomes of paramount importance.

There are also size and geography biases. The materiality of an ESG issue can vary depending on the country or the region [9] of the firm. For instance, while issues such as “worker health and safety” are very important in developed markets, developing markets may favor job creation over health and safety, so as to reduce poverty levels [10]. This may create biases in the ratings of firms in developing countries (unless normalized for regional differences), since most ESG rating agencies tend to be from Western countries.

In addition, given the size and geography bias that can exist in ESG company scores, a reliance on ESG ratings for investment decisions may bias investment portfolios to larger cap companies and to regions with higher levels of regulatory reporting for sustainability issues. This does not necessarily mean that these are the most sustainable companies in the investment universe.

Therefore, by solely relying on an ESG data provider’s score, investors are “taking on the perspectives of that provider without a full understanding of how the provider arrived at those conclusions.

A thorough understanding of the methodologies, as well as the types and sources of data used by the ESG data providers is fundamental for investors if they want to use this data as an input to their investment process and create truly sustainable investment portfolios.
Tamara Close

Some investors will try to get around this issue by using the underlying raw ESG data from a provider as opposed to the aggregated scores. However, depending on where providers get their data, this can also create unwanted impacts. For instance, most self reported corporate ESG data is not subjected to audits or internal controls and therefore may pose a risk for errors and omissions.

ESG data and metrics should also be looked at in a holistic fashion and not in a siloed approach to ensure more clarity into the ESG risk of a company. For instance, comparing two firms’ health and safety processes by simply looking at their reported “near miss accidents” may create unintended consequences in a portfolio. One firm may have higher near misses than another but is this due to the fact that they have a worse health and safety record compared to another firm or is this because they have a very transparent reporting methodology and encourage their workers to report all near misses in an effort to constantly improve their health and safety record? A wider investigation and assessment of governance and oversight processes is hence required to obtain a more holistic and realistic view of a company’s health and safety process.

Sophisticated investors understand that given the inconsistencies between ESG data providers, they need to apply critical judgement and perform an in-depth analysis when integrating ESG ratings into their investment decisions. A crucial first step is the due diligence of any ESG data provider used [12]. These investors will then often overlay their internal knowledge (from internal ESG specialists and/or from engagement with investee companies), and insights from ESG technical experts (such as engineers, scientists, and researchers) onto the raw data from ESG data providers to create their own proprietary ESG scores.

ESG data and data providers certainly have a critical role to play in the investment industry.Providers are solving for the ESG data availability challenge in the capital markets by sourcing, analysing and aggregating this data. However, this is also creating subjectivity and biases in portfolios.

Recommended reading by the editors: ESG Rating - what is measured and how? | Intalcon

[1] Scm Direct. 2019. “Greenwashing: Misclassification and Mis-selling of Ethical Investments.”
[2] SustainAbility. 2020. “Rate the Raters 2020: Investor Survey and Interview Results”
[3] Opimas “Spending on ESG data could hit $1B in 2021”
[4] Berg, Florian and Kölbel, Julian and Rigobon, Roberto, Aggregate Confusion: The Divergence of ESG Ratings (May 17, 2020). Available at SSRN: or
[5] 2
[6] Source: Kim Schumacher, School of Environment and Society, Tokyo Institute of Technology, School of Geography and the Environment, University of Oxford, (
[7] The Alliance for Corporate Transparency’s 2019 report assessed 1,000 European companies and concluded that companies tend to report on policies and not on specific ESG data, citing that there is also a lack of ESG metrics and targets. It is difficult to determine clearly how a firm is performing if only policies are published, without specific metrics, targets and reporting on the implementation of those targets;
[8] Environmental Finance. 2020. “Pitfalls in ESG ratings requires investor caution”
[9] For a good overview of the regional differences of ESG issues, see:
[10] 3
[11] State Street. 2019. “The ESG data challenge”,
[12] This was highlighted as a recommendation in the recent consultation paper from IOSCO: “Financial market participants could consider conducting due diligence on the ESG ratings and data products that they use in their internal processes. This due diligence could include an understanding of what is being rated or assessed by the product, how it is being rated or assessed and, limitations and the purposes for which the product is being used.” Source:

Would you like to use this article - in full or in part - for your purposes? Then please consider the following Creative Commons-Lizenz.