ESG data providers are not all created equal, and as opposed to credit rating agencies, ESG data providers do not have methodologies that are in the least regulated. They tend to have their own proprietary rating and ranking methodologies, and while there is overlap between providers, they can use different sources to gather the underlying raw data for the same ESG issues. Some providers make their methodologies transparent, but most do not.
Differences in methodologies and data sourcing have caused the widely discussed differences in ratings between the providers and as a result, the correlations between ESG data providers can be very low. In an often-cited paper [4], these correlations can average around 0.60 which is significantly lower than the correlations between credit rating agencies, such as Moody’s and S&P, which tend to remain close to 0.995.
Given the differences in ESG ratings, investors would be better-off thinking of them as ESG data “opinions”.
Similar to sell-side research opinions which can vary from firm to firm, ESG data providers should serve as an input to the investment process rather than a determination of the underlying issuers’ ESG quality. In an upcoming academic paper, “Inconsistencies in methodologies of ESG data providers and green bond standards” authors Kim Schumacher and Tamara Close identify 11 major inconsistencies between different ESG data providers based both on the type of data provider and their underlying methodologies. These include differences in the definitions of materiality, normalisation techniques, aggregation and weighting, survivorship bias and missing data, use of standards and metrics, creation of benchmarking and peer groups, sources and timing of data collection, and conduct vs product-based scoring methodologies. The authors then also point out how these inconsistencies can have a material impact on an investment portfolio. This includes potential cherry picking of ESG ratings, as well as factor, geography, and size biases.
While ESG data can have a significant and necessary place in the investment process, a more holistic and in-depth analysis is required.
Using ESG data as an input to an investment process can strengthen the process and provide a wider lens for the investment process as it encompasses the assessment of the intangibles of an investment. However, to ensure credibility and integrity, ESG data needs to follow the same data management processes as other material investment data. This includes validation and data quality checks.
The quality of an ESG rating will also depend on the comparability of its source data and the methods used to analyse this data. For instance, data from developing countries, or countries with diverging regulatory standards, can be tainted by gaps or bias. Therefore, companies from the same sector may be assessed in “very different ways as the context in which the underlying data was produced was highly divergent.” [6]
In addition, simply taking these ratings/data at face value without a holistic view will potentially create unintended sustainability exposures in a portfolio and can actually create the opposite effects of what an investor originally intended.
For instance, if a company has a best practice diversity & inclusion, employee health and safety, or other type of ESG policy, but does not make the policy publicly available, certain ESG ratings providers will give that company a low rating (or the industry average) for that ESG issue. At the other extreme, companies that publish well written, comprehensive policies will see themselves receive higher ratings even though these policies may not be followed at the individual companies. Simply having a policy for an ESG issue does not mean the issue is being properly managed at a company [7].
Some ESG data providers can also be sector-neutral, meaning that companies even in sectors with significant ESG risks (such as the oil and gas sector) can still score high on ESG metrics. A high ESG rating therefore does not necessarily mean a company is more sustainable or takes “better care of the environment or society” [8]. Hence the peer group or benchmark that is used to determine the ESG score or metric becomes of paramount importance.
There are also size and geography biases. The materiality of an ESG issue can vary depending on the country or the region [9] of the firm. For instance, while issues such as “worker health and safety” are very important in developed markets, developing markets may favor job creation over health and safety, so as to reduce poverty levels [10]. This may create biases in the ratings of firms in developing countries (unless normalized for regional differences), since most ESG rating agencies tend to be from Western countries.
In addition, given the size and geography bias that can exist in ESG company scores, a reliance on ESG ratings for investment decisions may bias investment portfolios to larger cap companies and to regions with higher levels of regulatory reporting for sustainability issues. This does not necessarily mean that these are the most sustainable companies in the investment universe.
Therefore, by solely relying on an ESG data provider’s score, investors are “taking on the perspectives of that provider without a full understanding of how the provider arrived at those conclusions.
