Here’s a fun exercise. Take any ASX 200 company and look up its ESG rating from three different agencies. MSCI, Sustainalytics, and ISS, say. You’ll get three different scores. Sometimes dramatically different scores. A company that’s rated A by one agency might be rated average or below average by another.

This isn’t an edge case. Research from MIT found that the correlation between major ESG rating agencies is about 0.6 — compared to about 0.99 for credit ratings. That means ESG raters disagree with each other about 40% of the time.

And we’re using these ratings to allocate capital.

The correlation problem

When Moody’s and S&P rate a bond, they almost always agree. There are occasional divergences, but they’re newsworthy precisely because they’re rare. When ESG rating agencies rate a company, divergence is the norm.

Why? Because ESG rating agencies are measuring different things, using different methodologies, with different weightings. One agency might weight governance heavily. Another might focus on carbon emissions. A third might prioritise labour practices. They’re all calling it “ESG” but they’re not measuring the same construct.

This creates a serious problem for investors, companies, and anyone trying to use ESG ratings as a signal of sustainability performance. If the ratings don’t agree, what are they actually telling us?

What gets measured gets managed — badly

The perverse incentive created by ESG ratings is that companies optimise for the ratings rather than for genuine sustainability performance. If you know which metrics a particular agency weights most heavily, you can focus your efforts on those metrics and ignore everything else.

I’ve talked to sustainability managers at Australian companies who describe their work in terms of “which rating agency are we targeting this year.” That’s not sustainability strategy. That’s a game.

The worst version of this is when companies achieve high ESG ratings while doing real environmental or social harm. A mining company with excellent governance processes and a great workplace diversity score might still be destroying ecosystems. A bank with strong climate commitments might still be financing deforestation through its lending portfolio.

The rating captures what’s measurable and reportable. It misses what’s actually happening.

The data problem

Underneath the methodology disagreements is a more fundamental issue: the data that ESG ratings are based on is often unreliable, incomplete, or self-reported.

Most ESG data comes from companies’ own disclosures. Companies choose what to report, how to measure it, and how to present it. Independent verification is patchy. And the standards for disclosure have, until recently, varied enormously.

Mandatory climate reporting under the ISSB framework will improve this for climate-related data. But ESG covers far more than climate, and much of the social and governance data that feeds into ratings remains voluntary and unaudited.

The Australian context

Australian super funds are increasingly using ESG ratings to inform investment decisions, often under the banner of “responsible investing” or “sustainable finance.” This is broadly positive — investors should consider environmental and social factors. But the reliability of the signal they’re getting is questionable.

When a super fund tells its members that it applies ESG screening to its portfolio, what does that actually mean? It means it’s using ratings that might disagree with each other by 40%, based on data that’s mostly self-reported, measuring constructs that different agencies define differently.

That’s not nothing. But it’s a lot less than most members probably assume.

What would actually help

Let me be clear: I’m not arguing that ESG considerations are irrelevant to investment decisions. They’re critically important. I’m arguing that the current rating system is an inadequate tool for assessing them.

Several things would help.

Standardised disclosure. Mandatory, audited reporting against consistent standards — like what Australia’s climate reporting regime is starting to deliver — would dramatically improve the quality of underlying data.

Greater transparency from rating agencies. Investors should be able to see exactly what data points feed into a rating and how they’re weighted. Too much of the methodology is proprietary and opaque.

More specificity. Instead of a single ESG score, investors need disaggregated information about specific issues. A company’s climate performance, labour practices, governance quality, and community impact are different things and shouldn’t be mashed into one number.

And intellectual honesty about limitations. ESG ratings are one input among many. They shouldn’t be treated as definitive assessments of a company’s sustainability performance.

The bottom line

ESG ratings in their current form are better than nothing but worse than most people think. They provide a rough signal in a noisy environment, and they’ve helped move sustainability onto the investment agenda. That’s valuable.

But treating them as reliable measures of corporate sustainability is a mistake. Until the methodology, data quality, and transparency improve significantly, ESG ratings should be used with appropriate scepticism — not as the foundation for major capital allocation decisions.

Australia’s financial sector can do better than this. It just requires admitting that the current tools aren’t up to the job.