ESG Data Provider Demo Scorecard: Five Tests Using Your Own Data

Two large providers publish policy statements on how they manage ESG ratings data and how they review methodologies. Read both. Neither one says whether a published score is restated when the methodology changes, and neither says whether an estimated figure is flagged as estimated in the field you receive. Those documents were written for a new European supervisory regime, and they still leave the two questions a buyer most needs answered.

So the documentation will not settle it. A demo can, but only if the demo is run as a test rather than watched as a presentation.

The structure below flips who controls the session. You bring the data, you name the test, you score the result in the room. It takes about ninety minutes and it does not require you to understand the provider's model.

What To Bring Into The Room

One file of forty entities, assembled a week in advance, sent to nobody.

Thirty of them are ordinary companies you actually care about. The other ten are constructed: one legal entity written three different ways, including the registered name, a trading name and a version with a plausible typo; a parent and one of its subsidiaries with similar names; a company that changed its name in the last two years, supplied under the old name; two unrelated entities with near-identical names registered in different countries; a joint venture; and an entity that was acquired and delisted.

Shuffle them so the hard cases are not clustered.

Then set two ground rules. The provider shares their screen and works live, or you type into their interface yourself. No taking the file away to prepare an answer overnight, because an overnight answer tells you about their client services team rather than their product. And one person on your side records outcomes on the scorecard as they happen, because memory of a demo is reliably kinder than the demo was.

Test One: Duplicate And Related Entities

Paste the forty names. Watch what comes back before anyone explains it.

You are looking for four behaviours. Did the three versions of one entity collapse into a single record, and does the interface show you which identifier it matched on? Did the parent and subsidiary stay separate, with the relationship visible rather than merged? Did the renamed company resolve to its current record, and is the former name retained as an alias? And, most revealing, what happened to the two unrelated lookalikes.

The last one matters because a system tuned to collapse duplicates will also collapse things that are not duplicates. Ask to see the match confidence for each, and ask what the threshold is. A provider who cannot show you a confidence value is doing the matching somewhere you cannot inspect.

Expect to hear that legal entity identifiers solve this. They do not, and the numbers are public. The Global Legal Entity Identifier Foundation, which administers the ISO 17442 identifier, publishes monthly statistics on parent relationship data. In October 2025 around 2.89 million records counted as carrying complete parent information, about 93 per cent of the population, and only about 135,000 of them named an actual parent. The rest qualify as complete because the registrant declared an accepted reason for not reporting one, such as that no parent exists or that none consolidates them.

Fewer than one record in twenty, then, gives you a parent. If your use case is group level exposure, the identifier gives you a clean key and almost no hierarchy, and the hierarchy is the provider's own work. Test it rather than assume it.

Test Two: Methodology Changes And Point In Time

Pick one entity in the list and ask for its score as that score stood on a specific date eighteen months ago. Then ask for the same entity today. Then ask which methodology version produced each number.

Three things can happen. You get two values, each stamped with a methodology version and a date, which is the answer you want. You get one value presented as though it had always been that value, which means the history has been overwritten and your trend line is a reconstruction. Or you are told the request needs to go to a specialist team, which is an answer about the product's architecture.

This is not an edge case. Under Regulation (EU) 2024/3005, which has applied since 2 July 2026, providers must review their rating methodologies on an ongoing basis and at least annually, and may only change ratings in accordance with the methodologies they have published. An annual review floor means a buyer signing a three year contract should assume at least three methodology revisions inside the term.

What happens to published history across those revisions is the gap. LSEG's methodology review policy statement commits to reviewing methodologies at least annually, and to disclosing the rationale and effective date of material changes. It does not state a notice period, and it does not say whether prior published scores are recalculated. Its impact assessments may include historical back-testing, which is an internal analysis step rather than a commitment to recompute what you were already sent.

The version number matters more than the score. Berg, Kölbel and Rigobon found correlations between six major providers ranging from 0.38 to 0.71, and attributed about 56 per cent of that divergence to measurement, 38 per cent to scope and 6 per cent to weighting. Most disagreement sits in what gets measured and how, which is what a methodology revision changes.

Test Three: Missing Values And Modelled Figures

Choose three entities where you already know the disclosure is thin. A small private supplier, a recently listed company, an entity in a market with light reporting requirements.

Ask for a specific quantitative field on all three. Then ask three questions in order.

Is this number reported by the company or produced by your model, and where does the delivered record say so? The answer must be a field in the data, not a line in the documentation. A flag that exists only in a methodology PDF cannot travel into your spreadsheet, your supplier scorecard or your assurance file.

Does a zero in this field mean zero, or mean absent? Ask them to show you an entity where the value is genuinely zero and one where it is missing, side by side, and see whether the delivered records differ.

What share of the values in my forty entity list are modelled rather than reported? Ask for the count in the room. A provider who can produce that number immediately has built the distinction into the data; one who cannot has built it into the prose.

Here too the published policies stop short. LSEG's data management policy statement says information is sourced primarily from publicly available corporate disclosures, and that potential issues including missing information are investigated and validated against public sources. It does not describe how missing values are flagged, how estimates are distinguished from reported figures, or how zeros are treated.

Test Four: Export

Ask them to export the full result set for your forty entities during the demo, and time it.

Four details to check in the file that comes out. It should be a non-proprietary format you can open without their software. Every row should carry a standard identifier alongside the provider's internal key, because an internal key makes you dependent on them for every future join. The provenance and modelled data flags from Test Three should still be present as columns, since flags that survive the screen but not the export are decoration. And there should be a methodology version and an as-of date on each value.

Then ask what the licence says about the exported file after the contract ends. Some agreements require deletion of historical extracts on termination, which means the trend data you built over three years leaves with the vendor. That clause belongs in the commercial discussion, but it is discovered in the technical one.

If the export has to be scheduled and delivered later, record that as the result. The delay is the finding.

Test Five: Audit Trail

Pick one value from one entity and ask them to trace it to its source in front of you.

A complete trace shows the source document, the page or section where the figure appears, the date the value was collected, the methodology version applied, and who or what made the judgement if a judgement was involved. Anything less than that is a number with a story attached rather than a number with evidence attached.

This test has become harder to skip. ISSA 5000, the general standard for sustainability assurance engagements, is effective for sustainability information reported for periods beginning on or after 15 December 2026. It requires the practitioner to evaluate the relevance and reliability of information used as evidence, including information from outside the entity, and to evaluate the competence, capabilities and objectivity of any management's expert who produced it. It also requires an understanding of how information from external sources in the value chain is recorded, processed, corrected and incorporated, and of how the entity identifies the methods, assumptions and data sources behind estimates and forward-looking information.

Read that as a list of things somebody will eventually ask you about this provider's data, and ask it yourself first.

The provider has a parallel obligation. Article 18(2) of the European regulation requires records to be kept for at least five years and in a form that makes it possible to replicate and fully understand the determination of a rating. LSEG states it maintains records supporting reproducibility of its ESG ratings for a minimum of five years. Replicable to the provider and visible to you are different things, which is why the test is to watch the trace, not to read the commitment.

Scoring The Session

Score each row pass, partial or fail, and write one sentence of evidence next to it while the call is still running. Five partials is a different procurement decision from two passes and three fails, and you will not reconstruct which you saw a week later.

Two Questions To Ask Before You Finish

Ask whether they have applied to the European Securities and Markets Authority for authorisation under the ESG ratings regulation. Providers operating in the EU on 2 January 2025 that relied on the transitional regime had to notify ESMA before 2 August 2026 and apply for authorisation or recognition before 2 November 2026; small providers had to notify by that same November date. The deadline is under a fortnight away as this is published, so the question has a definite answer.

Then ask about your own company, if a rating of it exists. Article 15(12) requires a provider, on request by the rated item or its issuer, to make available free of charge and on a non-commercial basis the information used, including the date of the last update of the data and any other data collected, estimated or computed about that entity. The rated entity must also be notified at least two full working days before a rating is first issued, so it can flag factual errors.

That gives you a route to see a provider's raw inputs on an entity whose true figures you already know, without buying anything. Whatever the demo showed, this is the sharper test, and most buyers never use it.

Interpreting The Results

A provider who fails Test One has an entity resolution problem, which is fixable with your own crosswalk file and some ongoing effort. A provider who fails Test Three or Test Five has a different problem, because flags and provenance cannot be retrofitted by the customer. If the distinction between reported and modelled does not exist in the delivered data, no amount of work on your side recreates it.

Rank the failures that way rather than by count. And if the session produces a polished walkthrough of a different dataset than the one you brought, that is also a result, recorded as one.

Position as of 21 October 2026. Regulatory references are to Regulation (EU) 2024/3005 and to ISSA 5000 as published; dates, article numbers and effective dates should be confirmed against the current official texts, and national adoption of ISSA 5000 varies by jurisdiction and may differ in scope and timing. Provider policy statements are cited as they stood when read and are revised by their publishers without notice; LSEG is referenced as a published example because its policies are public, not as a comparative judgement, and other providers' practices may differ in either direction. The Global Legal Entity Identifier Foundation figures are monthly statistics for October 2025 and change with each publication. The test protocol described is a procurement aid, not a legal, assurance or audit methodology, and passing it is not evidence that any dataset is fit for a regulatory filing. Nothing here is legal, accounting, investment or procurement advice.

Sources

Global Legal Entity Identifier Foundation, ISO 17442, EUR-Lex, European Securities and Markets Authority, International Auditing and Assurance Standards Board, LSEG, Berg

This article is intended for general professional information and does not constitute legal, financial, or investment advice.