Honest feedback…. on the approach

A few weeks ago, Maxwell presented its data-driven financial materiality methodology to an industry workshop of reporters, assurers, investors and data platforms. On the day, the feedback was positive and the consensus was to move from proof-of-concept pilots to real world Beta projects.

As we roll out the Beta projects, we wanted to collect and share honest feedback from users.

This article sets out the feedback on Maxwell’s general approach. Subsequent articles will focus on the reporter experience and the investor experience.

We welcome all feedback as we build out the product, please do let us know….. good and bad both welcome!

 
  1. Which audience do you address?

A key point made by many at our industry workshop is that there are multiple use cases for a data-driven approach to financial materiality. While the underlying data, curation, sentiment analysis are the same, we are working with six distinct and separate types of users:

Role and profile. Typically a Head of Sustainability Reporting or ESG Reporting Manager at a large listed company, often sitting in Finance under the CFO or Group Financial Controller. They are senior manager to director level, with 8–15 years' experience spanning financial reporting, audit or sustainability. They control a reporting budget of roughly £100k–£500k a year for software, data and advisors, shared with investor relations and legal.

Motivation. They need a materiality conclusion that survives challenge from the audit committee, the external assurer and investors. A data-driven approach lets them show why a topic is material with numbers, not workshop opinions. It also cuts the weeks spent re-running the assessment each cycle.

Current approach and why data-driven is better. Most still run a stakeholder survey, a handful of interviews and a scoring workshop, consolidated in spreadsheets. Results are subjective, hard to reproduce year on year and weakly linked to financial effects. A data-driven approach anchors each topic in quantified exposure, peer disclosures and financial metrics, giving a consistent, auditable trail.

Key output. A defensible, documented list of financially material sustainability topics, with thresholds and evidence for each. It must slot directly into the annual report and satisfy ISSB- or ESRS-aligned disclosure requirements.

Role and profile. Usually the Finance Director, Financial Controller or an operations manager who has inherited sustainability, reporting to the CEO or owner. In smaller firms it may be the founder. They are senior but generalist, with little formal sustainability training, and the budget is small: often under £10k–£25k a year, with almost no advisory spend.

Motivation. They are being asked for sustainability data by banks, large customers and insurers, often through supply-chain questionnaires. They want to answer once, credibly and quickly, without hiring a consultant. A data-driven approach gives them a starting point built from sector benchmarks rather than a blank page.

Current approach and why data-driven is better. Many have no formal assessment; they answer questionnaires ad hoc or copy a template such as the voluntary SME standard (VSME). That produces inconsistent answers and wasted effort across requests. Sector and peer data pre-populate the likely material topics, so the SME only validates and adjusts, saving time and money.

Key output. A short, proportionate materiality summary they can reuse: the handful of topics that matter financially for their sector and size. It should feed straight into lender and customer questionnaires and a lightweight sustainability report.

Role and profile. A Sustainability Assurance Director or Senior Manager in a Big Four or mid-tier audit firm, reporting to the assurance partner. They have 7–15 years in financial audit or ESG assurance. Budget is set per engagement through fees, with firm-level spend on methodology and tools.

Motivation. They must test whether the client's materiality process is complete, consistent and supported by evidence. Workshop-based assessments are hard to audit, which raises engagement risk and hours. Independent data lets them benchmark the client's conclusions and spot omitted topics quickly.

Current approach and why data-driven is better. Today they review the client's process documentation, interview staff and sample stakeholder inputs, largely manually. Coverage depends on the team's sector knowledge. A data-driven benchmark of what peers and the sector treat as material gives an objective reference point, focuses testing on gaps and scales across a client portfolio.

Key output. An evidence-backed view of whether the client's material topics and thresholds are reasonable, with flagged gaps against peers and sector norms. It should be documentable in the working papers to support the assurance opinion.

Role and profile. A Portfolio Manager, ESG/Stewardship Analyst or Head of Responsible Investment at an asset manager, pension fund or insurer. Analysts report to the Head of Responsible Investment or CIO; heads report to the CIO. They have 5–20 years in investment research, with data budgets often £250k–£1m+ a year across ESG and market data subscriptions.

Motivation. They want to know which sustainability issues will move cash flows, cost of capital or valuation for a given holding. Generic ESG scores often fail to answer that and disagree with each other. A data-driven, financially focused view helps them price risk, prioritise engagement and defend decisions to clients.

Current approach and why data-driven is better. Many rely on third-party ESG ratings, static industry maps such as SASB-style materiality frameworks and their own analyst judgement. These are broad-brush and update slowly. Company-specific data linking issues to financial metrics gives sharper, comparable signals across a portfolio and flags where disclosures diverge from real exposure.

Key output. A comparable, company-level view of financially material issues across their holdings, quantified where possible. It should plug into valuation models, risk dashboards and engagement priorities.

Role and profile. A Head of ESG Data, Product Manager or Head of Research at a ratings agency, index provider or market-data firm. They report to a Chief Product Officer or Chief Data Officer. They have 8–15 years in data products or research and manage product budgets of £500k to several million a year, including data sourcing and engineering.

Motivation. Their clients want materiality signals that are company-specific, explainable and comparable at scale. Analyst-built mappings are costly to maintain and hard to justify when challenged. A data-driven method lets them extend coverage, refresh faster and explain every score.

Current approach and why data-driven is better. Most use fixed sector materiality maps, set by in-house analysts and reviewed periodically, then apply them uniformly to every company in a sector. That misses company-specific exposures and ages quickly. A data-driven model derived from disclosures and financial data adapts to each issuer, scales to thousands of companies and gives a transparent methodology clients can trust.

Key output. A scalable, machine-readable dataset of financially material topics per company, with weights and a documented methodology. It should feed their ratings, indices and client APIs.

Role and profile. A Sustainability Strategy Partner, Director or Senior Manager at a consultancy, from a Big Four practice to a specialist boutique. They report to a practice lead or managing partner and have 10–20 years in strategy, risk or sustainability consulting. Budgets are client-funded, with materiality projects commonly £30k–£150k; internal spend goes on tools and methodology.

Motivation. They want to deliver faster, more defensible materiality work and move clients from compliance to strategy. Manual assessments compress margins and look similar across firms. A data-driven approach differentiates their offer, frees senior time for insight and turns a one-off project into a repeatable service.

Current approach and why data-driven is better. Most run bespoke projects: desk research, stakeholder interviews, surveys and facilitated workshops, rebuilt for each client. This is labour-intensive and hard to benchmark. Data-driven tools give a quantified baseline in days, peer comparisons out of the box and more time for scenario analysis and strategic recommendations.

Key output. A client-ready, evidence-based materiality assessment that links material topics to strategy, risk and value creation. It should include peer benchmarks and clear recommendations the board can act o

 

2. How does sentiment translate to financial materiality?

Sentiment is where we start, not what we conclude. We use machine learning to tag hundreds of thousands of stakeholder conversations against the ISSB taxonomy every day, and repeat the process for the company's peer group and industry. That gives us a net weekly sentiment score for each sustainability driver. On its own, the score only tells us what people are saying and how they feel about it. It becomes financial materiality only when we can show it moves the company's value. To test that, we treat closing share price as the dependent variable, which isolates the sustainability drivers that actually affect investors' capital allocation decisions. The result is a signed coefficient for each driver.

For BP as an example, a one-standard-deviation rise in net weekly sentiment on employee engagement was associated with a 0.24-standard-deviation rise in BP's weekly stock return, making it the company's largest internal value driver. The sign shows whether a topic is a risk or an opportunity, and the size shows how much it matters.

Which criteria distinguish the one from the other?

A topic can be heavily discussed and still not be financially material. To count as material, a driver has to pass four tests.

First, it must be specific to the company. We use the Frisch–Waugh–Lovell theorem to partial out the sector and index effects from both the returns and the drivers, which gives separate coefficients for the company, its peers and its benchmark. This matters because around 41% of price movement is company-specific, around 32% is industry-specific and 27% is index-specific [taking a randomised portfolio of large cap UK, European, US companies].

Second, it must hold up once overlap between drivers is controlled for. Ledoit–Wolf shrinkage handles the overfitting that comes from combining many sustainability drivers with a limited price history, and it does so without cross-validation.

Third, it must clear a threshold that comes from the data, not from a judgement call. Sparsemax projects the driver weights onto a probability simplex and zeroes out everything below the resulting cut-off. Any driver left with a non-zero weight is material, and every other driver is not.

Fourth, the signal must be robust. For BP we reported a directional hit rate of 65.38%, a rank IC of +0.34 and a Pearson IC of +0.36 between sentiment and returns. Materiality is also allowed to change over time. In quiet periods such as 2016–17, sector and index explain most of the movement and fewer drivers show up; in event-heavy periods such as the 2023–24 energy crisis, specific drivers activate immediately.

Are there different data sets feeding them?

Yes, there are two distinct data sets, and keeping them separate is what makes the method auditable. The sentiment side is unstructured text: publicly available content from traditional, trade and social media. The materiality side is market data: the company's share price, plus a peer-group series and a benchmark index used as controls. Neither data set relies on the annual report. The results come from correlating all publicly available data with share price rather than last year's disclosures, and they are updated weekly.

 

3. Which open-sourced content is deemed relevant?

We are not trying to capture everything said about a company online. We want what is said by the stakeholders whose views shape its value. In practice, that means publicly available conversation about the sustainability matters relevant to the company, its peers and its industry, collected continuously from people who know those issues. It comes from three channels:

  • Traditional media: national and financial press. For example, Financial Times, Economist, Guardian, BBC, Wall Street Journal, New York Times

  • Trade media: industry and sector publications, where specialists discuss operational risks before they reach the mainstream. For example, Fierce Pharma, Health Service Journal, Aviation Week, The Banker

  • Social media: where employees, communities, activists and investors react in real time. For example, Reddit, Quora, blogs, forums. But not Twitter as there is no signal value left here.

Content counts as relevant only if it can be tied to a sustainability topic. Every item is tagged by machine learning against the ISSB/SASB-aligned taxonomy of 26 sustainability drivers. Precision and recall for the classifier against the taxonomy are monitored closely to ensure that relevance filtering is itself reliable. Anything that doesn't map to a driver plays no further part in the analysis, however much attention it gets. Entities are resolved manually using bespoke Boolean strings together with entity-specific exclusion clauses in the local language. Bot activity is monitored but seldom makes an appearance in our specific search results. Annual reports and the company's own disclosures are deliberately excluded. Using them would bring back the time lag and self-assessment bias a data-driven approach is designed to remove.

Does Maxwell use thresholds?

Yes, but not where people usually expect them. There are two kinds.

The first is a data-sufficiency threshold. A company is added to the database only if there is enough conversation about it to produce robust results. Where there isn't, typically for smaller or private companies, its peer group and industry act as proxies. As a rule-of-thumb, the minimum number of tagged conversations is about 100 weekly but this is highly industry- and country-dependent. This is a quality gate on the input, not a judgement about materiality.

The second is the materiality cut-off, and this is where we differ most from the traditional approach. Ambiguous thresholds are one of the most common complaints about conventional assessments, which typically score topics and then draw an arbitrary line. Our early machine-learning pipeline had the same weakness. When we tested it at scale, we still had to set thresholds by hand. The current method uses Sparsemax instead. It projects the driver weights onto a probability simplex, which produces its own cut-off, and every driver below that cut-off is set to zero. The line is still there, but the data draws it, not an analyst. The same inputs always produce the same cut-off, and that is what makes the result auditable.

We also apply conventional statistical tests to the wider picture. In the BP example, the sector's dominance of returns was significant at p < 0.05, while the index had no significant effect. Individual drivers also carry significance tests or confidence intervals. We continue to work on this area to improve accuracy and efficiency.

Previous
Previous

Honest feedback…. on the reporter experience

Next
Next

Why Financial Materiality is like an IKEA chest of drawers