Jessica Mullan, Senior Measurement Systems Manager

As many companies and platforms scale up their sustainability data, using these four factors will help ensure that you get it right.

Hidden problems that cost dearly 

From blockchain to big data, there are hundreds of shiny apps and software on offer in today’s data-centric world. Every month new technology companies sprout up promising solutions for supply chain management, risk mitigation, and sustainable sourcing. How can you make the right decision? 

It appears simple but failure rates are high if you underestimate the very first step: a clearly articulated use case. There are many facets of data capture, storage-query, analysis, and reporting or display. We have evolved a 160+ point process to help clients have the right software that suits their needs. And yet, we engage an independent pro for complex choices. No matter how you proceed, we suggest having a professional who is not affiliated with a technology seller guide the process – you may be amazed at how much time and money, not to mention mistakes, you can save.

The second step is to realize that neither software nor hardware can solve the core issue of having useful and reliable content. Content is king and remember that the GIGO acronym (Garbage In, Garbage Out) became popular for a reason. Companies are filling servers with data that can easily be inaccurate, is not comparable to any other data (no benchmarking options), and is not suited to modern decision-making needs. If you cannot figure out how to measure what matters, then why bother spending money on data that may mislead you? 

It is perfectly clear from our many years of experience with leading global brands and institutions that data grows very fast, but growing on a flawed foundation is a bad idea. Unfortunately, many managers don’t discover that data is flawed until one or more years later and then it is too late. This is like discovering that your building was constructed with poor cement. As data streams can easily house many thousands of lines of data, it is difficult to determine what went wrong as you try to solve problems. Like relying on poor financial data, flaws in sustainability data can do more than damage a reputation.

So how to get the right data for good decision-making?

The genesis of viable solutions

One of the COSA objectives is to transform data into knowledge for better decision making. We’ve distilled some pragmatic and uncomplicated lessons from our work with hundreds of organizations and experts plus more than 100,000 field surveys. 

Our well-established systematic guidance can help nearly any company or institution to significantly improve their sustainability content. Beyond the rigor of data used for impact evaluations, we also have a strong focus on the lean or day-to-day data used for monitoring performance or compliance. 

One clear thread runs through this work: understanding the elements of data that matter most. Knowing how to measure what matters can continuously improve effectiveness, increase efficiency, and even help manage costs. 

Guiding Insight #1: Is your data both simple and SMART? 

The combined experience within the COSA consortium results in indicators and metrics that represent the best balance of scientific rigor and business-driven pragmatism. They are aligned with international norms to ensure validity and credibility, follow SMART principles so that they can be applied meaningfully, and they are developed in a multi-dimensional framework to ensure that gains in one aspect of sustainability do not come at the expense of another. 

As part of our Performance Monitoring work with many supply chains and projects globally, we have learned that it is important to have a pragmatic approach that can be readily applied during normal business processes and thus eliminate additional costs for data. The William Davidson Institute at the University of Michigan recently demonstrated the value of this simplicity in Danone’s urban supply chains discussed in Performance Monitoring: An Agile New Tool. 

As part of the Lean Research Field Guide published with MIT Design Lab, Sustainable Food Lab, and others, we outline the value of respectful simplicity when engaging farmers or suppliers. Across cultures and functions, it is easy to misinterpret complicated questions and it is easy to waste the time of your supply partners with unnecessary questions.

Guiding Insight #2:  Is your data really comparable across geographies, year to year, or to benchmark with others? 

We learn a lot from comparisons. For example, one of the most important economic indicators of smallholder farming systems is the cost of production. When measured in the same way each time, it can be an invaluable indicator of profitability and income and can help understand the main cost drivers in a farming system. All too often, we see this indicator inconsistently applied within and across populations leaving managers with a meaningless range of data. 

To do this well, you have to consider the “what” to measure and then “how specifically” it is going to be measured. While most approaches consider the basic costs of fertilizers, pesticides, hired labor, and planting material/renovation, fuller approaches may factor in other direct costs: energy, rent of land, irrigation, equipment, etc. or even indirect costs like capital assets, training costs, credit costs. Some will factor the value or the opportunity costs of unpaid (household) labor. It can get complicated if those are measured in different ways.

Contexts change, so you may not want to measure aspects such as credit costs, amortization, or mechanization in some places. Regardless of what you include, the important thing is to compare the same components, measured in explicitly defined and standard ways to get an accurate picture of the production costs. Otherwise, even small variants carried across data sets, add up and easily mislead decision-makers.

Whether it is cocoa, cotton, or coffee, we know that farming systems in Africa can differ from those in Latin America or Asia, but with a standardized approach, you can compare the key components (oranges to oranges) across geographies and year-to-year, to reveal the key cost differences and generate useful learning. 

Guiding Insight #3: Are your metrics specific enough for good decision-making?

Organizations often want to quantify the portion of their volume sustainably sourced. We have seen this as a blanket question with little clarifying information, e.g., asking “What was the volume of sustainable product that you sourced this year?” which is a surefire way to get poor data. 

‘Do you have a sustainability management plan?’ is another common trap. It misses many important factors such as what is included in the plan and who, if anyone, applies it. Questions that focus on intentions or mere activities fail to capture applied realities.

Such broad and generic questions make it difficult to have accurate or consistent information. They defeat the potential for any serious management decision-making, are considered weak as due diligence, and reduce the possibility of continuous improvement. Such questions or indicators can hold back the sustainability journey of an organization. Data and evaluation professionals also know that they are a red flag for potential greenwashing. Consider that sloppy data from sloppy questions are easy to expose.  

Guiding Insight #4: Do your metrics capture and encourage areas of continuous improvement?

Some metrics, despite good intentions, fail to deliver actionable information that can drive improvement or even better decision-making.  

Binary ‘yes/no’ questions are rarely useful beyond their simplistic function for compliance and yet they are simple to improve. Of course, they do not capture incremental changes in the amount or quality of the topics in question, so any changes can easily be missed and thus they fail to stimulate improvements year to year. 

Noise in the data is a bit harder to address. Noise or error is introduced when questions have multiple objectives such that they cannot be disentangled in the response. To illustrate simply, questions such as: “What percent of farmers in your supply had access to inputs, equipment, and extension services?” poses a data problem. If a producer had access to one or even two of the three items, they also would have to answer negatively thereby conveying a message of less access to services than may be true. Viable alternative questions would treat each component of the question separately and allow enough sensitivity to capture changes in sustainable outcomes over time. 

These and a number of other guidelines can help you to develop useful data. Data can be actionable to help you calculate returns on sustainability investments, to improve programs or services, and to deliver value to farmers as well. If you are investing in software and various technologies, you would benefit from also developing the right content with science-based and SMART indicators. COSA’s basic work in standardized global metrics is freely available to help ensure that managers have data that can make a difference.