How to quantify the ROI of data using decision playbooks

I was talking with a marketing VP at a large retailer about their challenges with data. I expected them to ask me about how to get more (or better) data, recommendations on AI or what technologies he should be exploring. Instead, he surprised me when he told me his team wanted less data and more insights.

I’m seeing a similar shift at other companies. It is increasingly clear collecting more data won’t move the needle. Companies want their data to help them make better decisions and more of it isn’t always the answer.

Here are some ideas on getting more insights without needing more data. 

The broken link in data

The obsession with data has protected this area from deep scrutiny. Few people would argue against the importance of data, even though it’s not always clear if it’s actually helping. 

That’s because there’s a missing link between it and business outcomes. To be fair, the connection isn’t always obvious. Unlike other things, such as paid advertising or sales reps, causation can be difficult to prove. Did data help you increase sales or was it something else?

The fuzziness has prevented companies from quantifying the true ROI of data. Since ROI is tricky to measure, many companies avoid the discussion altogether. Data is seen as an operating cost that doesn’t require a cost analysis. 

It shouldn’t be this way.

Macy’s is a great example of data driving better decisions and tangible value. Specifically, it let them avoid the inventory glut that plagued other retailers. Looking at their customers’ spending habits (through co-branded credit cards) they noticed more spending going into services. This aligned with other economic indicators and the surge in spending in other service industries like travel.

Macy’s leadership team has a monthly meeting where they discuss all of these data points and focus on determining the “so what” behind the data. At one of these they decided to change their upcoming orders and buy more products used for going back to the office or on vacation. 

Data helped Macy’s avoid costly inventory waste and shows how the broken link in data can be fixed. In essence, Macy’s has a clear process for making data-driven decisions. While you may not have the same resources as Macy’s, you can achieve many of the same benefits by taking advantage of a simple idea: decision playbooks. 

Dig deeper: Why we care about data-driven marketing

Quantifying data ROI using decision playbooks

Data is supposed to help individuals make better decisions. The problem is that how individuals make decisions is opaque at many companies.

This is where decision playbooks come in. A decision playbook is a document outlining the exact steps needed to make a decision. Let’s say that your team wants to run an A/B test on a key part of the conversion funnel. Do you know all the decisions are needed to launch and evaluate any given test?

Here’s a short list of what you would need to decide.

• Step 1: Determine the biggest opportunity area.
• Step 2: Find a KPI to measure success.
• Step 3: Design a test, including multiple variations.
• Step 4: Evaluate the test impact.
• Step 5: Capture learnings for future tests.

You can already see where data comes in. Steps 1-3 will use some kind of data analysis. Other decisions may also include some data once you get deeper into things.

A decision playbook would outline all the decisions needed to run a successful A/B test. More importantly, a clear playbook makes it easier to know where data is playing a role. After you run through the playbook, you can determine the value generated (e.g., a 5% increase in sales).

Data isn’t the only thing that drives success so we can’t attribute 100% of the value to it. In my experience, an attribution of 10%-20% is reasonable. If we take the low end of 10%, we can start to see the value of data to your organization.

The process here isn’t complex. If you consistently capture the value generated by key decisions and then attribute a portion of it to data, you will start to justify the value of your data. Best of all, you can see how this number changes over time. 

Dig deeper: Why testing is a marketer’s most powerful tool

Designing your own playbooks

Every marketing team can start taking advantage of decision playbooks. However, you need a proven approach to get up and running quickly. Here’s how I typically help companies bring consistency to their decision-making.

1. Raise awareness of consistent decision-making

Many teams often experience inconsistent results. Some campaigns are great while others miss the mark. The distinction comes down to how decisions were made. Without a playbook, it can be difficult to optimize these kinds of decisions.

2. Determine the most important decisions your team makes

A/B tests, new campaigns, go-to-market strategies and others typically make the top of the list. They are decisions that happen regularly and require a significant investment of time or resources. 

3. Design what the ideal decision-making process should be and compare it to what is right now

You will find some steps are often skipped and others aren’t fully completed. Common mismatches I see include not validating the data used, lack of alignment with the overall strategy and failing to get feedback from the rest of the team.

4. Train all relevant people on the decision playbook

The playbook is something anyone can reference when confronting these decisions. Just like doctors or pilots use a checklist — despite thousands of hours of training — marketing teams also need guidance on how to make the best decisions, every single time.

You can’t rely on luck

Marketing teams cannot rely on luck when making decisions. Hoping that they looked at all the relevant data and interpreted the results correctly.

Decision playbooks will start to make the link between data and insights clearer. It will allow teams to justify their spending in this area and best of all, drive data-driven decision-making.

It all starts by making the decision to get more value out of your data.