The Real Story on MarTech: Never skip a bake-off when selecting a new platform

I have never been accused of being a chef, cook, or baker. So, I am probably the least likely to write a food-related blog post. However, as an enterprise martech advisor, I increasingly find myself defending the practice of the “bake-off.”

What’s a bake-off?

In the context of an agile martech selection project, a “bake-off” is really a competitive proof of concept where the two vendor finalists are asked to mimic a real sprint, and then show you how to get hands-on with at least some parts of their proposed solution.

The key to a bake-off is customizing it to your requirements, with your ingredients (content and data), your bakers (participating employees), and your ovens (your real environments) — although typically you’ll employ the vendor’s kitchen (a.k.a., cloud environment).

Sometimes labeled a “sandboxing” or “prototyping” exercise, this approach is particularly well-suited to vetting marketing technology, since marketers should be familiar with both the empirical nature of the effort, and the general idea of A/B testing.

A bake-off story

Consider this experience I had at Real Story Group helping one of the largest hospitals in North America select a new digital asset management platform. After a productive vendor demo phase with five competitors, the selection team identified two finalists. However, “Vendor A” emerged the clear leader among the team.

It was tempting to just abandon the plan to conduct a bake-off and simply start the implementation with Vendor A. However, in the interest of full due diligence, the selection team went ahead with a bake-off where Vendor A and second-place “Vendor B” each had to implement three high-priority use case scenarios. During the bake-off weeks, the hospital’s marketing and IT team worked side-by-side with the vendor’s implementation team, learning how to use the platforms and execute some tasks themselves.

At the end of this phase, the team assessed their findings, and here’s what they learned:

• While Vendor A’s user interface had dazzled during the vendor demos, when it came to building out the hospital’s most common use case, it was Vendor B’s workflow and interface that resonated most with the hospital’s marketing team;
• When it came to using the hospital’s real content (high-res photos from numerous photo shoots), Vendor B was able to demonstrate much faster ingestion times;
• When it came to accessing each systems via mobile devices (a common request), certain key functionality was not available using Vendor A’s solution (this never came up in the vendor demos);
• When the hospital’s marketers reached out to each of the vendor’s help-desk, Vendor A’s help-desk never responded, but Vendor B’s help-desk addressed an issue immediately;
• And, something that should never be underestimated: the hospital’s team liked working with Vendor B’s team more than Vendor A’s team.

As you will often hear RSG analysts say, there is no perfect vendor or perfect product. And I don’t mean to cast Vendor B as a perfect fit in this case. But they were a really good fit for this client — something the enterprise would never have known without engaging in the bake-off. Vendor A was partially reimbursed for their time and Vendor B used the bake-off work as a head-start on the actual implementation project. The hospital has since implemented their new DAM and the project has gone well.

Addressing the nay-sayers

If you still get resistance to this approach, consider switching the analogy from food to cars. Let’s say you’re buying a new vehicle. Would you purchase it after watching the salesperson drive around the dealer’s lot? If you’re going to be using this car for the next five to ten years, you need to get behind the wheel yourself. The only question then is: how long a test drive?

In the case above, a one-week head-to-head bake-off proved sufficient time to get the information they needed to make a decision. In some cases, you’ll want a longer time and in others, a shorter time will suffice. The key determinant in terms of level of effort should be: how critical is this platform in your stack? Something that’s an “anchor tenant” in your martech mall will require at least a week of testing per bake-off finalist. More “boutique” solutions may only need a couple days of hands-on testing.

Business value

Like all good things, a useful bake-off takes time and attention, since the mechanics can get tricky. But the payoff is worth it:

• By engaging your end-user colleagues in a way that feels relevant and useful to them, via hands-on testing, it bodes well for their support during tough implementation times;
• Testing your own requirements against reality enables you to reprioritize your objectives midstream, before you’re locked into a particular supplier;
• This sort of prototyping allows you to jump-start your initial implementation and reduce the time from contract to value;
• By learning (most of) the shortcomings of the winning vendor, you can plan around them;
• You get a better sense for realistic implementation costs before you select a solution, and you also gain some valuable time and space to negotiate the best deal;
• You obtain a more accurate preview of likely operational, process, and staffing implications for any new platform — implications too often underestimated in the martech world.

So if you are in the market for new technology, don’t skip the phase where your people test the new system with real-life use cases. As we’ve seen with dozens of successful selections, devoting the proper resources to conducting a proper bake-off before you commit to a solution will ultimately leave a much better taste in your mouth.

Real Story on MarTech is presented through a partnership between MarTech and Real Story Group, a vendor-agnostic research and advisory organization that helps organizations make purchasing decisions on marketing technology applications and digital workplace tools.