A fresh look at the tech review space

It hardly needs to be said that the B2B buyer journey is around three-quarters complete before there’s any contact between buyer and sales. This is as true for martech, as for any other considered and expensive business purchases. What are some of the touch-points on that independent journey? Research and contact with peers, certainly: and one place those two things come together is at a software review site.

Anyone working with martech should already be familiar with CabinetM, a site dedicated exclusively to marketing tools: but there are other software review sites out there, with a much more general perspective, but which still stock a large volume of martech information. We wanted to catch up with what some of these sites offer, how they’re monetized, and how they weed out the shills.

G2: One million reviews

“We hit the milestone some time last year, and it’s a pretty cool milestone to hit as far as B2B user reviews go,” said Yoni Solomon on G2 (formerly G2 Crowd) passing one million reviews. “I spent several years on the B2C side, where obviously it’s much more prevalent.” Solomon is Director of Product Marketing for the platform, which includes marketing automation, ecommerce and CRM in its searchable categories.

SoftwareReviews: Depth over volume

SoftwareReviews has its roots in the research and analysis space, having originally been a division of Info-Tech Research Group. As Alan Neal, Chief Growth Officer explained: “Info-Tech has been around for 23 years. Software selection is one of the core issues that Info-Tech helped its members with. SoftwareReviews stems from Info-Tech’s goal of changing the way it helped its members select software.”

Info-Tech had gone the traditional research route of holding briefings with vendors and coming up with its own quadrant based its analysis. “People wanted to get a little further away from the analyst’s perspective and pivot more towards what peers were saying,” said Neal. “Five years ago we switched towards a peer-evaluation platform, and it has recently spun out into its own entity.”

Origins affect output. “Our reviews are almost like a diagnostic of the software experience, because they’re extremely in depth; we cover about 130 points of data in every review,” Neal told us. “Our reports and reviews are only one component in the software selection process. How we leverage the reports and work with end users to make those decisions is the other component, and has roots in our research and advisory.”

What’s more, the platform isn’t trying to be encyclopedia. “We have vendors that want to be featured on the platform, and will give us the perspective of some of their customers. We also have associations we partner with that survey technology users. But our goal isn’t to identify every single category that exists; Info-Tech analysts help us understand what technology users are requesting, what categories are really critical, and that allows us to target our data collection efforts.”

TrustRadius: The current environment

Budget pressures in the software buying space are by no means bad for review platforms. Said Russ Somers, VP Marketing at TrustRadius: “No-one welcomes a pandemic, but it’s been good for our business in a couple of ways. We tend to have a very enterprise audience, so our customers kept spending and our business kept growing.”

The reason was clear. “It’s a tough business environment no doubt, but people are having to be more careful about decisions they make in spend, and therefore we become more valuable to the buyers, and therefore also more valuable to the vendors.”

On a number of key issues — monetizing the platform, authenticating (and retiring) reviews, and providing services to vendors — G2, SoftwareReviews and TrustRadius have some commonalities, but also some interesting differences.

The shelf-life of reviews

Of course, having over a million reviews at G2 does imply that many are now old, and possibly out-of-date. “We’ve made a push toward review recency,” he said, “because that clearly does matter. If a vendor has ten-plus reviews, it’s going to make it onto our grid; but if those reviews are old, you’re going to see conversion on your profile suffer significantly.”

Last year, G2 acquired Advocately, a solution which automates the soliciting of reviews. “What that allows a company to do is sync an NPS tool they’re using, or sync to Salesforce, and to trigger review collection emails. We hope to see more review recency on G2 as customers need to run fewer one-to-one review campaigns.”

At TrustRadius, with around a quarter of a million reviews, there’s also no expiration. “A review is like a journal,” Somers explained. “Probably 20% of our reviewers will at some point come back and update a review. Knowing we have that kind of engagement with our community, we keep them live, we don’t have a time-span when they will expire, but we weight them differently in the algorithm.” SoftwareReviews currently counts its volume of reviews in tens of thousands, and emphasizes their comprehensiveness — but older reviews do get “de-weighted,” Neal told us.

All about the gift card?

It’s no secret that some reviews appearing on these platforms are incentivized — but it’s not universal, and not necessarily a bad thing. “There are three basic ways reviews get onto G2,” said Solomon. “The platform will collect them on the vendor’s behalf by running review campaigns. The vendor can collect them, although there can be a lower response rate.”

Vendors can choose to incentivize reviews with something like a gift card. “I think we’re seeing that drop across the board. Vendors are getting more sophisticated about who they’re reaching out to, and the times they’re reaching out, and as a result we’re seeing big increases in review collection that isn’t incentivized.”

“We do offer five to ten, sometimes 20 dollars,” Neal told us, “and the reason is the depth of data we’re asking for. It takes ten to fifteen minutes to complete. But every review passes through a rigorous data quality check.”

Somers thinks incentives actually reduce bias. “I am going to guess 25% of our reviews are organically generated by people in the community. Some of them, we reach out to people who have shown expertise, and we’ll offer a gift card or some such. We’ll also do it on behalf of the vendor, with the caveat that we’ll be sure we’re touching the whole customer base, not just the happy customers.”

Although a fair number of TrustRadius reviews are incentivized, Somers thinks that’s less of an issue than it is in B2C. “We’ve found it produces less bias because unincentivized reviews, especially of business software, [tend to be] written when really angry or ecstatically happy. By using incentives we get a fuller distribution.”

How reviews are authenticated and validated

There are automatic checks for inconsistencies in responses at SoftwareReviews. “Our data quality team looks at the actual quality of responses to make sure they come from someone who would actually use the platform, and every review has to be LinkedIn-verified. You need to have a LinkedIn profile and come from an organization that would use the platform. We need to make sure that we can stand behind any recommendations we are making.”

It’s worth noting that SoftwareReviews customizes its survey form according to the role of the reviewer. “Based on your specific role, the survey is going to change. If you’re the purchaser, we’ll ask you about costs and the negotiation experience,” said Neal. “If you’re on the IT side, there will be discussion of implementation. We have a whole report that deals specifically with the emotional footprint – what’s that actual experience like dealing with a vendor and what’s the long-term relationship like.”

“Every review which comes through G2 is authenticated and validated through a process that includes people looking at it and checking it, as well as automation we put into place,” Solomon told us. Curiously, the closer a vendor gets to five stars, said Solomon, the less trustworthy the ranking appears. “At my previous company, we ran a study with Northwestern, and found that the perfect star rating was somewhere between a 4.2 and 4.5; anything after that looks a little too good to be true. We’re rolling out education to vendors that it’s actually not in their best interest to stack the deck and get as close to a perfect five as you can.”

“One thing we have that other sites don’t – that I’m aware of,” said Somers, “is our algorithm for calculating what we call TR score. It’s based on a number of factors, including review source. If [a vendor] decides to send only their happiest customers to TrustRadius, we will detect that those are single-sourced, that they’re not organic. We’re not necessarily going to deny the reviews, but we’ll weight them a lot less heavily than if they were organically generated, or generated through a campaign we know touches all customers.”

TrustRadius also promotes a program called ‘True’ (for transparent, responsive, unbiased and ethical) that rewards vendors for that type of behavior. “Brands like IBM, Oracle and Cisco raise their hands and enthusiastically sign up,” said Somers.

Monetizing the review space

“Vendors pay for a contract with G2 which includes things like an upgraded profile with branding and demo video downloads, and buyer intent data across the market place.” This includes tracking target accounts which view the profile or just the category, as well as competitor comparisons.

“We sell a couple of other things,” said Solomon. “One is a content subscription that allows you to license G2 badges, awards and reports for you to use within your own landing pages and websites. Lastly, my favorite product, our insight subscriptions, which is deeper level competitive insights you can pull for the top competitors on your category. This allows you to find product and pricing gaps and compete more effectively.”

At SoftwareReviews, Info-Tech members get access to reports and analysis as part of their subscription package. The platform also charges for additional services: “We have anything from a series of calls to on-site workshops,” which were placed on pause during the pandemic. “We look to reduce the time and the number of people it takes to select software, and increase overall satisfaction with the roll-out of the software.”

SoftwareReviews also drives revenue from vendors. “We offer subscription services to our vendors if they want to leverage any of the reports and data.” Neal believes the emotional footprint is something which really stands out for vendors. “27 data points that look at what the long-term experience is like in using the software. What drives satisfaction, features or emotional aspects.” SoftwareReviews offers “board-ready” reports and other collateral which vendors can integrate those into their GTM strategies.

TrustRadius has a different model, sometimes characterized as selling leads — but inaccurately, Somers would say. “We don’t sell leads directly, but we do help vendors drive demand with a couple of things. One is intent data, helping them to understand which companies are shopping in their category. By the time somebody is shopping on a review site, the intent is very specific: I’m considering these three marketing solutions. Additionally, there’s the ability to build audiences based on that, and then target those audiences through your traditional platforms.”

He also points out that TrustRadius is focused on companies and accounts, not individuals. “We’re not selling you Joe Smith at Microsoft; we’re letting you know that someone at Microsoft’s New York office is researching solutions.”

Like G2 and SoftwareReviews, TrustRadius offers vendors ways to, as Somers put it, “weaponize the content. We provide tools for you to break that content up, pipe it where it needs to go in your eco-system to influence buyers. We sell this as a subscription package.”

Reliant on trust

Ultimately, whether the word “trust” is in the name or not, review sites have an investment in credibility. Two comments summarize that:

“We want to make sure we get the good, the bad and the ugly,” said Neal.

“We hold ourselves to standards of neutrality,” said Somers. “We’ll help you put your best foot forward, but it has to be an honest foot.”