Earlier this year, we published writings on our meetings and discussion with Frank Cespedes, Senior Lecturer of Business Administration, Harvard Business School, where he shared some great insights related to B2B selling, technology, and trends. Don’t worry if you missed the webinar; we’ve summarized our learnings and points of discussion here, and here. We talked about his book “Sales Management That Works” and also:
- Rise of the expert systems and digital transformation (CPQ, QTC, Billing systems and sales enablement systems)
- Selling in a B2B world
- Omnichannel
Wondering what Frank’s up to today?
Today, Frank continues to share his wisdom, knowledge, and research with the world on how to improve B2B selling organizations with new strategies and ideas. In his recently published article “How Managers Can Build a Culture of Experimentation”, Frank discusses his thoughts and findings about how business can put their innovative and evolving ideas to test, and use the results to make strategic changes. He highlights common best practices as well as stories of companies that have suffered due to poorly conducted B2B and sales experiments.
“Experimentation encourages innovation, but it can also be time and resource-draining.”
While every word of that sentence is true, Frank wrote, to make your experiments worthwhile, it is of paramount importance to conduct experiments with the following in mind: What are you looking to learn from the experiments? Frank highlighted how “most managers are good at asking questions, but not as good at specifying what would constitute a feasible answer to those questions”, How will you apply your learnings? Will your learnings yield any opportunities? What kind of conversations are you intending to have with your colleagues about your findings and their impact on organizational decision-making?
Conducting experiments in the ever-changing world of business requires one to be extremely cautious; as can be seen from the various stories that Frank Cespedes presented in the article.
No guinea pigs
Frank expressed that experiments in the business environment are not the same as medical experiments “Testing in business presents qualitatively different challenges than those in most academic and medical research. There are few opportunities for randomized control trials in a competitive market. You must typically repair the ship while it’s sailing on open waters in weather conditions that you do not control. This is especially true in an era of big data and artificial intelligence”.
Use what you can for today while investing in finding answers for tomorrow
Using existing data can help you with your tests and experiments, and even in determining whether an experiment is needed at all. Sometimes, just going back to the existing data can give you enough information for making business decisions; experiments and tests in such cases are an unnecessary expenditure.
Frank demonstrated the validity of this statement by sharing an example;
“In developing its digital strategy, a prominent retailer did not consider existing research, including a published and peer-reviewed study on cross-channel consumer behavior across more than 7 million purchases by nearly 1 million customers. Insisting that all evidence be “first-hand” data, it commissioned a test using six company-owned locations.”
The result?
- Time to take important decisions and actions was extended by 8 months
- Legacy biases arose, which negatively impacted them; while their competitors focused on multi-channel initiatives that were successful, and enabled them to gain market share.
Keep a close eye on the opportunity costs inherent in testing
“A B2B SaaS company was presented with evidence that a traditionally unprofitable customer segment was starting to shift its purchasing behavior, and a relatively modest marketing investment could accelerate that shift. But the legacy of losses loomed large, so decision-makers set a high bar in terms of experiment duration, sample size, and methodology to overcome organizational disbelief, when much simpler means were available to test the ROI of new initiatives in this segment.”
The outcome?
- The larger tests cost nearly 5 times more
- The business’s actions were significantly delayed while the market continued to change fast.
Agile systems are easier to experiment with and run tests on. Systems like servicePath CPQ+ can help you build products and pricing models that you can easily test.
Garbage-in garbage-out
Frank elaborated on the fact that having the right, reliable data can save you millions of dollars and months of time. “In most machine learning projects, as much as 80% of the time and costs of data scientists and IT groups is spent cleaning up the data, due to things like inconsistent inputs, outdated views of buyer behavior, and legacy assumptions.
Common examples involve tests driven by data in customer relationship management (CRM) systems. The inputs are noisy because the system reports the aggregate result of what, in reality, is multiple people using different criteria. Most CRM software also weights revenue expectations by pipeline stage on the assumption that the odds of closing increase in reported successive stages. But rather than moving sequentially through a linear funnel, omni-channel buyers now move from online to physical to influencer channels multiple times in buying journeys. Once the system is in place, however, tests are then designed to optimize for the software parameters, reinforcing an outdated view of consumer behavior. The test becomes a self-fulfilling prophecy, not a window on market realities. More generally, as others have noted, as easy access via mobile devices makes just-in-time information a growing factor in purchase decisions, many traditional research techniques like conjoint analysis do not reflect how buying decisions are made.”
The article continued this statement with a story of its co-author’s – who was once associated with a company that had a churn rate of 3% each year, according to the marketers. What’s interesting is that this number was established back in the 1990s – and it was being used ever since; regardless of repeated variations in products, fluctuations in prices, competition, substitutes, and consumer choices – not to mention all the technological advances that must’ve occurred in the interim. This goes to show that experimenting with systems that allow easy access to values and variables, and wherein it’s easy to manipulate entries and compare results is much better than experimenting with systems whose values are difficult to modify. Agile sales enablement systems like servicePath CPQ+, which is seamlessly integrated with leading CRMs, ensure complete transparency and easy viewing and access to values and variables like prices, and products, and thus facilitate real-time feedback from your customers and teams quickly, and safely. servicePath’s low-code/no-code framework enables users from sales teams, product managers and sales engineers to execute these real-time quote models and study the potential outcomes independently, without relying on, or involving IT; thus building the culture of experimentation and innovation. Sales is the lifeblood of most businesses and therefore it becomes imperative for these organizations to constantly get feedback from customers on new products and solutions as market dynamics change.
Frank and his co-author Neil Hyone added that testing should only be done with data that you are confident reflects the needs of your business. They referred to Amazon Prime’s winning “free returns” policy as an example. Product returns, they quoted, go way beyond “sending a parcel back”; its a trillion-dollar issue for retailers worldwide. How can this information be used for building reliable tests and ensuring business agility?
Frank adds “You can ask customers if they plan to return their purchase, but ex-ante surveys are a poor basis for predicting this behavior, and some companies now offer discounts to customers who give up their right to return a product — an inhibition to buying in many categories. A buyer’s order history is a firmer basis for testing. One study found that when shoppers interact with products, zooming in to see the texture of the fabric or rotating it to see its appearance from multiple sides, they are less likely to return the purchase. Conversely, those who order in a scattering of sizes are more likely to return products. This data can provide hypotheses for relevant tests that, in turn, generate dialogue about website design, pricing, order-fulfillment policies, and terms and conditions.”
By being mindful of, and entering this data into your systems, you can be assured that your tests will yield results that are reliable as well as useful.
Experiments are loaded with challenges
It is crucial that you be extremely careful with the data and about any assumptions you build your tests from. Then, knowing exactly what kind of results to extract and then how to leverage them in order to improve business functions are also imperatives of business success. Experiments in the world of business, by virtue of being experiments, are loaded with challenges. The greatest of which, Frank and Neil wrote from their experience, are internal processes and the need for well-established communication between departments and stakeholders. If the “right answer” is not explicitly defined by the manager and communicated with the teams, who would know what to do with the test results? Obviously, computers cannot be relied upon a hundred percent; not for taking business decisions on behalf of humans at least. Unless we set the computers up with the right parameters, the systems are not going to deliver the desired quality outcomes.
“Data, even allegedly self-correcting data as in some AI programs, is never the same as the answer to a management issue. Years ago, Peter Drucker emphasized this: “The computer makes no decisions; it’s a total moron, and therein lies its strength. It forces us to think, to set the criteria.” Data is crucial, but it’s mute. Managers must always interpret data with an end in mind. Pricing is an example. A price has multiple dimensions: base price, discounts off list price, rebates tied to volume, special offers, price for additional services, willingness to pay depending upon the product application, and so on. Further, price information is now often a click away for customers. Sites including Edmunds.com, and Kayak facilitate price comparisons in multiple categories. And inertia is rarely the profit-maximizing option for sellers. Notice, for instance, how Amazon distills thousands of SKUs for consumer-packaged goods into price-per-ounce comparisons on its website. Price testing should be an ongoing part of effective marketing, but first clarify the evaluation criteria because testing in business ultimately means evaluating alternatives. There’s a big difference between using profit increase or revenue lift, for instance, as the criterion, and price changes typically have an impact over multiple time periods, not just in the short term. Yet, most companies fail to specify the criteria they will use to interpret pricing tests and they spend time and money in an unfocused fishing expedition that goes nowhere.”
Experimenting with different numbers and pricing models, therefore, is not an easy task for manual and slow methods or even traditional software systems that are not as agile as some advanced systems. servicePath CPQ+ is an advanced and agile CPQ system where testing with different prices and complex product bundles is made very simple for customers and sales teams to get feedback quickly and adjust as needed to leverage the opportunity.
In addition, features like servicePath’s Deal Dashboards allow for real-time business intelligence and feedback. Deal Dashboards allows the sales teams to toggle between different aspects of the deal like payment models, product types, and duration and to quickly model inputs, requests from their customers and their desired configurations and purchasing volumes. Deal Dashboards provide complete transparency, show you key deal metrics and allow you to make changes according to the requirements of your customer. Deal Dashboards give you visibility, and enable you to obtain feedback for quote parameters and key metrics, and make adjustments quickly across multiple dimensions like cost impact, margin analysis, and payback. With Deal Dashboards, management and sales operations can quickly validate profitability analysis instantly and ensure that the deal you’re signing is a good one. Furthermore, its quick and easy to modify the requests and changes in inputs of your customers into these solutions. servicePath’s Deal Dashboards give a comprehensive, holistic and detailed view of the deal; and the interface is extremely easy to use. Deal Dashboards are also used by key stakeholders for example CFO and Head of Operations; because it can enable them to conduct a very detailed deal analysis; and the view and usability of Deal Dashboards varies with user roles. Deal Dashboards enable users to build the configurators very quickly with different pricing for example. Its then really easy for sales teams to get feedback and adjust as needed to help leverage the opportunity.
Frank and Neil also in their blog shared an example of a company that was clear about why they wanted to conduct the test, as well as the criteria they wanted to use in evaluating the results, before starting the experiment.
“….Yet, most companies fail to specify the criteria they will use to interpret pricing tests and they spend time and money in an unfocused fishing expedition that goes nowhere. An exception is Basecamp, the collaborative software provider whose products span a wide range of users, applications, individuals, and large corporations. When it introduced its Basecamp 3 product, it conducted a combination of price surveys, A/B tests, various offers, and specified its criteria up-front for making decisions.”
What a well-put-together experiment it must’ve been!
The outcome?
Well-informed business decisions were made and as Basecamp had intended, maximum LTV was achieved. “These criteria helped the organizational dialogue and improved cross-functional efforts to evaluate the data and implement options. There’s a tradeoff between LTV (lifetime value) pricing opportunities and maximizing initial customer acquisition. Different functions (sales, marketing, operations, finance, investor relations) usually have different views of that trade-off, and in many firms valuable options are stopped by managers who optimize their function’s metrics, not enterprise value.”
servicePath’s advanced CPQ platform enables managers to conduct tests for different configurations, solution offerings and quotes, where users can split test various offers and configurations and see which one gets better uptake. Product portfolios continuously change which makes it important for organizations to add new inputs and have new options, changing price points, and more functionality. Businesses, in order to succeed, need to constantly innovate by pulling those solutions together in different ways so they’re most desirable and impactful for their customers. A system that makes it easy to experiment with these variables can help businesses stay ahead of the game. Innovation has become the key to success because customer needs are also always changing and knowing what products and offerings work for your customers enables you to always be the one that they buy from. Managers can experiment with combining new products and inputs into their solutions and analyzing the impact it has on their sales.
Testing various product configurations, for example, enable managers to understand what’s selling and what’s not.
Pay attention to seemingly “small” details
The authors Frank Cespedes and Neil highlighted that billion-dollar companies oftentimes tend to overlook “small” ideas that, when aggregated, have the potential to make an even bigger impact, and reduce risk significantly; and they again gave “pricing” as an example.
“The impact varies by industry, but studies indicate that for a global 1000 firm, a 1% boost in price realization — not necessarily by increasing price on every order, but averaging out to 1% more and holding volume steady — typically means an 8% to 12% gain in operating profits. These results have been consistent for decades —before the internet became a commercial medium, since then, and for both online and offline firms.”
Pricing, hence proven, is the most easily experimented with, when making huge business decisions. It kind of makes sense if you think about it; pricing after all is the main determinant of macroeconomics. Pricing is also the most fragile element of a deal; pricing can make or break a deal.
The duo closed the article with some words of valuable advice;
“Seek progress, not perfection, and invest in processes that allow employees to submit seemingly small ideas. Online channels make testing these ideas feasible and inexpensive when you know how to ask questions. Here are three straightforward approaches:
- Mine your website purchase interactions. When airlines add a question asking if a trip is for business or personal, they have insight into price sensitivity for upgrades.
- Rotate periodically the questions you ask, gathering insights that are missed when the same questions are unchanged for months or years.
- Engage users and non-users. There’s now a class of tools that enable you to engage directly with customers and prospects in real-time and at different points in their buying journeys.”
Buying behaviors have changed drastically over the past years; with many businesses offering a compelling omnichannel experience, customers are exposed to nearly infinite options. Experimenting and testing in an omnichannel world is a whole different kind of challenge.
“As the pandemic demonstrated, markets move faster than ever and it’s your job to adapt. Talk about “big data” and “digital transformation” has many managers obsessing about how to store data. But the best firms obsess over how they can use their data in actionable tests of new ideas. Think of testing in your organization as part of an ongoing conversation with your market — a motion picture, not a selfie or snapshot, in a world that never stops changing.”
If you are interested in learning about Leading in the Digital Era from experts of Harvard Business School, register for the program here:https://www.exed.hbs.edu/leading-digital-era/
