Whitepapers

Best Practices Guide to Benchmarking - Debunking Ineffective Comparisons

by Agota Alvarez
on

Introduction
Clients frequently ask us how their employees, company performance, or survey results match up with others outside their organization (i.e., “How does my company compare to other companies in my industry?”) We frequently get this question because we are in a great position to answer it. Caliper is an international talent management consulting firm that has worked with over 30,000 companies worldwide, across all sizes, sectors, and industries, to help them make optimal selection and organizational development decisions. However, part of being a partner and advisor for our clients is not simply answering their questions, it is guiding them toward the questions they should be asking and getting them on the path that will enable them to achieve their true goals and maximize their success.

When a company wants to make a benchmark comparison, it is usually on at least one of the following factors: how does my company’s satisfaction, engagement, or culture survey results compare to other companies in my industry; how does an external candidate compare to my group of people within the department I am looking to hire him or her into; or the organization’s current performance to past performance.

Let’s take a look at each of these comparisons individually:

Industry Comparison
Organizational surveys are one of the most common methods for understanding how an organization is performing on some important aspects that lead to healthy organizations (e.g., communication, leadership, employee engagement and satisfaction, teamwork). Asking employees to rate the company’s performance on some key factors can provide a company with an approximate idea of how well they are doing on those factors. Having a score for each metric tends to drive people toward a competitive, as opposed to an ideal, way of thinking that leads them to ask, “If only 55% of my company feels we do an adequate job of communication, how does that compare to other companies in this industry?”

Comparing your organization to other companies in your industry may enable you to find out you are slightly ahead or behind of your competitors on that metric, but such a comparison ultimately is likely to be of less value than identifying and implementing targeted solutions to help you reach your ideal state—i.e., how your organization should look and where do you want to be on a metric such as communication. If you want your organization to have open and straightforward communication from the top down, then you should be striving toward 100% of people providing favorable responses about the company’s communication strategies.

The purposes of comparing yourself to other companies are often logical enough on their face: it may help you sleep better at night knowing where you are in relation to your competitors, and in the competitive environment most of the world is living in, you may feel a natural need to compare yourself to others.

However, it is important not to let such comparisons distract you from striving to reach your ideal state. Apart from being in the same industry, two companies might have very little in common: different missions, values, competitive advantages, etc. It may not even be an apples-to-apples comparison since your organization may define an area like communication differently than other companies (e.g., open and in-person communication versus thorough, detailed, and communicating in methods that provide a paper trail).

In our experience, companies are best served by making a comparison based on specific criteria that are related in the marketplace. For example, instead of asking how your company compares to the industry at large, the question that you should be asking is, “What are the best companies doing to get such high employee engagement, or to have such low turnover?” You might find that great companies structure their onboarding and training a certain way that encourages engagement and therefore reduces turnover.Candidate to Team Comparison

The problem with comparing a new candidate to your internal employees is that your existing employees may not all be ideally matched to the job. A candidate may look similar to a composite of incumbents, but without doing a study you will not know what your ideal performer in the role looks like. The effectiveness of comparing people against a composite is dependent on how well you can quantitatively define what success and top performers look like in the role. Oftentimes subjectivity plays a factor when considering people to be top performers, which may not be tied to performance—such a comparison would not be a strong legally defensible way to make a selection decision. Even if you are looking at someone in relation to the top 20% of your team, you still do not have a comparison group to identify which attributes from that group actually correlate with performance. Without performing
a scientific study, you cannot confidently say what is driving that performance because it is likely not every aspect that is captured in that top 20%—our personality assessment, the Caliper Profile, measures 22 personal attributes, and there may only be six that correlate to performance in a given role. It could be pointless to try to match a candidate to the other 14 attributes as well, and doing so may actually introduce negative aspects to the team, such as group-think.

Interpreting the Caliper Profile on the basis of single traits is another issue that we hear about. Some people will notice that their top performers in a role are high in Urgency and consequently think that everyone they hire into that position needs to be high in Urgency. However, one cannot know whether this trait is actually linked to performance without looking at the rest of the range of performers in the role. If low performers are also low in Urgency and moderate performers are moderate in Urgency, then you begin to have support to say that Urgency may be linked to performance.

Current to Past Performance
On the surface, a desire to compare your current performance to your past performance in organizational survey results is a good thing. Performing this type of comparative analysis every year can enable you to see the progress the organization is making and how you are moving the needle; but the problem is stopping the analysis at this point. The primary recurring problem with comparing your current performance to past performance is that companies fail to consider how their ideal state relates to those two benchmarks. You might hear anecdotally that “it’s way better here than it used to be” or “if you think this is bad, you should have seen us 10 years ago” without any follow-up on where they want to be or how they intend to get there.

Furthermore, what you considered to be a key metric of success last year may not be as relevant this year or 3 years from now. The dynamic environment we live and work in should result in regular investigation to understand the most relevant metrics that should be used to measure success.At the same time, business metrics are not the only thing that should be assessed and updated as needed. Selection systems, job descriptions, performance appraisals, and bonus criteria in most industries should be updated every 2 years to reflect the changing marketplace in which businesses today operate.
It is unfortunately common for companies to perform an organizational survey or review overall performance year after year solely because they think they should or because it satisfies some curiosity or provides a general benchmark for where the company is now. Making an exploratory analysis of the data, as opposed to taking a targeted, confirmatory approach, can often be a symptom of a benchmarking error. That is, rather than asking specific questions before reviewing the results of a comparison, companies may simply dive into the data just to see what’s there. The problem with an exploratory approach is that it lacks strategy and could draw attention, focus, effort, time, and money to unnecessary or irrelevant findings that aren’t part of the company’s strategy. This is not to say there is no value in an exploratory analysis, but that reviewing results with the organization’s strategy, mission, and past initiatives is more valuable of a place to start.

The following steps are the basics for an effective comparison for past to current performance:
• Identify where you want to be in terms of performance or on specific organizational aspects such as satisfaction, engagement, or culture items.• Determine the best way to assess the organizational aspects of interest (e.g., metrics, surveys, focus groups, etc.)

• Collect data

• Create targeted questions to ask once the data are collected. These questions should be based on targeted initiatives within the past year or two that were
meant to impact the organization. For example, if Collaboration was rated low on an organizational survey last year and then you put in place initiatives to increase a perception of Collaboration in the organization, you would expect to see those numbers increase the following year.

• Action Planning. Action planning is one of the most important, yet most neglected, steps in this process. To continue with the previous example, if your collaboration numbers only moved up slightly, what should you do? Even though you have increased perceptions in the past year, you are still far away from your goal of having a truly collaborative culture.

Summary
At the end of the day the right question to be asking is, “Where are we compared to where we want to be?” or “What do I need from my people and how are they performing compared to how we want them to be performing?” The most important part of this process is the action planning that comes after you ask these questions and get answers.

If you are going to compare yourself to others, you should be asking, “How does my company compete in the marketplace?” and then “How can we align our human capital to drive that strategy?” Answering these questions may involve taking a deeper dive to understand how to compare yourself to your competitors and show your value and understand how you can show competitive advantage to your customers that make your organization or product more favorable. This requires understanding your competitors’ products and how they compare to your products and services, and not necessarily the results of their organizational survey on their safety-focused environment.

Investing in the proper types of analyses will ultimately save you time and effort over relying on bad or misleading information. If cost is still a concern, there are a lot of low-cost initiatives (e.g., employee recognition, wellness initiatives, etc.) you can implement to improve your organization and move the needle toward ideal.