Performance Zone is brought to you in partnership with:

Reimagining the way work is done through big data, analytics, and event processing, Chris is the cofounder of Successful Workplace. He believes there’s no end to what we can change and improve. Chris is a marketing executive and flew for the US Navy before finding a home in technology 17 years ago. An avid outdoorsman, Chris is also passionate about technology and innovation and speaks frequently about creating great business outcomes at industry events. As well as being a contributor for The TIBCO Blog, Chris contributes to the Harvard Business Review, Venture Beat, Forbes, and the PEX Network. Christopher is a DZone MVB and is not an employee of DZone and has posted 276 posts at DZone. You can read more from them at their website. View Full User Profile

Using Benchmarking Vendors (Part 2 of 2)

04.23.2013
| 1047 views |
  • submit to reddit

This post comes from  at the Successful Workplace blog.

In part 1, of this benchmarking series, I covered a few of the instances where you might want to engage a benchmarking vendor to assist you with your projects. In this installment, I outline my recommendations around what to do when you find a vendor (or short list of vendors) that you might want to engage to help you with your project.

When you find a vendorvendor, partner, benchmarking, metrics

If you choose to work with a vendor there are a few things you should make sure you investigate in some depth about them.

  • Comparability – If you are conducting a predominantly metric benchmarking project, ask about comparability. If they tell you they have data that is truly “apples to apples”, they are lying. Dig deeper. There is no other organization just like you, so there isn’t an exact data match. All data will have some variation in it. Besides, it isn’t about finding an exact comparison, it’s about understanding the data as I discussed in this post.
  • Methodology or approach – They should have a fairly standard and robust way they execute benchmarking projects that has been tested, refined, and proven. Granted, it may need to be adjusted based on needs, but if they are inventing the approach each time they conduct a project, Consistency and validity will suffer.
  • Validation approach – This is a subset of methodology, but one that deserves some dedicated attention. If you are collecting any type of data or information, there should be validation applied to that information. We promote looking at validation in two ways.
    • Logical – A set of logical standards you should place on any data or information set. This is usually the first level of validation and includes steps like confirming the person providing the information actually make sense (e.g., you are not getting procurement data from someone in marketing). This is especially important as a first pass if you are doing an online survey. Many people will fill out a survey just to get a report.
    • Detailed – When you are conducting a highly metric benchmarking project, this may take the form of statistical validation where you look for outliers in the data. If it is more qualitative, it may include triangulating information provided with other public sources that might be available to add a layer of confirmation. It could also include follow-up questions to a specific organization to ask for examples of where they’ve actually demonstrated something they reported they do in the public domain.
  • Peer groups – Talk through how they will define relevant peer groups for your specific situation. If they say they are going to focus just on your industry peers, move on our push back. Here is more information that topic.
  • Assistance beyond just learning – You should discuss how they plan to help you develop action plans based on your benchmarking findings. If they don’t have any and leave it up to you to figure out what to do with the findings, just know what you are getting into and that you can handle the change phase of the project. If they do the actual implementation of the findings, make sure you keep an eye on the benchmarking phase. A truly disinterested benchmarking vendor doesn’t care what the findings are as long as they are the best they can find for you. A benchmarking vendor that has to implement the findings may not want to accept the challenge of getting you to the “best practice” standard uncovered during the benchmarking phase.

Just like any area, caveat emptor. Benchmarking is no different. I try to live by a rule I learned long ago. I like to work with vendors that don’t make my tummy hurt. I hope these steps help you find the benchmarking vendor that is right for you.



Published at DZone with permission of Christopher Taylor, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)