This post started life as a response the TSIA forums (registration required, free).
Setting up a social media support forum and trying to measure its effectiveness is an interesting challenge. Expecting it to reduce the volume of incoming cases is a long term proposition that has to be managed carefully before you have any chances of success at all.
Consider that in order for the forum to become a successful support tool, you will have to create traffic, first browsers and then contributors. In the mean time it will be up to your staff to encourage the traffic, respond to questions and keep the forum alive. Therefore in the first period of its life, measuring volume of traffic, repeat visitors, time spent, number of responders from outside your organization is exactly the right thing to do. You want to make sure there is interest and value to the visitors, regardless of the reduction in case volume you experience.
Eventually, you’d want to think about measuring the impact the social media initiative has on your incoming workload – now the main effect you are hoping for is reduced number of cases. This can be manifested in either of these two ways:
- Fewer new cases
- New cases are better documented and therefore more efficiently managed and resolved
The main challenge in directly measuring the impact your social media initiative (or knowledge base, or anything else designed to reduce case load) is that you can’t measure the things that did not happen. In other words, there are multiple reasons a customer did not open a case, for example: no problem happened due to improved product quality based on support feedback, they are phasing your product out due to reduced quality and are not interested in investing time in resolving minor problems, your knowledge management initiative is highly effective and the customer found a solution there, and so on.
What you can do is deduce the effectiveness based on several other measurements, for example:
- How does your workload develop based on your revenues, if you are growing revenues and reducing workload you are likely doing something right
- Ask your customers when they open a new case whether or not they have tried to use the website / forum / whatever and failed to find a solution
- Measure the number of visitors to the site that open a new case within 24 / 48 hours after their visit
- Ask your customers how useful they think the tool is during customer survey
- Measure the value to your organization through the content and information you were able to retrieve from the forum into your own systems and processes
There is an excellent book you may want to look at, it’s called Collective Wisdom, written by Françoise Tourniaire and David Kay, they have section discussing metrics for Knowledge Management initiatives which applies equally well to an social initiative. Also, there have been several valuable discussions on John Ragsdale’s blog,Esteban Kolsky’s blog, and the eVergance blog (which seems to have been deserted).
Was this post useful for you? What’s your experiences in implementing and measuring social media initiatives in support organizations?