I often come across references in articles to Eric Enge’s 2007 Web Analytics Shootout. I actually had the pleasant experience of working with Eric on behalf of ClickTracks while he was researching this study. I think the study points out some extremely useful information. The three most important findings are:

1. By far, the biggest source of error in analytics is implementation error (which is why we offer the analytics audit).
2. There are many things that influence the accuracy of the data (not the least of which is the placement of the java script on the page).
3. But the most important, I think, is that it is a waste of time to compare single data points from one analytics package to another. And, trying to choose which one is accurate and which one is inaccurate. There are so many reasons why the numbers are different which is the overall point of the survey.

This is especially true because so many aspects of analytics are configurable either in the java script that is generated and/or in the configuration or profile settings. You can create your own version of ‘accurate’ based on session lengths, visitors who are included in the stats, and visitors who are filtered out.

Countless times when customers were comparing the numbers of an older version of analytic software that did not filter out bot and spider traffic, they questioned the accuracy of ClickTracks. I would then change the settings to not filter out the bot and spider traffic and the numbers would line up. Just because data can make you feel good, does not mean that it should.

The key point if you do have two sets of analytic data to report is to ensure that the patterns and trends are in line over time and by segment. If you see a spike in traffic in December, it doesn’t matter that much whether ClickTracks says by 2,000 and WebTrends says 2,500, as long as the trends are in line.

Unless you have them configured to match each other, the numbers won’t match. As long as you can see the percentage of increased matches, that is what is important.

Filed under: Analytics on Tuesday, June 3rd, 2008 by Joy Brazelle | 1 Comment
You can follow any responses to this entry through the RSS 2.0 You can leave a response, or trackback from your own site.