Commentary
Web Analytics: Easy? Hard? Complex? It Depends
- by Judah Phillips , September 8, 2009
Yourtools. While Web analytics tools all share common functions, like counting, at a minimum, views, visits, referrers, time-based metrics, and allowing you to view the data across various time ranges, it's clear that not all analytics tools were created equal. Some tools allow for advanced drill-down and slicing and dicing of data from directly within the interface with little to no backend customization or advanced tagging required to do so. Other tools are simple static dumps of canned reports that have little to no ability to become dynamic or enable interactivity with the tool. They just report pre-programmed numbers. Some tools allow for customization of data directly in the tool, while others require changes to tags. Still other tools collect data differently - some sniff packets, some use javascript tags, others use log files, while others allow you write directly to a database and expose the data in reporting.
As you might imagine, the functionality afforded in your tool will define your judgment of Web analytics difficulty. When you have the ability to collect data the way you want and report data the way you want, Web analytics becomes easier. When you can't, it's hard. When you want to integrate data from internal systems, third parties, or qualitative studies, it can get complex.
Yoursite. Some sites are harder to instrument and more complex to analyze. Sites within the same company can vary in their user experience, backend architecture, information architecture, functionality, and goals. Sites, such as craigslist, still focus around the paradigm of a page view, whereas other sites have moved to rich internet applications (RIA) and AJAX functionality where the page view is largely irrelevant. Some sites make use of widgets, video, and provide mobile-only functionality. Certain more "rich" and "web 2.0" sites make data collection and tagging implementation and instrumentation hard and complex, while more static page-view driven sites are easier to instrument.
Your company's requirements. How your company decides to use Web analytics and their goals for the data impact an analyst's assessment of whether doing analytics is hard. All companies have different analytics requirements. Some companies regulate analytics to the discipline of marketing and emphasize the importance of using the data for campaign tracking and optimization, reach, frequency, attribution, and landing page optimization. Other companies seat analytics in product development and emphasize the data be used for functionality assessment, error detection, and overall usability analysis. Still other companies use analytics for ecommerce, and look at shopping cart performance, funnel analysis, and product performance. Other companies want to understand how specific visitors behave on the site and identify their propensity to convert to leads. Analytics can even be nested in a research group. Some larger companies have all these requirements and more.
Your team's skill sets. Not all teams were created equal. Some companies choose to homegrow Web analysts and give them training. Other companies choose to homegrow and say RTFM. Certain companies hire skilled analysts with years of experience at other companies. People with backgrounds in database marketing, business intelligence, financial analysis, business analysis, systems analysis, or even no analysis (entry-level) can be on an analytics team. Some analytical teams require only skills with Web analytics tools. Other teams require people to have skills with Customer Experience and Voice-of-Customer data. While certain teams need people who can create custom, proprietary tools around web data residing in internal databases. The often varied and sometimes motley skill sets of web analysts dictate the how difficult or not it is to do the job required by the company.
Yourprocesses. Successful Web analytics teams have defined processes for performing critical activities, such as collecting data, requesting reporting and analysis, and investigating data anomalies and discrepancies. Regardless of process, there is always a certain amount of chaos that revolves around Web analytics: the site is always changing; the reporting requirements can constantly shift; yesterday's site goals have largely become irrelevant; your tags have been pulled off the pages. By placing process around the chaos, the job of a Web analyst can become easier.
At the end of the day, is Web analytics easy, hard, or complex? I would argue that it depends on a combination of the factors I've presented above. There is no simple one-size-fits-all answer. So what do you think, good reader?
Great post! In my experience, capturing all this web analytics data can be overwhelming and actually lead to frustration. First, some tools are more difficult to navigate than others, 2nd, there is always some kind of obstacle like the site constantly changing or something not properly being tagged right.
However, I think the worst problem is if there is no one on the client side who is there to support web analytics efforts. Don't get me wrong, there is tons of useful information to pull from these tools. But after hours, days or even weeks pulling reports, how is knowing that visitors stemming from organic search stay 30 seconds longer on the site versus paid search visitors going to change the way you buy media or site design? How is knowing that visitors from Fargo tend to have a higher bounce rate going to change the way you design your website?
Eric
www.OnlineMediaAnalyst.com
Good reading! I've been approached by a company called Addison Klein and they are very big into web metrics and internet positioning in Europe. Anybody know anything about this company and how they operate?
Thanks
Excellent post. Along with the right tools and direction, it takes a person who loves data. You can have an ok tool and still get great results. Or, you can have a great tool and get mediocre results depending upon who's piloting the ship. A lot of people assume that once you tag your site, all of the answers are just going to jump off the page. Unfortunately, we all know that just isn't going to happen.
In my experience with publishers a big analytic challenge is marrying together in a cohesive and apples-to-apples way data from several disparate sources. Usually it's necessary to combine data from the website analytics package, the ecommerce and/or fulfillment system, the email vendor system, internal financial reports and the search advertising tool to get a complete conversion funnel analysis. The data never quite synching up can drive you crazy!
Judah - excellent points. Definitely depends. Responded here (http://tr.im/ycfm) that things can be made easier if time is taken to ask good questions to gain context about what matters and what people will take action on.
Eric - I hear your frustration. Saw your comment before I responded and added to the post here http://tr.im/ycfm. In a nutshell, I suggest asking questions about what matters to the client, framing more relevant questions for analysis, before diving into the data. Easier said than done, but it gets easier each time.
Good post Judah. Just one issue that niggles at me.
You refer to this as "web analytics". I also know that this is what the common parlance is, but truth be known it is actually "website analytics". "web" is a truncation of "world wide web" which is the aggregation of billions of websites. These tools do not analyse the "web", but merely individual nominated "websites" that collectively make up the "web". I know this is semantics ... but we as an industry should get it right.
Great post Judah, in fact, you identify 5 of the 6 critical success factors I have identified in my research on the Web Analytics Maturity Model. The 6th one, the most important, being "Management, Governance and Adoption".
My take on the hard vs easy debate: it's not any harder than other major initiatives undertaken by organizations...
Hopefully, my MBA research on the Web Analytics Maturity Model will help make it easier! Please take a look at http://immeria.net/wamm for a series of blog posts on the topic. Of course, I would love to get your own and your readers feedback.
Stéphane http://immeria.net
Judah,
It's a great list! As you say in your final lines, it's likely interrelationship among the capabilities / factors you cite that matters most; they're all only as strong as the weakest among them.
I'm also curious about the order in which you presented them -- starting with tools instead of with requirements and then perhaps data, followed by people-tools-process. Any particular logic?
Cesar
John Grono raises an interesting point in claiming that it should be "website analytics" rather than "web analytics." I agree...except that was a good point 3 or 4 years ago. Another factor that makes "web analytics" harder/more complex is when the web analyst is expected to incorporate off-site activity/data, be it off-site/offline or off-site/online. If anything, more and more companies are looking to do "web analytics" with John's *proposed* definition of the term.
Hi Tim.
I agree that more and more companies are trying to do "web analytics" as extensions of thei existing "website analytics". That is the very reason for me raising the distiction. The problem is that "website analytics" is a very server-side metric. There are major issues about aggregating sufficient server-side traffic that produce unbiased "web analytics". Sure these systems produce data per se, but the quality, accuracy and representativeness of that data needs to be deeply and thoroughly tested before use.
From both a researcher and a buyers perspective, we're really after "audience" data - that is we're after "people data" and not "server data". Cookies are not a robust proxy for people for any time period of more than a day - and the longer the time period the worse that proxy becomes.
I suppose it all depends on whether you want to measure the people on the computers who use the web, or the computers themselves. I know which is easier and which is attracting companies like moths to a flame. I also know what media agencies need in order to create communications campaigns. Caveat emptor.