Web Analytics in China

Web Analytics and the Scientific Method

I am currently reading Robert Pirsig's book Zen and the Art of Motorcycle Maintenance. And it's description of Scientific Method made me think that practitioners of Web Analytics really are scientist is a way that the good ones follow the scientific method closely.
The book describes 6 steps that are followed to find "truth" (some use 4 or 8 steps, but the underlying process remains the same)

1. Statement of the problem (scientific question)
2. Hypothesis of the cause of problem
3. Experiments designed to test the hypothesis
4. Predicted results of experiments
5. Observed results of experiments
6. Conclusion from the results of experiments

The key insight from using this method is that each step is critical and build on the concise execution of the previous steps. Any mistake or oversight along the way will result in a wrong conclusion. What is interesting is that I often find the first step most difficult. In order to state a problem effectively we need to be able to answer to questions I have posted before in a previous attempt to create a more formal methodology:

1. What is the current performance?
2. Is it good or bad?

Only if we answered question 2 well, we will be able to formulate an effective problems statement. That might sound easy at first, the devil is in the detail.

If a man does not know to what port he is sailing, no wind is favourable (Seneaca)

In a previous post I expanded on my methodology to reach actionable insights through web analytics by answering the 4 questions:

1. What is the current performance?
2. Is it good or bad?
3. What are the root cause and key drivers that lead to the current performance?
4. Which steps can be taken to improve the current performance?

While discussing the importance of context to answer question 2, I did not touch one key point:

• Did we reach our target?

Setting and measuring on targets is critical to all analytics efforts since, as Seneca said in my post title, "If a man does not know to what port he is sailing, no wind is favorable". Yet often targets for web analytics remain elusive.

Online marketing and campaign objectives are often vague and unspecific. Take the objective of this blog for example: "Provide relevant web analytics insights and best practices to local practitioners and drive the adoption of an analytic mindset for online marketing in China". All the detailed numbers Google Analytics and Feedburner can provide cannot tell me if I reached this objective.

To make the objective measurable, we need to break it down into measurable Key performance indicators (KPI) and to measure success we need to set target values for these KPI's. If that sounds more like business school gibberish than down to earth web analytics (?) remember the old adage "You get what you measure". So lets break the objective down into pieces to see which KPI's we can identify.

relevant content:Relevance is always in the eye of the beholder, but I can track the number visitors who identify this blog as relevant by tracking their loyalty e.g. repeat visits and subscription to the RSS feeds.
local practitioners: Since this blog is targeted at Chinese (what a shame I can't write Chinese), the KPI should be Visits from China / Visits.
Adoption of analytic mindset: While I can't measure what is on peoples mind (though not for the lack of trying), I can measure actions. In order to have an impact on the ecosystem, I need to reach a large number of people and engage them in a discussion about analytics. Both are measurable. For reach I will track Unique visitors. And for engagement tracking comments, trackbacks & emails received is relevant.

Now that we have a spanking new set of KPI's, we need to take the next step and set targets for each KPI.  Many web campaign and website owners do have a set of KPI's (not necessarily the right one), but don't have a target. The analytics team then end up in a desperate search for context and comparables to tell them is they are doing well. There is no context like a wisely set target, since the target setting process should take all available context into account.

For my little red blog I set the targets for October like this

Reach: Unique Visitors: 200
Loyalty:  Repeat Visits + Feed subscriptions: 100
Engagement: Comments + Trackbacks + emails received: 15
Visits from China: 70%

Since I am new at blogging, I am not sure how realistic these goals are. But I will certainly report my progress. If you have any thought, comments or recommendations. Please share them with me and my readers in the comments.