Saturday, April 30, 2011

User Experience vs. Accuracy

We have this little note on statistics pages on AdDuplex:

* - in order to maximize user experience we are tracking clicks on marketplace ads only when user comes back to the app where the ad was clicked. This results in some clicks not being counted.

I get emails regarding this line periodically. So I decided to address it in this blog post in order to explain the decision in greater detail and hopefully get a valuable feedback from you.

Technical Difficulties

As you probably know, there’s no multitasking in current version of Windows Phone. So as soon as you open a marketplace app via MarketplaceDetailTask the control is transferred to the MP app and the call to register the click on your server gets killed.

There are 2 general ways to overcome this:

  1. Route all the taps through your web server.
  2. Delay opening the marketplace app until the click is registered with you server.

We’ve dismissed both of these approaches because of the effect they have on user experience and based on the understanding of what is the end goal for our clients.

The User is More Important than the Stat

We believe that the most important asset for our users (apps on the AdDuplex network) is a new customer they can get through their ads. Tracking that user is important but secondary.

This is why when user taps on an ad we save this event to Isolated Storage and take them to the Marketplace immediately. When (if) the user comes back to the app where he/she clicked the ad we check if there’s a saved click and report it to our server. Thanks to RenĂ© Schulte for helping me crystalize this concept.

Obviously accuracy suffers with this approach. The user is not guaranteed to ever return to the same app. So some clicks stay unregistered. However she gets to your app in the fastest possible way, doesn’t get a chance to hit some button and drift away in the process, doesn’t get frustrated with waiting and no additional points of failure (web server) are introduced.

Believe me, I’m the last person who wants to artificially decrease CTR on the network. But since I’m a developer and not a marketing guy, I want the experience be the best. Even at the expense of some marketing (bragging) points.

So, How Much Do We Loose?

I’ve been running a campaign with 2 identical ads for some time now. One of them goes directly to the Marketplace and the other one goes through the web. The first one has a CTR of 0.97% and the second one – 1.22%. So based on this experiment about 20% of the Marketplace clicks are not registered.

I plan to continue this experiment for a little longer to get this number based on a larger data set and then I’ll probably add something like “estimated real CTR” to the statistics pages.

Conclusion

We, as technical people, often pay more attention to numbers and technical details rather than what’s really important. I think it’s really important to concentrate on our end goals. I’m sure that the end goal of all the participating AdDuplex apps and advertisers is to get more users.

Sure, if you work in a marketing department of a bigger company, your success metric could be a cost-per-click you got from the ad campaign. You can estimate this using the 20% number above. Still the end result of your campaign are the new customers for your app and I strongly believe you can get more of them by sacrificing a little accuracy in the statistics.

Do you agree? Do you have and idea on how to resolve this dilemma on both counts? Let me know in the comments.

2 comments:

  1. This seems quite reasonable to me. I am currently using a third party ad provider on WP7 but plan on abandoning this in favour of Ad-Duplex as soon as I get a chance. (In terms of VFM Ad-Duplex looks like it will win hands down).

    ReplyDelete
  2. Thanks. I'm sure you will be pleased with your results.

    ReplyDelete