Paul Holstein Weblog at Web Analytics Demystified

Paul Holstein is Co-Founder, Vice President and COO of CableOrganizer.com, Inc., now among the world's leading purveyors of cable and wire management-related products. In these capacities, Holstein oversees the company's strategic planning and day-to-day company operations, including web analytics and multivariate testing.

Subscribe to Paul Holstein weblog

Archive for 'General Web Analytics'

« Previous Entries

Those Annoying 404 Errors

Few things drive me more nuts than broken links.  Especially when those links are on your own site.  Here you are with a real live person browsing your site.  They are engaged and click on a link to learn more or purchase an item when, anticlimactically, they get a 404 page not found error.  Another form of this problem is almost as bad.  You have a beautiful page that’s supposed to have great images on it, but some or all of them are replaced by white boxes with x’s in the corner.  Yuck.

Nothing screams “amature” more than broken links.  404 pages are some of the most destructive pages or elements you can have on a site yet they are often one of the easiest things to find and fix.

Here’s how we attack ours:

  1. First, we crawl our own site weekly.  We use a program called Web Link Validator.  In addition to crawling your site for broken links, it also looks at spelling and can create site maps for you.  There are a lot of programs just like it available on the web today.  If your site uses static html anywhere on it, I would recommend crawling your own site on a regular basis.
  2. Every time a browser encounters a broken link, I get an e-mail.  That’s right.  I had our developers program our site to send me an e-mail if a 404 error is generated from a page on our site.  This has saved our bacon several times between crawls when we didn’t realize we had a problem.  Suddenly, my e-mail box was filling with error reports.  It’s a great help.
  3. Finally, we analyze our logs.  Page tagging solutions such as Omniture will only tell you about broken links to pages.  If you want to see your broken links to images, you need to look at the log files.  This can get a bit tricky if you are using a CDN (Content Delivery Network), such as Akamai, but to combat this, the CDN will usually provide its own reports to help you.

Given the tools available, there is no need for you to ever suffer major problems with broken links.

Web Analytics - The Kaizen Way

I got an e-mail today from Sitelab listing the top 10 reasons to redesign your company website.  In reading this e-mail, I thought to myself how overwhelming this process seams.  Here you are with dozens of priorities you’re responsible for and then someone suggests a redesign.  Yuck.

The idea is that once you’ve fixed your site, everything will be OK.  But it doesn’t really work that way, does it?  A site redesign is usually a major undertaking.  Often, it is so overwhelming that it never gets done or it is not fully implemented.  Is there a better way?

You bet there is.  The Japanese call it Kaizen.  Tony Robbins calls it CANI - Continuous and Never ending Improvement.  I just call it continuous improvement.  It works like this.  You simply make improvements to your website on a continuous basis.  Nothing dramatic, really.  Just make small improvements on a daily basis.  There’s a great book out there called “Web Analytics: An Hour a Day” written by Avinash Kaushik — perhaps you’ve heard of it.  Basically, the book details small steps you can take every day to improve your website. 

Here are some of my suggestions for using analytics to make small changes to your website:

  • Look at your top keywords.  Are all of them being advertised with PPC?
  • What are your worst PPC campaigns?  Do they need negative keywords or alternate ads?
  • Are you advertising all your top pages?
  • Click map study.  Can you re-arrange your navigation to be better matched with actual clicks?
  • Study your popular pages.  Are they optimized?
  • Meta tag study.  Do all pages have proper meta tags?  Any duplicates?
  • Tagging Study.  Have all pages been properly tagged?
  • Referrers Study.  Who are our top referrers?  Can you do better?  Who are the new referrers?  Can you advertise on them?
  • Advertising Study.  How are your ads doing.  Find new places to advertise.
  • Product feed study
  • Best performing vs. worst performing pages.  Both bounce rate and exits on path.
  • Shopping cart analysis.  Fallout.
  • Affiliate study
  • 404 errors review

What are some of the small things you do for your website?  I’d love to hear your suggestions.

Honored to be Blogging

Hello all, I’m the new blogger in town.  My name is Paul Holstein and I’m one of the founders of CableOrganizer.com.  One of our former employees, Dan Shields, was writing for this blog, but he’s moved on to his own practice.  Eric Peterson has been kind enough to extend his invitation to write for this site and I was honored to accept the invitation.

You can expect me to write about analytics and other subjects from a retailer’s perspective.  Our company has been on the IR 500 list for two years running at around position #441.  I’m very interested in improving our conversion rate, multivariate testing, usability testing and PPC among other subjects.  I hope you enjoy the posts and if you have anything you’d like to know about from my perspective, please feel free to reach out and let me know.

Thinking Outside of Out of the Box in Commercial Web Analytics

Settling for Packaged Metrics and Reports Can Be a Dangerous Game

If you were a kid once, and I suppose we all were, you may have learned that no single manufacturer of bikes got every part right. So, in order to improve the quality of the finished unit, you have to purchase and install superior components which lighten the bike, make it operate more efficiently, offer comfort or convenience to the skillful rider. In the end, you might have upgrades and customization over 30-35% of your bike. The same idea applies across the board for just about everything.

So, why then, settle for the factory state of your web analytics tool?

Looking around for people to discuss many of their calculated metrics can be difficult. Joseph and Eric and their ongoing banter on “Engagement” certainly merits its share of attention. This was the primary topic recently at the eMetrics Marketing Optimization Summit in San Francisco and a tremendous panel discussion between the aforementioned as well as Gary Angel from Semphonic and moderated by Robbin Steif from Lunametrics. Aside from the ubiquitous topic of Engagement, there is so little available on what to do to hammer stats together for better understanding. People seem to guard their special configurations very closely. A few resources, however, are making their way into the world.

Dustin Wallace is at least one resource whom appears to want to share. Dustin is, according to what information is available on his posts, a relatively new blogger working in analytics at Sun Microsystems. 2 of his 5 posts available discuss manipulating packaged metrics for a better understanding of performance indications. In these, he discusses a formula to help draw comparisons over particular time frames and goes in closer on Bounce Rate (he breaks these up into ‘Exit Rate’, ‘Soft Bounce’ and ‘Hard Bounce’ and lays out there craft and execution in Omniture SiteCatalyst).

Another guy, Vijay Bathula, is doing analytics for Hewlett-Packard, publishing a blog discussing ‘advanced web metrics‘ and trying to develop some interesting concepts. A particularly attractive metric which Vijay brings to light is what he calls ‘Time2Click’. He states:

Time2Click is the metric that tells the average amount of time that was taken in order to click a link on a web page. This metric helps web masters and marketers on how much time at an average, visitors are taking for making a decision to click a link or button on the web page. This metric is simple to calculate and great use to optimize the call-to-action buttons, positioning, anchor text in the links and much more without using any other expensive Split-Testing or multivariate testing tools.

This promotes a clever angle for thinking about testing and a great sense of what I’m trying to relay by posting here. Formal metrics a great for the purpose of executive reporting, however, when being used by the analyst, especially one involved in heavily process-oriented practice, thinking outside of the box means going beyond what comes in the box.

An Invitation to a Discussion and Pending Project to Build Collaborative Set of eCommerce Calculated Metrics

My work recently, and in the context of this highly charged discussion of engagement, has focused on trying to build a series of powerful calculated metrics to try to get a better understanding of how people are interacting with eCommerce websites. It has brought me to the point where I’m willing to assert that, with regard to online retailers, a series of operations and statistics can be gathered and placed into major commercial reporting interfaces to ad value to the suite as well as promote a more complete functional model of interaction.

Among these I have decided to research, study, and pursue defining the following:

  • Appraisal - This is the behavior of seeking information on product queue for a potential purchase. These are broad strokes in navigation based on general term and phrase usage, non-transactional focus and
  • Acceleration - The point where the brain begins to move from a general information processing state to a more focused channel. During acceleration, a subject should (ideally) only move toward action and curtail further lateral navigation.
  • Impulse - Having collected and been presented with one or several points where a call-to-action or option to execute an objective occurs, the confidence and slightly adrenal motivation carries a subject through an action and transition to a state of risk assessment.
  • Commitment - Acting on the information and the excited state of having executed a checkpoint in a system of obligation; this measurement should seek to imply reduced regressive states when continuing through to additional actionable areas and streamlined return to transactional navigation due to acquired trust and familiarity.
  • Conviction - The completion of the desired final action. Conviction should be measured by the degree to which a subject does or does not participate in building trust by gathering information as to policies, security, examining financial options, and other potentially pertinent information associated with going forward with a purchase.
  • Affirmation - Presumed to be existing in both a natural and provoked state, affirmation is post-transactional and is the return to the site with an abbreviated or non-existent appraisal process. The degree to which the initial experience was positive and powerful should, hypothetically shorten time to accelerate and kindle the impulse state.

I have spent a fair amount of time looking into how these particular points work into the process of making a decision to invest money in return for a product with a perceived value. My resources in being able to do this, at this time, are limited. For that reason, I would love to hear from anyone who is willing to allow me or our team to look at site metrics and frequencies to identify these points in their business model and help build support for a larger, more universal application of these proposed transcendant metrics.

Anyone who is interested, please comment or write to me directly using the contact information provided. Any company who submits for the ability to aggregate and scruitinize data will receive a complementary copy of the publication as well as promise that any sensitive data used in the studies will be protected by the appropriate legal instrument.

Web Analytics Should Drive Research to Foster IP Innovation

Mature Process Reporting Should Become Seedbed for Discoveries, Patents, and Broad Practice Foundations

With constant data streams passing over the minds of analysts everyday, it seems higher than likely that, over time, the team which deals with this data should start to construct ideas and applications built on consistent experiences. The amount of information which is aggregated about visitor and customer information over time gives a specific picture about who needs what with regard to navigation, structure, and assistance within the website. That information, paired with analysis, should produce a list of items for IT and stakeholders to brainstorm on solutions and short-cuts to help users achieve navigational nirvana.

The idea of the web analytics process is the continued production, analysis, and decision support for the business in which it has been installed; as well as the obligatory feedback from the action which takes root within the agency. Therefore the establishment of process is, at least in my eyes, a goal for online businesses with more recent adoption. In the case of mature and refined analytics process practitioners, this should be a means to success of a single entity, and feed into the larger sphere of markets, industries, and eventually, universally adapted technologies which could feed macro-improvement of the user experience globally.

Look Past Process: Visitors Deserve Your Attention and Devotion

We are all users who are visitors. As a collective group, we can attest to the number of half-baked ideas which are thrown on the net for one reason or another. Many of us who matured working with computers have a few we hide away from the world. Hell, I have ten of my own. We all find sites which we thought were going to meet our expectations and fail miserably. Our arrival on pages often misses the original intent of the search or slightly misdirects the concept. We blame the search results, but, broadly, this is the output from lousy understanding of SEO taxonomy (i.e. Silos) and/or usability issues.

With usability testing, search marketing and SEO information as widely available as they are, the only explanations for a lack of progress in these subjects are:

  1. Finances: Marketing budgets fail to consider the possible positive primary outcomes of efforts to improve these aspects of a site, much less the periphery value of the research contributions and application value abroad
  2. Hesitation: Generally, businesses in practice tend to be ‘bearish’ when it comes to expenditure on technology which discusses returns. Intent might be pure, but the risk clearly does not merit the restraint.
  3. Ego: In contrast to the previous item, trendy businesses based on whimsy and willingness to engage in risk view using data as somehow ‘cheating’ on creativity or lacking in true inspiration. Therefore, they overlook or suppress buy-in through a flurry of logical fallacies.
  4. Perceived Complexity: To some, creating an in-house usability program, building a lab, standardizing reporting and inciting discussions on stats and quantified experience data is an ENORMOUS undertaking. It becomes a sociological hurdle for the company and eventually, with enough steam, a self-fulfilling prophecy.

With little or no participation in gathering an understanding of how YOUR site visitors respond to content, you, in essence, neglect their voice. By neglecting them, you inhibit your growth and deprive a community of people dedicated to evolving our world wide web experience of data which can contribute to greater understanding. Gaining this perspective is the key to unlocking the true potential of a process-driven analytics practice and a wildly valuable means to building the new architecture of the world wide web.

A Lofty Prediction with Exponential Considerations

If I had to place a bet on the future of web analytics, it would be on services and solutions. Commercial vendors like Omniture and Coremetrics (and Google Analytics when/if that time comes) will provide a series of broad tools, which, at least Omniture, has started doing. As the analysis and application of insight suggested actions cascades down from those platforms, industries, and markets will eventually respond with vertical groups and practices zones. Some will split specifically into advertising, others into expertise on eCommerce, still others into Social Media or Branding or what have you.

Over time, probably a couple years before enough talented analysts can blossom and contribute, markets and geographies will develop their own identities where conversations and comparing notes will likely drive highly-predictive analytics and scaled systems for buy-in by the most frugal small/medium business ventures. Shared resources will again be the natural progression (outsourcing is a natural evolution of industry). Small but incredibly effective micro-solutions based on research compiled by ambitious teams driven by numbers will dominate each niche feeding more information back into the collective process.

Introducing: The Worst-Case Scenario Recession Web Analytics Survival Handbook

The idea of preparedness for unexpected events or peril is nothing new. The anxiety which mounts upon each published story seem to falsely disseminate that the corrections in commerce markets are a) unprecedented and b) inextricably linked to a slippery slope of economics which will plunge our company’s infrastructure and personal lifestyles into the pits of a depression.

The ‘Perfect Storm’ which COULD create widespread economic issues is not the natural and cyclical corrections taking place in the global or capital economies, but the poor approach to reacting to this new by forcing bad decisions. In order to avoid bad decisions as an ‘economy’ we, as analysts and gilded protectorate of the data, need to empower our companies with good information. We need to understand the business as well as the business we are in. Having a grasp on this will help us navigate the choppy water of volatility.

BACK TO BUSINESS WEB ANALYTICS

If there is a time to put being an evangelist aside, it is now. This is about business. Business is, essentially, a series of decisions being made by different people and teams to produce optimal return on finances invested. Continuously improving business means becoming more efficient in action and more adept at making decisions. Making better decisions means having good, current data which can be correlated to the way in which a business makes decisions. Good data comes from a diligent analyst plucking the groupings out of raw web analytics output:

  • Filtering out domains, IPs, internal users, bots, script-executing robots and anything else that will give a skewed picture of who comprises your audience.
    • Both Google Analytics and Omniture SiteCatalyst (and I’m sure the same is true of Coremetrics, ClickTracks etc.) provide a means to do this. While SiteCatalyst only boxes 5 filters, you can add many more with a Vista Rule ($)
    • Jot this down as a task that requires regular management. Simply doing this once is equivalent to taking inventory once.

When an analyst goes to a meeting to provide information to the people who need it, it should be without reservation. Confidence in the data which you are presenting or using to support avenues for business should have all the questioning done in the locker room and not on the field. Paying close attention to your data and watching out for invasive issues is key to building good decisions.

  • Correlating trends in the marketplace which impact your business model as they relates to your web analytics data
    • Learn about your customers, your vendors, and your financiers and see how they are impacted by market forces.
    • Get competitive intelligence on the checklist to see how the news is impacting the market and where opportunities exist. Big economic consequences tend to create consolidation at the top, and liquidation at the bottom.
    • Get to know your Geo-locations. Big cities produce a great deal of information about topics, but their volatility depends on their ‘perceived’ collective investment in the actions creating market fluctuation
      • example: Cities where housing markets are suffering might be more aware of the local impact of the lull. The result might be that individuals whom have been exposed to larger concentrations of for sale signs might not be as willing to buy the ‘handy-dandy three speed deluxe vacuum’ of the week.
    • Build presence in areas where its necessary, and expand presence in areas which have economically insulated populations. By the time the numbers stabilize, the brand identity value should be starting to trickle down.
  • Draft Momentum from Strong Brand Manufacturers or Business Affiliates
    • Whenever possible, spelunk the data on the websites of companies who provide your products, inventory, services or solutions.
    • Speak to your partners and affiliates and try to gauge when big media events are coming up.
    • Cross-pollinate their PR with your own to help feed each other energy around a product or idea.
    • Trade some venues with them. Inevitably people who are landing on your site might be trying to communicate with the manufacturer in circumstances, and vice versa. In all likelihood, you should be able to give each other ideas without cannibalizing the market. If anything, you both might benefit from knowing the mindset of the user.
  • Understand Web Analytics as a Function of Marketing; As a Function of Business
    • Take the time to understand how your analytics solution measures all marketing venues (referring domains, campaigns, SEM etc.) and know how the solution is attributing the success.
      • These methods might include tricky subjects like First, Recent, Linear, or Participation attributions, or, worse: acscription which essentially provides little or no scientific basis for the explained success
    • Every dollar which is spent through a marketing budget, either directly or indirectly, should have a highly scrutinized figure of return attached to it.
    • Know where your money is going and how much is coming back.
    • Help managers and executives make good decisions on where and how to spend money to:
      • Increase likelihood of success
      • Trim costs on resources
      • Quantify return on investment
  • Measure and understand the “other” web and business analytics - take the time to think through ways to get more data. Sometimes you have to get creative.
    • If you plan on using print media and have not yet done so, discuss test markets with publishers. You can spend 10% of the costs of a broad publication to test reactions in small markets. This requires benchmarking areas prior to injecting media and measuring the blip and tail in that market.
    • Independently measure personalization applications. Taking the word of the provider should never be enough. The fact that you do this might even win you points with the solution vendor. Ever since we really pressed SiteBrand, they’ve been extremely active and interested in the developments which we have with CableOrganizer.
    • Get more information about your customers through what happens when they pick up the phone. Our research company has developed a way to measure phone call web metrics using something we call Session ForeSite™. We liked it so much we filed a patent application and developed a brand around it.

While the graphic at the top of this post is a joke, on some level, I’m seriously considering putting together some sort of pamphlet or compendium on this topic for upcoming events. If anyone is interested in contributing, I’d be interested in hearing and discussing some of what experienced professionals have found is working, or has worked in the past, with regard to offsetting damaged markets.

Please feel free to comment or discuss any point here and continue to check back for updates to this particular post in case revisions are made to provide more useful information. I personally view this as a ‘living post’ instead of a regular article. Also, no information will ever be provided in any work from me on how to land a plane…just so you know…;)

Analytics Optimization - Sowing Seeds for Process, Buy-In, and Participation

This post is something akin to a rant. It is about the web analytics process and the ability to make it more functionally efficient. Its something we can all relate to:

It crossed my mind recently that there is so many facets of the analytics programs at maturity that there must be some means to internally optimize the process. I’m not talking about software or the many hundreds of small start-up companies or analysts throwing their hats in the ring, or any feature which they can add to our decision machines. I’m talking about grassroots analysis to grow business success through decision empowerment and process efficiency.

Quite simply, the steps to getting a data process start with the willingness to accept certain truths about the processes:

Knowing the objective of a website, and how that goal ties into the business model is essential in understanding where to plant an water web analytics. The connection between these can be elusive. Often, this is an aspect which comes to late in implementation or is overshadowed by a perceived need to begin producing reports. Unfortunately, this is good for the ego of the champion for analytics, but a detriment to properly installing functional analysis at the core of the human resource ecosystem.

Fueling a websites design and development machine requires making powerful assertions which can win support alone, or banish dissent with pure statistical science. Coming up with ideas is easy. Coming up with great ideas can be challenging. Coming up with great ideas and getting administration, colleagues, and the nay-sayer to go along with it can be impossible. There is a reason for this. It has to do with appeals. Appeals are the way in which information is communicated to reach the part of the receivers brain which helps them build a concept for themselves and the group. Everyone has a way they WANT to hear or see your information presented and how it affects their world. In that respect, you can either be a communications genius, or, build the case for math…which appeals to everyone. (I will write an entire post on appeals and how to use them some other time. This is a fascinating area which I think has very high relevance to analytics) Math is the key to making the process sprout. It is the foundation, the proof, and the method of delivery.

If making math work for the business routine of web analytics is the goal, we should break down the subroutine. This seems to be the main point of struggle for recent practice adopters and parties interested in participating in analytics. Each component is arguably a necessary step to producing useful reports which detail insights and suggest action, based on math. Here is HOW, piece by piece, you create action from analytics.

Graphic Representation of Correllation to Insight Accelleration Distribution Just looking at reports day to day, any analyst should start to recognize patterns. The human mind is amazing with regard to this. We see lines and columns with comparable trends and past performance which either fit or do not fit. Here, you begin to ask questions. Remember this part, because this is the first major function of any analytical process. You find things that intrigue you, and seek to explain them by using the data.

With continued observation and investigation, you might begin to see correlations. Some might be as simple as every weekday visits decrease consistently by 60% between 12-2pm EST. You might be in the business of selling medical supplies. Where you might already know it is a lunch related subsiding, why does this happen? Why is it that consistent? What do these people DO during that TIME which prevents them from visiting and buying between 12-2pm?

If you’ve ever worked with a pharmacist or in a doctor’s office, or a hospital, you’ve likely experienced a fairly standard phenomena of their lunch habits. For 2 hours everyday, administrative work in the medical world is reduced to a skeleton crew. This impacts the amount of work which can be done outside of necessities, which, sadly, does not include purchasing catheters or bidets from BioRelief.com. What this does, though, is provides a certain correlation for, or explanation to the dip.

High value correlations should be based on the number of times when the desired action occurs, the consistent recurrence of that action, and how many different places this same action can be observed. These collected actions are not, in themselves, the goal, but rather, checkpoints which may or may not positively indicate some motive to achieve, or explain diversion from, the goal. The correlations give us an idea of how to cater our information to and find more avenues to improving the experience for the user.

Correlations lead to understanding and understanding should lead to hypothetical assertions. The place for making bold statements is in this stage. Take the pieces of correlated information yielded from curiosities and chasing rabbits and start to use them in building profiles or saying things like, to use the example above: “Shutting down our Search Marketing campaigns between 12-2 pm daily will save a total of $95 in fruitless clicks per day, which, when calculated is an estimated $25,000 in savings per year.

Assertions should be elaborated on; then tested for validation and range. The world of online testing does not exist to give us something to talk about. The discussion is, instead, the result of seeing value in testing. A/B testing, using a control group versus variables, can be useful in all aspects of campaign or site optimization. Multivariate testing is an extremely efficient component for providing statistical validation of elements, as well as uncovering additional unintended correlations for further discussion. (Element design, additional perspectives, and other assertions from the design and development owners should help provide resources and add value to multivariate testing). Usability testing can place a single session under such scrutiny as to not only validate an assertion, but provide an immediate solution. (and, by the way, another great way to get people involved in the process).

Test results need to be scrutinized by analysts AND potential dissenters prior to reporting. Have colleagues and co-workers pick apart the outcomes. Get more input. This has the amazing effect of not only involving common adversaries in the project, but giving them an sense of where you are coming from. It helps them understand you. In doing to, you break down uncertainty and increase communication. (also a subject which I discussed at this venue). The business will begin to build around research based decisions.

By the end of this round of web analysis, you should have answers to report to the group, participation from people whom have been involved in the process. and the data and statistics to back up your position on any arguments you encounter. This will win favor, promote action, place value or at least validity behind statements, and, hopefully, spawn more questions, deeper analysis and better understanding.

The Web Analyst Case for Acceleration to Experimentation

Early in my training as a ‘web analyst’, Paul (my boss and the COO from CableOrganizer.com) set out some goals for me which included checkpoints to my actually being considered ‘mature’ in my designation. Those steps included tracking and reporting KPIs and insights on a weekly basis, regularly scheduled usability tests, and the successful completion of a multivariate test. I set my sights on those goals while Paul prepared the road for installing a data decision process in the business.

The purpose of this post is to explain the progression from carbon-based report handler to full scale and skill realized web analyst. It will attempt to point to areas which may help new and developing analysts gain momentum in the perfecting of their craft. Lastly, it will outline a few quick experiments which might help prime the process engine. (If you’re looking for ways to improve your value, I’m handing you the recipe)

For the purpose of support, the testing I will be describing includes:

  • A/B or Split Testing Using Google Website Optimizer - testing single variant elements per URL which traffic is directed into by a script. Goal success is then attributed to the page.
  • Full Factorial Multivariate Testing with GWO - testing multiple elements per single URL, simultaneously, where the items tested are identified within the page and rendered randomly. Goal success is distributed across the table of elements per variation and aggregated statistically to determine relevance of each element to success and compared to all variations and the original.
  • Usability Testing -qualitative test to gain insight into a user session experience on a given site where a professional, random, or representative subject is observed performing tasks and navigating a web site. These can be performed by usability experts in labs, by simple observation and analysis, or through a handful of testing services whom use and relay audio/video files back to the customer.

These experimentation methods, in their simplest form, provide an analyst with a set of tools to validate their assertions. The outcome of GWO tests which are set up correctly and run to completion provide invaluable statistical justification for keeping or replacing elements within a website. Usability testing these same areas and elements should augment the data by providing a qualitative perspective to the findings.
Making experimentation the goal forces developing creative hypotheses. Looking back this seems to be the most essential right of passage into the world of active practice of analytics. Where measurement, assertions, and hypotheses are part of the analytical psychosis; knowing there are systems in place to support or diminish statements forces us to think forward. It is by this means that assertions, testing, and ultimately improvements to the site become more innovative and increase the chances of greater success.

Continued success in testing and measuring site improvements for a primary goal (i.e. - conversion) increases the value of an analyst and their merit as authority among colleagues. It has been my personal experience that the more you test and objectively report complete results, the more weight your contribution is given. Sometimes things do not support your hypothesis and it is equally as important that these results are given to the appropriate people. Should the analyst be lucky enough to be surrounded by highly intelligent peers, the resulting discussion from success or failure from each hypothesis should be equally as fruitful in insights on which to base future hypotheses.

Google Website Optimizer Global Elements Testing Screenshot

GWO Experiments need not be enormous and complicated from the start. Get into testing by making up four or five alternative headlines for a high traffic page. Try each of the testing methodologies. Here’s a quick test to try just to get the mechanics down:

  1. Identify and analyze a page for testing with decent traffic and lackluster performance. (*This will help benchmark performance to understand impact more clearly.)
  2. Create four or five suitable alternatives (with at least one marginally poor headline to create divergence).
  3. Make an appropriate number of copies of pages to match the number of variant headlines
  4. Rename pages of variations and supply new URLs to Google
  5. Install GWO provided scripts on original page
  6. Install GWO provided script on variant pages
  7. Install GWO conversion goal script on goal page
  8. Test the scripts
  9. Execute test

After a few days, or weeks, depending on the level of traffic and the apparent difference in your variations, you should experience some divergence which can begin to allude to validating and supporting, or, possibly diminishing your claim. (again, regardless of the outcome, so long as you have clear data, the test should be considered a success)

(I’ll publish an edit with some photos and some tips on usability here when I have the time.)

Getting a test under your belt can be an enormous benefit. Just knowing you can perform a test makes you think differently. That aspect of perspective is a huge step in getting to where the real analysis takes place.

Measuring Success of Targeted Content Delivery with Omniture

If you’re planning on making some movement toward delivering targeted content, and by current estimates of interest in the topic I’d imagine sooner or later you would, you’ll eventually have to make some motion to measure the actions and the return on costs. In the past year, CableOrganizer.com installed a collaborative project to increase our ability to prepare and deliver targeted content based on rules and dispositions for arriving traffic. We used SiteBrand as our solution and measured the results both in their tool and using our Omniture SiteCatalyst interface.

This article should provide insights into the following assertions:

  • Targeted Content was successful at minimal application based not solely on statistical regression but in measurement on a per click basis.
  • Common and steady revenue returned on investment remains consistently above 500-700%. (Based on our costs by the SiteBrand implementation)
  • Time and effort to create and deliver targeted content is worth the resource investment, at first, second and final pass when based on good analysis
  • Omniture, though not yet partnered with SiteBrand, can efficiently and effectively measure this solution without extra costs or set up fees.

Targeted Content delivery is, although dynamic and requiring moderate to advanced analysis, not very difficult to implement. The process is relatively simple. You find areas on your landing pages which you would find highly visible and actionable and create alternative code which can be applied. Alright, maybe not quite that simple, but conceptually conceivable.

To determine what needs to be placed in that area, do some analysis. Take a good look at what is bringing people into the landing pages which you’re having problems with. In SiteCatalyst you can harvest a world of data at the page level. The same is true of Google Analytics. Find out why your users are landing on that page and create for them a page which they can relate to and act on based on their arriving disposition. In May 2007 at the eMetrics Summit, a gentleman named Christopher from Microsoft gave an incredible diagram of how to achieve this using dedicated servers. For most of us, this is cost prohibitive. For the rest of us, we can easily break down our 8-10 most important means to arrive at a page and concentrate on that.

Getting in the groove is half the obstacle to making a great, thoughtful, reactive landing page. Don’t sit and wonder for months trying to break into delivery. Sit down, hammer out a bunch of alternative code and images, and get the stuff out there. You will truly be surprised at the value of the pressed effort upon measuring. There is always room to refine and rework problem areas when you get moving. You’re mantra here should be something akin to: “Jimi Hendrix wasn’t born with a guitar and a bandana” or “Michael Jordan was cut from his High School Basketball Team”.

Measure your work; and don’t make one source your text on philosophy. Its a funny thing, but SiteCatalyst was not really set up to measure applications like SiteBrand out of the box. As a result of our content being served offsite, or at least the ‘object’ being served in and out by SiteBrand, we’re able to count these creative ‘Zones’ as campaigns. No eVar necessary. Caution: think about your campaign architecture for this before you input all your tracking codes. I would strongly urge you to consider numbering zones, naming creatives and delegating versions, ownership, and any partner agencies. The more complete and uniform these are when you set them up, the better off you are later in follow-up analysis. The following diagram should show how our zones behave and what type of value we’ve seen returned from our foray in to measured targeted content:
Targeted Content Delivery Performance Charts and Diagram

As you can see, there is a fairly solid level of performance in some zones over others. It is interesting to note, that, while the Zone 3 banner shoved into the corner here above is in what most consider a ‘Cold’ area of the site, its since had tremendous success. This feeds interesting insights back into the process. Suddenly, we’ve found ways to make areas of our site previously invisible to the user seem very relevant. Click rates and participation from this zone currently have participatory value in more orders than any two. I guess that means a possible cruise is a powerful ‘click-motivator’.

The big picture is that targeted content is affordable. It can be done with relatively little resource consumption. It can be tested. It can be measured (at least with SiteCatalyst). It can drive facets of the analytics process. It can raise conversion. You have that much information. I would be interested to hear anyone’s accounts of similar experiences.

For more information on any of what is used or discussed above, please feel free to contact me by comment or email. You can contact SiteBrand here. The publication from a Case Study with CableOrganizer.com is available on their site.

Organization Dialects as an Indicator of Process Adoption

Commonly, in human communication theory, people of a like group develop what is referred to as a closed language or closed dialect. This is as true of a generic dating couple, transcending nation, culture, or race, as it is of entire socially isolated tribes in say…, Papua New Guinea. We use words within the context of our group which represent the understanding upheld by those within the borders defined by that mass. For example, I can get away with telling my good friends to “Go to Hell”…and they’ll get the implied meaning of the phrase without becoming anxious or guarded. Not that my imposing 5′9 frame should present such threat, but, you get the idea. This is very much part of the same idea that a company which has become ‘Process-Aware’ would begin to use certain specific dialect or language as an internal sign that practice is maturing.

If you have made the decision to include ‘Process’, as defined by Eric, as a means to get the most from your web analytics investment, then you probably have an idea about the steps. The blueprint is simple but the work is hard, and knowing how to see where you’ve brought your practice to may require metrics of its own. The purpose of this post is to help illustrate a behavior which people exhibit in their communication which may help to define the progress of your adaptation by simple observations.

In eCommerce; the marketing department, business intelligence department, or an entire small business usually has a group of people who share similar goals. As a retailer, the primary goal is to sell a product above its costs to the extent that when all considerations are made, a portion of money remains to sustain the livelihood and lifestyle of the investors, shareholders, and resource contributors. With these shared goals come discussions about goals and how success is measured. With discussions come descriptive, business and operations specific language which affirms goal alignment and participates in something called “uncertainty reduction“. By reducing risk and uncertainty, communication becomes more clear and ideas flow freely without the risk of tangles in semantics or other noise. Language, essentially closes within the group.

Simply stated, this is the cycle in an interpersonal relationship when two persons are getting to know each other. When this is applied to a group, it is referred to as ‘Organizational Learning’. This is defined as an organization which..”actively creates, captures, transfers, and mobilizes knowledge to enable it to adapt to a changing environment.”(2) This, itself is a process.

Cycle of Communication

As a web analyst, the idea of instituting process to gather, organize, and disseminate information for its use for valuable improvement is both cyclical and self-seeding. Our reports are built on information which we have ensured for accuracy, structured and analyzed for purpose. Our scheduled delivery becomes essential and regular for the fulfillment of the practice expectations from our group of readers. It meets with its audience on a level with which they can both comprehend, and inspires action for the best outcome of the group. (This is John Stuart Mill’s principle of Utilitarianism) The result is, that, with consistent value being wielded for the improvement of their own success and the success of the organization (’Game Theory’), that its content becomes useful and relative. As these messages are reinforced, the behavior of the social structure continues to align. With the alignment of goals and the advent of group communication, feedback begins to trickle in to the analyst. The feedback ought to reflect a dialect which was initially sown. (NOTE: Any feedback is a sign of adoption, but valuable feedback is a sign of adoption & success.)

« Previous Entries