Paul Holstein Weblog at Web Analytics Demystified

Paul Holstein is Co-Founder, Vice President and COO of CableOrganizer.com, Inc., now among the world's leading purveyors of cable and wire management-related products. In these capacities, Holstein oversees the company's strategic planning and day-to-day company operations, including web analytics and multivariate testing.

Subscribe to Paul Holstein weblog

What to do if your company is failing

Yesterday, I wrote about the reasons that companies hire and fire people.  Basically, if a company can make money on you, they’ll hire you and if they are losing money on you, they will consider letting you go.  Eric Peterson read my post and asked me to consider a question he gets regularly:

I guess the one thing I wonder about, and this is probably because I hear from a lot of folks who aren’t getting the management support they’d like, is what do you do if the existing chain-of-command simply doesn’t support the analyst “making money?”

Does that make sense? This is the age old complaint about “we provide insights that go nowhere …”

Are any of you in this position?  Do you know of anyone in this position?  It’s not uncommon at all.  In fact at the recent xChange conference, I sat in on a session dedicated to this exact subject.  It was called, “Getting Analysts to Produce Analysis and Getting the Business to Listen.”  One solution the group came up with was to use what we call an “Upfront Contract” with the decision makers.  You can read more about that in the post.

But what do you do if the company you work for simply won’t listen to you?  They pay you a decent salary to analyze the website, but they won’t implement your suggestions.  In other words, the company is not making money on you.  Now, you are in the difficult position of being considered an unnecessary cost.  Seeing as the economy is heading South quickly, this is not a position you want to be in.

As a business owner, I feel that every employee has a fiduciary responsibility to the company that pays their salary to provide the best work they can and to produce a profit for their company.  Therefore, I feel the most ethical think to do would be to march into your bosses office and announce in a loud steady voice that you feel your contributions are not valued and that you don’t think it is appropriate for the company to waste it’s hard fought earnings on you.  You should immediately resign.

Ha, Ha, just kidding.  Of course you’re not going to resign.  You’ve got a spouse and kids at home who couldn’t care less that your company doesn’t listen to you.  They want luxuries such as food, shelter and clothing.  They are not interested in you taking a moral stand at work.  It’s time for an alternative approach.

At this point, you need to evaluate how much time you have.  If you are working for the federal government, you can relax.  They are unlikely to stop spending taxpayer money any time soon.  Just keep cruising.  However, if you work for GM, you may need to start planning yesterday.

How much clout does your boss have?  How much clout does your department have?  Who do you need to suck up to?  Start perusing the job boards at your company.  If you work for a Fortune 500 company, you can probably move laterally to another department that is more stable and secure.

Do you have colleagues in competition with you?  Perhaps you could sabotage their work or position yourself in a more favorable light.  After all, if the company execs call for a 50% cut, you don’t have to be great, you simply have to be better than the person standing next to you.

One thing you should seriously consider, however, is starting your own business.  Now, given the fact that you’ve got payments to make on your sub-prime loan, I wouldn’t start your new business without a steady stream of customers.  Therefore, you’ll need to moonlight.

I keep reading tons of job postings on the Web Analytics Forum.  Perhaps some of those employers would be happy to have an experienced part-time contractor rather than an expensive full time employee.  It’s time to call in your chips at your favorite web analytics vendor.  They surely know who is looking for support with their products.  Find out and make some inquiries.

OK, I know that most of you will not follow my advice on this one.  Frankly, I’m only kidding about most of this anyway.  However, the sad truth is this; there is no good solution if you are in a bad situation at work.  My only serious advice is to do the best you can, including:

  • Do the best you can
  • Find persuasive ways to make money or save money at your company.
  • Use the upfront contract
  • Network with others.  Get to know Eric Peterson
  • Attend the WA shows
  • Try your best to improve your companies profitability
  • Be a pleasant person at work
  • Keep your eyes open for other internal opportunities
  • Absolutely consider doing freelance work
  • Pay attention to the job postings at the WAA
  • Put your resume out there.
  • Keep your spending low at home.  Don’t make major purchases.
  • Pay off your high interest debts
  • Sell off unneeded assets on ebay.
  • Find out about the unemployment benefits your state offers
  • Get additional training from your company or other sources if you can.
  • Be a continuous student.  Keep learning.
  • Appreciate every day that you still have a job.
  • Don’t give up and don’t stress too much about this.  If you’re reading this post now, you’re probably smart enough to figure this all out in the end.

Good luck.  Feedback welcome.

Contribute to this conversation now!

Post Date:
Thursday, November 13th, 2008 at 2:34 pm
Categories:
Subscribe:
Interact:

Analyzing Layoffs (From a Business Owner’s Perspective)

Eric Peterson just wrote an outstanding piece about the relative job security of web analysts in a recession.  In one of the comments, he asked me if our company would consider cutbacks in analytics if times got tough.

To answer this question, I think we need to look at why companies hire people and why they sometimes have layoffs and firings.  First the good part.  Companies hire people when they think they can make or save an amount of money greater than the cost of hiring a new person.  Sounds obvious, doesn’t it?  Let me give you an example.

Say you are growing electrical distributor and your phones are ringing off the hook.  There are so many calls that your current staff can not answer them all.  Eventually, you may figure out that if you hire one more customer service rep, that you’ll be able to pick up enough additional calls and sales to cover the cost of the new rep.  You may even make a profit.  Therefore, you will hire.  The same holds true for every position in your company — Accounting, IT, warehouse, marketing, etc.

Now here comes the bad part.  Sometimes the economy shrinks or the employee you hired doesn’t do a good job.  Now what do you do?  That person may have a family and is certainly depending on their salary to pay the bills at home.  As a human being, you have a natural resitance to letting them go — especially in advance of a holiday and certainly not in the face of growing unemployment.  So you keep them and take a loss or you bite the bullet for the good of the company.

Some companies hold on way too long.  For example, we are constantly hearing about companies laying off thousands of employees at one time.  It happens so often now, that we just accept it as normal business practice.  But it isn’t good or even appropriate.  It is usually a reflection of poor leadership and failure.

Why would you lay off a thousand people at a time?  Tell me, what catastrophic event occurred today that you didn’t need a thousand people when you needed them yesterday?  In very few instances does this actually occur.  If your plant burned down, I could see it, but most layoffs don’t work that way.  The obvious truth is that if you don’t need the extra thousand people today, you probably didn’t need them yesterday either.  Usually, declines are slow and not catastrophic.  It simply means that you didn’t adjust fast enough and now you must take drastic action to try and catch up.  Total failure.  Management should be ashamed.

Good companies have the right number and quality of people on the job at all times.  If they are growing, they are looking for quality people to hire.  If an employee fails, they don’t cross their fingers or wish the problem away.  And, unfortunately, if the phones stop ringing, you will need less customer service reps.

Every single employee in your organization must be a profit center.  Otherwise, why are they there?  Think about this for a moment.  Why would you keep anyone in your organization that doesn’t pay for their own salary in some way or another?  Do you have a corporate chef?  If you do, I hope you’ve figured out how that chef will save your company money in lost productivity and increased morale.  Google has great free food.  I’m sure it’s a great investment for them.  However, if you only have 25 employees, a corporate chef may not realize a positive ROI.

OK, so what about web analysts?  Are they safe?  It really depends on how valuable they are to the organization.  If an analyst is regularly saving your company more money than they cost you, why would you let them go?  Obviously, you wouldn’t.  It would be foolish to fire someone who is making you money.  In fact, it would hurt your company more than it would help it.

Now if your traffic were suddenly to drop to zero, then it doesn’t really matter how good the analyst is, does it?  No analyst would be able to offer a positive ROI.  Conversely, even if times are tough, if hiring another analyst can make you more money than it costs you, by all means hire them — even if you are letting go of other employees.  It only makes sense.

So there you have it.  Your strategy for survival is obvious.  Make money for your organization or at least save them money in an amount greater than your salary.  Make sure the decision makers in your organization know how much money you are making them.  In the long run, if you’re making them money and the organization survives, you’ll be secure for a long time.

Contribute to this conversation now!

Post Date:
Wednesday, November 12th, 2008 at 9:44 pm
Categories:
Subscribe:
Interact:

Why do we still need Omniture?

Lately two things have occurred around here that have made me ask the question, “Why do we still need Omniture?”  First, we installed an amazing analytics package called Tea Leaf.  Secondly, Google has improved its free analytics offerings tremendously.  Truthfully, there is one other event that has prompted me to ask the question as well — the stock market is melting down and we are facing a deep recession.  Do I really need to spend $40k + for a largely redundant analytics package?  Incredibly, we have paid Omniture over $176,000 over the past two years.  Granted, there was some training in there, implementation, custom programming and Search Center, but that’s still A LOT of money.

So I asked the WA Forum for their thoughts.  I asked them only to compare Google and Omniture.  Here are the things they mentioned that Omniture has but Google is lacking:

  • Lots of custom Variables - This was a problem in the past, but I’m not sure it will be going forward.  When I logged in to Google this morning, I noticed that Google has added “Advanced Segments” to the package.  They look fantastic but I don’t know yet if you can really create custom ones.  In any case, Tea Leaf will certainly allow you to create all the custom variables (called events) you want.
  • Flash Tracking component - My company doesn’t make extensive use of flash so this is not a big issue for me.  If it were, I’m sure there are some free Flash tracking packages out there or we could even write our own.  In any case, this seams trivial in Tea Leaf.
  • Classifications (SAINT) - This sounds better than it really is.  We’ve never had a lot of luck using this in Omniture and found that updating it was a pain in the rear.  Today, we rarely use it.
  • Calculated Metrics - This is actually very cool and seams better implemented than Google which appears to not have any calculated metrics at all.  I created one metric called “exits on navigation” and a corresponding “exit rate on navigation” that I use in conjunction with bounce rates.  Omniture wins on this one.
  • Export and Import API - I’m torn on this one.  First of all, my exports have never really worked from Omniture.  Unfortunately, I’ve not used their API.  I was simply exporting reports from the interface.  Oddly enough, the exports never matched the online reports.  Nevertheless, Google has announced a new export API for GA.  As far as Imports are concerned, I think Omniture has this covered.  Do you really need this feature?  If you want to see your data in Omniture, yes, however, if you want to see your data outside Omniture, it is not important.
  • Advanced Dashboarding - Yes, Omniture wins on this one as well, but Google is no slouch either. 
  • Collaboration features - Yawn.
  • Integration - This could be very cool, but I haven’t done it.  Seams overly complex and expensive.  In general, Omniture’s partners have a lot of low cost / high value competition that I tend to favor.
  • Excel Plug-in - Yes, Omniture has a plug-in.  But Google has the export and a new API.  I don’t see this as being worth the money.
  • Discover - Hands down, the coolest analytics tool I’ve ever seen.  Unfortunately, it’s not part of the base package.  Even though we bought and paid for Discover when we bought the system, Omniture refused to upgrade us to the new version without a steep charge.  I’m very upset about that and will obviously not miss something I never had.
  • Search Center - Again, its not included in the base package.  Furthermore, I don’t think it’s very good at all.  Lastly, Google Analytics is doing a fantastic job of integrating adwords with GA.  Unless you advertise a lot on Yahoo and MSN, Google wins on this one.
  • Datawarehouse - This is a good feature in Omniture that we were forced to use because SiteCatalyst didn’t have some of the metrics we needed such as Bounce rate by keyword and bounce rate by exact search term.  I wouldn’t look at Datawarehouse as a separate application.  In my opinion, it’s more of a band-aid.
  • ASI filters - Despite a lot of calls to the support desk, our ASI filters have never worked.  I’ve got three separate filters that all return the same results.  I gave up on this a long time ago.
  • Outstanding Documentation - True, Omniture’s docs are better.  But resources are available for GA. 
  • An official support desk - My experience with Omniture’s support has lagged my expectations.  But even so, bad phone support is usually better than no phone support.  Google does have e-mail and forum support and it has 62 Authorized Consultants.
  • Performance - Omniture has SLA agreements and sometimes near real time reporting.  Google has neither.  I like the real time results.  Fortunately for us, Tea Leaf is as live as it gets.

What are your thoughts?  Given the fact that we already have Tea Leaf and Google Analytics, do you think we really need Omniture?

Contribute to this conversation now!

Post Date:
Friday, November 7th, 2008 at 12:05 pm
Categories:
Subscribe:
Interact:

Growing your way Through a Recession

If you are a web analyst, your business probably relies a great deal on the Internet.  As an analyst, you probably don’t limit yourself to online data either.  You may be looking at overall sales, the stock market indexes, unemployment, housing stats and other economic indicators to try and predict your company’s future.

Other than the online data, however, the news is glum.  Markets are falling, credit markets are tight, unemployment is rising and the world appears to be in or heading towards a recession.  So what can you do?

First of all, don’t panic.  A recession simply means that the economy is shrinking.  But it never really shrinks that much, does it?  The US has been growing at around 3% for the past 5 years.  If the economy sank more than 10% it would be considered a depression.  Since no one is predicting a depression, you should be prepared to deal with a drop of less than 10%.

Here’s the trick, though.  There is no reason you can’t grow your way out of a recession.  Take a look at your growth rate up until now.  If you’re reading this blog, chances are high that you’ve had some fairly good growth in the past few years.  Will it continue?  That depends on your company’s growth potential, but there is no reason to believe that our current economic woes will hurt online businesses worse than offline businesses.  In fact, I think online business will probably fare better than offline.

Our company grew 34% this year.  I’m not sure what it would have been if the economy had been better, but I am confident that we control our own destinies.  We are growing because we are adding more products to our website.  We are expanding our advertising and carefully measuring what works and what doesn’t.  We run multivariate testing to improve our conversion rates.  We support our affiliates to increase our traffic.  We do all sorts of things to prosper. 

All you need to do is grow faster than the economy shrinks.  Can you increase your efforts by 10% next year?  As a web analysts, your job is to figure out what works and what doesn’t.  It’s up to you to make sure the folks get the information they need to do more of what works and do less of what doesn’t.

We are all uniquely positioned to take advantage of the coming challenging economy.

Contribute to this conversation now!

Post Date:
Monday, October 20th, 2008 at 1:43 pm
Categories:
Subscribe:
Interact:

Is Performance an Analytics Issue?

One thing I constantly fret over is website performance.  I have a theory that is commonly accepted by most experts — that is that the slower a website performs, the worse it will convert.  Unfortunately, the hard data is difficult to come by.

First of all, where do you gather this information?  If you have Google Analytics or Omniture, you will note that nothing in those services tell you anything about website performance.  In fact, tagging solutions are really not designed to gather this information at all.  As far as Omniture is concerned, either a page loads, or it doesn’t.  It doesn’t matter how long the page took to load or even if the entire page loaded at all.  As long as the pixel fires, it’s counted as a page view.

Log files are much better at this, but they can be difficult to analyze.  A log file will tell you exactly what time each element was called and it can tell you how much of each element was served.  From this information, you could, theoretically, come up with a baseline for each page and compare it to any given visitor’s results to determine if the entire page loaded and approximate the time it took to load.

Unfortunately, that’s wicked hard to do and I’ve never been motivated enough to do it.  Because of the complexity of the issue, we don’t ever end up doing it.  So is performance an analytics issue?  My gut tells me that it is and that the industry should look at this more closely.

I know that services such as Gomez and Alertsite exist that supposedly let you know about performance issues.  I say supposedly, because my experience with them is that they are simply robots that time a series of events on your website.  They won’t tell you if every element on your pages loaded.  They won’t tell you what your customers see.  They won’t run the javascript and flash on your site either.  And if you have a problem with your site, they aren’t very good at helping you solve the problem.  Knowing what I do about business, I can tell you that the performance monitoring sites make their money by providing you with automated information.  What we need are experts who can help us.  But experts are expensive and I don’t know if customers are willing to pay for them.

We are currently in the process of installing Tealeaf.  I’m hoping that it will shed some light on our performance issues.  In the meantime, what do you do to monitor the performance of your website.  How can we make it faster?

Contribute to this conversation now!

Post Date:
Friday, October 10th, 2008 at 10:42 am
Categories:
Subscribe:
Interact:

What do Analytics look like?

Argentina During one of the huddles at xChange, a good question came up.  What does Analytics look like?  Is it a report, a graph or even a conversation?

At first blush, this seams like a simple question.  It looks like Omniture or Google Analytics, right? Or does it look like Webtrends / Visual Sciences?  Wait a minute, I hired a web analyst and he gives me a report weekly with text on it.  Sometimes we even have a conversation about analytics.

During xChange, analytics looked a lot like conversations to me.  We all sat around tables and talked a lot about it.  The problem is, there is no answer for this.  I know what accounting looks like.  I know what my doctor’s office looks like.  I know what most other professions look like but there doesn’t seam to be a standard way to communicate analytics.

Just printing out a dashboard or special report doesn’t seem to cut it for me.  Neither does a dry report.  When I used to receive weekly reports from our web analyst, I used to insist that he sit with me and go over the report.  I wanted to hear a story about our traffic.  I wanted to speculate as to reasons why things were up or down. 

Data itself is not very interesting.  I think Analytics must tell a story.  It must tell us something we didn’t know before we started.  I want to know how analytics can help our user experience or how it can help us avoid problems.  Those are the stories that interest me.

What does analytics look like at your organization?

Contribute to this conversation now!

Post Date:
Monday, September 22nd, 2008 at 2:38 pm
Categories:
Subscribe:
Interact:

Tealeaf Update

Peppermint tea  

It’s been a few weeks since we signed up for Tealeaf and I wanted to give you all an update.  Here’s the situation.  In our environment, we require three new servers to make it work.  We need one server to collect the data on our Linux server, one to collect the data on our Windows server (for Endeca), and one to compile and serve the data so we can read it.

Our servers are actually in two different locations.  Our Linux servers are in Connecticut and our Windows server is at Rackspace.  So we physically must install new servers at both locations.  Well, Rackspace has been great.  They are a traditional co-location facility.  We’ve arranged to have our servers sent there and installed.

Unfortunately, our Connecticut host is not as hospitable.  They’re paranoid about allowing in foreign servers.  They only offer about 4 different configurations and everything must be purchased and installed through them.  If you are simply running a website and don’t need special hardware, that’s great; however, if you need a special configuration such as a special router or firewall, you’ve got issues. 

Moving out of Connecticut is not a simple option.  They are the most reliable hosts and experts for our shopping cart.  If we move from them, we’d need to bring in extra expertise to keep our shopping cart running.  Granted, we are in the process of changing our shopping cart and moving to windows, but that won’t be ready for at least 6 more months.

So what to do?  We are currently working with our hosts to see if they can come up with a solution.  I’d hate to delay our tealeaf implementation because of this.

Contribute to this conversation now!

Post Date:
Thursday, September 18th, 2008 at 11:08 am
Categories:
Subscribe:
Interact:

We are Drinking the Tea

A couple posts back, I mentioned that we are interested in using Tealeaf to analyze our visitor sessions with the hope of fixing any interface issues.  Well, we signed up for them on Friday.  I’ve decided to blog about our experiences using tealeaf and give you an inside look into the implementation.

Let’s start with the contract process.  Tealeaf isn’t cheap.  It’s a product that major companies use and it’s priced that way.  Fortunately for those of us on the bottom half of the IR 500, Tealeaf came out with an offering called the SMB Package.  If you’ve shyed away from them because of the price, this may be a good opportunity for you to take a second look.  I’m not going to get into the exact price we paid them because I don’t want to find out later that we paid too much or upset someone else because we got a better deal.

We purchased the SMB Package which includes Tealeaf CX (the part that gathers the data) and cxImpact (the part that allows you to see the data).  In addition, we added cxResults.  cxResults gives you multi session capability among other segmenting tools.  This is important for us because our average transaction takes 3 days from the time a visitor first sees our site.

Tealeaf also offers other products such as cxView which is their dashboard product and cxReveal and cxVerify (which are customer service optimization suites) and cxConnect which allows you to connect to third party applications such as Omniture.  However, since we are on a budget and only need the core functionality, we passed on those products.

In addition to the product costs, there is implementation and training which will certainly vary based upon the complexity of your architecture and your training needs.  Our environment is modestly complex because we have servers at Rackspace in Denver and also a Linux server in Connecticut.  Therefore, Tealeaf needs to combine those sessions into one coherent view.

I’m excited about this new implementation and will be happy to let you know how it goes.  If you have any advice for me or any questions, please feel free to comment.

Contribute to this conversation now!

Post Date:
Friday, September 5th, 2008 at 11:39 am
Categories:
Subscribe:
Interact:

Getting the Business To Listen

Woman shouting down megaphoneOne of the huddles I attended at xChange was entitled “Getting Analysts to Produce Analysis and Getting the Business to Listen.”  I was mostly interested in the first part on how to get the analysts to produce, but I was far outnumbered by the others who wanted to know how to get the business to listen.

This seams to be a super important issue among analysts and here’s why.  Real ROI comes from action.  There really is no ROI in analytics.  Unless your analysis results in a positive change, it doesn’t really help, does it?

So how do you get the changes implemented?  Many analysts don’t have front line control over their websites — particularly in large companies.  The website is tightly controlled by the managers or executives.  In those cases, the analyst needs to become a salesperson.  They have to sell the idea of changing a website. 

One interesting perception from this huddle is that there is often some level of hostility between the analysts and the decision makers.  They don’t always respect each other. 

So how do you get a hostile manager to listen to you and do what you want them to do?  Matt Crenshaw from howstuffworks.com had a brilliant suggestion.  Matt suggested a technique called the “Upfront Contract.”  If you’ve ever bought a new car, you’re probably familiar with this technique.  This is where the salesperson will say to you, What will it take to sell you a car today.  Once you tell them your requirements, they’ll say to you.  If I am able to fulfill those requirements, will you sign the paperwork today?

It’s a very effective technique that can save everyone time.  The salesperson doesn’t want to waste time dealing with someone who won’t buy a car, and the buyer can get all his or her requirements listed up front. 

You can do the same thing with your decision makers.  The next time they ask you for an analysis, or during your next meeting, just ask them what information would they need to see in order to make a change to the website.  If they say, I would want to see an A/B test were the proposed change beats the control by 40%, then you write that down and get them to sign the paper.

Then you run off and perform the test.  Assuming you get the result they were looking for, then you present the signed contract to whoever made the decision and inform them that you’ve met their criteria.  It’s time to make the change.

That’s just one great idea.  How do you get your managers to listen?

Contribute to this conversation now!

Post Date:
Wednesday, August 27th, 2008 at 4:21 pm
Categories:
Subscribe:
Interact:

Measuring the Customer Experience

One of the huddles I attended at xChange was titled: Integrating Customer Experience and Marketing Data with Web Analytics.  We went around the room and discussed what we do to measure the customer experience.  Here’s what we do at my company:

  • We review our Product Reviews to see the problems or successes our customers experience.
  • We have a suggestion box on every single page of our website where customers can tell us if any page is confusing or needs additional information.
  • We monitor blogs with a tool from bloglines.
  • We use Google Alerts to monitor new web pages that reference us.
  • We look at new referrers to our site to see what they say about us.
  • We have a self serve testimonial page where our customers can share comments.
  • We monitor our e-mail to sales@ or orders@ or webmaster@ using help desk software.
  • We regularly perform usability studies in our own lab.
  • We regularly perform multivariate testing using Google’s website optimizer.
  • We use Alertsite to monitor our website performanceGomez is a more expensive competitor.
  • We monitor our 404 errors by studying our log files, not just our Omniture reports.
  • We poll our customer service reps. to get an idea of our customer’s thoughts.
  • We monitor high bounce rate pages.
  • We monitor zero site search results.
  • We use a newspaper clipping service from Cison (formerly Bacons).  A competitor is Burrelles Luce.
  • We regularly crawl our own site looking for problems.

You may think this is excessive but I am paranoid about good customer service.  We are an e-commerce site in the Internet Retailer top 500.  We can’t afford to have unhappy customers.  Just one mistreated customer can affect thousands of future sales.

In the future, we are considering using Tealeaf to detect problems.

Any other suggestions on what we could do better?  I’m all ears.

Contribute to this conversation now!

Post Date:
Friday, August 22nd, 2008 at 3:14 pm
Categories:
Subscribe:
Interact: