Comments on: The Web Analyst Case for Acceleration to Experimentation http://paul.webanalyticsdemystified.com/2008/02/06/the-web-analyst-case-for-acceleration-to-experimentation/ Mon, 06 Oct 2008 12:03:34 +0000 http://wordpress.org/?v=2.3.2 By: Daniel http://paul.webanalyticsdemystified.com/2008/02/06/the-web-analyst-case-for-acceleration-to-experimentation/#comment-95 Daniel Fri, 08 Feb 2008 01:26:41 +0000 http://paul.webanalyticsdemystified.com/2008/02/06/the-web-analyst-case-for-acceleration-to-experimentation/#comment-95 Hey Bill. Thanks for the comment. What you've stated is true...and I agree with you. The purpose for this is to illustrate to a web analyst(whom may have relatively low frequency or experience with multivariate testing) the ability to create an understanding of how divergence can impact the test through the comparison of the elements being tested. Also, it might be a point of interest to people who've already been testing but unsure as to why their experiments are taking forever to reach statistical significance. GWO does not 'complete' a test until the relevance rating has reached a certain threshold of validity. Building in a dog element can help foster expedience. In addition, to respond to the latter statement, if all the tested elements are very strong, the divergence will occur versus the original. Again, thanks for the note. Daniel Hey Bill. Thanks for the comment.

What you’ve stated is true…and I agree with you. The purpose for this is to illustrate to a web analyst(whom may have relatively low frequency or experience with multivariate testing) the ability to create an understanding of how divergence can impact the test through the comparison of the elements being tested. Also, it might be a point of interest to people who’ve already been testing but unsure as to why their experiments are taking forever to reach statistical significance. GWO does not ‘complete’ a test until the relevance rating has reached a certain threshold of validity. Building in a dog element can help foster expedience.

In addition, to respond to the latter statement, if all the tested elements are very strong, the divergence will occur versus the original.

Again, thanks for the note.

Daniel

]]> By: Billy Shih http://paul.webanalyticsdemystified.com/2008/02/06/the-web-analyst-case-for-acceleration-to-experimentation/#comment-94 Billy Shih Thu, 07 Feb 2008 22:43:26 +0000 http://paul.webanalyticsdemystified.com/2008/02/06/the-web-analyst-case-for-acceleration-to-experimentation/#comment-94 I love this sentence: "Should the analyst be lucky enough to be surrounded by highly intelligent peers, the resulting discussion from success or failure from each hypothesis should be equally as fruitful in insights on which to base future hypotheses." It's often hard to convince people that results, whether their spectacular or lukewarm, always teach you something that can help you improve the page. I'm curious about this statement though: -Create four or five suitable alternatives (with at least one marginally poor headline to create divergence). Why is it necessary to create divergence? And why with a marginally poor headline? I believe if you test sufficiently different headlines, you will create some divergence regardless. Still, I don't see how if all of the titles are good, why that would negatively impact the test or your results? I love this sentence:
“Should the analyst be lucky enough to be surrounded by highly intelligent peers, the resulting discussion from success or failure from each hypothesis should be equally as fruitful in insights on which to base future hypotheses.”

It’s often hard to convince people that results, whether their spectacular or lukewarm, always teach you something that can help you improve the page.

I’m curious about this statement though:
-Create four or five suitable alternatives (with at least one marginally poor headline to create divergence).

Why is it necessary to create divergence? And why with a marginally poor headline?

I believe if you test sufficiently different headlines, you will create some divergence regardless. Still, I don’t see how if all of the titles are good, why that would negatively impact the test or your results?

]]>