Is the Long Tail Getting the Short End of the Stick?

By Kurt Cagle
July 5, 2008 | Comments: 8

The idea has gone from statistical curiosity to a deeply entrenched belief - especially in Silicon Valley: We are moving toward an economic state where it no longer becomes necessary to invest just in the top 1% of all ideas, but that instead an investor could create strategies using the web to take advantage of the statistical long tail - small niche markets (perhaps only one consumer deep) that in the aggregate can prove more lucrative than the blockbuster.

The Long Tail theory (which first appeared in a Wired article by Chris Anderson - later expanded to the book The Long Tail: Why the Future of Business is Selling Less of More in 2006 - has become in that interval nearly a mantra for small companies seeking venture capital or similar business funding. "Our strategy is to take advantage of the Long Tail by providing services to capture the broad range of niche markets through mass customization" has become almost boilerplate text in any high tech business proposal.


However, a recent report by Harvard Business School media analyst Anita Elberse has raised a number of significant questions about how viable this particular business model really is in a new web article Should You Invest in the Long Tail?. Anderson's basic thesis was that as the web has increased the availability of more obscure content (either that produced outside of traditional channels or that which was produced in the past and has largely disappeared from distribution) it has also made possible the notion that afficianados of particular genres, authors, musicians and so on could in fact take advantage of these new media and start consuming these products, giving them a second life and as such making them more profitable over time. His vision thus was that the long tail was becoming thicker over time, making blockbusters less important in the long scheme of things and ushering in an era where independent creators would reign supreme.

Not so fast, says Professor Elberse. In a six month study she evaluated sales figures from online music vendor Rhapsody, on-demand video vendor QuickFlix, and Nielsen VideoScan and SoundScan services, as well as performing less rigorous analyses on Amazon and other online "entertainment and publishing" vendors. What she found surprised her - far from becoming more robust, the evidence that she indicated that in fact the long tail is becoming both longer and flatter. In other words, rather than the more obscure content becoming more visible and affordable over time, what she discovered was that the blockbusters were in fact becoming more important to marketing, and more obscure content could in fact prove detrimental to effective sales because the cost per sale of each transaction actually goes up.

This, in fact, conforms remarkably well to William McPhee's Formal Theories of Mass Behavior, published in the early 1960s, which describe the traditional blockbuster pattern of marketing consumption, which argues that statistically, the top 1% of most large segmented markets (such as those found for entertainment products) typically does as well or better than the top 9% after it, and that there is a dramatic logarithmic falloff that makes the tail generally unprofitable.

One thing pointed out in the study was a curious pattern - Anderson described the connoisseur who would go and seek out the obscure titles in any given area, thus raising the profile of long tail purchases. However, Elberse sees an alternative pattern developing - that connoisseur is far more likely than the average to consume a significantly higher number of titles, but in general for every obscure title they choose, they are likely to choose two or more popular titles. The non-connoisseur, on the other hand, is far more likely to choose the popular titles almost exclusively.  This has the net effect not of boosting long-tail purchasing, but in fact of suppressing it.

Andersen and Elberse have carried this debate online, with questions being raised of differences in methodology and potentially of interpretation, but this debate carries over into the world of software development as well. Do the same rules and patterns apply?

Open source software tends to follow the same kind of logarithm decay rules that media in general apply, and one of the more vocal (albeit not necessarily well-reasoned) arguments for the adoption of open source software is that there is benefit for those people working in the long tail part of a software niche distribution by virtue of the breadth of "similar" adoption.

However, for software developers there is an investment (in time, tools, and focus) that's made when any piece of software is used, and if the benefits of the long-tail really do not apply (the tail becomes longer and flatter over time rather than shorter and fatter) then it is worth it for developers to examine their choices heavily if they had assumed Anderson's thesis.

On the other hand, it's worth noting that there are other relationships at play too - early adopters are almost always working with long-tail technologies at first by definition - they are too new to have achieved broad exposure in the market. As with nearly any economic decision (and the choice of software is ultimately an economic decision) there are typically multiple variables at work, including, most importantly time, and aLongTail.pngs a consequence, the long tail description is only a snapshot of a considerably more complex multivariate function that says that while the top 1% of a market becomes perhaps more significant over time, the specific constituents of that top 1% are in constant flux and follow power laws of their own (such as the illustration showcasing the trajectory of a given "product" up and down the long tail over time).


Thus, before necessarily throwing out (or embracing) long tail dynamics based upon the Anderson vs. Elberse debates, it is worth understanding that there is more to the story than a simple two dimensional graph. Popularity is a remarkably transienty property ... the Romans stated this as the phrase "Sic transit gloria mundi", but which today the same sentiment may best be expressed as "today at the head of the curve, tomorrow somewhere out there on the long, long tail".


You might also be interested in:


8 Comments

BTW, the image was rendered using Mathematica - I'd be interested to know if anyone has actually done studies tracing individual "items" up and down the tail, as this "function" is pure speculation on my part

Thanks Kurt, another great piece.

Another great article! This is a long but very interesting read.

We are moving toward an economic state where it no longer becomes necessary to invest just in the top 1% of all ideas, but that instead an investor could create strategies using the web to take advantage of the statistical long tail - small niche markets (perhaps only one consumer deep) that in the aggregate can prove more lucrative than the blockbuster.

I thought that long tail - small niche markets will let some people see a very big market that not been developed , and will create unexpected value. We can see it several years or several months later.

I like your book and the articles very much, I have also learned many things from inside. I am now in kswchina, a examinations site about education. I am very happy that I can know so many experts in here.

Very interesting story.

"what she discovered was that the blockbusters were in fact becoming more important to marketing, and more obscure content could in fact prove detrimental to effective sales because the cost per sale of each transaction actually goes up."

I would like to see Prof Elberse proof that the cost goes up. I do not have any proof one way or the other but as an economist am interested in the argument.

An important part of 'long-tail' theory rests upon changes in marketing and distribution costs which are thought NOT to go up by handling more product lines as they are related more to total volume not specific line volume. In many cases these will be based on marginal costs which will be decreasing as total volume climbs.

OK that is the theory - which seems reasonable but does anyone have any proof?

google is doing the long tail in software too, careless, ever-beta, use-it-as-it-is software, small-maybe-someone-needs-it utilities or services...

nothing new here, some other giant following a social pattern.

getting the tail FROM END TO END, that makes it profitable! aggregating! that is the key word, there's no magic there, people! wake up, who can do that? google, amazon & al., not me & you, the startup joe.

it's, again, about aggregating, having the power to get it from end to end, it's not about niches, but about wholesaling an entire range of niches!

Brian,

I read the initial paper (posted online) but didn't get a chance to peruse the full methodologies that she used. I have some questions myself about her conclusions, not least of which because she makes the assertion that the "maven" both makes up a comparatively small percentage of the overall market and spends as much time and resources getting the most popular picks as he or she does pursuing the more obscure ones.

One thing that isn't quantified in those assumptions is the comparative power that the maven has to influence others. Mavens, almost by definition, both have a higher degree of interest in a given subject and a greater incentive to evangelize that interest, and as such tend to have a disproportionately high ability to move an obscure title into popularity.

What this means (as I pointed out in the initial article) that there are other variables that Dr. Elberse apparently did not include in her methodologies that I think may be more significant than she believed. I'm not sure that I believe that she's completely wrong (or that Anderson is completely right), only that the space is more complex than it appears on the surface.

I read the initial paper (posted online) but didn't get a chance to peruse the full methodologies that she used. I have some questions myself about her conclusions, not least of which because she makes the assertion that the "maven" both makes up a comparatively small percentage of the overall market and spends as much time and resources getting the most popular picks as he or she does pursuing the more obscure ones.

Popular Topics

Archives

Or, visit our complete archives.

Recommended for You

Got a Question?