The Long Tail

From Amit’s Thoughts

There’s a lot of excitement about the Long Tail. In the Wired article there are references to Pareto distributions and Zipf’s Law. Zipf studied city sizes to come up with the law. However it looks like he only studied the largest cities [pdf], and most cities (the tail) do not follow Zipf’s Law. Pareto studied incomes to come up with the distribution. However it looks like Pareto’s distribution only applies to the top incomes, and most people (the tail) do not follow the Pareto distribution.

So we have the excitement over the long tail built on an expectation of how big the tail is. And the size of the tail is modeled using Zipf’s Law or Pareto distributions. And those two didn’t even fit the tail; they only fit the top few percent. I don’t really expect the excitement over the long tail to die down for a while though; people will find ways to make the data fit their beliefs and ignore the data that doesn’t fit.