Something I don't understand from Ray Kurzweil's arguments about the “Singularity”: he says that we're in the “knee” of the growth curve:

... exponential growth is seductive, starting out slowly and virtually unnoticable, but beyond the knee of the curve it turns explosive and profoundly transformative.

There's a big problem with this idea.

Exponential curves do not have knees.

Exponential curves are scale-free. If you replot them at a different scale, the knee will appear to be in a different place. Furthermore, the knee will always appear to be near the right edge of the curve, so you'll always think the knee just occurred recently.

I searched Google to find any page on how to define the “knee” of an exponential curve, but did not find one. I only found one page that even mentions the issue. Why aren't people pointing this out? Am I missing something? Only Steve Jurvetson, in a comment on his own blog, says:

For almost any issue, the “knee in the curve” occurred in the recent past, and history before that seemed pretty flat. But, of course, there is no knee or inflection point or “hockey stick” in an exponential curve (when plotted on log paper, this more obvious). Roll the clock forward 5 years, plot again, and the perceived “knee” on a linear graph will have moved forward 5 years.

I'm only on page 10 in Kurzweil's new book, The Singularity is Near (which I received at the Accelerating Change 2005 conference), and his opening argument is suspect. This is going to leave a bad taste in my mouth as I read the rest of the book.

Labels: ,

Anonymous wrote at Monday, September 26, 2005 at 3:38:00 PM PDT

Hi, I think when Kurzweil is talking about knees, he is talking about his specific graphs, in which he has arranged the graph scale to extend to cover the particular timeperiod he is interested in discussing.

So, on those particular graphs, there is a visible knee.

He has arranged the graphs to portray the particular period of time that he wants to discuss.

I don't think these "graphology" issues detract from his primary arguments and data which DO demonstrate exponential technological development. So don't let it get under your skin too much.

Anonymous wrote at Monday, September 26, 2005 at 11:13:00 PM PDT

This had vaguely bugged me too, so it's great to see formulated. A related point which should be even more obvious (but I can imagine missing it) is that exponentials don't have vertical asymptotes.

Mark wrote at Tuesday, September 27, 2005 at 11:16:00 AM PDT

I think the "knee" here is more of a psychological/perceptual one. I'm not a Kurzweil disciple, but I have to believe he understands the mathematics of exponentiation.

Anyway, to give a wishy-washy definition of what he might mean by the "knee": The knee is the point where the curve crosses the threshold where it begins to affect the daily life of most* people.

So, look at the internet through the 80s - no one on the street knew what it was, and then in the early 90s middle-aged engineers and nerds started paying for compuserve accounts, and now you can search google on your phone. The knee in this case happened sometime in between when the pop-science rags started doing "information superhighway" cover stories and the year 2000.

Computation is another easy example. Through most of this century, computing devices were used only by researchers, governments, and huge corporations. Then, some time in the 70s people started buying 4 function electronic calculators that cost \$100 and had to be plugged in to the wall.

So, obviously, this sort of knee eludes formalism, which makes it a bit of a shaky point to predict the future from. But I think there is something there...

* Where by "most" I mean "most people in the developed world". My big complaint with Kurzweil is that he forgets that most of the world is still living in squalor...

Anonymous wrote at Friday, November 18, 2005 at 7:25:00 AM PST

This had vaguely bugged me too, so it's great to see formulated.

Anonymous wrote at Saturday, December 17, 2005 at 11:33:00 AM PST

A minor correction. Exponential functions f(x)=exp(x) are not "scale-free". This is evidenced by the fact that the "knee" "moves" when you rescale. Were exp(x) scale-free nothing would move when you rescale. Mathematically "scale-free" for real functions is defined as f(x)dx=f(x')dx' for x'=a*x. In English scaling the variable scales the function in an equal and inverse manner. For example the function f(x)=1/x is scale-free

x' = a*x
f(x) = 1/x

f(x)dx = f(x')dx'
dx/x = f(a*x)d(a*x)
dx/x = a*dx/(a*x)
dx/x = dx/x

Now try this with exp(x)

x' = a*x
f(x) = exp(x)

exp(x)dx = exp(a*x)d(a*x)
exp(x)dx = a*exp(a*x)dx

which holds only for the special case of a=1 (no scaling) but is false otherwise.

Furthermore, if you allow for a constant (actually a function of the function f) c(f) scaling factor in the definition of scale-free f(x)dx=c(f)*f(x')dx' then the function f(x)=1/x is even scale-free for power scaling of the variable

x' = a*x^p
f(x) = 1/x

c = c(f)
f(x)dx = c*f(x')dx'
dx/x = d(a*x^p)/(a*x^p)
dx/x = c*a*p*x^(p-1)dx/(a*x^p)
dx/x = c*p*dx/x
c -> 1/p
dx/x = dx/x

Thus again scale free if we allow a constant 1/p which if a function only of the function f. This is one reason why 1/x is used in Bayesian analysis as a prior distribution for an unknown scaling factor, because it is scale free an thus imposes no information about the scale. In this usage it is called Jeffrey's prior.

Where is the hidden scale in exp(x)? It is that which multiplies x. In other words the "scale" or "lifetime" or "half-life" or "time constant" etc of the exponential function which is typically written as

f(x) = exp(x/t) = exp(k*x)

which shows up all over the place in science for one. Here is the exponential probability distribution

p(x|tau) = tau*exp(x/tau)

which endows the exponential distribution with the unambiguous scale tau which is also the mean. The fact that when tau = 1 exp(x/tau) can be written as exp(x) hiding tau does not make exp(x/tau) scale-free. The scale simply is 1.

Hmm ... I hope that long wind helped somewhat.

Keith H Duggar
duggar youknowwhattodo alum.mit.edu

Paul wrote at Friday, December 30, 2005 at 9:46:00 AM PST

Ok, everyone...Its not about "knees" and the calculus of exponential functions. In critiquing Kurzweil's thesis, we need to look at the big picture he is trying to lay out.

I believe the major weakness in his proposition, and we've seen this many times before in discussions about the brain and artificial intelligence, is that there is simply too much reliance on the information processing metaphor in describing the human brain.

For proponents of singularity theory to successfully build their case, they must make a compelling argument that the brain is simply an information processing machine.

This is because "information processing" is the stuff that computer scientists study.

But scientists who study the brain know that it is an oversimplification to think of the brain in this manner. The information processing model of brain function leaves too many unanswered questions about how the brain behaves.

I predict that this will be a major stumbling block for the singularity theorists and researchers going forward.

It may be fairly easy to get to a point of "near" singularity where the human/computer can do incredibly fast arithetic calculations, navigate an obstacle course in a vehicle at very high speeds with great accuracy or function as a very efficient accountant or bookkeeper with a fantastic memory, but these types of tasks are only a small fraction of what the brain actually does.

Unfortunately, these tasks are the most easy to understand and measure.

Without an acceptance of brain function as encompassing much more than simple information processing, the singularity Kurzweil speaks of will elude us.

Paul wrote at Friday, December 30, 2005 at 9:48:00 AM PST
This comment has been removed by a blog administrator.
Anonymous wrote at Thursday, January 5, 2006 at 2:11:00 AM PST

e^2x - e^x has a definite knee at x=-ln(2) irrespective of scaling.

The question is I suppose whether e^2x-e^x counts as an exponential expression. I don't see why not

Keith H Duggar wrote at Wednesday, November 3, 2010 at 10:50:00 PM PDT

A minor correction. Exponential functions f(x)=exp(x) are not "scale-free". This is evidenced by the fact that the "knee" "moves" when you rescale. Were exp(x) scale-free nothing would move when you rescale. Mathematically "scale-free" for real functions is defined as f(x)dx=f(x')dx' for x'=a*x. In English scaling the variable scales the function in an equal and inverse manner. For example the function f(x)=1/x is scale-free

x' = a*x
f(x) = 1/x

f(x)dx = f(x')dx'
dx/x = f(a*x)d(a*x)
dx/x = a*dx/(a*x)
dx/x = dx/x

Now try this with exp(x)

x' = a*x
f(x) = exp(x)

exp(x)dx = exp(a*x)d(a*x)
exp(x)dx = a*exp(a*x)dx

which holds only for the special case of a=1 (no scaling) but is false otherwise.

Furthermore, if you allow for a constant (actually a function of the function f) c(f) scaling factor in the definition of scale-free f(x)dx=c(f)*f(x')dx' then the function f(x)=1/x is even scale-free for power scaling of the variable

x' = a*x^p
f(x) = 1/x

c = c(f)
f(x)dx = c*f(x')dx'
dx/x = d(a*x^p)/(a*x^p)
dx/x = c*a*p*x^(p-1)dx/(a*x^p)
dx/x = c*p*dx/x
c -> 1/p
dx/x = dx/x

Thus again scale free if we allow a constant 1/p which if a function only of the function f. This is one reason why 1/x is used in Bayesian analysis as a prior distribution for an unknown scaling factor, because it is scale free an thus imposes no information about the scale. In this usage it is called Jeffrey's prior.

Where is the hidden scale in exp(x)? It is that which multiplies x. In other words the "scale" or "lifetime" or "half-life" or "time constant" etc of the exponential function which is typically written as

f(x) = exp(x/t) = exp(k*x)

which shows up all over the place in science for one. Here is the exponential probability distribution

p(x|tau) = tau*exp(x/tau)

which endows the exponential distribution with the unambiguous scale tau which is also the mean. The fact that when tau = 1 exp(x/tau) can be written as exp(x) hiding tau does not make exp(x/tau) scale-free. The scale simply is 1.

Hmm ... I hope that long wind helped somewhat.

Keith H Duggar
duggar youknowwhattodo alum.mit.edu

Gordon wrote at Tuesday, November 6, 2012 at 9:04:00 AM PST

Attempting to define "knee".

The knee point in a current transformer is the essentially point at which the transformer begins to saturate. At that point the transformer output becomes nonlinear.

In computer systems, the knee point is: "Computer systems often reach a point at which the relative cost to increase some tunable parameter is no longer worth the corresponding performance benefit."
--Finding a “Kneedle” in a Haystack: Detecting Knee Points in System Behavior by Ville Satop, Jeannie Albrecht, David Irwin, and Barath Raghavan

The knee point in an ecological system might be defined as the point at which a change in a specific variable the system is irreversible from a pragmatic human perspective.

Yes, there is no knee point one can mathematically specify on an exponential curve. Yet there may be a point of 'no return' when applied to a real-world system expressing exponential behavior.

See
http://en.wikipedia.org/wiki/Exponential_growth

where the story of the Lily Pond is presented:
French children are told a story in which they imagine having a pond with water lily leaves floating on the surface. The lily population doubles in size every day and if left unchecked will smother the pond in 30 days, killing all the other living things in the water. Day after day the plant seems small and so it is decided to leave it to grow until it half-covers the pond, before cutting it back. They are then asked, on what day that will occur. This is revealed to be the 29th day, and then there will be just one day to save the pond. (From Meadows et al. 1972, p. 29 via Porritt 2005)