Right now, we are studying the various root finding algorithms in Numerical Analysis (bisection, Newton’s, etc.)

Such algorithms yield a sequence of numbers, which (hopefully) converge to a solution: .

Of course, each point in the sequence is obtained by calculations and, if there were a way to combine these points so as to obtain a sequence that converges faster (while not adding much to the computation complexity), there is some benefit. And yes, it is possible to use the sequence points, do the series manipulation and use the manipulated points in the root finding algorithm itself (e. g. Steffensen’s method).

In this post, I’ll talk about Aitken’s method and how one can cook up examples that not only show that the method can work but give the students some intuition as to why it might work.

I’ll provide just a bit of background in the event that the general reader comes across this.

Let . If we have ( is positive, of course) we say that the convergence is LINEAR if and the inequality must be strict. If then we say that convergence is quadratic (regardless of .)

To see the reason for the terminology, just multiply both sides of the “defining equation” by the denominator. In the linear case : so one can think: “get new approximation by multiplying the old approximation by some constant less than one”. For example: exhibits linear convergence to 0; that is a decent way to think about it.

Now (in the linear convergence case anyway), suppose you think of your approximation having an error that shrinks with iteration but shrinks in the following way: the n’th iteration looks like where are constants strictly between 0 and 1. Of course, as goes to infinity, approaches the limit as the error terms die away.

Aitken’s method is this: let’s denote a new sequence by . (I am not using the traditional P-hat out of laziness) Then . To see how this formula is obtained, check out these excellent course notes. Or the Wiki article is pretty good.

Look at the numerator of what is being subtracted off: if we have the terms written (very roughly) as and if, say, is much larger than , then will be closer to than is to , hence more of this error will be subtracted away.

Yes, I know that his is simple-minded, but it “gets the spirit” of the process. I’ve set up some spreadsheets with “cooked” sequences linearly converging to zero and showed how the Aitken process works there. Note: the spreadsheet round off errors start occurring at the range; you can see that here.

(click to see the larger image)

To see an abstract example where where , go to the next post in this series.