Back in January 1991, I attended the American Mathematical Society meeting in San Francisco. I was there mostly to absorb the talks, and to help myself with the job search. It was there where I made my first contact with my current university.

One evening, I attended some point set talks. Since this was not my area, I kept my mouth shut; I didn’t ask many questions. In one talk, someone raised a question about a possible “bisection” type process in a certain case. I thought “that question makes no sense because there is no metric” but figured that I had missed something.

Then in the other corner of the room, a white haired old lady said something to the effect “that question makes no sense because there is no mention of a metric on this space.” π

Then I realized who this was: Mary Ellen Rudin.

Because she was known for topology and because she was a student at the University of Texas (Ph. D. in 1949), I had heard stories about her and her interactions with R. H. Bing. It turns out that her (late) husband was also famous, although in analysis (yes, he is THAT Rudin; the one associated with the analysis books that we studied for our Ph. D. comprehensive examinations.)

I also remember reading her paper on shellings; it was the first one that I tackled as a graduate student. The paper itself is terse (and very brief) but is not overly technical; it just requires time and effort to understand. You don’t have to spend half a decade learning gauge theory. π Here is a workthrough of her paper.

So, in her honor, I’ll talk about this topic. I’ll keep it as basic as possible and try not to lie too much. π

(“lie” as in, leaving out some details that a rigorous presentation would cover).

**Shellings: what are they?**

First of all, let’s discuss what topology is about. Two objects are said to be topologically equivalent if there exists a function where is a bijection (one to one and onto) and bicontinuous; that is is continuous and its inverse function is also continuous. Note: I am using “continuous” in a topological sense, but for the spaces we are talking about, the beginner can imagine that are subsets of for some and that “continuous” is the “calculus” definition of continuous. This is NOT always true in general, but it works for the kind of objects we discuss here. If two objects are topologically equivalent, we say that they are “homeomorphic” and call the “homeomorphism”.

Now geometric topology restricts itself to the study of certain nice objects, and often we require that be differentiable (“differentiable topology”) or take polygonal objects to polygonal objects (“piecewise linear topology”, denoted by p. l.). We will work with polygonal maps, but one can make the translation to differentiable maps (“smooth category”).

Now consider a disk in the plane; the calculus student might think of the unit disk: .

In the p. l. category these objects are represented by triangles. It is an exercise to see that a traditional “round” disk is homeomorphic to a triangle (with interior).

Now consider a polygonal disk which is the finite union of triangles put together “in a nice way” (e. g., two triangles touch along a common face or exactly at a vertex, every interior point has an open set in the disk that surrounds it and the boundary consists of a finite number of line segments; faces of the triangles.). This crude illustration shows a “triangulated p. l. disk”

(figures: click to see larger)

One technical problem is this: it is clear that the whole disk is topologically equivalent to any one of its triangles. However it is often useful to see WHAT homeomorphism gives us the equivalence we want and to be able to describe this homeomorphism in a finite number of steps. Homeomorphisms which can be described “triangle by triangle” are especially useful. Now look at the labeled disk on the left. It is clear that the whole disk is homeomorphic to the disk that has all of the triangles EXCEPT for the first one. True, the process of simply removing triangle 1 is not a homeomorphism, but there is a homeomorphism that starts with the whole disk and, in effects, “pushes” triangle 1 into the larger disk

In spirit, this is a bit like the one dimensional homeomorphism given by which compresses the unit square into a rectangle of half its original height.

So, in the disk on the left hand side of the first figure, we can find a series of homeomorphisms which, in step by step fashion, takes the whole disk to the triangle labeled 10. has the effect of taking the original disk and mapping it to the disk with triangle 1 missing. then starts with this modified disk and has the effect of removing triangle 2, and so on. One can check out that is a homeomorphism from the original disk to the triangle labeled 10.

So, we an think about these maps as a labeling of this triangulated p. l. disk and such a labeling is called a “shelling” of the p. l. disk with the triangle 10 saved for last. More precisely: a shelling is a labeling of a triangulated p. l. disk with triangles so that, for ALL is a p. l. disk.

Note: not just any labeling produces a shelling! For example, if you look at the first figure and look at the disk on the right, the removal of the triangle labeled “1” leaves us with an annulus (ring with a hole) and NOT a disk; hence there is no homeomorphism that does that!

So here is a question: does every 2 dimensional p. l. disk have a shelling? The answer: “yes” and not only that: we can choose, in advance, the ending triangle. There is always a path to the last triangle (though we’ve seen that we do NOT have our choice of starting triangle).

To see a proof, see Bing’s book The Geometric Topology of 3-manifolds:

This is an outline of the proof. It is fun to try it yourself.

Ok, good enough. But the follow on question:

if we now try to extend this “shellable 3-balls”: if we have a 3-dimensional triangulated (into tetrahedra) ball (homeomorphic to ) can you always shell it?

The answer is NO. That is the subject of Mary Ellen Rudin’s paper that I talked about at the start. However there are easier counterexamples, and I’ll close with my favorite, which is suggested by this figure:

Here is an idea: start with a cube (homeomorphic to a ball) and break this into a bunch of small cubes. Then start at the top of the cube and start removing a cube in the middle of the top face of the cube. Then work your way down, removing small cubes so as to leave a “knotted tunnel” from the top face of the cube to the bottom face. Of course what is left is NOT a ball due to this hole; you have “added genus”. Now we need to make this a ball, so we add the last “removed cube” back (“plugging the hole”); hence the “cube with knotted tunnel” is now a cube with an indentation in the form of an “almost hole”; hence it remains homeomorphic to a ball.

Now divide the remaining small cubes into tetrahedra to obtain a triangulated 3-ball.

It turns out that this ball cannot be shelled. Here is why:

1. If one removes the “plug” by shelling before ALL of the cubes above the plane of the plug cube have been removed (by removing tetrahedra) one changes the genus from zero (a ball) to non-zero (not a ball).

2. If one tries to remove all of the cubes in the plane above plug (by removing the tetrahedra), this has the effect of taking the knotted tunnel to the sides of the cube via a homeomorphism of space; this is impossible if the tunnel truly is knotted! (that is why having an honest to goodness knot is essential).

So the existence of knots in 3-space really complicates things. Hence “knot theory” remains an active field of research in mathematics.