A statistic is said to be sufficient for
if the conditional distribution
, that is, doesn’t depend on
. Intuitively, we mean that a given statistic provides as much information as possible about
; there isn’t a way to “crunch” the observations in a way to yield more data.
Of course, this is equivalent to the likelihood function factoring into a function of and
alone and a function of the
alone.
Though the problems can be assigned to get the students to practice using the likelihood function factorization method, I think it is important to provide an example which easily shows what sort of statistic would NOT be sufficient for a parameter.
Here is one example that I found useful:
let come from a uniform distribution on
.
Now ask the class: is there any way that could be sufficient for
? It is easy to see that
will converge to 0 as
goes to infinity.
It is also easy to see that the likelihood function is where
is the standard Heavyside function on the interval
(equal to one on the support set
and zero elsewhere) and
is the
of maximum magnitude (or the
order statistic for the absolute values of the observations).
So one can easily see an example of a sufficient statistic as well.