Let me try to explain what's going on here.
First look at a pair of discrete random variables, the scores on two dice.
X and Y each take values {1,2,3,4,5,6} with probability 1/6, and all
other values with probability 0.
Equivalently, the probability mass functions f_X and f_Y are each
_____|_|_|_|_|_|_______
1 2 3 4 5 6
Let Z = X+Y.
What is P(Z=1)? As there is no roll with X+Y=1, P(Z=1) = 0.
What is P(Z=2)? There is 1 roll with X+Y=2,
P(Z=2) = 1/6 x 1/6 = 1/36
What is P(Z=5)? There are 4 rolls with Z=5, namely
(1,4),(2,3),(3,2), and (4,1).
Each of these has probability 1/36, so the sum evaluates to
1/9. However, instead of just listing the events by eye, we can
be more systematic and write the sum as
sum_{i=1}^4 P(X=i)P(Y=5-i).
How do we know to start at 1 and end at 4? We don't need to.
As for all nonpositive values and values >6, P(X=i)=P(Y=i)=0,
we can let the sum extend further and all "extra" values will
be 0:
sum_{i=-infinity}^infinity P(X=i)P(Y=5-i).
It may also be convenient to think of this as
P(Z=n) = sum _{i+j=n} P(X=i)P(Y=j)
which is the same thing under a change of variables.
In general, the PMF for X+Y looks like:
|
| | |
| | | | |
| | | | | | |
| | | | | | | | |
___________|_|_|_|_|_|_|_|_|_|_|______
2 3 4 5 6 7 8 9 101112
This is clearly the discrete form of a convolution. Either by
working from this via Riemann sums, or directly from joint
distributions, we can show that, where X and Y are continuous,
the probability density function of their sum is given by the
convolution of their individual PDF's. Thus, if U and V have PDF's f and g
respectively, and U+V=W,
f_W(z)= lim P(z < W < z+e) / e
e->0
= lim P(z < U+V < z+e ) / e
e->0
= lim I(e)/e
e->0
where I(e) is the integral of the f(x)g(y) over the region where x+y are
between z and z+e:
\|
|\
|\\
| \\
0|__\\__
0 z\\
We can set this integral up in many ways, bt if we set it up so that one
variable goes along the diagonal while the second goes across the short
strip, the second integral becomes very easy in the limit, because we can
assume that the joint density does not change in the narrow width of the
strip, and moreover e drops out. This leaves us with the single integral
along the diagonal y = z-x
f_W(z)= lim I(e)/e
e->0
= Int f(x) g(y) dx
and eliminating y we get
= Int f(x) g(z-x) dx.
Once more - this is the same convolution that turns up in optics,
differential equations, time series analysis, image processing, etc. In
each case it arises for the same reason- we are totalling up contributions
from each case in which x+y adds to the desired value.
By the way:
> The convolution or Faltung integral is defined as follows:
> y(t) = {the integral from zero to t of} f(t-lambda) times g(lambda)
d(lambda).
Not in general. The bounds are canonically from -infinity to infinity;
in cases where they are given as something else, this usually means that the
functions being convolved have restricted support (eg, are nonzero only for
t>0) and the writer is trying to avoid having to write out "and 0 for t<0"
over and over again.
-Robert Dawson
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
