Modes of Convergence

Let Z_{1}, Z_{2}, ..., Z_{n} be a sequence of jointly distributed random variables defined on the same sample space Ω. Let Z be another random variable defined on this sample space Ω. We now consider what is meant by Z_{n} “tending to” Z. Def: (Convergence in probability or weak convergence) Z_{n} converges to Z in probability, Z_{n} ⟶^{p} Z, if for every ϵ > 0

lim_{n→∞} P (|Z_{n} − Z| > ϵ) = 0  (1)

Def: (Strong Convergence or almost sure convergence) Z_{n} converges to Z almost surely, Z_{n} ⟶^{a.s.} Z, if

P ( lim_{n→∞} |Z_{n} − Z| = 0) = 1  (2)

Def: (Convergence in Mean Square) Z_{n} converges to Z in mean square, Z_{n} ⟶^{m.s.} Z, if

lim_{n→∞} E[(Z_{n} − Z)^{2}] = 0  (3)

Thm: (Weak Law of Large Numbers) Let X_{1}, X_{2}, ..., X_{n} be a sequence of i.i.d. random variables, each having E[X_{i}] = μ and V ar(X_{i}) = σ^{2}. Let X̄_{n} = \frac{1}{n} ∑_{i=1}^{n} X_{i}. Then, for any ϵ > 0

P (|X̄_{n} − μ| > ϵ) → 0  (4)

as n → ∞. Proof: Use Chebychev’s Inequality.

So X̄_{n} ⟶^{p} μ.

Thm: (Strong Law of Large Numbers) Let X_{1}, X_{2}, ..., X_{n} be a sequence of i.i.d. random variables, each having E[X_{i}] = μ and V ar(X_{i}) = σ^{2}. Then, for any ϵ > 0

P ( lim_{n→∞} X̄_{n} = μ) = 1.  (5)

So X̄_{n} ⟶^{a.s.} μ.

Def: (Convergence in Distributon) Z_{n} converges to Z in distribution, Z_{n} ⟶^{d} Z, if,

lim_{n→∞} F_{Z_{n}}(t) = F_{Z}(t)  (6)

for all t where F_{Z}(t) is continuous. Thm: (Continuity Theorem) Let Z_{1}, Z_{2}, ..., Z_{n} be a sequence of random variables whose moment generating funtion, M_{Z_{n}}(t), existis for all |t| < ϵ, for every n. Then

F_{Z_{n}}(z) → F_{Z}(z),  (7)

Z_{n} ⟶^{d} Z, if and only if,

lim_{n→∞} M_{Z_{n}}(t) ⟶^{d} M_{Z}(t)  (8)

where M_{Z}(t) is the m.g.f. of F_{Z}(z). Thm: (Central Limit Theorem) Let X_{1}, X_{2}, ..., X_{n} be a sequence of i.i.d. random variables, each having mean μ and variance σ^{2}. The the distribution of

\frac{S_{n} − nμ}{σ\sqrt{n}},  (9)

where S_{n} = ∑_{i=1}^{n} X_{i}, tends to the standard normal distribution

lim_{n→∞} P (\frac{S_{n} − nμ}{σ\sqrt{n}} ≤ x) = Φ(x)  (10)

where −∞ < x < ∞ and Φ(x) is the c.d.f. of the standard normal distribution. Thm: Let F_{1}, F_{2}, ... be a sequence of c.d.f.’s and the corresponding p.d.f.’s be f_{1}, f_{2}, ... . If

lim_{n→∞} f_{n}(x) = f(x)  (11)

where f(x) is a p.d.f., then F_{n} converges (in distribution) to the c.d.f. F. Note: (Stirling Formula)

\frac{Γ(\frac{n+1}{2})}{Γ(\frac{n}{2})\sqrt{\frac{n}{2}}} → 1  (12)

which says Γ(n + 1) = n! ≈ \sqrt{2π} n^{n+1/2} e^{−n} = \sqrt{2π} n (\frac{n}{e})^{n}.