Lecture 3: Fréchet Means — Consistency and Convergence Rate

How fast do empirical Fréchet means converge?

1 Learning Goals

By the end of this lecture, learners should be able to:

State the strong consistency theorem for sample Fréchet means.
Outline the proof of consistency via the $\varepsilon$-net argument on compact sets.
Define covering numbers, packing numbers, and metric entropy.
Compare the metric entropy behavior of spheres, SPD matrices, BHV tree space, and Wasserstein space.
State the convergence rate result for Hadamard-space Fréchet means and explain the role of the entropy exponent $\beta$.
Describe the phenomena of stickiness and smeariness and how they differ from classical parametric asymptotics.

2 Strong Consistency

Let $X_1, \dots, X_n$ be i.i.d. copies of a random element $X$ taking values in a metric space $(\mathcal{M}, d)$. When $\mathbb{E}\,d^2(X, o) < \infty$ for some $o \in \mathcal{M}$, recall the population and sample Fréchet functions

\[ F(x) = \mathbb{E}\,d^2(X, x), \qquad F_n(x) = \frac{1}{n}\sum_{i=1}^n d^2(X_i, x). \]

A sample Fréchet mean is any measurable selector $\hat{\mu} \in \arg\min_{x \in \mathcal{M}} F_n(x)$. We now ask: does $\hat{\mu}$ converge to the population Fréchet mean $\mu$, and if so, how fast?

The following statement specializes Schötz’s strong-law results for generalized Fréchet mean sets (Schötz 2022).

Theorem 1 Let $(\mathcal{M}, d)$ be a complete and separable metric space, and $X$ a random element such that its Fréchet function $F(x) < \infty$ for some $x \in \mathcal{M}$. Let $\hat{\mu}$ be a sample Fréchet mean of $n$ i.i.d. copies of $X$. If both $\hat{\mu}$ and $\mu$ exist and are unique, then $\hat{\mu} \to \mu$ almost surely.

Probability refresher: spaces, measurability, and modes of convergence

Probability space. A probability space is a triple $(\Omega, \mathcal{F}, \mathbb{P})$ where:

$\Omega$ is the sample space — the set of all possible outcomes.
$\mathcal{F} \subseteq 2^{\Omega}$ is a σ-algebra — a collection of subsets of $\Omega$ that is closed under complement and countable unions (hence also countable intersections). Elements of $\mathcal{F}$ are called events.
$\mathbb{P} : \mathcal{F} \to [0, 1]$ is a probability measure satisfying $\mathbb{P}(\Omega) = 1$ and countable additivity: for any disjoint sequence $(A_n) \subset \mathcal{F}$, \[ \mathbb{P}\!\left(\bigcup_{n=1}^{\infty} A_n\right) = \sum_{n=1}^{\infty} \mathbb{P}(A_n). \]

Measurable maps and random variables. Let $(\Omega, \mathcal{F})$ and $(\mathcal{M}, \mathcal{B})$ be measurable spaces (sets equipped with σ-algebras). A map $X : \Omega \to \mathcal{M}$ is measurable (or $\mathcal{F}/\mathcal{B}$-measurable) if for every $B \in \mathcal{B}$,

\[ X^{-1}(B) = \{\omega \in \Omega : X(\omega) \in B\} \in \mathcal{F}. \]

A random variable is simply a measurable map defined on a probability space. When $\mathcal{M}$ is a general metric space, the σ-algebra $\mathcal{B}$ is usually the Borel σ-algebra — the smallest σ-algebra containing all open sets. A map $X : \Omega \to \mathcal{M}$ is then a random element (or random variable when $\mathcal{M} = \mathbb{R}$, random vector when $\mathcal{M} = \mathbb{R}^p$).

Throughout these lectures, we speak of a random element $X$ on $\mathcal{M}$: this always means a measurable map from some underlying probability space $(\Omega, \mathcal{F}, \mathbb{P})$ into $\mathcal{M}$ equipped with its Borel σ-algebra.

Modes of convergence. For a sequence $(X_n)$ of random elements and a limit $X$ (all taking values in $\mathcal{M}$ with metric $d$):

Almost sure convergence $X_n \xrightarrow{\text{a.s.}} X$: \[ \mathbb{P}\!\left(\lim_{n \to \infty} d(X_n, X) = 0\right) = 1, \] i.e., pointwise convergence on a set of probability one. Almost sure convergence implies convergence in probability.
Convergence in probability $X_n \xrightarrow{p} X$: \[ \lim_{n \to \infty} \mathbb{P}\bigl(d(X_n, X) > \varepsilon\bigr) = 0 \quad\text{for every }\varepsilon > 0. \]
Convergence in mean (or $L^p$-convergence) $X_n \xrightarrow{L^p} X$: for $p \ge 1$, \[ \lim_{n \to \infty} \mathbb{E}\bigl[d^p(X_n, X)\bigr] = 0. \] When $p = 2$ this is called convergence in quadratic mean. $L^p$ convergence implies convergence in probability (by Markov’s inequality).

In a Hadamard space, Proposition 4.3 of Sturm (2003) yields uniqueness of both population and sample Fréchet means. Thus, strong consistency holds in separable Hadamard spaces.

For spaces with the Heine–Borel property, we have the following earlier result.

Theorem 2 (Bhattacharya & Patrangenaru 2003, Theorem 2.3) Let $Q$ be a probability measure on a metric space $(\mathcal{M}, d)$ such that every closed bounded subset of $\mathcal{M}$ is compact. Assume the Fréchet function

\[ F(x) = \int d^2(x, y)\,Q(dy) \]

is finite on $\mathcal{M}$. Then:

Given any $\varepsilon > 0$, there exists a $\mathbb{P}$-null set $N$ such that for every $\omega \in N^c$ there is an integer $n(\omega) < \infty$ for which the sample Fréchet mean set $\mathscr{E}_n$ is contained in the $\varepsilon$-neighborhood of the Fréchet mean set of $Q$ for all $n \ge n(\omega)$.
If the Fréchet mean of $Q$ is unique, then every measurable choice from the sample Fréchet mean set $\mathscr{E}_n$ is a strongly consistent estimator.

Proof. Let $X_1,X_2,\ldots$ be i.i.d. random elements with law $Q$, defined on a probability space $(\Omega,\mathcal F,\mathbb P)$. Write

\[ F_n(x)=\frac1n\sum_{i=1}^n d^2(x,X_i),\qquad m=\inf_{x\in\mathcal M}F(x), \]

and define the population and sample mean sets by

\[ \mathscr E=\{x\in\mathcal M:F(x)=m\}, \qquad \mathscr E_n(\omega)=\operatorname*{argmin}_{x\in\mathcal M}F_n(x,\omega). \]

We prove four claims.

Step 1: The population and sample mean sets are nonempty and compact.

The hypothesis says that $F$ is finite on $\mathcal M$. By the Heine–Borel existence theorem from Lecture 2, $F$ attains its infimum and $\mathscr E$ is a nonempty compact set. In particular, $m<\infty$. Similar arguments show that $\mathscr E_n(\omega)$ is nonempty and compact for every $n$ and $\omega$.

Step 2: Almost surely, all sample mean sets eventually lie in one deterministic compact set.

Choose $p\in\mathscr E$ and any reference point $o\in\mathcal M$, and set

\[ R=\sqrt{m+1}+\sqrt{F(o)}+1, \qquad K=\overline B(o,R)=\{x\in\mathcal M:d(x,o)\le R\}. \]

The set $K$ is closed and bounded, hence compact by the Heine–Borel property. We first note that $p\in K$. Indeed, the triangle inequality and Minkowski’s inequality in $L^2(Q)$ give

\[ \begin{aligned} d(p,o) &=\left(\int d^2(p,o)\,Q(dy)\right)^{1/2}\\ &\le \left(\int d^2(p,y)\,Q(dy)\right)^{1/2} + \left(\int d^2(y,o)\,Q(dy)\right)^{1/2}\\ &=\sqrt{F(p)}+\sqrt{F(o)} =\sqrt m+\sqrt{F(o)} <R. \end{aligned} \]

Now fix $x\in\mathscr E_n(\omega)$. Since $x$ minimizes $F_n(\,\cdot\,,\omega)$,

\[ F_n(x)\le F_n(p). \]

For each $i$, the triangle inequality gives $d(x,o)\le d(x,X_i)+d(X_i,o)$. Taking the empirical $L^2$ norm and applying Minkowski’s inequality yields

\[ \begin{aligned} d(x,o) &=\left(\frac1n\sum_{i=1}^n d^2(x,o)\right)^{1/2}\\ &\le \left(\frac1n\sum_{i=1}^n d^2(x,X_i)\right)^{1/2} + \left(\frac1n\sum_{i=1}^n d^2(o,X_i)\right)^{1/2}\\ &=\sqrt{F_n(x)} + \left(\frac1n\sum_{i=1}^n d^2(o,X_i)\right)^{1/2}\\ &\le \sqrt{F_n(p)} + \left(\frac1n\sum_{i=1}^n d^2(o,X_i)\right)^{1/2}. \end{aligned} \]

Both random variables $d^2(p,X_1)$ and $d^2(o,X_1)$ are integrable because their expectations are $F(p)=m$ and $F(o)$, respectively. Therefore, by the strong law of large numbers, on an event $\Omega_1$ with $\mathbb P(\Omega_1)=1$,

\[ F_n(p)\longrightarrow m, \qquad \frac1n\sum_{i=1}^n d^2(o,X_i)\longrightarrow F(o). \]

Consequently, for every $\omega\in\Omega_1$, both

\[ F_n(p)<m+1 \quad\text{and}\quad \left(\frac1n\sum_{i=1}^n d^2(o,X_i)\right)^{1/2} <\sqrt{F(o)}+1 \]

hold for all sufficiently large $n$. Thus, for every $\omega\in\Omega_1$,

\[ \mathscr E_n(\omega)\subset K \]

for all sufficiently large $n$.

Step 3: $F_n$ converges to $F$ uniformly on $K$, almost surely.

First we obtain a common Lipschitz bound. For $x,z\in K$ and $y\in\mathcal M$,

\[ \begin{aligned} \left|d^2(x,y)-d^2(z,y)\right| &=|d(x,y)-d(z,y)|\{d(x,y)+d(z,y)\}\\ &\le d(x,z)\{d(x,y)+d(z,y)\}\\ &\le \{2R+2d(o,y)\}\,d(x,z). \end{aligned} \]

The last inequality uses $d(x,y)\le d(x,o)+d(o,y)\le R+d(o,y)$, and similarly for $z$. Averaging over the sample gives

\[ |F_n(x)-F_n(z)|\le L_n\,d(x,z), \qquad L_n:=2R+\frac2n\sum_{i=1}^n d(o,X_i). \]

Also, taking expectations gives

\[ |F(x)-F(z)|\le L\,d(x,z), \qquad L:=2R+2\,\mathbb E\,d(o,X_1). \]

The constant $L$ is finite because Cauchy–Schwarz implies

\[ \mathbb E\,d(o,X_1)\le \{\mathbb E\,d^2(o,X_1)\}^{1/2} =\sqrt{F(o)}<\infty. \]

The strong law applied to $d(o,X_i)$ shows that $L_n\to L$ almost surely. In particular, with the deterministic constant $C:=L+1$, there is an event of probability one on which $L_n\le C$ eventually.

We now give the finite-net argument explicitly. For each $k\in\mathbb N$, choose $\rho_k>0$ so small that

\[ (C+L)\rho_k<\frac{1}{2k}. \]

Because $K$ is compact, it is totally bounded. Hence there is a finite $\rho_k$-net

\[ \mathcal Z_k=\{z_{k,1},\ldots,z_{k,N_k}\}\subset K; \]

that is, every $x\in K$ satisfies $d(x,z_{k,j})<\rho_k$ for at least one $j$. The set

\[ \mathcal Z:=\bigcup_{k=1}^{\infty}\mathcal Z_k \]

is countable. For each $z\in\mathcal Z$, the random variable $d^2(z,X_1)$ is integrable because $F(z)<\infty$. Taking the countable intersection of the corresponding strong-law events, we obtain an event $\Omega_2$ of probability one on which

\[ F_n(z)\longrightarrow F(z) \qquad\text{for every }z\in\mathcal Z, \]

and on which $L_n\le C$ eventually.

Fix $\omega\in\Omega_2$ and $k\in\mathbb N$. Since $\mathcal Z_k$ is finite, there is an integer $N_k(\omega)$ such that, for $n\ge N_k(\omega)$,

\[ \max_{1\le j\le N_k}|F_n(z_{k,j})-F(z_{k,j})|<\frac{1}{2k} \quad\text{and}\quad L_n\le C. \]

For any $x\in K$, select $z_{k,j}\in\mathcal Z_k$ with $d(x,z_{k,j})<\rho_k$. Then

\[ \begin{aligned} |F_n(x)-F(x)| &\le |F_n(x)-F_n(z_{k,j})| +|F_n(z_{k,j})-F(z_{k,j})|\\ &\qquad +|F(z_{k,j})-F(x)|\\ &\le L_n\rho_k+\frac{1}{2k}+L\rho_k\\ &<\frac1k. \end{aligned} \]

This bound holds for every $x\in K$, so

\[ \sup_{x\in K}|F_n(x)-F(x)|<\frac1k \]

for all $n\ge N_k(\omega)$. Since $k$ is arbitrary,

\[ \sup_{x\in K}|F_n(x)-F(x)|\longrightarrow0 \]

on $\Omega_2$.

Step 4: A positive separation gap excludes minimizers outside the $\varepsilon$-neighborhood.

Fix $\varepsilon>0$ and define

\[ d(x,\mathscr E):=\inf_{p\in\mathscr E}d(x,p), \qquad \mathscr E^\varepsilon =\{x\in\mathcal M:d(x,\mathscr E)<\varepsilon\}, \qquad A_\varepsilon=K\setminus\mathscr E^\varepsilon. \]

Because $\mathscr E$ is nonempty, the map $x\mapsto d(x,\mathscr E)$ is $1$-Lipschitz and therefore continuous. Thus $\mathscr E^\varepsilon$ is open, so $A_\varepsilon$ is a closed subset of the compact set $K$ and is compact.

If $A_\varepsilon=\varnothing$, Step 2 already gives $\mathscr E_n\subset K\subset\mathscr E^\varepsilon$ eventually, and there is nothing more to prove. Suppose therefore that $A_\varepsilon\ne\varnothing$. Since $F$ is continuous, it attains its minimum on $A_\varepsilon$. Moreover, $A_\varepsilon\cap\mathscr E=\varnothing$: every point of $\mathscr E$ has distance $0<\varepsilon$ from $\mathscr E$. Hence the minimum over $A_\varepsilon$ is strictly larger than the global minimum $m$. In other words,

\[ \delta_\varepsilon :=\min_{x\in A_\varepsilon}F(x)-m >0. \]

Let $\Omega_*=\Omega_1\cap\Omega_2$, which has probability one. Fix $\omega\in\Omega_*$. By Steps 2 and 3, there is an integer $N(\omega,\varepsilon)$ such that, for all $n\ge N(\omega,\varepsilon)$,

\[ \mathscr E_n(\omega)\subset K \qquad\text{and}\qquad \sup_{x\in K}|F_n(x,\omega)-F(x)|<\frac{\delta_\varepsilon}{3}. \]

For every $x\in A_\varepsilon$, it follows that

\[ F_n(x) \ge F(x)-\frac{\delta_\varepsilon}{3} \ge m+\frac{2\delta_\varepsilon}{3}. \]

On the other hand, the fixed population minimizer $p$ belongs to $K$, and therefore

\[ F_n(p) \le F(p)+\frac{\delta_\varepsilon}{3} =m+\frac{\delta_\varepsilon}{3}. \]

Thus $F_n(x)>F_n(p)$ for every $x\in A_\varepsilon$. No point of $A_\varepsilon$ can minimize $F_n$. Since every sample minimizer is already in $K$, we conclude that

\[ \mathscr E_n(\omega)\subset K\setminus A_\varepsilon =K\cap\mathscr E^\varepsilon \subset\mathscr E^\varepsilon \]

for every $n\ge N(\omega,\varepsilon)$. Taking the null set $N=\Omega\setminus\Omega_*$ proves part (a).

Finally, suppose the population Fréchet mean is unique, so $\mathscr E=\{\mu\}$. Let $\hat\mu_n$ be any measurable choice from $\mathscr E_n$. Apply part (a) with the countable sequence $\varepsilon_k=1/k$. Outside the union of the corresponding null sets—which is still a null set—for every $k$,

\[ d(\hat\mu_n,\mu)<\frac1k \]

for all sufficiently large $n$. This is exactly $d(\hat\mu_n,\mu)\to0$ almost surely, or $\hat\mu_n\to\mu$ almost surely. This proves part (b).

The $\varepsilon$-neighborhood argument: the compact set $K$ (blue boundary) contains the sample Fréchet means eventually. The $\varepsilon$-neighborhood $\mathscr{E}^{\varepsilon}$ of the population Fréchet mean set $\mathscr{E}$ (green) is shown. The region $A_{\varepsilon} = K \setminus \mathscr{E}^{\varepsilon}$ has Fréchet function values strictly above the minimum by a gap $\delta_{\varepsilon}$, so the sample minimizer cannot land there for large $n$.

3 Metric Entropy

To study the convergence rate of the sample Fréchet mean, we need to quantify the complexity of a metric space. This is done via metric entropy.

Definition 1 Let $(\mathcal{M}, d)$ be a metric space, let $A \subset \mathcal{M}$, and let $\varepsilon > 0$.

An $\varepsilon$-cover of $A$ is a finite collection of points $x_1, \ldots, x_m \in \mathcal{M}$ such that $A \subset \bigcup_{j=1}^m B(x_j, \varepsilon)$. The covering number is

\[ N(\varepsilon, A, d) = \inf\left\{m \in \mathbb{N} : A \subset \bigcup_{j=1}^m B(x_j, \varepsilon)\right\}. \]

An $\varepsilon$-packing of $A$ is a subset $\{x_1, \ldots, x_m\} \subset A$ such that $d(x_i, x_j) > \varepsilon$ for all $i \neq j$. Its maximal cardinality is the packing number $M(\varepsilon, A, d)$.
The metric entropy of $A$ at scale $\varepsilon$ is

\[ H(\varepsilon, A, d) = \log N(\varepsilon, A, d). \]

Interactive Demo: ε-Covers and ε-Packings

ε = 0.120

Displayed cover size0

Displayed packing size0

ε-cover of rectangle A

ε-packing in rectangle A

Because $M(2\varepsilon, A, d) \le N(\varepsilon, A, d) \le M(\varepsilon, A, d)$, covering and packing formulations are equivalent up to constants.

The quantity $H(\varepsilon, \mathcal{M}, d)$ is the global metric entropy. For a point $\omega \in \mathcal{M}$ and radius $\delta > 0$, $H(\varepsilon, B_\delta(\omega), d)$ is the local metric entropy. Finite-dimensional spaces typically have logarithmic metric entropy in $1/\varepsilon$, while genuinely infinite-dimensional spaces often have polynomial or faster entropy growth.

Example 1 The spaces from the introduction have the following entropy behavior as $\varepsilon \downarrow 0$.

Sphere $\mathbb{S}^{p-1}$ (geodesic metric):

\[ H(\varepsilon, \mathbb{S}^{p-1}, d_{\mathbb{S}}) = (p-1)\log(1/\varepsilon) + O(1), \]

and for geodesic balls, $H(\varepsilon, B_\delta(x), d_{\mathbb{S}}) = (p-1)\log(\delta/\varepsilon) + O(1)$ for $0 < \varepsilon < \delta < \pi$ (Rogers 1963).

SPD matrices $\mathcal{S}_{++}^{m}$ (log-Euclidean metric): the matrix logarithm identifies $\mathcal{S}_{++}^{m}$ isometrically with symmetric matrices of dimension $q = m(m+1)/2$. Hence

\[ H(\varepsilon, B_\delta(A), d_{\mathrm{LE}}) = q\log(\delta/\varepsilon) + O(1) \]

on bounded metric balls. The affine-invariant metric has the same local finite-dimensional order (Arsigny et al. 2006, 2007; Pennec et al. 2006).

BHV tree space (fixed leaf set of size $m$): a finite union of Euclidean orthants of dimension $p_m = m-2$ (rooted case). Every bounded subset $A$ satisfies

\[ H(\varepsilon, A, d_{\mathrm{BHV}}) \le p_m\log(1/\varepsilon) + O_A(1). \]

Since BHV tree space is unbounded, its global entropy is infinite (Billera et al. 2001).

Wasserstein space $(\mathcal{P}_2([0,1]), W_2)$: the quantile map identifies this space with monotone functions in $L^2(0,1)$. The entropy is polynomial:

\[ H(\varepsilon, \mathcal{P}_2([0,1]), W_2) \asymp \varepsilon^{-1}, \]

consistent with bounds for monotone classes (Gao and Wellner 2007; Blei et al. 2007). In higher dimensions the entropy is even larger: for $T = [0,1]^p$,

\[ H(\varepsilon, \mathcal{P}([0,1]^p), W_1) \gtrsim \varepsilon^{-p}, \]

which shows that Wasserstein spaces are infinite-dimensional from the entropy viewpoint (Boedihardjo et al. 2024).

4 Convergence Rate in Hadamard Spaces

The following result applies the theory of Schötz (2019) to Hadamard spaces.

Proposition 1 Let $X$ be a random element on a Hadamard space $(\mathcal{M}, d)$ such that $\mathbb{E}\,d^2(X, x) < \infty$ for some $x \in \mathcal{M}$. Assume that there exist constants $c_e > 0$ and $\beta > 0$ such that, for all $\delta > 0$ and all $0 < r \le \delta$,

\[ \sqrt{\log N\bigl(\varepsilon, B_\delta(\mu), d\bigr)} \le c_e\left(\frac{\delta}{\varepsilon}\right)^{\beta}. \]

Define

\[ \eta_{\beta,n} = \begin{cases} n^{-1/2}, & \beta < 1, \\ n^{-1/2}\log(n+1), & \beta = 1, \\ n^{-1/(2\beta)}, & \beta > 1. \end{cases} \]

Then, for every $s > 0$,

\[ \mathbb{P}\bigl\{\eta_{\beta,n}^{-1}\,d(\hat{\mu}, \mu) \ge s\bigr\} \le c\,s^{-2}, \]

where $c > 0$ depends only on $\beta$, $c_e$, and $\mathbb{E}\,d^2(X, X')$ for an independent copy $X'$ of $X$. In particular,

\[ d(\hat{\mu}, \mu) = O_P(\eta_{\beta,n}). \]

Thus the distance rate is parametric, $n^{-1/2}$, when $\beta < 1$, acquires a logarithmic factor when $\beta = 1$, and is slower when $\beta > 1$.

Interpretation of the Entropy Exponent

$\beta = 0$ (finite-dimensional spaces like $\mathbb{R}^p$, sphere, SPD, BHV): parametric rate $n^{-1/2}$.
$\beta = 1/2$ (1D Wasserstein space $W_2$ on $[0,1]$): rate $n^{-1/2}$ since $\beta < 1$.
$\beta > 1$ (higher-dimensional Wasserstein spaces): slower than parametric rate.

Note that the above only provides a general upper bound on the convergence rate. Faster rates are possible in some cases, depending on the geometry of the space and the distribution of the data. For example, for spaces with nonnegative curvature (including high-dimensional Wasserstein spaces), Le Gouic et al. (2023) show that, under some conditions, $\mathbb{E}\,d^2(\hat{\mu}, \mu) = O(n^{-1})$, again yielding the parametric rate $d(\hat{\mu}, \mu) = O_P(n^{-1/2})$.

Key Insight

The entropy exponent $\beta$ determines the convergence rate. Finite-dimensional spaces have $\beta = 0$ (logarithmic entropy), yielding the parametric rate $n^{-1/2}$. Infinite-dimensional spaces with $\beta > 1$ yield slower rates.

5 Beyond Classical Asymptotics: Stickiness and Smeariness

In non-Euclidean spaces, sample Fréchet means can exhibit behavior that has no Euclidean counterpart. Two important phenomena are stickiness and smeariness.

5.1 Stickiness

Stickiness arises in singular stratified spaces (such as BHV tree space or open books). When the population Fréchet mean lies on a lower-dimensional stratum, the empirical Fréchet means can get trapped there after a finite random sample size, and small perturbations of the underlying law need not move the mean off that stratum.

The open-book analysis of Hotz et al. (2013) distinguishes sticky, partly sticky, and nonsticky cases. In the sticky case, the sample means eventually lie almost surely on the codimension-one spine, and the limiting Gaussian law is supported on that spine.

More recently, Lammers et al. (2023) systematize the phenomenon by distinguishing sample, perturbation, and topological stickiness and studying their relations in $\mathrm{CAT}(\kappa)$ spaces. In BHV phylogenetic tree space, Lammers et al. (2024) explain that stickiness typically manifests itself through unresolved or star trees and can create computational bottlenecks.

5.2 Smeariness

Smeariness occurs on smooth positively curved manifolds (not singular spaces). In smeary models the empirical Fréchet mean need not obey a classical root-$n$ central limit theorem: depending on the local geometry and the behavior of the distribution near the cut locus, the fluctuations can occur on a slower scale and may have a non-Gaussian limit.

Key developments:

On the circle, Hotz and Huckemann (2015) showed that the convergence rate can become arbitrarily slow as the distribution approaches uniformity near the antipode.
On higher-dimensional spheres, Eltzner and Huckemann (2019) established a smeary central limit theorem.
Eltzner (2022) showed that such behavior can arise purely from positive curvature, even without mass concentrated at the antipode — motivating the term geometrical smeariness.
Eltzner et al. (2021) and Tran et al. (2021) show that finite-sample smeariness is widespread near smeary models, leading to variance inflation and making naive tangent-space Gaussian approximations unreliable at moderate sample sizes.

Takeaway

Both stickiness (from singularities) and smeariness (from positive curvature) show that the geometry of the data space fundamentally shapes the statistical behavior of Fréchet means. Classical CLT intuition from $\mathbb{R}^p$ does not automatically carry over.

6 Key Takeaways

Sample Fréchet means are strongly consistent under mild conditions (complete, separable metric space).
The proof uses an $\varepsilon$-net argument: uniform convergence of $F_n$ to $F$ on a compact set containing the sample minimizers.
Metric entropy quantifies the complexity of a metric space and governs convergence rates.
Finite-dimensional spaces (sphere, SPD, BHV) have logarithmic entropy → parametric $n^{-1/2}$ rate.
Wasserstein spaces have polynomial entropy → potentially slower rates in higher dimensions.
Stickiness (from singularities) and smeariness (from positive curvature) are non-classical phenomena that invalidate standard CLT intuition.

7 Exercises

Covering vs. packing: Prove that $M(2\varepsilon, A, d) \le N(\varepsilon, A, d) \le M(\varepsilon, A, d)$. 📝 Show Solution
Entropy of the sphere: Derive the leading-order term $(p-1)\log(1/\varepsilon)$ for the metric entropy of $\mathbb{S}^{p-1}$. (Hint: the sphere is a $(p-1)$-dimensional manifold; compare volumes of balls.) 📝 Show Solution
Entropy of $\mathbb{R}^p$: Show that for a bounded subset $A \subset \mathbb{R}^p$, $H(\varepsilon, A, d) = p\log(1/\varepsilon) + O(1)$. What is $\beta$ in this case? 📝 Show Solution

8 Further Reading

Schötz (2019) — General theory of convergence rates for Fréchet means via metric entropy.
Schötz (2022) — Strong laws for generalized Fréchet mean sets.
Bhattacharya and Patrangenaru (2003) — Systematic consistency results for intrinsic means on manifolds.
Hotz et al. (2013) — Foundational analysis of stickiness in open-book spaces.
Lammers et al. (2023) — Systematization of stickiness phenomena in $\mathrm{CAT}(\kappa)$ spaces.
Eltzner and Huckemann (2019) — The smeary central limit theorem on spheres.
Le Gouic et al. (2023) — Fast rates for Fréchet means in nonnegatively curved spaces.

9 Self-Assessment Quiz

Test your understanding of this lecture with the interactive MCQ quiz:

👉 Lecture 3 Quiz — 10 Multiple-Choice Questions

References

Arsigny, V., P. Fillard, X. Pennec, and N. Ayache. 2006. “Log-Euclidean Metrics for Fast and Simple Calculus on Diffusion Tensors.” Magnetic Resonance in Medicine 56 (2): 411–21.

Arsigny, V., P. Fillard, X. Pennec, and N. Ayache. 2007. “Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices.” SIAM Journal on Matrix Analysis and Applications 29 (1): 328–47. https://doi.org/10.1137/050637996.

Bhattacharya, Rabi, and Vic Patrangenaru. 2003. “Large Sample Theory of Intrinsic and Extrinsic Sample Means on Manifolds. I.” The Annals of Statistics 31 (1): 1–29. https://doi.org/10.1214/aos/1046294456.

Billera, Louis J., Susan P. Holmes, and Karen Vogtmann. 2001. “Geometry of the Space of Phylogenetic Trees.” Advances in Applied Mathematics 27 (4): 733–67.

Blei, Ron, Fang Gao, and Wenbo V. Li. 2007. “Metric Entropy of High Dimensional Distributions.” Proceedings of the American Mathematical Society 135 (12): 4009–18. https://doi.org/10.1090/S0002-9939-07-08935-6.

Boedihardjo, Horatio P., Thomas Strohmer, and Roman Vershynin. 2024. “Private Measures, Random Walks, and Synthetic Data.” Probability Theory and Related Fields 190 (1–2): 471–543. https://doi.org/10.1007/s00440-024-01279-z.

Eltzner, Benjamin. 2022. “Geometrical Smeariness — a New Phenomenon of Fréchet Means.” Bernoulli 28 (1): 239–54. https://doi.org/10.3150/21-BEJ1340.

Eltzner, Benjamin, and Stephan F. Huckemann. 2019. “A Smeary Central Limit Theorem for Manifolds with Application to High-Dimensional Spheres.” The Annals of Statistics 47 (6): 3360–81. https://doi.org/10.1214/18-AOS1781.

Eltzner, Benjamin, Shayan Hundrieser, and Stephan F. Huckemann. 2021. “Finite Sample Smeariness on Spheres.” Geometric Science of Information, Lecture notes in computer science, vol. 12829: 12–19. https://doi.org/10.1007/978-3-030-80209-7_2.

Gao, Fang, and Jon A. Wellner. 2007. “Entropy Estimates for High-Dimensional Monotonic Functions.” Journal of Multivariate Analysis 98 (9): 1751–64. https://doi.org/10.1016/j.jmva.2006.09.003.

Hotz, Thomas, and Stephan Huckemann. 2015. “Intrinsic Means on the Circle: Uniqueness, Locus and Asymptotics.” Annals of the Institute of Statistical Mathematics 67 (1): 177–93. https://doi.org/10.1007/s10463-013-0444-7.

Hotz, Thomas, Stephan Huckemann, Huynh Le, et al. 2013. “Sticky Central Limit Theorems on Open Books.” The Annals of Applied Probability 23 (6): 2238–58. https://doi.org/10.1214/13-AAP918.

Lammers, Lars, T. M. W. Nye, and Stephan Huckemann. 2024. Statistics for Phylogenetic Trees in the Presence of Stickiness.

Lammers, Lars, Do Tran Van, and Stephan Huckemann. 2023. Sticky Flavors.

Le Gouic, Thierry, Quentin Paris, Philippe Rigollet, and Alexander J. Stromme. 2023. “Fast Convergence of Empirical Barycenters in Alexandrov Spaces and the Wasserstein Space.” Journal of the European Mathematical Society 25 (6): 2229–50. https://doi.org/10.4171/JEMS/1234.

Pennec, X., P. Fillard, and N. Ayache. 2006. “A Riemannian Framework for Tensor Computing.” International Journal of Computer Vision 66 (1): 41–66. https://doi.org/10.1007/s11263-005-3222-z.

Rogers, C. A. 1963. “Covering a Sphere with Spheres.” Mathematika 10 (2): 157–64. https://doi.org/10.1112/S0025579300004083.

Schötz, Christof. 2019. “Convergence Rates for the Generalized Fréchet Mean via the Quadruple Inequality.” Electronic Journal of Statistics 13 (2): 4280–345. https://doi.org/10.1214/19-EJS1618.

Schötz, Christof. 2022. “Strong Laws of Large Numbers for Generalizations of Fréchet Mean Sets.” Statistics 56 (1): 34–52. https://doi.org/10.1080/02331888.2021.2009868.

Sturm, Karl-Theodor. 2003. “Probability Measures on Metric Spaces of Nonpositive Curvature.” In Heat Kernels and Analysis on Manifolds, Graphs, and Metric Spaces, vol. 338. Contemporary Mathematics. American Mathematical Society. https://doi.org/10.1090/conm/338/06080.

Tran, Do, Benjamin Eltzner, and Stephan F. Huckemann. 2021. “Smeariness Begets Finite Sample Smeariness.” Geometric Science of Information, Lecture notes in computer science, vol. 12829: 29–36. https://doi.org/10.1007/978-3-030-80209-7_4.

--- title: "Lecture 3: Fréchet Means — Consistency and Convergence Rate" subtitle: "How fast do empirical Fréchet means converge?" format: html: code-fold: true code-tools: true code-copy: true pdf: documentclass: scrartcl pdf-engine: xelatex toc: true number-sections: true geometry: - margin=1in colorlinks: true bibliography: source/ref.bib --- ## Learning Goals By the end of this lecture, learners should be able to: - State the strong consistency theorem for sample Fréchet means. - Outline the proof of consistency via the $\varepsilon$-net argument on compact sets. - Define covering numbers, packing numbers, and metric entropy. - Compare the metric entropy behavior of spheres, SPD matrices, BHV tree space, and Wasserstein space. - State the convergence rate result for Hadamard-space Fréchet means and explain the role of the entropy exponent $\beta$. - Describe the phenomena of **stickiness** and **smeariness** and how they differ from classical parametric asymptotics. ## Strong Consistency Let $X_1, \dots, X_n$ be i.i.d. copies of a random element $X$ taking values in a metric space $(\mathcal{M}, d)$. When $\mathbb{E}\,d^2(X, o) < \infty$ for some $o \in \mathcal{M}$, recall the population and sample Fréchet functions $$ F(x) = \mathbb{E}\,d^2(X, x), \qquad F_n(x) = \frac{1}{n}\sum_{i=1}^n d^2(X_i, x). $$ A sample Fréchet mean is any measurable selector $\hat{\mu} \in \arg\min_{x \in \mathcal{M}} F_n(x)$. We now ask: does $\hat{\mu}$ converge to the population Fréchet mean $\mu$, and if so, how fast? The following statement specializes Schötz's strong-law results for generalized Fréchet mean sets [@Schotz2022]. ::: {#thm-schoetz-consistency .theorem title="Strong consistency of sample Fréchet means, Schötz"} Let $(\mathcal{M}, d)$ be a complete and separable metric space, and $X$ a random element such that its Fréchet function $F(x) < \infty$ for some $x \in \mathcal{M}$. Let $\hat{\mu}$ be a sample Fréchet mean of $n$ i.i.d. copies of $X$. If both $\hat{\mu}$ and $\mu$ exist and are unique, then $\hat{\mu} \to \mu$ almost surely. ::: ::: {.callout-note title="Probability refresher: spaces, measurability, and modes of convergence" collapse="true"} **Probability space.** A **probability space** is a triple $(\Omega, \mathcal{F}, \mathbb{P})$ where: - $\Omega$ is the **sample space** — the set of all possible outcomes. - $\mathcal{F} \subseteq 2^{\Omega}$ is a **σ-algebra** — a collection of subsets of $\Omega$ that is closed under complement and countable unions (hence also countable intersections). Elements of $\mathcal{F}$ are called **events**. - $\mathbb{P} : \mathcal{F} \to [0, 1]$ is a **probability measure** satisfying $\mathbb{P}(\Omega) = 1$ and **countable additivity**: for any disjoint sequence $(A_n) \subset \mathcal{F}$, $$ \mathbb{P}\!\left(\bigcup_{n=1}^{\infty} A_n\right) = \sum_{n=1}^{\infty} \mathbb{P}(A_n). $$ **Measurable maps and random variables.** Let $(\Omega, \mathcal{F})$ and $(\mathcal{M}, \mathcal{B})$ be measurable spaces (sets equipped with σ-algebras). A map $X : \Omega \to \mathcal{M}$ is **measurable** (or $\mathcal{F}/\mathcal{B}$-measurable) if for every $B \in \mathcal{B}$, $$ X^{-1}(B) = \{\omega \in \Omega : X(\omega) \in B\} \in \mathcal{F}. $$ A **random variable** is simply a measurable map defined on a probability space. When $\mathcal{M}$ is a general metric space, the σ-algebra $\mathcal{B}$ is usually the **Borel σ-algebra** — the smallest σ-algebra containing all open sets. A map $X : \Omega \to \mathcal{M}$ is then a **random element** (or random variable when $\mathcal{M} = \mathbb{R}$, random vector when $\mathcal{M} = \mathbb{R}^p$). Throughout these lectures, we speak of a *random element $X$ on $\mathcal{M}$*: this always means a measurable map from some underlying probability space $(\Omega, \mathcal{F}, \mathbb{P})$ into $\mathcal{M}$ equipped with its Borel σ-algebra. **Modes of convergence.** For a sequence $(X_n)$ of random elements and a limit $X$ (all taking values in $\mathcal{M}$ with metric $d$): - **Almost sure convergence** $X_n \xrightarrow{\text{a.s.}} X$: $$ \mathbb{P}\!\left(\lim_{n \to \infty} d(X_n, X) = 0\right) = 1, $$ i.e., pointwise convergence on a set of probability one. Almost sure convergence implies convergence in probability. - **Convergence in probability** $X_n \xrightarrow{p} X$: $$ \lim_{n \to \infty} \mathbb{P}\bigl(d(X_n, X) > \varepsilon\bigr) = 0 \quad\text{for every }\varepsilon > 0. $$ - **Convergence in mean** (or $L^p$-convergence) $X_n \xrightarrow{L^p} X$: for $p \ge 1$, $$ \lim_{n \to \infty} \mathbb{E}\bigl[d^p(X_n, X)\bigr] = 0. $$ When $p = 2$ this is called convergence in quadratic mean. $L^p$ convergence implies convergence in probability (by Markov's inequality). ::: In a Hadamard space, Proposition 4.3 of @Sturm2003 yields uniqueness of both population and sample Fréchet means. Thus, strong consistency holds in separable Hadamard spaces. For spaces with the Heine–Borel property, we have the following earlier result. ::: {#thm-bp2003 .theorem title="Bhattacharya & Patrangenaru 2003, Theorem 2.3"} (Bhattacharya & Patrangenaru 2003, Theorem 2.3) Let $Q$ be a probability measure on a metric space $(\mathcal{M}, d)$ such that every closed bounded subset of $\mathcal{M}$ is compact. Assume the Fréchet function $$ F(x) = \int d^2(x, y)\,Q(dy) $$ is finite on $\mathcal{M}$. Then: (a) Given any $\varepsilon > 0$, there exists a $\mathbb{P}$-null set $N$ such that for every $\omega \in N^c$ there is an integer $n(\omega) < \infty$ for which the sample Fréchet mean set $\mathscr{E}_n$ is contained in the $\varepsilon$-neighborhood of the Fréchet mean set of $Q$ for all $n \ge n(\omega)$. (b) If the Fréchet mean of $Q$ is unique, then every measurable choice from the sample Fréchet mean set $\mathscr{E}_n$ is a strongly consistent estimator. ::: ::: {.proof} Let $X_1,X_2,\ldots$ be i.i.d. random elements with law $Q$, defined on a probability space $(\Omega,\mathcal F,\mathbb P)$. Write $$ F_n(x)=\frac1n\sum_{i=1}^n d^2(x,X_i),\qquad m=\inf_{x\in\mathcal M}F(x), $$ and define the population and sample mean sets by $$ \mathscr E=\{x\in\mathcal M:F(x)=m\}, \qquad \mathscr E_n(\omega)=\operatorname*{argmin}_{x\in\mathcal M}F_n(x,\omega). $$ We prove four claims. **Step 1: The population and sample mean sets are nonempty and compact.** The hypothesis says that $F$ is finite on $\mathcal M$. By the Heine–Borel existence theorem from Lecture 2, $F$ attains its infimum and $\mathscr E$ is a nonempty compact set. In particular, $m<\infty$. Similar arguments show that $\mathscr E_n(\omega)$ is nonempty and compact for every $n$ and $\omega$. **Step 2: Almost surely, all sample mean sets eventually lie in one deterministic compact set.** Choose $p\in\mathscr E$ and any reference point $o\in\mathcal M$, and set $$ R=\sqrt{m+1}+\sqrt{F(o)}+1, \qquad K=\overline B(o,R)=\{x\in\mathcal M:d(x,o)\le R\}. $$ The set $K$ is closed and bounded, hence compact by the Heine–Borel property. We first note that $p\in K$. Indeed, the triangle inequality and Minkowski's inequality in $L^2(Q)$ give $$ \begin{aligned} d(p,o) &=\left(\int d^2(p,o)\,Q(dy)\right)^{1/2}\\ &\le \left(\int d^2(p,y)\,Q(dy)\right)^{1/2} + \left(\int d^2(y,o)\,Q(dy)\right)^{1/2}\\ &=\sqrt{F(p)}+\sqrt{F(o)} =\sqrt m+\sqrt{F(o)} <R. \end{aligned} $$ Now fix $x\in\mathscr E_n(\omega)$. Since $x$ minimizes $F_n(\,\cdot\,,\omega)$, $$ F_n(x)\le F_n(p). $$ For each $i$, the triangle inequality gives $d(x,o)\le d(x,X_i)+d(X_i,o)$. Taking the empirical $L^2$ norm and applying Minkowski's inequality yields $$ \begin{aligned} d(x,o) &=\left(\frac1n\sum_{i=1}^n d^2(x,o)\right)^{1/2}\\ &\le \left(\frac1n\sum_{i=1}^n d^2(x,X_i)\right)^{1/2} + \left(\frac1n\sum_{i=1}^n d^2(o,X_i)\right)^{1/2}\\ &=\sqrt{F_n(x)} + \left(\frac1n\sum_{i=1}^n d^2(o,X_i)\right)^{1/2}\\ &\le \sqrt{F_n(p)} + \left(\frac1n\sum_{i=1}^n d^2(o,X_i)\right)^{1/2}. \end{aligned} $$ Both random variables $d^2(p,X_1)$ and $d^2(o,X_1)$ are integrable because their expectations are $F(p)=m$ and $F(o)$, respectively. Therefore, by the strong law of large numbers, on an event $\Omega_1$ with $\mathbb P(\Omega_1)=1$, $$ F_n(p)\longrightarrow m, \qquad \frac1n\sum_{i=1}^n d^2(o,X_i)\longrightarrow F(o). $$ Consequently, for every $\omega\in\Omega_1$, both $$ F_n(p)<m+1 \quad\text{and}\quad \left(\frac1n\sum_{i=1}^n d^2(o,X_i)\right)^{1/2} <\sqrt{F(o)}+1 $$ hold for all sufficiently large $n$. Thus, for every $\omega\in\Omega_1$, $$ \mathscr E_n(\omega)\subset K $$ for all sufficiently large $n$. **Step 3: $F_n$ converges to $F$ uniformly on $K$, almost surely.** First we obtain a common Lipschitz bound. For $x,z\in K$ and $y\in\mathcal M$, $$ \begin{aligned} \left|d^2(x,y)-d^2(z,y)\right| &=|d(x,y)-d(z,y)|\{d(x,y)+d(z,y)\}\\ &\le d(x,z)\{d(x,y)+d(z,y)\}\\ &\le \{2R+2d(o,y)\}\,d(x,z). \end{aligned} $$ The last inequality uses $d(x,y)\le d(x,o)+d(o,y)\le R+d(o,y)$, and similarly for $z$. Averaging over the sample gives $$ |F_n(x)-F_n(z)|\le L_n\,d(x,z), \qquad L_n:=2R+\frac2n\sum_{i=1}^n d(o,X_i). $$ Also, taking expectations gives $$ |F(x)-F(z)|\le L\,d(x,z), \qquad L:=2R+2\,\mathbb E\,d(o,X_1). $$ The constant $L$ is finite because Cauchy–Schwarz implies $$ \mathbb E\,d(o,X_1)\le \{\mathbb E\,d^2(o,X_1)\}^{1/2} =\sqrt{F(o)}<\infty. $$ The strong law applied to $d(o,X_i)$ shows that $L_n\to L$ almost surely. In particular, with the deterministic constant $C:=L+1$, there is an event of probability one on which $L_n\le C$ eventually. We now give the finite-net argument explicitly. For each $k\in\mathbb N$, choose $\rho_k>0$ so small that $$ (C+L)\rho_k<\frac{1}{2k}. $$ Because $K$ is compact, it is totally bounded. Hence there is a finite $\rho_k$-net $$ \mathcal Z_k=\{z_{k,1},\ldots,z_{k,N_k}\}\subset K; $$ that is, every $x\in K$ satisfies $d(x,z_{k,j})<\rho_k$ for at least one $j$. The set $$ \mathcal Z:=\bigcup_{k=1}^{\infty}\mathcal Z_k $$ is countable. For each $z\in\mathcal Z$, the random variable $d^2(z,X_1)$ is integrable because $F(z)<\infty$. Taking the countable intersection of the corresponding strong-law events, we obtain an event $\Omega_2$ of probability one on which $$ F_n(z)\longrightarrow F(z) \qquad\text{for every }z\in\mathcal Z, $$ and on which $L_n\le C$ eventually. Fix $\omega\in\Omega_2$ and $k\in\mathbb N$. Since $\mathcal Z_k$ is finite, there is an integer $N_k(\omega)$ such that, for $n\ge N_k(\omega)$, $$ \max_{1\le j\le N_k}|F_n(z_{k,j})-F(z_{k,j})|<\frac{1}{2k} \quad\text{and}\quad L_n\le C. $$ For any $x\in K$, select $z_{k,j}\in\mathcal Z_k$ with $d(x,z_{k,j})<\rho_k$. Then $$ \begin{aligned} |F_n(x)-F(x)| &\le |F_n(x)-F_n(z_{k,j})| +|F_n(z_{k,j})-F(z_{k,j})|\\ &\qquad +|F(z_{k,j})-F(x)|\\ &\le L_n\rho_k+\frac{1}{2k}+L\rho_k\\ &<\frac1k. \end{aligned} $$ This bound holds for every $x\in K$, so $$ \sup_{x\in K}|F_n(x)-F(x)|<\frac1k $$ for all $n\ge N_k(\omega)$. Since $k$ is arbitrary, $$ \sup_{x\in K}|F_n(x)-F(x)|\longrightarrow0 $$ on $\Omega_2$. **Step 4: A positive separation gap excludes minimizers outside the $\varepsilon$-neighborhood.** Fix $\varepsilon>0$ and define $$ d(x,\mathscr E):=\inf_{p\in\mathscr E}d(x,p), \qquad \mathscr E^\varepsilon =\{x\in\mathcal M:d(x,\mathscr E)<\varepsilon\}, \qquad A_\varepsilon=K\setminus\mathscr E^\varepsilon. $$ Because $\mathscr E$ is nonempty, the map $x\mapsto d(x,\mathscr E)$ is $1$-Lipschitz and therefore continuous. Thus $\mathscr E^\varepsilon$ is open, so $A_\varepsilon$ is a closed subset of the compact set $K$ and is compact. If $A_\varepsilon=\varnothing$, Step 2 already gives $\mathscr E_n\subset K\subset\mathscr E^\varepsilon$ eventually, and there is nothing more to prove. Suppose therefore that $A_\varepsilon\ne\varnothing$. Since $F$ is continuous, it attains its minimum on $A_\varepsilon$. Moreover, $A_\varepsilon\cap\mathscr E=\varnothing$: every point of $\mathscr E$ has distance $0<\varepsilon$ from $\mathscr E$. Hence the minimum over $A_\varepsilon$ is strictly larger than the global minimum $m$. In other words, $$ \delta_\varepsilon :=\min_{x\in A_\varepsilon}F(x)-m >0. $$ Let $\Omega_*=\Omega_1\cap\Omega_2$, which has probability one. Fix $\omega\in\Omega_*$. By Steps 2 and 3, there is an integer $N(\omega,\varepsilon)$ such that, for all $n\ge N(\omega,\varepsilon)$, $$ \mathscr E_n(\omega)\subset K \qquad\text{and}\qquad \sup_{x\in K}|F_n(x,\omega)-F(x)|<\frac{\delta_\varepsilon}{3}. $$ For every $x\in A_\varepsilon$, it follows that $$ F_n(x) \ge F(x)-\frac{\delta_\varepsilon}{3} \ge m+\frac{2\delta_\varepsilon}{3}. $$ On the other hand, the fixed population minimizer $p$ belongs to $K$, and therefore $$ F_n(p) \le F(p)+\frac{\delta_\varepsilon}{3} =m+\frac{\delta_\varepsilon}{3}. $$ Thus $F_n(x)>F_n(p)$ for every $x\in A_\varepsilon$. No point of $A_\varepsilon$ can minimize $F_n$. Since every sample minimizer is already in $K$, we conclude that $$ \mathscr E_n(\omega)\subset K\setminus A_\varepsilon =K\cap\mathscr E^\varepsilon \subset\mathscr E^\varepsilon $$ for every $n\ge N(\omega,\varepsilon)$. Taking the null set $N=\Omega\setminus\Omega_*$ proves part (a). Finally, suppose the population Fréchet mean is unique, so $\mathscr E=\{\mu\}$. Let $\hat\mu_n$ be any measurable choice from $\mathscr E_n$. Apply part (a) with the countable sequence $\varepsilon_k=1/k$. Outside the union of the corresponding null sets—which is still a null set—for every $k$, $$ d(\hat\mu_n,\mu)<\frac1k $$ for all sufficiently large $n$. This is exactly $d(\hat\mu_n,\mu)\to0$ almost surely, or $\hat\mu_n\to\mu$ almost surely. This proves part (b). ::: ![The $\varepsilon$-neighborhood argument: the compact set $K$ (blue boundary) contains the sample Fréchet means eventually. The $\varepsilon$-neighborhood $\mathscr{E}^{\varepsilon}$ of the population Fréchet mean set $\mathscr{E}$ (green) is shown. The region $A_{\varepsilon} = K \setminus \mathscr{E}^{\varepsilon}$ has Fréchet function values strictly above the minimum by a gap $\delta_{\varepsilon}$, so the sample minimizer cannot land there for large $n$.](source/fig/generated/epsilon-net.svg){width=75%} ## Metric Entropy To study the **convergence rate** of the sample Fréchet mean, we need to quantify the complexity of a metric space. This is done via metric entropy. ::: {#def-metric-entropy .definition title="Covering numbers, packing numbers, and metric entropy"} Let $(\mathcal{M}, d)$ be a metric space, let $A \subset \mathcal{M}$, and let $\varepsilon > 0$. - An **$\varepsilon$-cover** of $A$ is a finite collection of points $x_1, \ldots, x_m \in \mathcal{M}$ such that $A \subset \bigcup_{j=1}^m B(x_j, \varepsilon)$. The **covering number** is $$ N(\varepsilon, A, d) = \inf\left\{m \in \mathbb{N} : A \subset \bigcup_{j=1}^m B(x_j, \varepsilon)\right\}. $$ - An **$\varepsilon$-packing** of $A$ is a subset $\{x_1, \ldots, x_m\} \subset A$ such that $d(x_i, x_j) > \varepsilon$ for all $i \neq j$. Its maximal cardinality is the **packing number** $M(\varepsilon, A, d)$. - The **metric entropy** of $A$ at scale $\varepsilon$ is $$ H(\varepsilon, A, d) = \log N(\varepsilon, A, d). $$ ::: {.callout-note title="Interactive Demo: ε-Covers and ε-Packings" collapse="true"} ```{=html} <div class="entropy-cover-demo" data-entropy-cover-demo> <div class="entropy-cover-demo__controls"> <label class="entropy-cover-demo__slider"> <span>ε = <output data-role="epsilon-output">0.120</output></span> <input type="range" min="0.055" max="0.24" step="0.005" value="0.12" data-role="epsilon-slider"> </label> </div> <div class="entropy-cover-demo__summary" aria-live="polite"> <div><span class="entropy-cover-demo__label">Displayed cover size</span><strong data-role="cover-count">0</strong></div> <div><span class="entropy-cover-demo__label">Displayed packing size</span><strong data-role="packing-count">0</strong></div> </div> <div class="entropy-cover-demo__panels"> <section class="entropy-cover-demo__panel"> <h4>ε-cover of rectangle A</h4> <canvas data-role="cover-canvas" width="460" height="320" aria-label="Epsilon cover of a rectangle"></canvas> </section> <section class="entropy-cover-demo__panel"> <h4>ε-packing in rectangle A</h4> <canvas data-role="packing-canvas" width="460" height="320" aria-label="Epsilon packing in a rectangle"></canvas> </section> </div> </div> ``` ::: Because $M(2\varepsilon, A, d) \le N(\varepsilon, A, d) \le M(\varepsilon, A, d)$, covering and packing formulations are equivalent up to constants. ::: The quantity $H(\varepsilon, \mathcal{M}, d)$ is the **global metric entropy**. For a point $\omega \in \mathcal{M}$ and radius $\delta > 0$, $H(\varepsilon, B_\delta(\omega), d)$ is the **local metric entropy**. Finite-dimensional spaces typically have **logarithmic** metric entropy in $1/\varepsilon$, while genuinely infinite-dimensional spaces often have **polynomial** or faster entropy growth. ::: {#exm-entropy-spaces .example title="Metric entropy of example spaces"} The spaces from the introduction have the following entropy behavior as $\varepsilon \downarrow 0$. 1. **Sphere $\mathbb{S}^{p-1}$** (geodesic metric): $$ H(\varepsilon, \mathbb{S}^{p-1}, d_{\mathbb{S}}) = (p-1)\log(1/\varepsilon) + O(1), $$ and for geodesic balls, $H(\varepsilon, B_\delta(x), d_{\mathbb{S}}) = (p-1)\log(\delta/\varepsilon) + O(1)$ for $0 < \varepsilon < \delta < \pi$ [@Rogers1963]. 2. **SPD matrices $\mathcal{S}_{++}^{m}$** (log-Euclidean metric): the matrix logarithm identifies $\mathcal{S}_{++}^{m}$ isometrically with symmetric matrices of dimension $q = m(m+1)/2$. Hence $$ H(\varepsilon, B_\delta(A), d_{\mathrm{LE}}) = q\log(\delta/\varepsilon) + O(1) $$ on bounded metric balls. The affine-invariant metric has the same local finite-dimensional order [@ArsignyEtAl2006; @ArsignyEtAl2007; @PennecFillardAyache2006]. 3. **BHV tree space** (fixed leaf set of size $m$): a finite union of Euclidean orthants of dimension $p_m = m-2$ (rooted case). Every bounded subset $A$ satisfies $$ H(\varepsilon, A, d_{\mathrm{BHV}}) \le p_m\log(1/\varepsilon) + O_A(1). $$ Since BHV tree space is unbounded, its global entropy is infinite [@BilleraHolmesVogtmann2001]. 4. **Wasserstein space $(\mathcal{P}_2([0,1]), W_2)$**: the quantile map identifies this space with monotone functions in $L^2(0,1)$. The entropy is **polynomial**: $$ H(\varepsilon, \mathcal{P}_2([0,1]), W_2) \asymp \varepsilon^{-1}, $$ consistent with bounds for monotone classes [@GaoWellner2007; @BleiGaoLi2007]. In higher dimensions the entropy is even larger: for $T = [0,1]^p$, $$ H(\varepsilon, \mathcal{P}([0,1]^p), W_1) \gtrsim \varepsilon^{-p}, $$ which shows that Wasserstein spaces are infinite-dimensional from the entropy viewpoint [@BoedihardjoStrohmerVershynin2024]. ::: ## Convergence Rate in Hadamard Spaces The following result applies the theory of @Schotz2019 to Hadamard spaces. ::: {#prp-schoetz-rate .proposition title="Rate for Hadamard-space Fréchet means, Schötz 2019"} Let $X$ be a random element on a Hadamard space $(\mathcal{M}, d)$ such that $\mathbb{E}\,d^2(X, x) < \infty$ for some $x \in \mathcal{M}$. Assume that there exist constants $c_e > 0$ and $\beta > 0$ such that, for all $\delta > 0$ and all $0 < r \le \delta$, $$ \sqrt{\log N\bigl(\varepsilon, B_\delta(\mu), d\bigr)} \le c_e\left(\frac{\delta}{\varepsilon}\right)^{\beta}. $$ Define $$ \eta_{\beta,n} = \begin{cases} n^{-1/2}, & \beta < 1, \\ n^{-1/2}\log(n+1), & \beta = 1, \\ n^{-1/(2\beta)}, & \beta > 1. \end{cases} $$ Then, for every $s > 0$, $$ \mathbb{P}\bigl\{\eta_{\beta,n}^{-1}\,d(\hat{\mu}, \mu) \ge s\bigr\} \le c\,s^{-2}, $$ where $c > 0$ depends only on $\beta$, $c_e$, and $\mathbb{E}\,d^2(X, X')$ for an independent copy $X'$ of $X$. In particular, $$ d(\hat{\mu}, \mu) = O_P(\eta_{\beta,n}). $$ Thus the distance rate is **parametric**, $n^{-1/2}$, when $\beta < 1$, acquires a **logarithmic factor** when $\beta = 1$, and is **slower** when $\beta > 1$. ::: ::: {.callout-important title="Interpretation of the Entropy Exponent"} - **$\beta = 0$** (finite-dimensional spaces like $\mathbb{R}^p$, sphere, SPD, BHV): parametric rate $n^{-1/2}$. - **$\beta = 1/2$** (1D Wasserstein space $W_2$ on $[0,1]$): rate $n^{-1/2}$ since $\beta < 1$. - **$\beta > 1$** (higher-dimensional Wasserstein spaces): slower than parametric rate. Note that the above only provides a general upper bound on the convergence rate. Faster rates are possible in some cases, depending on the geometry of the space and the distribution of the data. For example, for spaces with **nonnegative curvature** (including high-dimensional Wasserstein spaces), @LeGouicParisRigolletStromme2023 show that, under some conditions, $\mathbb{E}\,d^2(\hat{\mu}, \mu) = O(n^{-1})$, again yielding the parametric rate $d(\hat{\mu}, \mu) = O_P(n^{-1/2})$. ::: ::: {.callout-tip title="Key Insight"} The entropy exponent $\beta$ determines the convergence rate. Finite-dimensional spaces have $\beta = 0$ (logarithmic entropy), yielding the parametric rate $n^{-1/2}$. Infinite-dimensional spaces with $\beta > 1$ yield slower rates. ::: ## Beyond Classical Asymptotics: Stickiness and Smeariness In non-Euclidean spaces, sample Fréchet means can exhibit behavior that has no Euclidean counterpart. Two important phenomena are **stickiness** and **smeariness**. ### Stickiness **Stickiness** arises in singular stratified spaces (such as BHV tree space or open books). When the population Fréchet mean lies on a lower-dimensional stratum, the empirical Fréchet means can get **trapped** there after a finite random sample size, and small perturbations of the underlying law need not move the mean off that stratum. The open-book analysis of @HotzEtAl2013 distinguishes **sticky**, **partly sticky**, and **nonsticky** cases. In the sticky case, the sample means eventually lie almost surely on the codimension-one spine, and the limiting Gaussian law is supported on that spine. More recently, @StickyFlavors2023 systematize the phenomenon by distinguishing **sample**, **perturbation**, and **topological** stickiness and studying their relations in $\mathrm{CAT}(\kappa)$ spaces. In BHV phylogenetic tree space, @LammersNyeHuckemann2024 explain that stickiness typically manifests itself through unresolved or star trees and can create computational bottlenecks. ### Smeariness **Smeariness** occurs on smooth positively curved manifolds (not singular spaces). In smeary models the empirical Fréchet mean need not obey a classical root-$n$ central limit theorem: depending on the local geometry and the behavior of the distribution near the cut locus, the fluctuations can occur on a slower scale and may have a non-Gaussian limit. Key developments: - On the **circle**, @HotzHuckemann2015Circle showed that the convergence rate can become arbitrarily slow as the distribution approaches uniformity near the antipode. - On higher-dimensional **spheres**, @EltznerHuckemann2019SmearyCLT established a smeary central limit theorem. - @Eltzner2022GeometricalSmeariness showed that such behavior can arise purely from **positive curvature**, even without mass concentrated at the antipode — motivating the term *geometrical smeariness*. - @EltznerHundrieserHuckemann2021FSSSpheres and @TranEltznerHuckemann2021SmearinessBegetsFSS show that finite-sample smeariness is widespread near smeary models, leading to **variance inflation** and making naive tangent-space Gaussian approximations unreliable at moderate sample sizes. ::: {.callout-important title="Takeaway"} Both stickiness (from singularities) and smeariness (from positive curvature) show that **the geometry of the data space fundamentally shapes the statistical behavior** of Fréchet means. Classical CLT intuition from $\mathbb{R}^p$ does not automatically carry over. ::: ## Key Takeaways - Sample Fréchet means are **strongly consistent** under mild conditions (complete, separable metric space). - The proof uses an **$\varepsilon$-net argument**: uniform convergence of $F_n$ to $F$ on a compact set containing the sample minimizers. - **Metric entropy** quantifies the complexity of a metric space and governs convergence rates. - Finite-dimensional spaces (sphere, SPD, BHV) have **logarithmic entropy** → parametric $n^{-1/2}$ rate. - Wasserstein spaces have **polynomial entropy** → potentially slower rates in higher dimensions. - **Stickiness** (from singularities) and **smeariness** (from positive curvature) are non-classical phenomena that invalidate standard CLT intuition. ## Exercises 1. **Covering vs. packing**: Prove that $M(2\varepsilon, A, d) \le N(\varepsilon, A, d) \le M(\varepsilon, A, d)$. <a href="javascript:void(0)" onclick="showSolution('l3-sol-1')" class="solution-link">📝 Show Solution</a> 2. **Entropy of the sphere**: Derive the leading-order term $(p-1)\log(1/\varepsilon)$ for the metric entropy of $\mathbb{S}^{p-1}$. (Hint: the sphere is a $(p-1)$-dimensional manifold; compare volumes of balls.) <a href="javascript:void(0)" onclick="showSolution('l3-sol-2')" class="solution-link">📝 Show Solution</a> 3. **Entropy of $\mathbb{R}^p$**: Show that for a bounded subset $A \subset \mathbb{R}^p$, $H(\varepsilon, A, d) = p\log(1/\varepsilon) + O(1)$. What is $\beta$ in this case? <a href="javascript:void(0)" onclick="showSolution('l3-sol-3')" class="solution-link">📝 Show Solution</a> <style> .solution-link { font-size: 0.9em; text-decoration: none; white-space: nowrap; margin-left: 0.3em; } .solution-link:hover { text-decoration: underline; } .solution-dialog { padding: 0; max-width: 720px; } .solution-dialog-header { display: flex; justify-content: space-between; align-items: flex-start; border-bottom: 1px solid #dee2e6; padding: 1.25rem 1.5rem 1rem; background: #f8f9fa; border-radius: 8px 8px 0 0; } .solution-dialog-header h4 { margin: 0; font-size: 1.15rem; } .solution-dialog-close { background: none; border: 1px solid #adb5bd; border-radius: 4px; padding: 0.2rem 0.75rem; cursor: pointer; font-size: 0.9rem; color: #495057; white-space: nowrap; flex-shrink: 0; } .solution-dialog-close:hover { background: #e9ecef; } .solution-original { padding: 1rem 1.5rem; background: #f1f3f5; border-left: 4px solid #868e96; margin: 1rem 1.5rem; border-radius: 4px; font-size: 0.95rem; } .solution-original strong { color: #495057; } .solution-answer { padding: 0.5rem 1.5rem 1.5rem; } .solution-answer strong { color: #2b8a3e; } dialog { border: none; border-radius: 8px; box-shadow: 0 8px 32px rgba(0,0,0,0.22); padding: 0; max-width: 750px; width: 90vw; } dialog::backdrop { background: rgba(0,0,0,0.45); } </style> <dialog id="l3-sol-1"> <div class="solution-dialog"> <div class="solution-dialog-header"> <h4>Exercise 1: Covering vs. Packing</h4> <button onclick="closeSolution('l3-sol-1')" class="solution-dialog-close">✕ Close</button> </div> <div class="solution-original"> <strong>Exercise:</strong> Prove that $M(2\varepsilon, A, d) \le N(\varepsilon, A, d) \le M(\varepsilon, A, d)$. </div> <div class="solution-answer"> <strong>Solution:</strong> **Right inequality $N(\varepsilon, A, d) \le M(\varepsilon, A, d)$:** Let $P = \{x_1, \ldots, x_M\}$ be a **maximal** $\varepsilon$-packing of $A$. Maximality means no further point can be added while maintaining pairwise distances $> \varepsilon$. Therefore, for any $a \in A$, if $a \notin P$, then there exists some $x_i \in P$ such that $d(a, x_i) \le \varepsilon$; otherwise, $a$ could be added to $P$, contradicting maximality. Hence $A \subset \bigcup_{i=1}^M B(x_i, \varepsilon)$, so $P$ is also an $\varepsilon$-cover. Since $N(\varepsilon, A, d)$ is the minimum cardinality of an $\varepsilon$-cover, we have $N(\varepsilon, A, d) \le M = M(\varepsilon, A, d)$. **Left inequality $M(2\varepsilon, A, d) \le N(\varepsilon, A, d)$:** Let $C = \{c_1, \ldots, c_N\}$ be a **minimal** $\varepsilon$-cover of $A$, so $N = N(\varepsilon, A, d)$. Let $P = \{p_1, \ldots, p_M\}$ be any $2\varepsilon$-packing. We claim $M \le N$. Each $p_j \in P \subset A$ must lie in some ball $B(c_i, \varepsilon)$ of the cover. Moreover, no two distinct points $p_j, p_k$ can lie in the same ball $B(c_i, \varepsilon)$, because if $p_j, p_k \in B(c_i, \varepsilon)$, then by the triangle inequality $d(p_j, p_k) \le d(p_j, c_i) + d(c_i, p_k) \le 2\varepsilon$, contradicting the $2\varepsilon$-packing property $d(p_j, p_k) > 2\varepsilon$. Thus each ball of the cover contains at most one packing point, implying $M \le N$. Taking the maximum possible $M$ (the packing number) gives $M(2\varepsilon, A, d) \le N(\varepsilon, A, d)$. </div> </div> </dialog> <dialog id="l3-sol-2"> <div class="solution-dialog"> <div class="solution-dialog-header"> <h4>Exercise 2: Entropy of the Sphere</h4> <button onclick="closeSolution('l3-sol-2')" class="solution-dialog-close">✕ Close</button> </div> <div class="solution-original"> <strong>Exercise:</strong> Derive the leading-order term $(p-1)\log(1/\varepsilon)$ for the metric entropy of $\mathbb{S}^{p-1}$. </div> <div class="solution-answer"> <strong>Solution:</strong> The sphere $\mathbb{S}^{p-1} = \{x \in \mathbb{R}^p : \|x\|_2 = 1\}$ is a compact Riemannian manifold of dimension $d = p-1$ with the geodesic (great-circle) distance. **Volume argument.** The geodesic ball $B(x, \varepsilon)$ on $\mathbb{S}^{p-1}$ is (to leading order in small $\varepsilon$) isometric to a Euclidean ball of radius $\varepsilon$ in $\mathbb{R}^{p-1}$. Its volume is therefore $$\operatorname{vol}(B(x, \varepsilon)) \sim \omega_{p-1}\,\varepsilon^{p-1},$$ where $\omega_{p-1} = \pi^{(p-1)/2}/\Gamma((p+1)/2)$ is the volume of the unit ball in $\mathbb{R}^{p-1}$. The total volume of $\mathbb{S}^{p-1}$ is $\operatorname{vol}(\mathbb{S}^{p-1}) = 2\pi^{p/2}/\Gamma(p/2)$. **Covering number.** If $\{x_1, \ldots, x_M\}$ is a maximal $\varepsilon$-packing, then the balls $B(x_i, \varepsilon/2)$ are disjoint (by the packing property: $d(x_i, x_j) > \varepsilon$ implies $B(x_i, \varepsilon/2) \cap B(x_j, \varepsilon/2) = \varnothing$). Therefore $$M \cdot \operatorname{vol}(B(x, \varepsilon/2)) \le \operatorname{vol}(\mathbb{S}^{p-1}).$$ This gives $$N(\varepsilon, \mathbb{S}^{p-1}) \asymp \varepsilon^{-(p-1)}.$$ **Metric entropy.** Taking logarithms, $$H(\varepsilon, \mathbb{S}^{p-1}) = \log N(\varepsilon) = (p-1)\log(1/\varepsilon) + O(1).$$ The entropy exponent is $\beta = 0$, consistent with $\mathbb{S}^{p-1}$ being finite-dimensional. The prefactor $p-1$ equals the manifold dimension. </div> </div> </dialog> <dialog id="l3-sol-3"> <div class="solution-dialog"> <div class="solution-dialog-header"> <h4>Exercise 3: Entropy of $\mathbb{R}^p$</h4> <button onclick="closeSolution('l3-sol-3')" class="solution-dialog-close">✕ Close</button> </div> <div class="solution-original"> <strong>Exercise:</strong> Show that for a bounded subset $A \subset \mathbb{R}^p$, $H(\varepsilon, A, d) = p\log(1/\varepsilon) + O(1)$. What is $\beta$ in this case? </div> <div class="solution-answer"> <strong>Solution:</strong> Let $A \subset \mathbb{R}^p$ be bounded. Then $A$ is contained in a Euclidean ball of some radius $R$, or equivalently in a hypercube $[-R, R]^p$. **Covering construction.** Divide each coordinate axis into intervals of length $2\varepsilon/\sqrt{p}$. This partitions the hypercube into $$m = \left(\frac{2R}{2\varepsilon/\sqrt{p}}\right)^p = \left(\frac{R\sqrt{p}}{\varepsilon}\right)^p$$ smaller hypercubes, each of diameter $\le 2\varepsilon$ (the diagonal of each small cube is $\sqrt{p} \cdot (2\varepsilon/\sqrt{p}) = 2\varepsilon$). Therefore, each small cube is contained in a Euclidean ball of radius $\varepsilon$. The centers of these cubes thus form an $\varepsilon$-cover of $A$, giving $$N(\varepsilon, A, d) \le \left(\frac{R\sqrt{p}}{\varepsilon}\right)^p = C_p \,\varepsilon^{-p}.$$ **Lower bound.** A standard volume argument (disjoint $\varepsilon/2$-balls inside a slightly expanded set) yields the matching lower bound $N(\varepsilon, A, d) \ge c_p \,\varepsilon^{-p}$. **Metric entropy.** Taking logarithms: $$H(\varepsilon, A, d) = \log N(\varepsilon) = p\log(1/\varepsilon) + O(1).$$ **Entropy exponent $\beta$.** From the definition in the lecture, $\sqrt{\log N(\varepsilon, B_\delta(\mu), d)} \le c_e(\delta/\varepsilon)^\beta$. Here $\log N(\varepsilon) \asymp p\log(1/\varepsilon)$, which grows **logarithmically** in $1/\varepsilon$. Any $\beta > 0$ would suffice, leading to the parametric convergence rate $n^{-1/2}$. </div> </div> </dialog> <script> function showSolution(id) { const dialog = document.getElementById(id); if (dialog) { dialog.showModal(); if (window.MathJax && MathJax.typesetPromise) { MathJax.typesetPromise([dialog]).catch(function(err) { console.log('MathJax typeset error:', err); }); } } } function closeSolution(id) { const dialog = document.getElementById(id); if (dialog) { dialog.close(); } } document.addEventListener('click', function(e) { if (e.target.tagName === 'DIALOG') { e.target.close(); } }); </script> ## Further Reading - @Schotz2019 — General theory of convergence rates for Fréchet means via metric entropy. - @Schotz2022 — Strong laws for generalized Fréchet mean sets. - @BhattacharyaPatrangenaru2003 — Systematic consistency results for intrinsic means on manifolds. - @HotzEtAl2013 — Foundational analysis of stickiness in open-book spaces. - @StickyFlavors2023 — Systematization of stickiness phenomena in $\mathrm{CAT}(\kappa)$ spaces. - @EltznerHuckemann2019SmearyCLT — The smeary central limit theorem on spheres. - @LeGouicParisRigolletStromme2023 — Fast rates for Fréchet means in nonnegatively curved spaces. ## Self-Assessment Quiz Test your understanding of this lecture with the interactive MCQ quiz: 👉 **[Lecture 3 Quiz — 10 Multiple-Choice Questions](../quizzes/lecture-03-quiz.qmd)**