Expectation and Basics of linear algebra

大部分内容包括在我的这篇数据科学笔记

$\textbf{Definition 3.2.}\quad$ Two rvs $X$ and $Y$ are independent if $$ \forall i, j:\mathbb P(X = i, Y = j) = \mathbb P(X = i)\cdot \mathbb P(Y = j) $$

Note that $X, Y$ independent implies that $\mathbb E(XY )$ = $\mathbb E(X)\mathbb E(Y)$, but not the converse

$\textbf{Definition 3.3.}\quad$ Given a rv $X : Ω → I$ and an event $B ∈ \mathcal F$ in $Ω$ with $P(B) > 0$, the conditional expectation of $X$ given $B$ is defines as $$ \mathbb{E}(X \mid B)=\sum_{i \in I} i \mathbb{P}(X=i \mid B)=\frac{\sum_{i \in I} i \mathbb{P}(\{X=i\} \cap B)}{\mathbb{P}(B)} $$

$\textbf{Definition 3.4.}\quad$ Let $X : Ω → I$ be a rv and $\mathcal B ⊆ \mathcal F$ be a $σ$-algebra on $Ω$ generated by a countable partition $\Pi = \{Ei\mid i ⩾ 1\}$ of $Ω$. The conditional expectation of $X$ given $\mathcal B$ is a piecewise constant rv defined as $$ \forall \omega \in E_{i}: \mathbb{E}(X \mid \mathcal{B})(\omega):=\mathbb{E}\left(X \mid E_{i}\right) $$

Heuristically, this is the average of $X$ given the partial information through $B$. Note that $\mathbb P(A \mid \mathcal B)$ is a scalar, and so is $\mathbb E(A \mid \mathcal B)$. But, both $\mathbb P(A \mid\mathcal B)$ and $\mathbb E(X \mid\mathcal B)$ are RVs. One can check that if $X$ is measurable $w.r.t.$ $\mathcal B$, then $\mathbb E(X \mid\mathcal B) = X$ and vice versa.

$\textbf{Theorem 3.6.}$ (Law of total expectation). If $X : Ω → I$ is a rv and $\mathcal B ⊆ \mathcal F$ is a $σ$-algebra on $Ω$, then: $$ \mathbb E(\mathbb E(X | \mathcal B)) = \mathbb E(X). $$


$\textbf{Definition 3.17.}\quad$ The spectrum of a matrix $M$ is defined as $$ \operatorname{spec}(M) := \{\mu \mid \mu \text{ is an eigenvalue of } M\}. $$

$\textbf{Theorem 3.18.}$ (Gershgorin Circle Theorem). Let $A$ be an $n×n$ matrix. For each $i = 1, 2,\ldots, n$ we define $$ \begin{align} R_{i}&:=\sum_{j \neq i}\left|a_{i j}\right|\\ B_{i}&:=\overline{\mathrm{B}\left(a_{i i}, R_{i}\right)}=\left\{z \in \mathbb{C}\mid \|z-a_{i i} \|\leqslant R_{i}\right\} \end{align} $$ Then, $$ \operatorname{spec}(A) \subseteq \bigcup_{i=1}^{n} B_{i} $$