$$\huge\textbf{Probability and Random Variables}
$$
1. Fubini’s theorem
$\textbf{Theorem 2.1 (Fubini’s Theorem).}\quad$
If $I$ and $J$ are countable sets and $\forall i\in I, j \in J: a_{ij} ⩾ 0$ then
$$
\sum_{i\in I}\sum_{j\in J}a_{ij}=\sum_{j\in J}\sum_{i\in I}a_{ij}
$$
That is, we can rearrange the order of summing countable many non-negative terms.
$Proof.$ Enumerate $J = \{j_1, j_2, . . .\}$. Then
$$\sum_{i \in I} \sum_{j \in J} a_{i j} \geqslant \sum_{i \in I}\left(\sum_{j=1}^{n} a_{i j}\right)
$$
by dropping non-negative terms. Now one sum is finite, so we can exchange order to get
$$\sum_{i \in I}\left(\sum_{j=1}^{n} a_{i j}\right)=\sum_{j=1}^{n}\left(\sum_{i \in I} a_{i j}\right)
$$
The last sum forms an increasing sequence in n that converges to $\displaystyle \sum_{j\in J}\sum_{i\in I}a_{ij}$ by monotone convergence theorem. Thus
$$\sum_{i\in I}\sum_{j\in J}a_{ij}\geqslant\sum_{j\in J}\sum_{i\in I}a_{ij}
$$
By symmetry, the other inequality is also true, so equality holds.
2. Probability and Measure
$\textbf{Definition 2.2.}\quad$ A measure $\mu$ on $(E,\mathcal E)$ is a function $\mu: E\to [0, ∞]$ such that if $(A_n, n ∈ \mathbb N)$ are disjoint then
$$
\mu\left(\bigcup_{n \in \mathbb{N}} A_{n}\right)=\sum_{n \in \mathbb{N}} \mu\left(A_{n}\right)
$$
Then $(E,\mathcal E, \mu)$ is called a measure space. If $\mu(E) = 1$, then $\mu$ is called a probability measure
and $(E,\mathcal E, \mu)$ is called a probability space.
A probability space $(E, \mathcal E, \mathbb P)$ has outcomes $\omega \in E$ and events $A \in \mathcal E$.
$\textbf{Theorem 2.4.}$(Properties of probability)$\quad$
From the definition of probability measure, it follows that:
(1) $\mathbb P(\emptyset) = 0$
(2) $\mathbb P(A^c) = 1- \mathbb P(A)$
(3) $\mathbb P(A) +\mathbb P(B) = \mathbb P(A\cup B) + \mathbb P(A\cap B)$
3. Random Variables
$\textbf{Definition 2.5.}\quad$ Given a probability space $(\Omega, \mathcal F, \mathbb P)$ and a measurable space $(E, \mathcal E)$, an $(\mathcal E, \mathcal F)$-measurable random variable $(rv)$ is a measurable function $X : Ω \to E$, $i.e.$,
$$
\forall A\in \mathcal E: X^{-1}A\in \mathcal F
$$
$\textbf{Special case}$: If $E$ is countable, then $X : \Omega \to E$ is a $(2^E, \mathcal F)$-measurable rv $\iff \forall i ∈ E\quad X^{−1}(i) \in \mathcal F$ . The probability mass function (pmf) or distribution of $X$ is defined by $$ \lambda(i):=\mathbb{P}(X=i)=\mathbb{P}(\{\omega \in \Omega \mid X(\omega)=i\}) $$
4. Condition Probability
$\textbf{Definition 2.7.}\quad$ If $A, B$ are events with $\mathbb P(B) > 0$, the conditional probability of $A$ given $B$ is defined as
$$
\mathbb{P}(A \mid B):=\frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)}
$$
Note that $\mathbb P(B \mid B) = 1$
$\textbf{Theorem 2.8.}\quad$(Law of Total Probability)
If $B_n$, $n \in I$ is a partition of $E$ then for $A \in \mathcal E$ we have
$$
\mathbb{P}(A)=\sum_{i \in I} \mathbb{P}\left(A \mid B_{n}\right) \mathbb{P}\left(B_{n}\right)
$$
$\textbf{Definition 2.9.}\quad$(Independence) Two events $A, B$ are independent $\iff \mathbb P(A \cap B) = P(A) P(B)$, equivalently $P(A \mid B) = P(A)$ when $P(B) > 0$.
Let $(E, \mathcal E, \mathbb P)$ be a probability space. Assume that $E$ is countable. Thus there exists a partition $\Pi=\{E_i, i \in I\}$ generating $E$.
$\textbf{Definition 2.10.}\quad$ Given a $\sigma$-algebra $B \subseteq E$ and an event $A \in E$, the conditional probability of $A$ given $B$ is a $rv$ that is constant on each $E_i$ and
$$
\forall \omega \in E_{i}:\mathbb{P}(A \mid \mathcal{B})(\omega):=\mathbb{P}\left(A \mid E_{i}\right)
$$