$ \newcommand{\Dice}{\mathrm{Dice}} \newcommand{\D}[1]{\mathcal{D}(#1)} \newcommand{\E}[1]{ \langle #1 \rangle} \newcommand{\G}{ \mathcal{N}\, } \newcommand{\Var}{ \mathrm{Var}\, } \newcommand{\std}{ \mathrm{std}\, } \newcommand{\Cov}{ \mathrm{Cov}\, } \newcommand{\Corr}{ \mathrm{Corr}\, } $

Random variables:
modeling imperfection

Florian Angeletti
NITheP

A random variable is an mathematical experiment protocol
$\Dice = \text{"Throw a dice and note the output"} $
$\Dice$ is a random variable
output $\Dice \in \{1,...,6\}$

There is many way to throw a dice
$ \Dice: \Omega \rightarrow \{1,\ldots,6\}$
$\Omega$ all the possible configurations of dice throwing
$X(\omega)$ an observation/realization of the random variable

Computing $\Dice$: Chaos
Modeling the global behavior: random distribution
Probabilities: \[ p_m( \Dice = n ) \equiv p_{\Dice}(n) = \frac 1 6 \]

Probabilities property

\[ p_\Dice > 0 \] \[ \sum_{n=1}^{6} p_\Dice(n) = 1 \]

Continuous distribution

$T_{\lambda} = \text{Time before an } {}^{235}U \text{atom disintegrates}$
density of probability : $ p_T (t) = \lambda e^{- \lambda t}$

$U$, uniform distribution on the interval $[0,1]$
```
  rand()  
```
density of probability : \[ p_U (u) = \begin{cases} \frac 1 {b-a} & \text{if a < u < b } \\ 0 & \text{otherwise} \end{cases} \]

$\G_{\mu,\sigma^2}$, gaussian or normal distribution
```
  randn()  
```
density of probability : \[ p_{\G} u) = \frac{1}{\sqrt{2 \pi \sigma^2}} e^{- \frac{(x-\mu)^2}{2\sigma^2}} \]

$k$ constant

$\D k = \text {"Output k"}$

  
      function rv = determinist()
        rv = 1
      end

Non-random random variables

Analysis

Synthesis

Random variable $X$

\[ \E{f(X) } = \int f(x) p_X (x) dx \]

$\E{\cdot}$ is linear: \[ \E{ X + \lambda Y} = \E X + \lambda \E Y \]
$\E{\cdot}$ is monotonic: \[ X > 0 \implies \E X > 0 \]

Scatter plot : plot many realizations of $X$

Uniform

Exponential

Gaussian distribution

$X_k$ series of random variables with

same distribution
no interactions

Example: A series of dice throw

Law of large number

\[ A_n = \frac 1 n \sum_{k=1}^n X_k \rightarrow \D{ \E X } \]

 
  function f=plot_avg(n,rv,avg) 
    xs = rv(n);
    ns = 1:n;
    s=0;
    for i=1:n
      ss(i) = xs(i) + s;
      s = s + xs(i);
    end
    ss = ss ./ ns;
    f=plot(ns,ss);
    hold on; 
    plot(ns, 0*ns + avg);
    hold off;
  end

$\E{\Dice} = \frac 7 2$

Uniform: $\E{U} = \frac 1 2 $

Exponential: $ \E{T} = \frac 1 {\lambda} $

Gaussian: $\E{G} = \mu $

Determinist : $\E {\D k } = k$

Characterizing distribution

mean $\E X$
$\Var X = \E{X^2} - \E{X}^2$ fluctuations around the mean

$ \Var X = \E{(X - \E X)^2} \geq 0 $
$ \Var \D k = 0 $
$ \Var ( X + \D k ) = \Var X $
$ \std X = \sqrt{\Var X} $

$\Var \Dice = \frac {35} {12}$

$\Var (2\, \Dice) = \frac {35} 3$

$\Var (\Dice_1 + \Dice_2) = \frac {35} 6 $

 function f=plot_var(n,rv,var) 
  xs = rv(n);
  ns = 1:n;
  s=0;
  s2=0;
  for i=1:n
    s = s + xs(i);
    s2 = s2 + xs(i) .^ 2;
    ss(i) = s2 ./ i - (s ./ i) .^ 2 ;
  end
  f=plot(ns,ss);
  hold on; 
  plot(ns, 0*ns + var);
  hold off;
end

$\Var{\Dice} = \frac {35} {12}$

Uniform: $\Var{U} = \frac {1} {12} $

Exponential: $ \Var{T} = \frac {1} {\lambda^2} $

Gaussian: $\Var{G} = \sigma^2 $

Determinist : $\Var {\D k } = k$

$\Var X > 0$

$\Var Y > 0$

$\Var (X+Y) \lt \Var X $

Scatter plot realisations of $(X,Y)$

$ \Cov(X,Y) = \E{XY} - \E X \E Y$
$\Var (X+Y) = \Var X + 2 \Cov (X,Y) + \Var Y $

Independence:

\[ \Cov (f(X), g(Y)) = 0 \iff p_{X,Y}(x,y) = p_X(x) p_Y(y) \]

Correlation

\[ \Corr(X,Y) = \frac{\Cov(X,Y)} {\sqrt{\Var X \Var Y}} \in [0,1] \]

Scatter plot realisations of $(X,Y)$

$X_i$ independent random variables \[ \Var (\frac 1 N \sum_{i=1}^{N} X_i ) = \frac{\Var X}{N} \]

How to measure $p_X$? Discrete variables:

\[ p_X (k) = \E { 1(X = k) } \]

    
        clf();
        nh = 100;
        h = zeros(1,6);
        for i = 1:nh 
          d = dice(1);
          h(d) = h(d) + 1;
        end
        h = h ./ nh;
        f = stem(h);
        hold on;
        plot(1:6, ones(6,1) / 6, 'k--' )

Continuous variable?

Discretize the observation space in $n_h$ bins
$x_k, x_{k+1} = x_k + \Delta $
$ P( X \in [x_k,x_{k+1}] ) = \int_{x_k}^{x_k+1} p_X(x) dx \approx \Delta p(x_k) $

Histogram on $[0,1]$

 
      n_b=10;
      delta = 1/n_b;
      n_sample = 1000;
      h = zeros(1,n_b);
      for i = 1:n_sample;
        x = rand();
        k = ceil( x * n_b );
        h(k) = h(k) + 1;
      end
      h = h ./ ( n_sample * delta )
     for i = 1:n_b 
      rectangle('Position', 
      [ (a + (i-1) * delta) 0 delta h(i)], 
      'FaceColor','b' );
      end

How many bins to choose

Too large bins

\[ P( X \in [x_k,x_{k+1}] ) \ne \Delta p(x_k) \]

Too narrow bins

High variance \[ \Var h(k) \approx \frac{ p(x_k) } {N \Delta} \]

Starting point: $U$

Uniform on [a,b] :

\[ a + (b-a) * U \]

 
      
        function d = dice()
          u = rand();
          for k=1:6 
           if u < k / 6
            d=k
            return
          end  
        end

Transforming random variable: $Y = f(X)$, f bijection \[ p_Y(y) = \frac{ p_x(f^{-1}(y)) } {| f'( f^{-1}(y) ) |} \]

$f(x) = - \ln x$, $p_Y(y) = e^{-y}$
$f(x) = \sqrt{x} $, $p_Y(y) = 2 y$

Uniform (X,Y) random variables on the unit disc

  
      function [x,y] = disc_rand()
        while true
        %draw x,y in the square [-1,1]^2
        x = 2 * rand() - 1
        y = 2 * rand() - 1
        % if they are in the disc, exit the loop
        if x^2 + y^2 < 1
          return          
        end 
       % otherwise try again
       end
      end

Uniform (X,Y,Z) random variables in the rectangular unit tetrahedron

  
      function [x,y,z] = tetrahedron_rand()
        while true
        %draw x,y in the cube [0,1]^3
        x = rand()
        y = rand()
        z = rand()
        % if they are inside the tetrahedron, exit the loop
        if x + y + z < 1
          return          
        end 
       % otherwise try again
       end
      end

$X \in [0,1]$ random variable
probability density $f$, $f([0,1]) \lt m$

 
      function x = uniform_reject(f,m)
        while true
        %draw x, y in [0,1]*[0,m]
        x = rand()
        y = m * rand()
        % if they are in the disc, exit the loop
        if y < f(x)
          return          
        end 
       % otherwise try again
       end
      end

Random variables: modeling imperfection