Stochastic OR/QTS

Nicky van Foreest


Table of Contents

1. Design

1.1. Course design

  • Make simple stochastic models for real cases, e.g., psychiatrist planning.
  • Code, in Python, models to simulate, compute KPIs, and analyze control policies
  • Develop exact models for simple cases, e.g., waiting time formula for \(M/G/1\)
  • Code, in Python, the simple models
  • Make approximate models
  • Code, in Python, these approximations

1.2. Approach

  1. Start with memoryless property of exponential, Poisson.
  2. Use simulation to construct sample paths for stochastic process; develop intuition for such systems.
  3. Compute KPI's based on sample paths
  4. Develop a discrete event simulation from scratch with heapqueues

1.3. Approach 2

  1. Analyze the sample paths analytically to get theoretical tools:
    1. renewal reward theorem,
    2. Little' law
    3. Pasta
    4. Level crossing (build up to Markov chains)
  2. Applications of theoretical tools: \(M/M/1\), \(M/G/1\), basestock and \((Q,r)\) policy
  3. Provide underlying motivation for approximate models

2. Examples

2.1. Example in Discrete time

\begin{align*} d_i &= \min\{L_{i-1}, c_i\}, \\ L_i &= L_{i-1} + a_i - d_i \end{align*}
  1. Use this recursion to model a priority queue with blocking
  2. Use the recursion to model radiation reflection in the atmosphere due to \(CO_2\).

2.2. Examples continuous time

\(D_i = \max\{A_i, D_{i-1}\} + S_i\)

  1. (Most) analytic tools apply in continuous time
  2. Simulation of priority queue with discrete event simulation.
  3. \(M/M/1\), etc, are continuous time models

2.3. Example code:

num = 10
X = rng.exponential(scale=1 / labda, size=num)
S = rng.exponential(scale=1 / mu, size=num)
X[0] = S[0] = 0
A = X.cumsum()

D = np.zeros_like(X)
for i in range(1, num):
    D[i] = max(A[i], D[i - 1]) + S[i]

J = D - A  # sojourn times
W = J - S  # waiting times
print(J.mean(), J.var())

2.4. Sample path proof, PASTA


\begin{align*} A(n,t) &:= \sum_{k=1}^{A(t)}\1{L(A_k-) = n} , \\ Y(n,t) &:= \int_0^t \1{L(s) = n} d s \end{align*}

2.5. Sample proof, PASTA

\begin{equation*} \frac{A(n,t)}{Y(n,t)}\frac{Y(n,t)}t = \frac{A(n,t)}t = \frac{A(t)}t \frac{A(n,t)}{A(t)}. \end{equation*}

Assuming the limits for \(t \to \infty\) exist

\begin{equation*} \lambda(n) p(n) = \lambda \pi(n) \end{equation*}

2.6. Examples: approximate models

Expected waiting time for \(G/G/1\) queue: \(\E W \approx \frac{C_a^2+C_e^2}2 \frac\rho{1-\rho} \E S\)

Modify \(C_a^2\) and \(\E S\) for systems with

  1. batch services
  2. setup times
  3. Failures