# Stochastic OR/QTS

## 1. Design

### 1.1. Course design

• Make simple stochastic models for real cases, e.g., psychiatrist planning.
• Code, in Python, models to simulate, compute KPIs, and analyze control policies
• Develop exact models for simple cases, e.g., waiting time formula for $$M/G/1$$
• Code, in Python, the simple models
• Make approximate models
• Code, in Python, these approximations

### 1.2. Approach

2. Use simulation to construct sample paths for stochastic process; develop intuition for such systems.
3. Compute KPI's based on sample paths
4. Develop a discrete event simulation from scratch with heapqueues

### 1.3. Approach 2

1. Analyze the sample paths analytically to get theoretical tools:
1. renewal reward theorem,
2. Little' law
3. Pasta
4. Level crossing (build up to Markov chains)
2. Applications of theoretical tools: $$M/M/1$$, $$M/G/1$$, basestock and $$(Q,r)$$ policy
3. Provide underlying motivation for approximate models

## 2. Examples

### 2.1. Example in Discrete time

\begin{align*} d_i &= \min\{L_{i-1}, c_i\}, \\ L_i &= L_{i-1} + a_i - d_i \end{align*}
1. Use this recursion to model a priority queue with blocking
2. Use the recursion to model radiation reflection in the atmosphere due to $$CO_2$$.

### 2.2. Examples continuous time

$$D_i = \max\{A_i, D_{i-1}\} + S_i$$

1. (Most) analytic tools apply in continuous time
2. Simulation of priority queue with discrete event simulation.
3. $$M/M/1$$, etc, are continuous time models

### 2.3. Example code:

num = 10
X = rng.exponential(scale=1 / labda, size=num)
S = rng.exponential(scale=1 / mu, size=num)
X[0] = S[0] = 0
A = X.cumsum()

D = np.zeros_like(X)
for i in range(1, num):
D[i] = max(A[i], D[i - 1]) + S[i]

J = D - A  # sojourn times
W = J - S  # waiting times
print(J.mean(), J.var())


### 2.4. Sample path proof, PASTA

Define:

\begin{align*} A(n,t) &:= \sum_{k=1}^{A(t)}\1{L(A_k-) = n} , \\ Y(n,t) &:= \int_0^t \1{L(s) = n} d s \end{align*}

### 2.5. Sample proof, PASTA

\begin{equation*} \frac{A(n,t)}{Y(n,t)}\frac{Y(n,t)}t = \frac{A(n,t)}t = \frac{A(t)}t \frac{A(n,t)}{A(t)}. \end{equation*}

Assuming the limits for $$t \to \infty$$ exist

\begin{equation*} \lambda(n) p(n) = \lambda \pi(n) \end{equation*}

### 2.6. Examples: approximate models

Expected waiting time for $$G/G/1$$ queue: $$\E W \approx \frac{C_a^2+C_e^2}2 \frac\rho{1-\rho} \E S$$

Modify $$C_a^2$$ and $$\E S$$ for systems with

1. batch services
2. setup times
3. Failures