- Make simple stochastic models for real cases, e.g., psychiatrist planning.
- Code, in Python, models to simulate, compute KPIs, and analyze control policies
- Develop exact models for simple cases, e.g., waiting time formula for \(M/G/1\)
- Code, in Python, the simple models
- Make approximate models
- Code, in Python, these approximations

- Start with memoryless property of exponential, Poisson.
- Use simulation to construct sample paths for stochastic process; develop intuition for such systems.
- Compute KPI's based on sample paths
- Develop a discrete event simulation from scratch with heapqueues

- Analyze the sample paths analytically to get theoretical tools:
- renewal reward theorem,
- Little' law
- Pasta
- Level crossing (build up to Markov chains)

- Applications of theoretical tools: \(M/M/1\), \(M/G/1\), basestock and \((Q,r)\) policy
- Provide underlying motivation for approximate models

\begin{align*}
d_i &= \min\{L_{i-1}, c_i\}, \\
L_i &= L_{i-1} + a_i - d_i
\end{align*}

- Use this recursion to model a priority queue with blocking
- Use the recursion to model radiation reflection in the atmosphere due to \(CO_2\).

\(D_i = \max\{A_i, D_{i-1}\} + S_i\)

- (Most) analytic tools apply in continuous time
- Simulation of priority queue with discrete event simulation.
- \(M/M/1\), etc, are continuous time models

```
num = 10
X = rng.exponential(scale=1 / labda, size=num)
S = rng.exponential(scale=1 / mu, size=num)
X[0] = S[0] = 0
A = X.cumsum()
D = np.zeros_like(X)
for i in range(1, num):
D[i] = max(A[i], D[i - 1]) + S[i]
J = D - A # sojourn times
W = J - S # waiting times
print(J.mean(), J.var())
```

Define:

\begin{align*}
A(n,t) &:= \sum_{k=1}^{A(t)}\1{L(A_k-) = n} , \\
Y(n,t) &:= \int_0^t \1{L(s) = n} d s
\end{align*}

\begin{equation*}
\frac{A(n,t)}{Y(n,t)}\frac{Y(n,t)}t = \frac{A(n,t)}t = \frac{A(t)}t \frac{A(n,t)}{A(t)}.
\end{equation*}

Assuming the limits for \(t \to \infty\) exist

\begin{equation*}
\lambda(n) p(n) = \lambda \pi(n)
\end{equation*}

Expected waiting time for \(G/G/1\) queue: \(\E W \approx \frac{C_a^2+C_e^2}2 \frac\rho{1-\rho} \E S\)

Modify \(C_a^2\) and \(\E S\) for systems with

- batch services
- setup times
- Failures