An introduction to stochastic modeling / Mark A. Pinsky , Samuel Karlin.

CONTENTS

1. Introduction

1.1 Stochastic Processes
1.2 Probability review
1.3 The Major Discrete Distributions
1.4 Important Continuous Distribution
1.5 Some Elementary exercises

2. Conditional Probability and Conditional Expectation

2.1 The Discrete Case
2.2 The Dice Game Craps
2.3 Random Sums
2.4 Conditioning on a Continuous Random Variable
2.5 Martingales

3. Markov Chains: Introduction

3.1 Definitions
3.2 Transition probability Matrices of a Markov Chain
3.3 Some Markov Chain Models
3.4 First step Analysis
3.5 Some special Markov Chain

4. The long Run Behavior of Markov Chains

4.1 Regular Transition Probability Matrices
4.2 Examples
4.3 The Classification of states
4.4 The Basic Limit Theorem of Markov Chains
4.5 Reducible Markov Chains

5. Poisson Processes

5.1 The Poison Distribution and the poison Process
5.2 The Law or Rare Events
5.3 Distributions associated with the Poisson process
5.4 The Uniform Distribution and Poisson processes
5.5 Spatial Poisson Processes

6. Continuous Time Markov Chains

6.1 Pure Birth Processes
6.2 Pure Death Processes
6.3 Birth and Death Processes
6.4 The Limiting behavior of Birth and Death Processes
6.5 Birth and Death Processes with Absorption into State 0

7. Renewal Phenomena

7.1 Definition of a renewal processes and related Concepts
7.2 Some Examples of renewable Processes
7.3 The Poison Process Viewed as a Renewal process
7.4 The Asymptotic behavior of renewable Processes
7.5 Generalizations and Variations of Renewable processes

8. Brownian Motion and Related Processes

8.1 Brownian Motion and Gaussian Processes
8.2 The maximum Variable and the Reflection Principle
8.3 Variations and Extensions
8.4 Brownian Motion with Drift
8.5 The Ornstein-Uhlenbeck Process

9. Queueing Systems

9.1 Queueing Processes
9.2 Poisson Arrivals, Exponential Service time
9.3 General Services Time Distributions
9.4 Variations and Extensions
9.5 Open Acyclic Queueing Networks

10. Random Evolutions

10.1 Two-State Velocity Model
10.2 N-State Random Evolution
10.3 Weak Law and Central Limit Theorem
10.4 Isotropic Transport in higher Dimensions

11. Characteristic Functions and their Applications

11.1 Definition of characteristic Function
11.2 Inversion Formulas for characteristic Functions
11.3 Inversion Formula for general Random Variables
11.4 The Continuity Theorem
11.5 Proof of the Central Limit Theorem

Includes Index : p.557-563