Notes on Markov models in pyton - with emphasis on Health Economic Evaluation

Markov models and health economic evaluation in python

Intro
Markov models are commonly used to do health economic evaluation. In this post I'll explore some tools and options for doing this kind of analysis in Python.

NetworkX
A good toold to construct a model formally, is to use the python package networkX. It contains more than you need, but it is quite easy to construct a model with nodes and edges (a directed graph). It is also easy to add properties to nodes and graphs (probabilities, utility, costs associated to states and events). It also allows you to export the model to other packages (like pandas). The model can also be exported to a standard language for networks (dot).

NetworkX is a tool for creating a model, but in order to visualize it, you may want to install some other tools and packages.

Visualizing the model
This turned out to be a little more tricky than I thought. In short you need to install something called Graphviz AND a Python wrapper, for instance: graphviz python

To visualize the markov model in a jupyter notebook nxpd works fine.

You may also want to consider xdot for more interactive visualizations (but it requires some quite heavy dependencies).

"Solving" the model
There are lots of tools for markov models in python, but they are usually not designed to handle the typical workflow in health economics.

In health economic models, the Markov model is used to analyse how an intervention affects the outcome. This is done by assigning utilities and costs to each state and transition probabilities between each state. An intervention could, for instance, reduce the probability of becoming sick. We then run a simulating with a given number of individuals, observe how many end up in each state in each time period (after each cycle of the model), calculate costs/utilities and go on for another step for the life span of all individuals. The results are summrized and we can compare the total  costs and benefits in models with different costs and transition probabilities.

Here are some tools that migh be used to do the simulation described above:

http://quant-econ.net/py/finite_markov.html

https://github.com/gvanderheide/discreteMarkovChain

http://pomegranate.readthedocs.io/en/latest/markovchain.html

https://github.com/riccardoscalco/Pykov

http://pymc-devs.github.io/pymc3/index.html


The quant package seems to be the one that fits best for our purpose. Alternatively pykov might also be used.

Here is a notebook that describes markov modelling using the quant package:

http://nbviewer.jupyter.org/github/QuantEcon/QuantEcon.notebooks/blob/master/markov_chain_ex01_py.ipynb