Markov Chains

In the previous analyses of the NSAID-induced ulcer, the response to therapy with COX-2 was treated as a dichoto-mous yes-no variable. Patients were considered responders to COX-2 therapy only if they stayed completely free of recurrent ulcerations. In all other instances, COX-2 therapy was regarded a failure and the patients were automatically switched to a combination of a conventional NSAID plus PPI. In clinical practice, however, patients may be maintained on a COX-2 and the therapy would still be considered a success if the number of ulcer recurrences were halved or if the patient stayed ulcer-free for a longer time period than without a COX-2. The Markov chain provides a means to estimate the ulcer-free time and compare the success rates of competing treatment strategies.

Any acute ulcer can go in two directions: It can heal or it can stay acute. Similarly, any healed ulcer can become acute again or stay healed. The natural history of PUD can, therefore, be conceptualized as ongoing transitions between two health states (ie, acute and healed peptic ulcer). All that one needs to know to be able to set up the Markov model are the healing rate (HR) of acute ulcers and the relapse rate (RR) of healed ulcers. For the model of Figure 2-6, it was assumed that during a 1-month time period 40% of all acute ulcers would heal spontaneously and 8% would relapse. Because the entirety of possibilities to exit any given health state needs to add up to 100%, the monthly rate of patients remaining unhealed equals 100% - HR = 60%, and the monthly rate of healed ulcers without recurrence equals

acute ulcer

healed ulcer

100

0

acute ulcer 60

healed ulcer 40

>

f

x1

>

f

acute ulcer 39

healed ulcer 61

1

f

><

>

r

acute ulcer 28

healed ulcer 72

>

f

r

acute ulcer 23

healed ulcer 77

FIGURE 2-6. Markov chain (drawn in extensive form) for the natural history of an ulcer.

1st month

2nd month

3rd month

4th month

FIGURE 2-6. Markov chain (drawn in extensive form) for the natural history of an ulcer.

100% - RR = 92%. The same set of transition rates acts on the patient population with acute and healed ulcers every month. In essence, patients are shifted back and forth between the two Markov states of acute and healed ulcers until a steady state is reached. In the example of Figure 2-6, the analysis was started with 100 patients in the state of acute ulcerations. By looking at the numbers of patients in each two boxes of consecutive months, one can appreciate that after 4 months the numbers of patients in the acute and healed ulcer states start to approach some steady state. If the chain is continued for a few more months, a steady state is achieved with 17 patients staying continuously in the acute state and 83 patients in the healed ulcer state. In other words, at any given point in time after the chain has been allowed to settle, about 83% of ulcer patients will be ulcer-free even without medical intervention. One could also say that 83% of the time during the natural history of untreated ulcer disease is spent ulcer-free.

How does ulcer prevention with COX-2 affect its natural history? COX-2 maintenance therapy may half the RR = 4%, but it will probably leave the HR = 40% unaffected. Under these conditions the Markov chain of Figure 2-6 yields steady state conditions of 91% healed and 9% acute ulcers. This outcome represents an overall improvement by 8% compared with the 83% fraction of ulcer-free patients without any therapy at all. How would maintenance therapy with PPI affect the natural history of peptic ulcer? In addition to halving the RR by inhibiting acid secretion, PPI may also double the HR. With a RR of 4% and an HR of 80%, a steady state is reached with 95% of all subjects having a healed ulcer and 5% having an acute ulcer. Compared with no therapy or COX-2 therapy, this represents overall improvements of 12% and 4%, respectively.

It is important to realize that Markov chains address different questions and yield different answers compared to decision trees. Decision trees are centered on the concept of the expected value and how to select the best expected value from a multitude of alternative options. In Markov chains, by contradistinction, the analysis is focused on the average or cumulative amount of time spent in different medical states and how to select a management strategy that provides the most time spent in a healthy state.

Figure 2-6 depicts the extensive form of a Markov chain. Although the chain is stopped after 4 months, it could have been continued for an endless time. This type of drawing provides an intuitive explanation for why the analysis is referred to as a "chain," and it also allows the model to be transformed directly into a spreadsheet calculation. From a mathematical perspective, however, all the relevant information necessary to calculate the chain outcome is already contained in the transitions drawn for the first month only. Frequently, this relevant information can be condensed into a short form of a Markov chain, as shown in the upper drawing of Figure 2-7. The arrows pointing towards the same health state, from which they originate, indicate the fractions of patients who remain in the same health states during the monthly cycle. If a Markov chain is made up of more than a few health states, the extensive form becomes cumbersome to draw and difficult to appreciate with its many crisscrossing arrows. The short form lends itself to be used for the depiction of more elaborate Markov chains comprising multiple health states. As an example, in the bottom part of Figure 2-7, the basic model was expanded by two additional health states.

Was this article helpful?

0 0
Breaking Bulimia

Breaking Bulimia

We have all been there: turning to the refrigerator if feeling lonely or bored or indulging in seconds or thirds if strained. But if you suffer from bulimia, the from time to time urge to overeat is more like an obsession.

Get My Free Ebook


Post a comment