Markov Chain Question

[JackJill]

Suppose you have two players. Player 1 has $1 and Player 2 has $2. In any game, Player 1 has 2/3 chance of winning and Player 2 has 1/3 chance of winning. A win will give you $1 and a loss will take away $1. What is the probability that Player 1 end up with all of the money?

 

[Tsotne]

4/9. Construct a binomial tree

I am editing now...
No correct I'll correct it now. Sorry that was a quick response

 

[ValueSeeker]

We know:
P(0,0)=1
P($3,$3)=1
B/c they are absorbing states

Set up these independent equations:

Probability start at $1 and go to $3=(2/3)*Probability start at $2 and go to $3

Probability start at $2 and go to $3=(1/3)*Probability start at $1 and go to $3+(2/3)

By Solving for the unknowns, we have:

Probability start at $1 and go to $3=(4/7)

Thus, the probability he will get all the money is (4/7)

 

[Tsotne]

Through the Monte Carlo I got 0.572...

Do we know the correct answer to check?

 

[pathfinder]

4/7 is the correct answer

4/9 + 2/9 (4/9) + (2/9)^2 *(4/9) + 4/9 *(2/9)^3 .....

4/9 ( 1+ 2/9 + (2/9)^2 ....)


i.e 4/7

 

[pathfinder]

@Tstone....draw a binary tree you will get the right answer

 

[ValueSeeker]

Thats correct (at least very accurate),

(4/7)=.571429

 

[iHateVariance]

http://en.wikipedia.org/wiki/Gambler's_ruin

 

[Tsotne]

@Tstone....draw a binary tree you will get the right answer

You have to follow till the infinity to get the reasonable answer then. Monte Carlo is the better approach if you don't want to go through the analytical solution. If you have programmed it, then it simplifies your deal.

 

[pruse]

This sort of problem lends itself well to conditioning.

Let \(p\) be the probability that player 1 ends up with all the money. Then conditioning on the first two outcomes for the second player,

\(1-p=\frac{1}{3}\cdot 1+\frac{2}{3}\cdot \frac{1}{3}\cdot (1-p)\)

solving gives \(p=\frac{4}{7}\)

 

[AlexandreH]

This sort of problem lends itself well to conditioning.

Let \(p\) be the probability that player 1 ends up with all the money. Then conditioning on the first two outcomes for the second player,

\(1-p=\frac{1}{3}\cdot 1+\frac{2}{3}\cdot \frac{1}{3}\cdot (1-p)\)

solving gives \(p=\frac{4}{7}\)

Either Peter's solution or solve for P1 (P of Player 1 winning) = P(Player 1 winning twice in a row) + P(Player 1 winning once then losing once)*P1
P1 = (2/3)^2 + 2/3 * 1/3 *P1
solving gets P1 = 4/7