Quite simply it’s down to the likelyhood of the hand occuring, with rarer hands worth more.  We can calculate the probability of being dealt a hand by looking at the number of ways five cards can make that hand.

There are 311,875,200 ways (permutations) of being dealt five cards from a 52 card deck.  However since a poker hand has the same value whatever order the cards are arranged in we are interested in the number of unique ways (combinations).  To find this we divide by the number of ways a five card hand can be arranged, which is 5! = 5*4*3*2*1 = 120.

Thus the number of distinct hand combinations is 311,875,200 / 120 = 2,598,960.

Since a royal flush is just a particular (the best) straight flush we start by enumerating the straight flushes.

A straight flush is a hand of five cards in sequence, all of the same suit.  When comparing straight flushes, the hand with the highest card wins.  Suits have no relative value and are all ranked equally.  Obviously there are 4 royal flushes, and 36 other possible straight flushes.  So the probability of being dealt a straight flush is 40 / 2,598,960 ≈ 0.0015%.

Four of a kind (or quads) is fairly self explanatory - four cards of the same rank plus one more unmatched card.  Comparing quads the highest matched set wins.  If two players have the same four of a kind (i.e. in a community card game) then the player with the highest kicker (the unmatched card) wins.  There are 624 different ways to make a hand with four of a kind, hence the probability of being dealt one is 624 / 2,598,960 ≈ 0.024%.

A full house is a hand containing three cards sharing one rank, and two cards sharing another rank.  Between two full houses the hand with the higher ranking set of three wins.  If the set of three mayching cards is the same then the hand with the higher pair wins.  There are 3,744 possible full houses and so the probability of being dealt one is 3,744 / 2,598,960 ≈ 0.14%.

A flush is five cards all of the same suit, but not in sequence (otherwise it would be a straight flush).  As the suit has no value a flush with a higher ranking high card is considered superior.  There are 5,148 possible hands with all five cards sharing the same suit but 40 of these are straight flushes, so the probability of being dealt a flush is 5,108 / 2,598,960 ≈ 0.20%.

A straight is a hand of five cards in sequential order but of more than one suit (otherwise it would be a straight flush).  Note that as with straight flushes an ace may be high or low but not both (i.e. a straight can not be “around the deck” and include both a king and a 2).  Straights are compared by looking at their highest card - remember that if the ace is low it counts as a one!  There are 10,240 hands with all five cards in sequence but 40 of these are straight flushes so the probability of being dealt a straight is 10,200 / 2,598,960 ≈ 0.39%.

Three of a kind (also known as trips or a set) is a hand containing three cards of the same rank plus two unmatched.  As usual a higher ranked three of a kind beats a lower valued one, and if the rank is the same then kickers decide.  There are 54,912 hands containing three cards of the same rank which are not full houses.  Thus the probability of a three of a kind is 54,912 / 2,598,960 ≈ 2.1%.

Two pair is a hand containing two pairs of cards of the same rank, plus one unmatched card.  If two players both have two pair then the win is decided first by the high pair, then the low pair, then the kicker.  There are 123,552 two pair hands that are not full houses.  The probability of being dealt one is 123,552 / 2,598,960 ≈ 4.75%.

One pair is a poker hand containing a single pair of cards of the same rank, with three unmatched cards.  If two players both have one pair then the higher pair wins.  If they both have the same pair then it is decided by the kickers.  There are 1,098,240 different hands with a single pair.  Thus the probability of being dealt one of them is 1,098,240 / 2,598,960 ≈ 42.26%.

A high card hand is one with none of the above properties.  It does not have any cards of the same rank, they are not all in sequence and they are of more than one suit.  When comparing high card hands the one with the largest high card wins.  Since we have eliminated the other options we know there are 1,302,540 different ways to be dealt a high card hand and so the probability of being dealt one of them is 1,302,540 / 2,598,960 ≈ 50.12%.

Hope that clears up why the hand rankings are the way they are and how to compare hands of the same type.

Lee Jones

A great beginner’s book on hold’em.  The emphasis is on solid fundamental play to beat loose, low limit games.  It is an easy read and is particularly successful in pointing out the big mistakes which can cost a new player dearly.  A perfect choice for newcomers to hold’em.

Content

The first section “Hold’em : The Game” serves as an introduction and gives an overview of the fundamentals of the game.  Jones covers topics such as calculating pot odds and reading the board.  The focus is very much on bricks and mortar poker, so some adjustment to online stakes is required.  I would suggest that the games he is discussing would be roughly analogous to $0.5/$1 tables online.  The section ends with a quiz on the material it has covered.

The majority of the book is within the second section “Play of the Hand from Deal to Showdown”.  The author details the basics of winning play from preflop to showdown.  The advice will be no surprise to veterans (although they may feel it is not aggressive enough), advocating an ABC game of tight hand selection and betting for value.  I particularly like the preflop treatment - it’s a good introduction to starting hand selection, which is probably the single most important thing for beginners to learn.  There are preflop and post-flop quizzes, which I feel are both valuable.

The third section “Miscellaneous Topics” contains a number of short essays.  Some are more useful than others but I think the discussions of bluffing, bankroll, player types and discipline will be interesting to the new player.  For more advanced players the essay on implicit collusion will probably be the biggest draw.

Presentation

The book is well presented with an easy writing style.  Complicated concepts are well described, again making it ideal for newer players.

Perhaps not a presentation problem but unfortunately the book’s discussions of online play (or lack thereof) have not dated well.

Conclusion

A good buy for beginners.  However if you already play and win at small stakes then the book has less to offer you.  For instance Small Stakes Hold’em is much better suited to the intermediate player.

The players bet on the value of their hand which is the best 5 card combination they can make from those available to them.  Some games only allow players to use cards dealt to them while others allow a combination of those held by players individually and those shared amongst all the players (board or community cards).

Gameplay is usually similar with the dealership rotating amongst the players, some or all players being subject to forced betting (e.g. ante or blinds) each hand and a round of betting on each street (i.e. each time cards are changed/revealed).  Betting follows the usual protocols with each player being required to fold or match the current level of betting (calling) before optionally increasing the bet (raising).  A betting round ends when either all players have contributed equally to the pot or there is only one player remaining.  Note that players are not normally able to “bet out” other players due to lack of funds.  A player with all his money in the pot is all in and may continue without contributing more.  However if he wins he does not take any money contributed after his all in has been matched by the other players - the remainder goes to the player with the second best hand.

What is shared amongst all poker variants are the established hand rankings:

]]> http://www.smash-vegas.com/2008/08/poker-basics/feed/ http://www.smash-vegas.com/2008/08/anatomy-of-a-pokerbot-room-for-improvement/ http://www.smash-vegas.com/2008/08/anatomy-of-a-pokerbot-room-for-improvement/#comments Mon, 18 Aug 2008 11:39:14 +0000 jamie http://www.smash-vegas.com/?p=326 So you’ve tried coding your first bot along the lines of our previous post.  If you’ve gone to those lengths then you’ve probably also watched it do some stupid things.  So where can we improve upon our basic design?

Preflop

Probably the biggest single area for improvement is the preflop play.  If you get preflop right then it can really help the rest of the bot’s game.  Every good poker player knows that if you play correctly preflop you cut down on mistakes in later streets.

It’s quite hard to get a dynamic preflop system working.  One of the reasons is that preflop win percentages tend to be all bunched up.  Against nine opponents even pocket aces only have a 30.8% chance of winning (if all opponents go to showdown).  The other reason is the lack of information, particularly if we are not in late position.  These two factors make a dynamic preflop very sensitive.  Often it will perform well in late position when we have a good idea of how many players will hit the flop and what the pot size is, but poorly in early position.

A popular solution is to implement a static preflop strategy before switching to a dynamic strategy postflop.  Personally I would advocate Sklansky groups but I’ve also had some success with Lee Jones’ preflop strategy.  Basically any of the established preflop systems will work fine.

Pot odds calculation

Our pot odds calculation was very simplistic.  We can improve it in several ways.  If we can accurately estimate how many callers we will get then this will yield an immediate improvement.  A loftier goal is to change from estimating the pot odds for the round to estimating the implied odds for the rest of the hand.  Obviously the further away from the present we predict the less accurate our predictions will become, however it can still be very worthwhile to look at what the pot will be worth.

Prwin

We’re currently using (prwin + prtie/2) to estimate our chance of winning.  The problem with the default prwin is that it assumes a random distribution of hole cards in our opponents.  In other words it assumes that an opponent is just as likely to go to the flop with AA as he is with 72o (7 and 2 offsuit).  This is obviously incorrect as opponent hands are self selecting - only the better hands will reach the flop and later streets.

A simple way to make it better is to assume that all players will only go to the flop with the top x% of pocket hands.  This generic opponent modelling can be implemented very easily in openholdem as a weighted prwin.  We could pick an arbitrary value for x (say around 30%), although a more sophisticated approach is to collect data (perhaps by direct collection within our bot, or by averaging reach flop percentages from pokertracker) and calculate an average for how often players reach the flop.  As a rough approximation it is fair to say that if a player goes to the flop 20% of the time then he must only play the top 20% of hole cards.

An even more sophisticated approach is to weight prwin at the chair level instead of the table level.  We call this specific opponent modelling.  Back in winholdem this was only possible by writing your own prwin function with the capability to weight each combination of hole cards for each opponent.  By adjusting the weights with every action observed quite an accurate picture of which cards an opponent might hold can be built up (e.g. player A only plays the top 16% of pockets but has not bet the flop so he’s probably holding something in this range…).  With an accurate estimation of the range of hole cards opponents may be holding we can produce a very accurate prwin.

Fortunately openholdem has included an enhanced prwin (prw1326) which makes it much easier to implement this.  I’d recommend trying out the simpler weighted prwin before moving on to this though.

Predicting opponent actions

Opponent modelling isn’t just for predicting hand ranges.  It’s also to predict opponent behaviour.  Our first attempt did very little in the way of prediction (we simply assumed that half the opponents would call a raise) so there is plenty of room for improvement here.

One of the most useful things we can predict is whether an opponent will fold to a raise.  If we can predict this then we can include it in our expected value calculations.  A simple approach is to assign an arbitrary percentage chance based on observation.  More subtle would be to use pokertracker data on each player to determine how often they will fold to a raise on a particular street.  At the top of the spectrum we could train an artificial neural network to model opponents based on their pokertracker statistics and the current state of the game, and then use this to simulate the outcome of a raise (this can also be useful for predicting the pot odds).

Training a neural network is not particularly difficult.  The trick is identifying which factors are important to prediction and collecting enough good data to build an accurate model.  Once the neural network is built it can be exported (to a dll for example) and then accessed by your bot as required.

Conclusion

Our simple dynamic bot was a good start but has plenty of space for growth.  The key is to improve small areas at a time otherwise the task can seem overwhelming.

I hope this gives you some ideas.  If there’s enough interest I’ll go into more detail on some of the improvements and maybe mention some more.

The problem with all betting systems is that bet sizing does not affect the house edge.  It can increase your chance of winning in the short term, but that will always be balanced out by a large loss later (as in the martingale system).  The longer you play, the closer your loss will be to the house edge.

The reason that bet sizing cannot affect the house edge is simple - expected value is linear.  This means that:

expected value (A+B) = expected value (A) + expected value (B)

In other words it doesn’t matter what the individual amounts you wager are, overall your loss will equal the total you wager multiplied by the house edge.  There cannot be a clever way of sizing bets which would change this - otherwise expectation would not be linear.

Thus we find (although we already knew) that the equation for your loss at a gambling table is:

loss = (total amount wagered) * (house edge)

No betting system can alter that.

In fact the martingale has been around for hundreds of years.  The system is simple - by increasing the stake with each loss, when the gambler eventually makes a win it will cover all the previous losses and make a profit.

For example if we are flipping a fair coin with 1:1 payout if we guess correctly, our starting bet would be $1.  After each loss we double our stake and when we win the stake is reset to $1.  So if we lose once before winning we gain $2 - $1 = $1.  If we lose twice before winning we gain $4 - $2 - $1 = $1.  Losing three times before winning nets us $8 - $4 - $2 - $1 = $1.  And so on.

As you can see the win always nets us $1 - so mathematically it is unbeatable!

Of course there is a problem which I hope you can see.  The size of the bets grows exponentially and so the stakes quickly get very large indeed.  This means that sooner or later we will either run out of money to make the next bet, or hit the limit of what the casino will let us wager.

This happens sooner than you’d think.  In the example above after 10 losses in a row we would be betting over $1000 to win $1.

The martingale is only an unbeatable strategy if:

1. You have infinite wealth
2. There is no limit to the size of bet you may make

It tricks a lot of people because if you try it for a short period of time you are likely to finish ahead.  Unfortunately if you keep playing you will eventually hit a big loss which will wipe you out.

What the martingale really does is trade a lot of small wins now for a big loss later.

No betting system can change the expected value of a game.

Of course the answer is no.  No matter how many times you toss a fair coin it is still 50/50 that it will land heads or tails.

That’s because the coin has no memory and each throw is independent of the last.  Not recognizing this and thinking that a result is “due” to “balance out” the probabilities is the gambler’s fallacy.

It’s an easy mistake to make because we assume that short term trends should be representative of long term trends.  We know that in the long term that coin will come up heads about 50% of the time so we expect deviations from the average to even out.  We fail to understand that in the short term a run of coin tosses is unlikely to “look random” and clustering of results is the norm.

The key to avoiding this fallacy is recognizing when a game has memory and when each round is independent of the last.

Throwing a coin obviously has no memory.  While playing texas hold’em getting dealt pocket aces twice in a row does not change the odds of you being dealt them again in the next hand (provided the deck is shuffled between hands).

On the other hand since the deck in blackjack is not shuffled every hand, a high occurrence of low cards does mean that more high cards are “due”.  This is because the hands are not independent, and is the basis of card counting.

The gambler’s fallacy is at the root of many fraudulent betting systems, don’t get caught out by it.

David Sklansky, Ed Miller, Mason Malmuth

This book on small stakes should not be mistaken for a beginners book.  Although it focusses on the smaller stakes where beginners tend to dwell the concepts and analysis are applicable to any game with loose players - this makes it one of the best books available for improving your online play.

Content

Small Stakes Hold’em (SSH) begins with a discussion of gambling concepts such as expectation, pot odds and pot equity.  It’s a well written introduction which I thoroughly approve of because I (like the authors) believe that every poker decision boils down to “which choice maximises my expected value?”.  As you’d expect with Sklansky on board the authors do a great job of explaining these concepts in simple terms and detailing why they are important.

The next chapters cover the theory behind playing each of the streets (preflop, flop, turn, river) with an emphasis on recognizing the type of hand you hold and comparing your odds to the pot odds.  Hand types and the strategy for playing them are discussed for each street - e.g. play speculative preflop hands like low suited connectors or pocket pairs if they can see the flop cheaply, try to win straight away on the flop by protecting made hands that are good but not great, etc…

The advice is good and general, emphasising the critical importance of pot odds.  There is also some discussion of awareness of the game structure.  For instance the authors argue that because of the minimum bet increase in many games on the turn and river it is sometimes useful to forego betting on the flop to reduce the pot odds being offered to an opponent on the turn.  This is thought provoking stuff and the many sections like this (along with the lack of a basic “these are the rules of poker” chapter) ensure this is definitely not a beginners book and would be useful to a wide range of players.

The last part of the book is an extensive test yourself chapter, with questions covering all the topics discussed in the book.  It is one of the largest question and answer sections I’ve seen in a poker book and is very comprehensive.

If I had to sum up the theme of the book it would be to play tight and aggressive in a loose game, being constantly aware of the pot odds.  The authors argue (correctly in my view) that in a loose game where players will call with rags the correct attitude is not to become defensive because of “bad beats” but instead to punish the opponents for their constant weak calls.

Presentation

The book is generally well laid out, although some of the tables could be a little clearer.

There is some discussion of poker mathematics which does not go into too much depth but may put off some readers with the display of equations.

The writing style is good and the book is a quick read despite its length (over 350 pages).

The test yourself sections are a useful way of absorbing the theory by practical examples.

Summary

I like SSH and regularly recommend it to friends wanting to improve their online play.  I think it does a good job of teaching players to rely on playing a solid “by the numbers” game of poker instead of fancy hollywood poker plays.  There’s a lot here even for advanced players who are perhaps not as familiar with the lower stakes and the book does a good job of explaining not only the differences in strategy at this level but the reasons for the differences.

They’re a method of computation which use repeated random sampling to produce results.  A simpler way of putting it would be “you try the thing lots of times to see what happens”.

A simple example would be calculating the probability of flipping two coins and getting two heads.  We know the probability is 1/2 * 1/2 = 1/4.  What if we couldn’t work that out?  A monte carlo method to calculate the probability would be to try the experiment many times, record the results and see what proportion of the time both were heads.  Obviously the more trials we perform, the more accurate a result we will get.  We will never get a perfectly accurate result but as the number of attempts increases, the result will get closer to 1/4.

This example probably seems silly because the answer is so obvious.  Where monte carlo methods come into their own are in problems which are too complex to calculate exactly.  This is why they are so crucial to pokerbots.

Consider a game of heads up holdem.  If we want to calculate our probability of winning a particular hand then we must compare it to all possible opponent hands.  Suppose we are preflop -

• there are 50×49/2 = 1,225 possible opponent hands
• there are 48×47x46×45x44/120 = 1,712,304 combinations of board card
• that leaves us with 1,225×1,712,304 = 2,097,572,400 possibilities to consider

That’s not too bad for a modern computer to calculate, but this is the simplest case.  Things quickly get out of hand as the number of opponents increases - e.g. two opponents have 690,900 possible hand combinations while nine opponents have 6.2211×10²⁰ (622 quintillion).

Much better would be to simulate the outcome of the hand many (10,000 - 1,000,000) times.  It is less computationally intensive and so will take less time.  100,000 trials will give us a winning percentage to an accuracy of several decimal places and we can adjust the number of trials to meet our desired accuracy.

Another advantage is that we can distribute the cards in the simulations in a non-random way.  Suppose we know that an opponent only goes to the flop 20% of the time, we could include this information in the simulation and only deal him the top 20% of hole cards.

Probably best to continue that train of thought in another post though.

I hope everyone likes the new design.  There are still a few bugs (the archives button is playing up for instance) but overall it wasn’t too painful a move and I think it’s quite an improvement.

We’re now hosted by DreamHost and I’ve just found out that all their hosting is carbon neutral, a nice bonus to what seems like a great company.  No complaints so far.

All that remains is to apologise again for the disruption and let you know it’s back to business as usual.