Smash Vegas

So you’ve tried coding your first bot along the lines of our previous post.  If you’ve gone to those lengths then you’ve probably also watched it do some stupid things.  So where can we improve upon our basic design?

Preflop

Probably the biggest single area for improvement is the preflop play.  If you get preflop right then it can really help the rest of the bot’s game.  Every good poker player knows that if you play correctly preflop you cut down on mistakes in later streets.

It’s quite hard to get a dynamic preflop system working.  One of the reasons is that preflop win percentages tend to be all bunched up.  Against nine opponents even pocket aces only have a 30.8% chance of winning (if all opponents go to showdown).  The other reason is the lack of information, particularly if we are not in late position.  These two factors make a dynamic preflop very sensitive.  Often it will perform well in late position when we have a good idea of how many players will hit the flop and what the pot size is, but poorly in early position.

A popular solution is to implement a static preflop strategy before switching to a dynamic strategy postflop.  Personally I would advocate Sklansky groups but I’ve also had some success with Lee Jones’ preflop strategy.  Basically any of the established preflop systems will work fine.

Pot odds calculation

Our pot odds calculation was very simplistic.  We can improve it in several ways.  If we can accurately estimate how many callers we will get then this will yield an immediate improvement.  A loftier goal is to change from estimating the pot odds for the round to estimating the implied odds for the rest of the hand.  Obviously the further away from the present we predict the less accurate our predictions will become, however it can still be very worthwhile to look at what the pot will be worth.

Prwin

We’re currently using (prwin + prtie/2) to estimate our chance of winning.  The problem with the default prwin is that it assumes a random distribution of hole cards in our opponents.  In other words it assumes that an opponent is just as likely to go to the flop with AA as he is with 72o (7 and 2 offsuit).  This is obviously incorrect as opponent hands are self selecting - only the better hands will reach the flop and later streets.

A simple way to make it better is to assume that all players will only go to the flop with the top x% of pocket hands.  This generic opponent modelling can be implemented very easily in openholdem as a weighted prwin.  We could pick an arbitrary value for x (say around 30%), although a more sophisticated approach is to collect data (perhaps by direct collection within our bot, or by averaging reach flop percentages from pokertracker) and calculate an average for how often players reach the flop.  As a rough approximation it is fair to say that if a player goes to the flop 20% of the time then he must only play the top 20% of hole cards.

An even more sophisticated approach is to weight prwin at the chair level instead of the table level.  We call this specific opponent modelling.  Back in winholdem this was only possible by writing your own prwin function with the capability to weight each combination of hole cards for each opponent.  By adjusting the weights with every action observed quite an accurate picture of which cards an opponent might hold can be built up (e.g. player A only plays the top 16% of pockets but has not bet the flop so he’s probably holding something in this range…).  With an accurate estimation of the range of hole cards opponents may be holding we can produce a very accurate prwin.

Fortunately openholdem has included an enhanced prwin (prw1326) which makes it much easier to implement this.  I’d recommend trying out the simpler weighted prwin before moving on to this though.

Predicting opponent actions

Opponent modelling isn’t just for predicting hand ranges.  It’s also to predict opponent behaviour.  Our first attempt did very little in the way of prediction (we simply assumed that half the opponents would call a raise) so there is plenty of room for improvement here.

One of the most useful things we can predict is whether an opponent will fold to a raise.  If we can predict this then we can include it in our expected value calculations.  A simple approach is to assign an arbitrary percentage chance based on observation.  More subtle would be to use pokertracker data on each player to determine how often they will fold to a raise on a particular street.  At the top of the spectrum we could train an artificial neural network to model opponents based on their pokertracker statistics and the current state of the game, and then use this to simulate the outcome of a raise (this can also be useful for predicting the pot odds).

Training a neural network is not particularly difficult.  The trick is identifying which factors are important to prediction and collecting enough good data to build an accurate model.  Once the neural network is built it can be exported (to a dll for example) and then accessed by your bot as required.

Conclusion

Our simple dynamic bot was a good start but has plenty of space for growth.  The key is to improve small areas at a time otherwise the task can seem overwhelming.

I hope this gives you some ideas.  If there’s enough interest I’ll go into more detail on some of the improvements and maybe mention some more.