Smash Vegas

So you’ve tried coding your first bot along the lines of our previous post.  If you’ve gone to those lengths then you’ve probably also watched it do some stupid things.  So where can we improve upon our basic design?

Preflop

Probably the biggest single area for improvement is the preflop play.  If you get preflop right then it can really help the rest of the bot’s game.  Every good poker player knows that if you play correctly preflop you cut down on mistakes in later streets.

It’s quite hard to get a dynamic preflop system working.  One of the reasons is that preflop win percentages tend to be all bunched up.  Against nine opponents even pocket aces only have a 30.8% chance of winning (if all opponents go to showdown).  The other reason is the lack of information, particularly if we are not in late position.  These two factors make a dynamic preflop very sensitive.  Often it will perform well in late position when we have a good idea of how many players will hit the flop and what the pot size is, but poorly in early position.

A popular solution is to implement a static preflop strategy before switching to a dynamic strategy postflop.  Personally I would advocate Sklansky groups but I’ve also had some success with Lee Jones’ preflop strategy.  Basically any of the established preflop systems will work fine.

[Read more]

With all this talk about pokerbots I’ve mentioned monte carlo simulations quite a lot.  But what are they?

They’re a method of computation which use repeated random sampling to produce results.  A simpler way of putting it would be “you try the thing lots of times to see what happens”.

A simple example would be calculating the probability of flipping two coins and getting two heads.  We know the probability is 1/2 * 1/2 = 1/4.  What if we couldn’t work that out?  A monte carlo method to calculate the probability would be to try the experiment many times, record the results and see what proportion of the time both were heads.  Obviously the more trials we perform, the more accurate a result we will get.  We will never get a perfectly accurate result but as the number of attempts increases, the result will get closer to 1/4.

This example probably seems silly because the answer is so obvious.  Where monte carlo methods come into their own are in problems which are too complex to calculate exactly.  This is why they are so crucial to pokerbots.

Consider a game of heads up holdem.  If we want to calculate our probability of winning a particular hand then we must compare it to all possible opponent hands.  Suppose we are preflop -

• there are 50×49/2 = 1,225 possible opponent hands
• there are 48×47x46×45x44/120 = 1,712,304 combinations of board card
• that leaves us with 1,225×1,712,304 = 2,097,572,400 possibilities to consider

That’s not too bad for a modern computer to calculate, but this is the simplest case.  Things quickly get out of hand as the number of opponents increases - e.g. two opponents have 690,900 possible hand combinations while nine opponents have 6.2211×10²⁰ (622 quintillion).

Much better would be to simulate the outcome of the hand many (10,000 - 1,000,000) times.  It is less computationally intensive and so will take less time.  100,000 trials will give us a winning percentage to an accuracy of several decimal places and we can adjust the number of trials to meet our desired accuracy.

Another advantage is that we can distribute the cards in the simulations in a non-random way.  Suppose we know that an opponent only goes to the flop 20% of the time, we could include this information in the simulation and only deal him the top 20% of hole cards.

Probably best to continue that train of thought in another post though.

Now that we have decided on our strategy it’s time to start coding the bot.  We’re going to use the default formulae, so I’d recommend you take a look at this.

We will need to be familiar with some of the symbols openholdem uses.  You can find a complete list here but these should suffice for now;

• call tells us the current cost to call.
• bet is the amount of a single bet for the current round.
• pot is the current amount in the pot.
• br is the current bet round (1=preflop,2=flop,3=turn,4=river).
• prwin is our probability of winning the hand (estimated with a monte carlo simulation).
• prtie is the probability of a tie for the win.
• prlos is our probability of losing the hand.
• callshort is the total amount which will be added to the pot if all players call.
• raisshort is the total amount which will be added to the pot if all players still in the hand call one bet.  I.e. it is callshort + bet * (number of players still in the hand).

Let’s start with coding the formulae for the expected values of calling and raising; f$evcall and f$evrais.  Going with the strategy we decided last time we have;

##f$evcall##
0                         // start at 0
- call                    // my cost to call
+ (pot+call)           // the minimum possible pot
* (prwin+prtie/2)    // my chances of winning

[Read more]

In this post we are going to design the logic we want our bot to follow.

Openholdem works by looking in turn at several functions which you design.  It stops to act on the first one which evaluates true.  First it checks the all in function, then the stated wager function, then the raise function, then the call function.  Since we are programming a limit bot we can ignore the first two (and make them always return false).  That leaves us with a raise function and a call function to design.

Our bot is going to play simple ABC poker.  We want it to evaluate the expected value of raising and calling, and then choose the greatest out of raising, calling and folding (the EV of folding is always 0).

A quick and dirty estimate of our EV of calling is:

(current pot + our call + other callers)*(our probability of winning) - (our call)

Similarly our estimate for the EV of raising is:

(current pot + our call + our bet + other callers)*(our probability of winning) - (our call + our bet)

Basically both cases are “what we could win multiplied by our chance of winning, minus what it costs us to call”.  They are analogous to looking at the pot odds and comparing them with our chance to win, but instead give us a numerical output of how much we expect to win or lose with a play in the long term.  These EV formulae are very basic, but later on we’ll look at ways to make them a bit better.

Next time we’ll look at how to code this as an Openholdem formula.

Over this series of posts I’m going to talk you through building a basic bot to play limit holdem. It won’t be a world beater, in fact it won’t even break even long term, but it should serve as a demonstration of the principals and a base to build on if you take up the hobby. I think you’ll be surprised how sophisticated a bot can be with very few lines of code.

Like building anything else a good plan helps immensely. We’re going to use a premade program to interface with our chosen poker site, so that we can concentrate on the logic the bot should follow.  The program in question is called Openholdem, and will take care of watching the poker site window to see what’s happening (screen scraping), evaluating the logic we have programmed and then pressing the relevant buttons in the poker window.  It will also provide us with many premade functions for evaluating the situation.  Certainly for our basic bot it’s all we’ll need.

Part of openholdem is called openscrape and is responsible for the screen scraping.  You provide it with a table map of the poker site which shows it where buttons and text are.  You can either download these table maps or create your own (it’s very easy but takes a little time and patience).  Openholdem also has the ability to import data from Pokertracker if you have it installed.  It’s easy to do but beyond the scope of this little project.

I would be remiss if I didn’t mention a topic of utmost importance to botting - stealth.  If you’re doing this for real then you need to use some clever tricks to prevent the poker window from knowing that it’s being operated by a bot.  The poker window can see what processes you’re running, where you’re clicking and gather all sorts of data on you to report back to the casino.  Since this is a basic tutorial I’m going to recommend “running naked” (as it’s known in the community) and using a disposable account.  Either pick a poker site you’re never going to use again or sign up for a play money only account.

Continue to Anatomy of a pokerbot - strategy.