Computer vs Poker Professionals

Four of the top professional world poker players spent the majority of January holed up at the Rivers Casino, Pittsburgh. They showed up before 11 am, wearing fancy sweatpants and stylish sneakers. They sat down in front of the computer screens. Each one of them was supposed to play 1,500 hands of heads-up no limit, Texas Hold ‘Em, online before they would be allowed to go back to the hotel nightly

This many times meant they worked past 10 p.m. during the course of the day. Mostly there were Starbucks cups and plenty of water bottles that were piled up next to the player's keyboards. The Chipotle bags lay at their feet.

Every time a player made a move, the action was transmitted to a computer server that was sitting five miles away at the Carnegie Mellon University. From that point, a signal would be travelling to another 12 miles to a opponent. It is a piece of software that is called Libratus which is running at the Pittsburgh Supercomputing Center, Monroeville, in a nearby suburb. The Libratus played eight hands at once — the two of them against each of the opponents. this moved at a deliberate pace, which was slow enough to drive Jason Les, their opponents are athletic-looking men that all seemed very eager to take some time off. One of the players said the waiting should not affect him, but that sometimes you feel like, 'well OK, is this going to be over soon?''

Bots Don't Need a Break

Libratus, never needs a break. It is very different from the human players in many ways, too. The computer plays slowly on the small pots, this is a result of the computer having to scroll through all additional possibilities that come from more chips that remain in its hand. Libratus takes opportunities to make huge, sudden wagers, which is a violation of the standard betting conventions by throwing its money into the pot with irregular amounts and at odd intervals.

If this behavior came from a human player, it would be irritating, reckless and, also over the long run, very expensive. However Libratus’s main attribute, as a poker player, is that it is inhumanly good. When the 20-day tournament, at Rivers, came to an end, the humans had lost $1.8 million. (The players did not actually have to cough up the cash; the money serves as the way of keeping score in poker.) The two scientists, Thomas Sandholm and Noam Brown, from Carnegie Mellon who had built Libratus, celebrated the win as it was the first time that a computer had beaten the top poker players at a variant of the unlimited Texas hold’em, this is the world’s most prominent poker game.

Artificial intelligence experts, have always used games as a method of developing and testing their creations. Computers have proven to surpass the best human players when playing against players on chess, checkers, backgammon. Poker, however, is a distinct challenge because it has an element of chance, and that the players don’t know what cards their opponents might be holding. The so-called imperfect information games, need a sort of human intelligence, such as, deceiving an opponent and being able to sense when a player is deceiving you, that is something that computers lack.

Perfect Bet's Author

The author of The Perfect Bet, Adam Kucharski, sand that the No limit hold’em is the game that one mostly sees in tournaments, and that it has a reputation of it being more of an art than of a science. How Science and Math Are Taking the Luck Out of Gambling. He continued to say that there was an idea that the game would be safer for longer from these machines.

That idea has recently been blown up. During January, the researchers at the University of Alberta had released a paper that is based on a contest where their own AI, named DeepStack, had beaten 11 professional poker players.

As to whether DeepStack would beat Libratus to the punch is currently a matter of debate. Mr. Sandholm stated that the pros who played against the bot were much better than those the DeepStack had defeated. The head of University of Alberta’s computer program, Michael Bowling, has conceded on this point. However, he questioned if humans are at their best when they have been playing continuously for almost a month.

Both of these men have agreed that poker’s AI has just crossed a significant threshold. For them it has very little to do with poker itself. The Hold'em is just a method of finding sparring partners for their artificial intelligence (AI) programs, and the gains that were made by the game-playing bots will filter back into the applications like cybersecurity.

Mr. Sandholm said that this is the main benchmark that the community had settled on, however, these algorithms are not for poker. Mr Sandholm was once known as one of the world's top-ranked windsurfers and he kinda looks like Bill Gates.

The DeepStack and the Libratus play an unusual version of poker. The computers were matched up against one opponent, as opposed to a group of players. The number of chips that each player holds was reset after each hand, this eliminated the psychological game where the players that have more chips can intimidate the poorer players by forcing them to make bigger bets.

Eric Hollreiser, who is a spokesman for Amaya Inc., said this limited any threat that the AI posed to the poker industry. He said that while on a functional hand-by-hand basis, it mimicked poker play, that it was by far removed from reality of what really happens at Poker tables.

There are less controlled experiments going on. The Poker bots have been playing on online cash games for almost as long as the scientists have been building them in the labs. Historically they have played low-stakes games and have not been considered to be very skilful. A gambling industry analyst, Chris Grove said that the but bots are spreading into the higher-stakes contests, if you are an online poker operator, then this is probably the number one fraud concern, and he continued probably by a very wide margin.

Collaboration

The poker industry and academic poker world have quietly been collaborating over many years. Everyone that is involved remains quite sketchy on details. However, both of the people who build the commercial bots and those that are trying to combat them have a close eye on the academic work. Many of the Mr. Bowling’s former students have, in fact, gone on to work for the online poker companies.

Mr Sandholm said that people are worried it would kill internet gambling for money, because they worry about the bots being so good they will be had. But he said that was not his concern.

In poker industry slang, a computer program that is able to do your playing for you is called a “dream machine.” Many of he participants who write on online forums are swapping notes about suspicious activities that might indicate robotic play also ‘war stories’ about how they have made their own bots.

Amaya, who is the operator of online poker operations, PokerStars and Full Tilt, currently employs 70 who work at combating this kind of fraud. The PokerStars employees call on players and ask them to describe the strategies they use on certain hands. The company has also sent e-mails to the players that require them to make a video where the users need to rotate the camera 360 degrees to show the surroundings they are playing in, then they have play for over an hour with their hands and the keyboards fully visible.

Bots Don't Need to be Highly Skilled

The bots don’t need to be wildly skilled at poker for them to be profitable for their operators. A program that is able to make a modest profit by exploiting the mediocre players might be worth it. Darse Billings, who is the head of poker strategy, at Gamesys, which is a UK-based online gaming company, said that the dream machines and the academic AI’s are making use of different techniques and that they are trying to solve different challenges.

Mr. Billings understands both poker worlds. He has studied the game whilst he was getting a master's degree in computer science during the 1990s, then he became a professional poker player which his winnings helped him to pay off his student loans.

In later years he went back to school in order to work with Jonathan Schaeffer, who is a computer scientist at the University of Alberta and is best known for his writing of software that could play checkers perfectly. Mr. Billings convinced Mr. Schaeffer to focus on poker next.

To solve the checkers question, Mr. Schaeffer used a method that essentially tried to calculate the best move in any of the relevant situations, without consideration of what had happened up to that point.

It did not make any sense to for the bot to think about each move, as an isolated problem, in a game such as poker, where luck is involved and that not everyone has access to the important relevant information. The researchers at the University of Alberta set out to develop an overall strategy. In Game theory this entailed looking at what is known as a Nash equilibrium, an approach used for playing in a two-person game, that cannot lose over the long run, regardless of what one's opponent does to respond.

The Nash equilibrium is not a single ideal style of play. A key to the equilibrium strategy in poker is to play the potential strongest hands possible at the same time remaining unpredictable. When betting a strong hand, there needs to remain some doubt. said Mr. Billings. The team has developed a cautious AI,which is as dubbed Mr. Pink, and also a very aggressive one, called Agent Orange. It’s not easy to talk about a computer program is capable of this without sounding like one is talking about something that to actual thinks.

The Equilibrium Approach

The equilibrium approach got the attention of Mr. Bowling of the University of Alberta, whose speciality is game theory, towards poker in 2003. Mr. Sandholm, who sat on Mr. Bowling's thesis committee at the Carnegie Mellon, got involved in poker in the following year, and he has taken on a similar approach. Mr. Sandholm and Mr. Bowling then started the Annual Computer Poker Competition together during 2006, and they have periodically played against many top human players. While they were competing the labs have been gaining insights from each other’s research ever since.

Both of the programs took huge steps toward the endgame over the past several years. During January of 2015, Mr. Bowling’s team published a paper that showed how they had solved heads-up limit hold’em, which is a two-person poker game which is simpler to the no-limit hold’em because of the restrictions on the way players can bet.

Mr. Sandholm and Mr. Brown, a Ph.D. student who has worked with him on the poker AI over the past five years, held their first “Brains v. AI” competition against the top humans at the Rivers Casino several months later. Their bot, which was named Claudico, lost $732,000 in 80,000 hands that were played against four known professional players. Mr. Sandholm said the match was very close to be considered a draw, one of the players disputed this.

Mr. Sandholm and Mr. Brown said that there were several general areas that their AI had improved since then. Claudico played very well in the early stages of the game, however it tended to make mistakes towards the end of the hands. The AI bluffed at the wrong moments, and it had trouble accounting how the odds of the game had changed based on the cards that had been removed from the deck. The reasoning behind this is if there are two kings on view on the table and your hand has two kings, your opponent cannot have any.

Libratus has improved on all these areas. Its creators are remaining coy on some of the other areas, such as, how it chooses to make adjustments that are based on what it learns each time it plays.

When the publishers reveal their findings all will be revealed. Mr Bowling said that his research papers were on message forums where people build their own bots.