Monopoly AI

I built this because I have played Monopoly and could see where an AI might do something interesting with it. Mostly I was curious whether a program could beat my friends, my family, or me.

Monopoly suits the idea. Apart from the dice and the order of the cards, everyone can see everything, so the only thing hidden is what the other player will do next. Some players accept any trade. Some bid far too much at auctions. If you can tell which kind you are sitting across from, you can use it, the way you would if you noticed a friend always buys the first property they land on. I wanted to see whether a program could work that out mid-game, and whether it would win more because of it.

To try it I first had to write the game. I built a Monopoly simulator in Python with the standard rules: auctions, mortgages, trades, houses and hotels, bankruptcy. It passes 37 rule tests and played a thousand random games without breaking. Then I built the opponents, a set of scripted bots that each have their own habits, some careless and some careful.

The program that came out of it watches what an opponent does and learns to guess what they will do next. Before each big decision (buy this, bid that, build here) it plays the game forward a few moves in its head, with the opponent doing what it guesses, and picks the option that works out best. The guessing is the part I cared about. If it is useless, the program should do just as well by assuming the other player moves at random.

What happened

The guessing helped. Against a mix of twelve different bots it won about 83 games out of 100 when it was guessing, and about 73 when it assumed random moves. Against four bots it had never met, it won 69 against 56. That is a real gap, not a huge one. Against one careful, well-played bot, guessing made no difference I could measure.

Getting there took most of the time. My first approach was to let a neural network learn by playing itself. It learned to beat a random player nearly every time and almost never beat a decent scripted one. When I gave it a way to read opponents it read them fine, then ignored what it had read. Giving it ten times more practice games per round, on a rented, much more powerful computer, changed nothing. Then the first version of the "think ahead" step played worse than one that assumed random against bots that accept any trade, because it kept grabbing trades instead of building. I fixed that, but the guessing still did not help, because it only changed which trades I offered and sensible bots refuse lopsided trades. The gain showed up once the guess shaped every decision: buying, bidding, building and trading.

All of this is two-player games against bots I wrote. I have not played it against a real person yet, so I cannot say how it would do at a family table.

What's next

First, a bigger test with more bots, including ones that play more like people, and games with more than two players. What I really want is a program that plays better than any of us, and this one is not that yet. If I get there, Christmas Monopoly with the family is where I will find out.

The problem

I picked Monopoly because I have played it, and I could see room for an AI to do something interesting. I was curious whether it could beat my friends, my family or me.

I wanted a Monopoly agent that works out who it is playing and exploits it, instead of playing one fixed style. Apart from dice and card order the game is perfect information, and auctions and trades give a careless opponent plenty of ways to lose. The reward comes only from the final result (placement, later blended with final net worth). Nothing rewards a strategy directly.

First I needed something to train against. The engine is about 1,150 lines of plain Python with the standard US rules: house and hotel shortage, even building, auctions, mortgages, bankruptcy, trades as structured objects. It has 37 rule tests, and 1,000 random games ran with no exceptions. I checked edge cases against two public implementations, GNOME-p3 and fj317/Monopoly-RL, as a reference only. I did not copy code from either. It is an unofficial simulator, Monopoly is a Hasbro trademark, and I use no board art or card artwork.

How it works

The first agent was self-play PPO: a small network over a 691-number observation and 135 masked actions. The later one adds a transformer with a per-opponent memory. Each opponent's recent moves become a "style" vector, and a side head is trained to predict that opponent's next move. The head only shapes the representation and never touches the reward. The idea follows published opponent-modelling work (Hernandez-Leal et al., arXiv 1907.09597; LIAM, arXiv 2006.09447).

What is PPO?

Proximal policy optimisation is a reinforcement-learning method. The agent plays, finds out whether it won, and nudges its policy toward the moves that went better, with a cap on how big each nudge can be so training does not blow up. In self-play the opponents are copies of the agent itself.

What is a side head?

A network has a main job, here choosing a move, and can be given a second job on the side. Mine guesses the opponent's next move from the style vector. That forces the vector to carry real information about that opponent, without tying it to the reward.

The version that finally used the read is a search, not a network. At each buy, auction or post-roll decision it copies the game, tries a few candidate moves, plays 30 decisions forward and scores the result by net-worth margin. My scripted HeuristicAgent plays my side in that forward play, and the opponents follow the learned prediction. The search only leaves the heuristic's move for an alternative that scores more than 20 better.

Results

The read itself works. The more of an opponent's moves the model has seen in a game, the better it predicts the next one. In one run that goes from 47% with one to three moves seen to 87% with 23. Drag the slider to walk along it, or switch to the second run, which has a point for every move count and is less tidy.

What do "read on" and "read off" mean?

Both versions run exactly the same search. In "read on" the opponents in the forward play do what the model predicts. In "read off" they move at random. So the gap between them is how much the guess about the opponent is worth. The same games are played both ways, starting from the same seeds, to keep the comparison fair.

Win rate against scripted bots, two players, paired games. The read-off column is the same search with opponents assumed random.

Opponents	Games	Read on	Read off
Diverse league, 12 bots the model trained on	240	83%	73%
League, 4 held-out bots	240	69%	56%
GNOME-style bot	about 112	56%	59%
Fixed-policy bot	about 112	79%	68%

What is a standard error?

If you replayed the same number of games, the win rate would wobble a little just from the luck of the dice. A standard error is the typical size of that wobble. A gain of three or four of them is very unlikely to be luck. A gain of one is something luck could easily produce.

The two league gains are roughly three to four standard errors, and the fixed-policy gain about two. Against the GNOME-style bot, a tight and competent player, the read adds nothing I can measure. That bot is a reimplementation of the shape of the public GNOME baseline, not its code. The fixed-policy bot is just that: buy above a flat cash buffer, build when able, turn down every trade. The model never trained against either. The league bots are 16 draws from one generator of six style numbers, mainly trade willingness, bidding and jail habits, so "held-out" means unseen draws, not a different kind of player.

Reading opponents beat assuming they are random by 10 to 13 points against a diverse bot league. That is a weaker comparison than it sounds, because the league results are all against 16 draws from one bot generator, and everything here is two players and one checkpoint.

What didn't work

Almost everything before the last step. Plain self-play won about 100% against a random player but 0 to 4% against the heuristic: an outcome-only reward gives almost no signal when you never win. A curriculum fixed that in two players, starting from a near-random heuristic and sharpening it, and the agent reached about 51% against the full heuristic (42 to 58% across evaluations). That is parity, and more training did not move it. In four players the curriculum failed (about 0% against full heuristics). A heuristic-heavy league reached an average placement of 0.74, third to fourth, with almost no outright wins.

The opponent-reading network then hit another wall. With its memory on or off it won the same share of games on the hard bot panel (26.0% and 26.7%, 150 games). Its policy drifted to near-uniform whatever I changed, including batches ten times larger on a rented 36-core GPU machine. The read worked. The network never used it.

What does "drifted to near-uniform" mean?

A policy is a spread of probability over the legal moves. One that has learned something puts most of it on a few moves. One that has learned nothing spreads it evenly. Entropy measures that spread, and mine sat close to its maximum, so the agent was in effect choosing moves at random and had nothing to adapt.

Search against opponents that accept any trade, 2 players, small samples. Win rate by what the search believes about the opponent.

Opponent model in the search	Before fix	After fix
Learned (aux)	0.75	0.60
Random (null)	1.00	1.00
Perfect (oracle)	0.75	1.00

Search first looked like it made things worse: 21 points below the random-opponent baseline (0.73 against 0.94 over 100 games), and a perfect opponent model did no better. The cause was my scoring. One-step net worth made grabbing a property always beat building, and in a move-by-move comparison against a competent opponent the agent went for a trade on 95% of turns. Rolling the game forward fixed the oracle, which now matches the baseline. It did not help the learned model's own row, which went from 0.75 to 0.60. That is within noise on 10 games, but it is not an improvement. All three after-fix cells are 10-game runs, so only the direction counts. A 60-game run had shown +28 points on held-out bots, and that shrank to +13 at 240 games.

The learned read was also weakest on the rare moves that matter. In an earlier diagnosis, before the league-trained model, it predicted "reject" 68 times out of 68 against a bot that accepts any trade.

What is unfinished

Every result for the search and the opponent read is two players against scripted bots. The only four-player numbers are the earlier PPO runs above. The design covers two to six players, but I have not evaluated the search beyond two. My evaluation caps games at 35 rounds and ranks survivors by net worth, so a "win" can be a lead at the cap. I have no human play data and have not played it against a person yet.

A larger run is scoped (about 40 bots, GNOME-style and fixed-policy bots inside training, two to six players) but has no results. Trades still come from a handful of candidate offers rather than being built freely, and a faster engine in a compiled language would help at league scale.

The point of all this is a super-human player, and I am not there. Once I am, I will try it on the family at Christmas, which is a good excuse for a few Monopoly games.