Computer aided board-gaming

When I was first exposed to augmented reality, I couldn’t help immediately thinking about how it could be applied to games. I was intrigued by the technology which connects our real world, to a virtual one. Somewhat naively, I set about implementing my own 3D pose estimation algorithm in order to create a simple augmented reality (AR) board-game.

One of my inspirations for this project was Eye of Judgement, a Playstation 3 title. Despite its modest 330,000 unit sales, it was still one of the first commercial games to heavily integrate augmented reality, as a core game mechanic. Since then, many notable video games have leveraged augmented reality as part of the experience. Pokemon Go undoubtedly being the most popular.

Augmented reality enables unique video-board-game features, such as:

Embedding hidden information in the environment which one player can see, but the others cannot.
Automatically apply complex rules or calculate outcomes of player actions.
Seamless integrate computer-controlled player(s) that require no human support.

Publication and resources

Computer aided board gaming using computer vision
Lasse Farnung Laursen, Master thesis,
Local PDF | Published PDF

Quick Facts

Implementation and publication written from scratch over a six month period in 2008
- Game Engine written from scratch in C++
- 3D Pose Estimation written from scratch in C++
- Ogre3D used as graphics engine
Quantitative study comparing 3D pose estimation algorithm to OpenCV counter-part. Comparable performance.
Qualitative user-study of implemented game-demo with 10 participants
Informal comparison to commercial counter-part (Eye of Judgement)

Implementation journey

By 2008, I had several group projects of 2-3 spanning a few months under my belt. The master thesis was set to last 6 months and for personal reasons, this one was going to be a solo undertaking. Sufficed to say, this project represented a substantial increase in scope and complexity to anything I had undertaken before. When presenting my initial ideas to my supervisor, he seemed cautiously optimistic. Up to this point, my university education had covered a lot of 2D/3D graphics and general computer vision. However, I had never specifically worked with pose estimation before. The few available open-source projects – like the ARToolKit – as well as Multiple View Geometry the supposed bible for pose estimation, made me hopeful that I would not be biting off more than I could chew.

Oh the hubris…

Approximately 3 months into the project, I was torn. On the one hand, I felt I had a significant grasp on Ogre3D, the 3D engine I had decided to use for the project. The basic game engine I was building was also beginning to take shape, and my thesis was even coming along nicely. However, a single huge unknown still loomed at the back of my mind, unsolved. Specifically, proper 3D pose estimation.

I was worried. Multiple View Geometry, the supposed bible for all things “estimatory” in 3D, had been on my desk for the past 3 months, and I still felt like I only understood half of the problem I was trying to solve: Estimating a real-world camera position from 4 known, projected points. I wish I could tell you that the book probably explained what I needed in terms that I simply did not understand. But I can’t. For this particular case, the book remains, in my humble opinion, incredibly vague and downright lacking. Even after I had the good fortune of conversing with Daniel Wagner over a period of several weeks, helping me tune into what exactly wasn’t working as expected and helping me with the final pieces of the puzzle, I still could not find a complete solution in the book. To be fair, the book touches upon the solution I ended up implementing, but fails to mention key details which I – to this day – still do not believe are trivial.

But enough about my personal struggle with this particular book. If you’re ever in need of understanding how to solve 3D pose estimation from 4 known, projected points, I have gone to great lengths to detail every single step of the process in my master thesis. I encourage you to peruse at your leisure. Once I’d finally gotten all the pose estimation pieces into place, I could move onto creating a use case scenario. A playable game demo.

Augmented reality board game

To explore applying 3D pose estimation, I opted to implement a new board-game. Originally, I had envisioned a simple ‘tower defense’ game using augmented reality, but my initial prototypes (see videos below), made me realize two important facts. First, a good ‘tower defense’ game is anything but simple. Even one that only needs to hold the players attention for about 15 minutes would have to be thoroughly play-tested and balanced to ensure an interesting progression. Second, the playing field simply wasn’t grand enough for my visions. At most, I could reliably fit approximately 10 by 6 tags on the playing area.

With time quickly slipping through my fingers, I had to reduce scope. The ‘tower defense’ game was replaced with simple train-track game, ala. Pipe Mania. While much simpler than the initial ‘tower defense’ game concept, it still served as a useful case study of player experience when interacting with augmented reality game mechanics.

Prototypes

These prototype videos show the gradual progress towards the final working prototype, showcased in the video at the top of this page. As previously noted, originally I intended the game to be a simple tower-defense clone. However, lack-of-time and technical constraints forced me to change the game into a simpler Pipe mania inspired game.