I've played a lot of games in my time. So when I first saw augmented reality my brain started to wrap itself around how it could be used in a gaming context. Having never worked with any sort of pose estimation techniques before, I decided to (perhaps foolishly) embark on turning my masters project in to a simple augmented reality board-game. One of my inspirations for this project came from the (back then) contemporary Playstation 3 title, Eye of Judgement. Despite it's modest 330,000 unit sales, it was still one of the first commercial games to heavily integrate augmented reality, as a type of game mechanic.
Augmented reality can help properly mesh video and board games. It enables us to retain the hidden information - something one player should know, but another should not - which is crucial when several people play together. Additionally, the computer can handle tedious tasks that the players would prefer to avoid, like tedious math or convoluted rules. The cherry on top is the fact that the computer can also manage non-player elements in a much more interesting manner. For example, the computer could control an additional Ai player for the other players to play against.
This project was 6 month project that I ever undertook. Although most of the projects I worked on during my time at the University of Copenhagen involved a group of 2-3 people, this one ended up as a solo project. After presenting my initial ideas to my supervisor, he seemed cautiously optimistic. Although my education had covered a lot of 2D/3D graphics, as well as general computer vision, I had never specifically worked with pose estimation before.
The few available open-source projects (like the ARToolKit) as well as the supposed bible for pose estimation (Multiple View Geometry) made me hopeful that I would not be biting off more than I could chew.
Approximately 3 months into the project, my feelings were divided. On the one hand, I felt I had a significant grasp on the 3D engine (Ogre3D) I had decided to use for the project. I also felt the the basic game engine I was building was beginning to take shape and my thesis was even coming along nicely. However, the single biggest unknown in my project was still constantly looming at the back of my mind, unsolved. Specifically, the problem of proper pose estimation.
I was worried. Although the supposed bible for all things "estimatory" in 3D had been on my desk for the past 3 months, and I still felt like I only understood half of the problem I was trying to solve: Estimating a real-world camera position from 4 known, projected points. I'd like to pretend that there are two sides to the story, and that the book probably explained what I needed in terms I simply didn't understand. But I can't. For this particular case, the book remains, in my humble opinion, incredibly vague and downright lacking. Even after I had the good fortune of conversing with Daniel Wagner over a period of several weeks, helping me tune into what exactly wasn't working as expected and helping me with the final pieces of the puzzle, I still could not find this solution in the book. Actually, that's somewhat incorrect. The book does address this case, but fails to mention key details which I - to this day - still do not believe are 'trivial'.
But enough about my personal struggle with this particular book. If you're ever in need of understanding how to solve this particular pose estimation problem, I have gone to great lengths to detail every single step of the process in my master thesis which I encourage you to peruse at your leisure. Having finally gotten all the pieces into place, I could move onto creating a use case scenario.
Originally, I had envisioned designing a simple tower-defence game using augmented reality, but my initial prototypes (see videos below), made me realize two important facts. First of all, a good tower-defence game is anything but simple. Even one that only has to hold the players attention for about 10-15 minutes would needed to have been properly play-tested and balanced to ensure an interesting progression. Second, the playing field simply wasn't grand enough for my visions. At most, I could reliably fit approximately 10 by 6 tags on the playing area.
With time quickly running through my fingers, I bit the bullet and made the decision to instead implement a very simple train-track game. The new game was overly simplistic compared to my initial aspirations, but still served as a useful case study of the users experience of interacting with augmented reality elements.
Gallery Function to return as soon as possible. Basically, Foundation 6 still needs to port this functionality from Foundation 5. Early adopter risk/reward.
These prototype videos show the gradual progress towards the final working prototype, showcased in the video at the very top of this page. As previously noted, originally I intended the game to a simple tower-defence clone. However, lack-of-time and technical constraints forced me to change the game into a simpler 'pipe-dream' clone.