Situation Awareness and Attention Guidance / cARcassonne

We have built an AR version of the world-acclaimed tabletop game Carcassonne, called cARcassonne, using Unity and for the Microsoft Hololens 2.

This project has been funded by the Crafoord Foundation, and it’s led by Arezoo Sarkheyli-Hägele, started in cooperation with the former Egocentric Interaction Research Group.

Publications

cARcassonne gameplay features

AR board placement by user. Place the board wherever feels best!

Game start and main board UI.

Play cARcassonne!

Current lines of research:

-Player modeling in cooperative/competitive environments.

-Detecting player intention through gaze tracking and gameplay data.

-Guiding player attention for strategy tips.

A player playing cARcassonne

Board and gamestate representation

The game state in cARcassonne is encoded as a graph of interconnected game elements that allows us to easily perform complex computation over the board.

A Convolutional Neural Network agent learns to play the game from gameplay data, and a situation model attempts to model the upcoming moves and flow of the game.

Gaze tracking

We capture player gaze while playing cARcassonne. This can be displayed as a heatmap over the board to clearly see which areas players focus their attention to.

Gaze tracking heatmap
Board
Gaze tracking heatmap over board
Gaze tracking over board represented as a graph that indicates gaze order and duration.