Situation Awareness and Attention Guidance / cARcassonne

We have built an AR version of the world-acclaimed tabletop game Carcassonne, called cARcassonne, using Unity and for the Microsoft Hololens 2.

This project has been funded by the Crafoord Foundation, and it’s led by Arezoo Sarkheyli-Hägele, started in cooperation with the former Egocentric Interaction Research Group.

Sample early cARcassonne gameplay

Current lines of research:

-Player modeling in cooperative/competitive environments.

-Detecting player intention through gaze tracking and gameplay data.

-Guiding player attention for strategy tips.

A player playing cARcassonne

Board and gamestate representation

The game state in cARcassonne is encoded as a graph of interconnected game elements that allows us to easily perform complex computation over the board.

A Convolutional Neural Network agent learns to play the game from gameplay data, and a situation model attempts to model the upcoming moves and flow of the game.

Gaze tracking

We capture player gaze while playing cARcassonne. This can be displayed as a heatmap over the board to clearly see which areas players focus their attention to.

Gaze tracking heatmap
Board
Gaze tracking heatmap over board
Gaze tracking over board represented as a graph that indicates gaze order and duration.