Artemisia's Revenge

Experimental Gaming Experience
Projection Mapping + Physical Movement + Story-telling

This project is funded by The Deans' Undergraduate Research Fund of New York University Shanghai and instructed by Prof. Xingchen Ian Zhang

This project was exhibited in the gallery of Interactive Media Art as its second exhibit.

Over 100 individuals, including professors, game industry insiders, artists, and fellow students, visited and provided their insights.

This project took part in the Undergraduate Research Symposium of NYUSH

Based on the experience of building this project, we held a Popcorn Session to introduce and share our experience.

My Role:

Lead Gameplay Designer, Programmer, and most work in Unity Engine. I also took part in reference searching and part of stroy writing.

Playthrough Video

Design Focus

Use Projection Mapping techniques and Physical Model as the medium to convey the Game as a Experimental Experience.
Building the story based on an Italian Baroque painter Artemisia Gentileschi who had a Feminist Perspectives in 17th century.

Description

In a fusion of art and technological innovation, we utilized Projection Mapping to develop a Game inspired by Baroque female painter Artemisia Gentileschi. Players use a controller to guide an avatar in a virtual 'house,' with the visuals projected onto a physical model. Our research focuses on three areas: the integration of MadMapper with Unity for reliable projection, the adaptation of gameplay mechanics to suit the unique capabilities of projection mapping, and weaving in historically accurate storytelling. The project expands traditional gaming boundaries and offers a unique tribute to a historically significant female artist.

Inspiration on GamePlay

CUBE ESCAPE: ARLES
Game by Rusty Lake
A simple click-based escape room game based on van Gogh's work, spirit world, and hobbies. It makes good use of van Gogh's classic paintings and uses them as game scenes to tell a story.

Bedroom in Arles (1888) by Vincent van Gogh

Status Quo of Projection Mapping

Most of today's interactive projection mapping is limited to either projecting games onto irregular surfaces or projecting simple interactions into larger spaces.

Challenge:

We experiment with projecting interactive games onto non-regular canvases and trying to transcend the limits of space by having the movement of the player's physical space as an input.

Experience Design

Planning

Building a physical model to project on:

We initially intended to build a model of a more common house and project the virtual content onto the model's canvas via projection.

At this stage, we are considering the possibility of "breaking out of space" by somehow rotating the table.

Physical Model

As an installation, in order to bring a more stunning visual experience, our model needs to be:

1. large enough to catch the eyes of passers-by and clear enough for each room to be projected.

2. find suitable materials for projection while controlling the cost and the difficulty of making.
After comprehensive consideration, we chose to use simple and easily available cardboard boxes as the base material and fix them with glue and tape. We then ase material and fix them with glue and tape.
The areas of the canvas that would get projected are thick white paper, the areas of the roof that would not be projected are black paint, and the other areas that would not be projected are white paint.

Technical Test

Unity <=> MadMapper
We used Spout as the video “router” and Unity plug-in KlakSpout to convey the views from Unity to MadMapper.

1. Unity Editor & “KlakSpout” Spout Sender

2. Test in MadMapper: successfully received the view from Unity

3. Multiple Views with senders in Unity Engine Editor

At this stage, we realized that using a rotatable table plus a single projector and tracking the rotation of the model was not quite as easy to achieve. So we moved to using multiple projectors and fixing the model on the platform. From there, all we had to think about was making the experience more special for the player through Gameplay, visuals, etc. without having to do more technical testing.

Story-telling

Consideration of Methods:

The problem we have to consider is that after video mapping, the resolution obtained for each scene is not very high. So, using text or dialog boxes can be difficult to read in bright ambient light.

It is similar if we rely too much on the content of the image, where insufficient resolution can make it impossible to recognize the content in detail.

Result:

Our storytelling is told in a meta way through the mouth of Artemisia, the original owner of the house, who guides the avatar to each room and tells the story behind each room.

Visual and Gameplay Design

Visual

For the visual design, we referenced 17th-century Roman furniture styles, including the popular clothing of the time, beds, dressers, etc., and stylized them. We ended up with 11 rooms and two stairwell rooms.

Route

The projection medium we use is characterized by the physical movement of the player's body as an input in addition to the controller. To emphasize this, I needed to design a path that would allow the player to move physically and experience the entire story of the game linearly.

Lights

To help the player know if they've been to a room in the linear flow of the game: if they haven't, the room's light will be the dimmest; if the avatar is in a room, it will be the brightest and flicking; if they've been in a room and left, it will be dimmer.

Input of the game

The Hanging Controller

We don't have the usual wired controller setup: a fixed-length cable to connect to the console; nor do we have the usual wireless controller setup: the freedom to move around within the range. And they both sit on a table etc. in their idle state.

Instead, we hung a wireless handle by a retractable pull box. In this way:
1. the audience and the player will immediately understand that this is a game they can interact with when they enter the dark area where the curtains are built up.
2. the pull box has a length limitation that will naturally guide the player around our physics model.

A player is holding the hanging controller

Physical Movement of the players

Unlike the usual on-screen games, our image medium is a projections mapped on a physical model of a house. No matter which direction the player is facing, there may be something going on on the other side of that direction as well. Also, after the player enters some doors, they will have to move around the model to keep track of the avatar.