Sazerac House
Interactive Media Table

Deploying machine learning to improve upon an existing interactive exhibit.

Background

Unique to New Orleans, Sazerac House is a museum, showroom, and micro-distillery of the 174 year-old Sazerac Co. brand of cocktails. An immersive tour of the facility includes interactive exhibits that help tell the story of the brand, incorporating elements the existing built environment.

Astound Group was contracted for the creation of a media controller that could activate motion graphic content on Sazerac Houses’, interactive media table. A computer-vision solution was developed in Touchdesigner that could detect objects and gestures.

*Note that due to the nature of this projects awarded contract agreement, only details during the productions cycle related to development can be released. Information related to concept, asset design, and final delivery are under non-disclosure agreement.

Client

Sazerac House

Studio

Astound Group

Roles

Touchdesigner/ML Developer
Asset Manager

Timeline (Development Cycle)

Site Visit - (Sep 2023)
Prototyping- 1 mo. (Oct 2023)
Development - 2 mo. (Nov 2023 - Dec 2023)
Testing/Debugging - 1mo. (Jan 2024)

Location

New Orleans, LS.

One of the exhibits during the immersive tour of Sazerac House, is an interactive media table where touchpoints allow guests to activate historic-themed, animated content with information about the history of the brewery. First, guests would trigger a narrative pathway using on of three different styled drink coasters. When placed in a trigger zone within the top archway image, png animation sequences with embedded touchpoints appear. Guests can tap these touchpoints to play through the animations and learn about the museum.

Guest interaction with projected animations

Prototype with camera scan zone in red.

Development

Initially, the media table was running on a surface level depth-sensor in combination with a FLIR IR camera, mounted above. This setup was recreated and tested in-office for long term feasibility. Due to issues found with touch detection accuracy, future hardware upkeep cost, and LiDAR’s ability to interfere with smartphone cameras, I opted to redevelop the table using machine-learning (computer vision specifically).

Computer vision only requires a low cost, conventional webcam and does not cause image blowout on smartphone cameras.

LiDAR removes the tables animated content when photographed using an iPhone 15 Max Pro.

Through Touchdesigner, two python packages helped to drive the two types of guest interaction:

  • MediaPipe, for tracking hand placement and determining when/where touch interaction occurs. This is used when guests tap on a projected image to trigger it to animate.

  • openCV, for detecting placement and shape of a drink coaster on the table, which initialized 1 of 3 different animated sequences.