Gemini Golf Coach

Problem

Google wanted its new Gemini audio models shown live at their launch event. Handstand built three walk-up demos, and golf was the hardest. A guest who may never have held a club steps into a netted bay, says their name, swings once, and needs coaching that is actually right. It has to be spoken aloud within seconds, while a queue waits behind them.

Everything was built around one rule. The cameras measure, Gemini interprets. The model never invents a number, a body position or a fault. If it can’t get a clean read, the coach says so.

Constraints

• Four weeks from the first line of code to event night. Hardware arrived mid-build, and the final launch models landed about 48 hours before doors.
• Many guests had never swung a club, so each tip had to be one simple feel they could try on the very next swing.
• The voice model can’t return structured data, so one model decides what to say and another one says it.
• Analysis takes several seconds, and nobody wants to stand there in silence.

What I owned

Handstand built the installation as a team. Hardware, the recap video and the cloud platform were others’ work. I joined the golf build on the first day of code and owned everything the guest sees, hears and steps through.

• The UI. The kiosk screens guests watch, including the replay that jumps to the exact moment the coach is talking about and zooms in on that part of the body.
• The workflow. Every step of a guest’s visit, from saying their name to the second swing and the send-off, and what happens when a swing is missed or unreadable.
• The coaching. What the coach says and how it says it, from the prompts and the rules Gemini works within to the shape of every tip (problem, fix, why it helps, recap).
• The tests. A tuning lab that scored prompts and models against recorded swings, plus the tests that kept the experience working as it changed every day.
• Making it all work. Brought the pieces together and tuned both bays on site through rehearsals and event night, with a recorder capturing every session so nothing went unexplained.

How it works

Two 120 fps cameras → MediaPipe pose → Gemini analysis → Gemini Live voice coach → Replay and recap video

Computer vision does the measuring, with six swing phases and 24 body measurements per swing. Code then works out which faults those measurements actually support, and Gemini chooses from that shortlist. It can’t diagnose anything the cameras didn’t see, and its answer is checked before a word is spoken.

While Gemini thinks, the guest watches their own swing in slow motion from two angles, so the wait becomes part of the show. Then the coach talks them through one fix as the replay walks to each moment. On the second swing, the coach tells them whether it took.

The stats screen shows measured angles only, never a guess.

Results

191 guests were coached across two bays in roughly 9.5 hours. Along the way, a few findings changed what shipped.

Skills this shows