chat is not a neutral interface. it puts ideas in a queue, and people stop thinking while they wait for the next answer.
Designers reach for a whiteboard, not a word processor, when a problem gets hard. On a wall, ideas sit side by side without hierarchy, connections appear through proximity, and the whole picture stays in view. A chat with an LLM does the opposite: earlier branches scroll away, alternatives are never visible at once, and research on chat-based systems finds users skimming, disengaging and losing the thread of their own thinking.
Tangible Co-Ideation is a spatial canvas paired with objects you hold. Instead of typing prompts, you place, turn, rotate and tip things on a table. A figurine brings in a mentor's point of view, a lighthouse searches related work, a shelf judges an idea. Every move is a physical act your partner can see, and every idea stays on the table.
creativity alternates between opening up and narrowing down. each object on the table belongs to one side of that cycle.
02the set
a table, a canvas, seven objects.
A 50-inch screen lies flat under an infrared touch frame. Every object stands on three legs, and the frame reads each one by the triangle its legs make.
4ai mentorsabacus · piano · macintosh · mona lisa
1lighthousesearches related work
2goal shelvespublishability · commercialization
50″displaywith a 10-point infrared touch frame
03the canvas
move a mentor, move the thinking.
A working model of the table. The idea in the room is the one from the study's walkthrough: a video game controller. Drag a mentor into the room to branch three ideas. Then bring it up against an idea: the side you approach from decides whether it converges or diverges. Drag ideas onto each other to merge them, or into the bin to prune.
drag a mentor from the tray into the room
vocabulary
branch
0
step
0
merge
0
prune
0
mentors
selected idea
the room holds one idea: “i want to make a video game controller.”
best on a larger screen. On a phone, the log under the canvas reads out every move.
pre-written model. On the real table, mentors generate every idea live from their knowledge base, and the third side, talk, takes your own typed or sketched request. The first two branches quote the study's example outputs.
04ai mentors
four mentors, four value systems.
Design education runs on critique: a mentor responds to work in progress with questions, redirections and advocacy. Each mentor here argues from one point of view, and is represented not by a likeness but by an object its perspective is known for.
mentor_01 · abacus
the hci researcher
Modelled on Prof. Hiroshi Ishii's research side. He often cites the abacus as the embodiment of his research vision: computation you can touch.
mentor_02 · piano
a second hci voice
A persona built from another HCI researcher's published work: a second academic lens, so two researchers can disagree.
mentor_03 · macintosh
the entrepreneur
A Steve Jobs persona with an entrepreneurial lens: who it's for, why it matters, and what to leave out.
mentor_04 · mona lisa
the engineer
A Leonardo da Vinci persona: mechanisms, materials, and how a thing would actually be built.
perspectives, not people. A mentor is a slice of a value system, never a digital twin. The two living researchers consented to the use of their published work; the historical figures are built from public material.
05how a mentor is made
a knowledge base for every mind.
Each mentor is a retrieval-augmented model, combined with supervised fine-tuning and prompt engineering, so its answers stay grounded in the person's own material and speak in their voice.
src_01publicationsPapers and books as PDFs: the foundation of the expert's knowledge.
src_02talks + interviewsRecorded and transcribed, to tune alignment and tone.
src_03sketchesIn-progress thinking. OCR and image analysis, then clustering and manual tagging, linked to documents for cross-modal retrieval.
src_04conversation logsHow the person talks. Cleaned with a custom tool that removes personal exchanges, under human supervision.
unified, indexed knowledge base
rag + fine-tuning + prompts
Defined expertise, background and communication style. Prompts tested and refined for grounding and fidelity.
textideas · converge · diverge · talk
imagesketches on the canvasgenerated with Nano Banana
06debate mode
turn them to face each other.
Put two mentor figurines face to face and they start to argue about the nearest idea. Their orientation is the prompt: facing means build on each other, turned away means disagree, and one of each means one tries to convince while the other refuses. Gestures change the tone mid-argument.
on the table: “a controller with swappable haptic modules”
stance: facing · agree and build
illustrative persona lines, not real quotes.
07lighthouse
a beam through the related work.
Brainstorming, you rarely know what already exists around you. Place the lighthouse on a room and black smoke covers the canvas. An LLM sorts the idea's neighbourhood into three clusters (main themes, adjacent contexts, technical foundations) and fills each with at least five papers from Semantic Scholar, arXiv and Crossref. Rotate the tower and its 90° beam reveals one cluster at a time, so there is less to take in at once.
rotate the beam
drag across the dark, or use ← →
titles from the study walkthrough. The table lists five papers per cluster.
08goal shelves
tip the shelf, flip the verdict.
Mentors advocate; shelves judge. Place one on an idea to sharpen every mentor's view to a single criterion. A bookshelf reads it as research, a supermarket shelf as a product. Reports come as a notepad or a receipt, always four sections of three bullets. Tip the shelf over and the same idea is judged harshly.
idea under review: “a video game controller with swappable haptic modules”
illustrative report. Two lines are quoted from the study walkthrough: “the controller sells once; modules sell forever” and “friction at the moment of peak intent”.
09process
rethink, remake, redesign.
The objects, the sensing and the software were built in parallel, then rebuilt once they met. The biggest lesson: software designed without the objects in mind, and objects designed without the screen, break each other.
a form: finding the figurine
a person.Humanoid first. Black conductive filament only, so a second colour made it friendlier.
legs.Lifting the body off the glass frees the screen beneath for a digital shadow.
too abstract.Pure geometry lost the playful feeling that makes brainstorming fun.
made to hold.Shapes modelled in Play-Doh to invite a grip, within the limits of capacitive sensing.
an abacus.Moving beads, easy to grasp, with the three contact points hidden in the frame.
a symbol.Final form: an object the person stands for, not a portrait of them.
b sensing: how the table knows
v1markers + cameraArUco markers tracked by an overhead camera: reliable identity, height and 3D orientation. But the top of every object had to stay visible, which fights expressive handling.
v2conductive feetThree conductive contact points per base, read by an ordinary capacitive screen. The distances between them encode identity and orientation. Ten footprints were generated and tested. Ten touch points means three objects at once.
v3infrared frameA 50-inch infrared touch frame over a 50-inch display. Each object is a unique three-leg footprint, so the screen beneath stays free for shadows and gaze beams.
c software: from threads to a vocabulary
it_01rooms + threadsRectangular rooms held text, images and sketches. Mentors joined a room; you could open several conversations and merge their context by hand.
it_02bubbles, lens, backpackA mentor in a room produced three idea bubbles; a lens showed live sketches; a backpack carried threads across the canvas. Too many functions: it was hard to know where to focus.
finalfour moves, four modesRound rooms and bubbles, four moves (branch, step, merge, prune), and one job per object: ideate, debate, search, evaluate.
learned when the two halves met
Moving interface elements fight objects that sit still.
Every object needs a visible tie to the screen, like a digital shadow.
Legs that seem to rise out of the screen hide less of it than a flat base.
Objects must be easy to grab and stable, for the sensing as much as the hand.
10the study
16 pairs, two tables, one question.
Does holding the AI change how people think with it? Thirty-two participants brainstormed in pairs, each pair once with the objects and once screen-only, where every mentor and tool was a draggable icon of the same size.
intro5
onboarding15
task 120
break5
task 220
interview15
80 minutes per pair · condition and task order counterbalanced · tasks: “a physical tool for future programmers” and “a physical tool to talk with your younger self” · 8 pairs of friends, 8 of strangers
screen-onlytangible
show as a table
measure (median per pair)
screen-only
tangible
p
ideation actions per minute
8.36
10.72
.045
pause between actions (s)
2.85
2.41
.009
burst index
0.271
0.345
.009
human-authored ideas
0
1
.034
all ideas
9.00
7.19
.216 (n.s.)
speaking turns
59.5
70.5
.032
speaking time (min)
19.0
20.5
.008
paired Wilcoxon signed-rank tests, 16 pairs.
the rhythm changed.
With the objects, actions came in bursts with longer pauses between them, and the pauses were filled with talk: which mentor to place, which idea to chase. Screen-only, pairs fell into steady, continuous prompting.
screen-only
tangible
illustration of the pattern, not session data · 20 minutes
The dexterity allows you to think harder than just, you know, a quick type.p3
I think the objects keep you in this connection-making mode for longer.p6b
I love kind of offloading some of my mental processes in the space and organizing them.p4a
We're playing with action figures, which was just fun. What is brainstorming without fun?p15a
13 of 16 pairs called the objects satisfying, characterful or playful, and saw the AI less as an all-knowing tool and more as a collaborator with limited, specific knowledge.
AI abundance doesn't have to mean passive consumption. When generated material is spatial, partial, physically actionable and socially negotiated, it can generate freely without becoming the author.
first encounters, 20 minutes per condition. The tangible condition bundles material, form, persistence and play, and the study doesn't separate them. Idea quality wasn't measured.
11film
12specifications
tangible co-ideation.
type
tangible interface for AI-assisted, collaborative ideation
ArUco markers tracked by an overhead camera: reliable identity, height and 3D orientation. But the top of every object had to stay visible, which fights expressive handling.
Three conductive contact points per base, read by an ordinary capacitive screen. The distances between them encode identity and orientation. Ten footprints were generated and tested. Ten touch points means three objects at once.
Rectangular rooms held text, images and sketches. Mentors joined a room; you could open several conversations and merge their context by hand.
A mentor in a room produced three idea bubbles; a lens showed live sketches; a backpack carried threads across the canvas. Too many functions: it was hard to know where to focus.
