Cluster&Merge
2024-10
Cluster&Merge is a collocated collaborative brainstorming system that uses natural language processing to convert speech into live keyword streams, then allows participants to cluster and merge words into new prompts. By combining touch interaction and visual image generation, the prototype explores whether merged concepts can improve ideation quality and collaboration during brainstorm sessions.
Key Features
- Real-time speech-to-text keyword stream
- Direct-touch drag-and-place interaction
- Word clustering and merge-triggered concept generation
- Image querying for single and merged keywords
- Interaction logging and post-session analysis
# Research Focus
The project investigated how combining speech-to-text extracted keywords and visual merging influences brainstorming quality. The core research question centered on whether merged keyword prompts and generated imagery can stimulate richer or more varied idea generation in face-to-face collaboration.
# Prototype
The system was built in Processing and run on a touch-enabled LCD setup. Spoken input is translated into keyword streams that participants can drag, place, cluster, and merge. A merge action combines words into a new concept and allows image retrieval for additional inspiration. A trash boundary enables quick removal of irrelevant items.
prototype device (tv screen) that consist an LCD screen with a touch input layer on top
How the actual screen looked, dragging a word out of the stream on the prototype. Dragging multiple words over eachother would merge them and query images
# User Study
A field study was conducted with 15 graduate design participants in 4 groups. Sessions included an introduction, a timed brainstorm, and individual debrief interviews. Findings showed mixed but generally positive responses: participants appreciated playful collaboration and inspiration from images, while others reported distraction, pacing issues, and occasional disruption of conversational flow.
Picture of camera taped to the ceiling during field study.
The second computer is attached to the prototype to run the program.
# Findings & Reflection
Clustering and merging were often used out of curiosity and perceived as engaging, but their measurable effect on interactions was not consistently positive. Merged images were considered inspiring by many participants, while others felt they could constrain imagination. The study highlighted trade-offs between stimulation and distraction, and pointed to future improvements in pacing, organization tools, and interface adaptability for different collaboration styles.