WordBond
2025-01
WordBond is an AI-powered shared reading tool designed to support visually impaired parents when reading physical children's books with their sighted children. Combining OCR, LLM-based correction, hand-tracking, and text-to-speech, WordBond allows parents to narrate stories with their own voice while receiving contextual support during shared reading.
This project is actually getting published through ICHEC 2025. Right below here you can find the link to the paper and DOI, and at the bottom of this page you can find the paper as pdf directly
This project is actually getting published through ICHEC 2025. Right below here you can find the link to the paper and DOI, and at the bottom of this page you can find the paper as pdf directly
Github Repository: Here
Paper published on ACM: Here
# Problem & Design Goal
Many existing assistive reading tools prioritize information access, but often replace the parents' voice with synthetic narration. WordBond focuses on preserving the emotional voice bond in shared reading by helping visually impaired parents read physical books in their own voice while still receiving timely support when needed.
# System Overview
WordBond enables visually impaired parents to scan books in advance and read them aloud naturally with real-time support. The system uses easyOCR and gemini-1.5-flash to produce clean audio snippets, while openCV hand tracking supports pointing gestures during shared reading.
# Technical Pipeline
The technical pipeline has two phases. Phase 1 uses easyOCR to read the page, gemini-1.5-flash to correct OCR errors and segment text, and yaseenUOM to produce MP3 audio files. Phase 2 uses openCV to track pointing gestures, triggering contextual support and interactive feedback for parent-child shared reading.
# Evaluation Setup
WordBond was evaluated iteratively with blindfolded sighted participants simulating visually impaired use. A pilot study (n=4) identified core usability bottlenecks, then a formal user study (n=7) assessed improvements using mixed methods: observation, semi-structured interviews, UEQ for user experience, and NASA TLX for workload.
# Key Results
Results showed strong novelty and engagement (UEQ hedonic quality higher than pragmatic quality), with all UEQ dimensions scoring above 1.0 in the formal study. NASA TLX indicated medium-low overall workload and clear improvements after design iteration, especially through shorter segmented audio and replay controls. Remaining challenges centered on interaction clarity (the cue for pointing interactions) and supporting more expressive, less monotonous storytelling delivery.
# Next Steps
Moving from simulated users to on-site studies with visually impaired parents and children, improving on-device autonomy, adding richer contextual cues for interactive mode, and integrating image description support to help parents narrate with stronger emotional expression.
# WordBond Final Report (PDF)
Open or download PDF
WordBond Final Report (January 2025)