ALLPCB
Overview
Voice assistants are increasingly important in daily life, whether users cannot use a phone, or as interfaces for the Internet of Things. However, many people rarely use voice assistants because they prefer not to speak aloud while working or resting. Speaking or taking out a phone can break concentration and require time to recover focus.
AlterEgo is a wearable developed by MIT Media Lab master's student Arnav Kapur. It can detect the words you silently subvocalize without requiring explicit facial gestures, effectively interpreting your inner speech.
Principle: detecting subvocalization
AlterEgo does not use telepathy. Instead, it relies on subvocalization, a common phenomenon in which people internally "speak" without producing audible speech. Subvocalization is known to slow reading speed, and reducing it is a goal in speed-reading training.
Although subvocalization is a mental activity, small movements occur in facial muscles that are not externally expressed. AlterEgo captures these neuromuscular signals to infer the silently articulated words.
Subvocalization is an ability everyone has; with practice, users can become proficient at it.
Signal acquisition and neural network processing
The researchers initially searched the face for reliable neuromuscular signal sources and identified seven electrode locations that consistently produced high recognition rates. In the final prototype, they achieved good results with four electrodes placed on the palate, reducing the number of electrodes and making the device more acceptable to wear.
Note that AlterEgo only captures information conveyed by subvocalization. Random thoughts that are not subvocalized cannot be detected.
The system uses neural networks to map different electrical signals to the words a person subvocalizes. As with typical neural networks, the architecture consists of layers of simple processing nodes, each connected to nodes in adjacent layers. Data is fed into the lower layers; nodes process the data and pass it up the hierarchy, ultimately producing a self-organizing map or other output.
System architecture and feedback
AlterEgo transmits the recognized data to a computer via Bluetooth, where tasks are completed and results are sent back to the device.
The device also incorporates a bone conduction headset that transmits vibrations through the facial bones to the inner ear, similar to in-ear monitors used by singers. Because bone conduction does not block the ear canal, the system can communicate with the user without interrupting ambient hearing.
Use cases
Subvocalizing "down", "right", "select" can control a TV. Subvocalizing "time" can return the current time. These functions are part of a silent interaction system that allows users to obtain information or control devices without speaking aloud or appearing to do anything to bystanders.
In demonstrations, the researchers implemented two applications: arithmetic with large numbers, for example calculating total prices while shopping, and playing chess. During a chess game, the user subvocalizes chess moves using chess terminology, the computer evaluates the game and computes optimal moves, and the device silently informs the user of the recommended move.
Privacy and social benefits
AlterEgo only reads information that the user subvocalizes. This contrasts with approaches that place many electrodes directly on the scalp, which can expose broad brain activity and compromise privacy. Subvocalization-based systems can enhance privacy and reduce social awkwardness, for example by allowing a user to silently search their contacts when they cannot recall a name, or to look up definitions without asking others.
Limitations and future work
The main advantage of AlterEgo is that it enables seamless, efficient interaction with computing devices without extracting users from their current environment or interrupting real-world interactions. However, the system also has clear limitations. The current prototype requires personalized calibration and is limited to recognizing roughly 20 words. Its appearance is conspicuous, which may deter public use, although the researchers prioritized function over form during development. Advances in electrode design, electrophysiological modeling, materials, and industrial design could make future versions less noticeable.
The researchers state that AlterEgo is still a prototype and requires further development before any commercial deployment. They are working on generalized systems that do not need individualized calibration. In the future, similar technology might assist people with speech impairments.
