By Shelly Fan – Re-Blogged From Singularity Hub
At 29 years old, Canadian firefighter Rob Anderson lost his left arm and left leg to a harrowing helicopter crash into the side of a mountain. Although fitted with “top of the line” prosthetics for the last 10 years, he said, using them feels like “doing things with a long pair of pliers.”
Part of the problem is that he just doesn’t feel connected to his prosthetic hand. “There’s a disconnect between what you’re physically touching and what your body is doing,” he explained.
So when researchers at the University of Alberta offered him a place in a new prosthetics trial, he leaped at the chance.
The result, published this month in Science Translational Medicine, is a bionic arm strikingly different than anything that’s come before it. Once fitted, the arm uses vibrations and a sensory illusion to give wearers a natural sense of their robotic appendage moving through space. Even when blindfolded and wearing noise-canceling headphones, Anderson intuitively knew what his robotic arm was up to.
“It was kind of surreal,” he said.
The Sixth Sense
There’s no arguing that prosthetics have come a long way. Controlling a robotic limb with your brainwaves was impossible a mere decade ago; now it seems routine. More than ever, scientists are squeezing increasingly dense sets of motors and sensors into replacement limbs.
The result is sophisticated bionic appendages capable of fine, dexterous movement. Some can even send feelings of touch back to their wearer, closing the loop from action to sensation.
But there’s a problem: without a direct visual, the wearer has absolutely no idea what their bionic arm is up to. They don’t know where the arm is in space, how fast it’s moving, or where it’s going.
This intuitive sense of body positioning, dubbed kinesthesia, has been hard to build into prosthetics. It’s not touch—kinesthesia uses feedback from the joints and muscles to compute where your limbs are even without direct touch feedback.
Yet, like touch, kinesthesia is essential for fine motor control: this is the sense that lets you shove a handful of popcorn into your mouth while keeping your eyes on the big screen. It’s behind seemingly mundane actions such as scratching your back or catching a ball. You can’t multi-task without it.
“Somebody with a prosthetic hand, since they can’t feel the movement of their device, they essentially have to compensate [for] that with vision,” said lead author Dr. Paul Marasco at the Cleveland Clinic, who collaborated with the University of Alberta and University of New Brunswick. This kills any sense of ownership of the arm.
The new device restores kinesthesia using a seriously clever body hack.
When you vibrate a tendon at 70 to 115 Hz, it makes it feel like the associated joint is moving. The illusion is strong enough that the person thinks their limbs are contorted into impossible positions or that their nose is growing like Pinocchio’s. By vibrating multiple tendons, scientists can induce the sensation of complex arm movements in space without anything physically moving.
Scientists have known about this phenomenon—dubbed the vibration-induced kinesthetic illusion—since the 1970s, but no one’s ever tested it in amputees before.
The volunteers in this study had previously undergone surgery to rewire the remaining nerves in their upper bodies to other muscles. For example, the nerve that normally controls the elbow is hooked up to chest muscles. When the patient thinks about moving his elbow, the nerve sends the command to the chest muscle. This activity is then picked up by a sensor that, in turn, instructs the prosthetic arm’s elbow to move accordingly.
The team first vibrated the volunteers’ chest, bicep, and tricep tendons—where the remaining nerves were rerouted to—and asked them to mimic the perceived movements in their missing hands with their remaining one.
Incredibly, different vibration paradigms mapped onto a library of complex hand motions. For example, stimulating the biceps in most patients generated the “cylinder grip,” in which the hand is loosely clenched as if wrapping around a tube. Other motions included the thumb and index finger “fine pinch,” or the thumb, middle, and index finger “tripod pinch.”
In all, the team identified 22 different hand motions, or precepts.
“We thought that if it worked like it did in an able-bodied person, we’d just get one or two joints; we’d get a wrist—something basic,” said Marasco. “We were floored.”
A Kinesthetic Interface
The next step was to put this library to use. The team developed a neural-machine interface with two lines of communication. When the patient thinks about moving the bionic arm, the signal is picked up from the re-innervated muscle to control the prosthesis. At the same time, it also triggers a small but powerful motor to vibrate the muscle, generating the kinesthetic illusion.
The improvement was evident within minutes. Using computer simulation software, the volunteers could easily close their virtual prosthetic hands a quarter, half, or three quarters of the way without watching the hand. In contrast, with the vibrations turned off they performed significantly worse—one patient had nearly no sense of hand position without adding the hack.
Kinesthetic feedback was even more powerful than vision for fine motor control. When asked to catch a virtual ball using their virtual hands, kinesthetic reflexes kicked in far faster than visual feedback, allowing the volunteers to reach out precisely and intuitively.
Unsatisfied with just a proof-of-concept, the team next rigged a physical prosthetic hand with their vibration motor. The clinical standard for a bionic hand attachment is a rigid plastic socket with limited space, the authors explained. Real estate is tight, but it’s possible to squeeze all the components in.
The results were remarkable. Even with blindfolds and noise-canceling headphones on to block off the world, the volunteers easily followed instructions to close the bionic hand into a cylinder grip. What’s more, they had no trouble reporting the status of the prosthetics—whether they were open or closed.
“A normal amputee is completely lost with a blindfold,” explained Marasco, “and can’t even do the test.”
The Ultimate Replacement
With the restored sense of kinesthesia came a sense of ownership. The vibrations blurred the lines between what’s “self” versus what’s “machine” in the patients’ brains. The bionic hand didn’t just feel like a hand—it felt like their own.
“I did pattern recognition for years—training myself to operate a prosthetic hand—and that was great,” said one participant. “But this? This is going to take things to an entirely different level.”
Dr. Dustin Tyler, who had previously worked on prosthetic hands with a sense of touch, found the work fascinating. “If you’d asked me before, I would have said it’s unlikely to work,” said the bioengineer, who works at the Louis Stokes Cleveland Veterans Affairs Medical Center.
Marasco and his team are now working to extend the technique to other types of injuries, such as those who had lost a leg or those with movement issues due to stroke. He’s also working on integrating the vibration hardware with prosthetic arms into an easy-to-use system for patients to operate on their own.
The ultimate goal is to integrate bionic limbs as a part of the patients, Marasco said. That requires touch, movement, temperature, kinesthesia, and a slew of other senses to run simultaneously in the same bionic arm.
This way, “[when] they think about moving the arm, they feel that it’s moving through space and when they grab something, they feel that touch as though it’s theirs,” he said. “That’s the next piece of this, integrating these pieces into a whole.”