Raahil M. Sha ’18 observed quietly as the doctor he was shadowing carefully inserted a needle into his patient’s exposed abdomen. Watching as the doctor removed the needle and readjusted the angle, he envisioned a tumor ablation procedure that was both safer and more efficient.
“We thought that it would be cool if they were able to just see the tumor inside of the person, stick the needle in, hit the tumor, and ablate it,” Sha says, using hand motions to demonstrate the approach. “Without having to wait forever for the scan to go in and out of the room.”{shortcode-74dd2ceac9b3de99038e50a8dae9b79e67cfb618}
Together, Sha, Grigory Khimulya ’18, and Jose M. Amich ’18 have a startup called Z Imaging, which aims to cut the time it takes to do this procedure in half. Through the use of augmented reality headsets that superimpose digital images onto the real world, their system will allow doctors to see MRI scans overlaid onto the patient’s body.
“So what essentially happens is the surgeon puts on the headset, and they’re able to see the person’s liver and see where the tumor is within the liver,” Sha says. “As the needle is going through the body, they’re not able to see it in real life. But with the augmented reality overlay, they’re able to say, ‘Oh the needle is exactly 5 millimeters towards the target.’ Then the little simulated needle turns green and flashes whenever they’re on actual target, and they’re able to click their button and do the actual ablation from there.”
By shortening surgery time, Sha explains, these headsets allow the procedures to be done more safely and with less anesthesia, resulting in quicker recoveries, and limiting potential complications. He knows this because he’s seen it work.
“We already did tests on blocks of fake flesh,” he says. “There’s recipes you can find online for ‘Combine these things together in a pot and then put it into a bowl. Then it’ll have the same consistency as an abdomen,’ for example. So we made those and tested.”
This “home-cooked” flesh is soon to be replaced with professional dummies and eventually cadaver models. Then, they’ll work on getting FDA approval before taking their headsets into real-life operating rooms. The only thing holding them back? Existing tech.
“HoloLens,” Sha says, referring to the headset the company uses, “is limiting because your field of view is small and it doesn’t track your head as well. So if I turn my head 5.6 degrees, the Hololens might say that was 5.65 degrees or something inaccurate. So we had to make our own tracking system that was more accurate than the built-in one in the headset.”{shortcode-f2f49c94637278749b0f453b990c225f637a2d9b}
Sha and his friends work in eight-hour blocks, two to three days per week, developing their technology at Harvard’s Innovation Lab. When he’s not working, he’s spending time with friends in his “deep underwater themed” room or visiting the tallest buildings and monuments in Boston to get the best views of the city skyline.
“We usually have tea parties by the full moon,” Sha says. “Whenever there’s a full moon and it’s not cloudy, we go out to the Science Center observation deck and drink tea and, after a while, come back in, open up the telescope and look at the moon.”
Immersed in a virtual world, Sha draws a flower pattern in three-dimensional space. This VR headset, he explains, is no longer useful to their project so it has been repurposed as a gaming device. Sha reaches into his backpack and pulls out the HoloLens. After waiting a few seconds for it to turn on, he places it over his head and adjusts the machine’s position over his eyes.
“If you put it on, you can kind of put a cat in the middle of the room,” Sha says, as he extends his arm and flicks open his fist. “It won’t be a real cat, it’ll be like a virtual cat and it’ll just walk around or whatever.” He then hands me the headset. “Wanna try?”
I place the device over my eyes and look around, but nothing abnormal seems to be within my field of view. Confused, I look over at Sha, who smiles and points at the ground underneath my feet. As I glance down, suddenly, a small cat comes into focus, tail wagging and sitting attentively.
As I pet its soft-looking orange fur, Sha says, “But instead of just placing things in the room, we decided to place them inside the person’s body.”
