Un settore altamente innovativo come quello dell’ industria elettronica, può beneficiare enormemente della sinergia tra il microscopio digitale e gli smartglasses. Nel nostro progetto realizzato per la IMW abbiamo connesso i due device in modo tale che il tecnico possa vedere direttamente dove sta operando senza necessariamente alzare ogni volta lo sguardo verso il monitor. Questo ha comportato un forte riduzione del carico cognitivo da parte dell’operatore con la conseguente riduzione degli errori.
Augmented Reality tech is helping hospitals to deliver better patient care while at the same time cutting down on Covid infection danger, as Giacomo Lee reports.
Alder Hey Children’s Hospital in Liverpool has added to a long list of “firsts” in its century-long history by making innovative use of Augmented Reality (AR) technology to help its staff deliver better care.
“I was in the operating theatre performing a complex repair on a little heart, no bigger than the size of a strawberry,” says Rafael Guerrero, heart unit director and chief of congenital cardiac surgery at Alder Hey, one of the largest children’s hospitals in the UK.
Guerrero, who is also clinical director of innovation at Alder Hey, is recounting how he strapped on a pair of Microsoft’s HoloLens 2 smartglasses during surgery last year, bringing a new kind of cutting edge into the operating theatre – that of augmented reality. Transmitting the view of the patient’s heart live through HoloLens to colleagues nowhere nearby, as well as reviewing the echocardiogram at the same time, the procedure was another example of groundbreaking healthcare in Alder Hey’s wards.
In those wards there’s an obvious risk that Covid stalks the corridors, but older stalwarts such as MRSA remain equally infectious. As such, Alder Hey, the first hospital to test penicillin and to establish a neo-natal unit in the UK, wanted to use immersive, augmented and mixed reality (MR) technology to reduce the amount of physical contact between hospital staff and the public.
Working with Insight’s FastStart service to find technological solutions, the hospital primarily needed a way to communicate and share images and real life scenarios with other practitioners, regardless of location. This had to be done through video and audio in real time 24/7, not only for remote patient care, but also virtual ward rounds and staff training.
Wearing a HoloLens 2 device running Microsoft’s remote assist software Dynamics 365, Guerrero could perform his rounds alone. Other specialists joined in using the Teams app, seeing exactly what the clinician saw, along with three-dimensional (3D) echocardiography.
Alongside this augmented reality including real-life video, there was the ability for 2-way audio/video communication and sharing other visual content. Other important tasks – such as updating patient records – could also be redistributed among the team to save time as Guerrero worked.
“Using HoloLens, we have already demonstrated the potential of mixed-reality healthcare applications to minimise the number of staff required to be in contact with patients; share expertise in the shortest possible time and from any location; and, ultimately, increase the quality of care,” Guerrero tells Verdict.
Guerrero believes these applications would allow Alder Hey to support clinicians and patients in other hospitals and in the community, without the need for travel in either direction, thus helping efficiency and expertise-sharing in the NHS.
“While Covid-19 has acted as a catalyst, mixed reality devices that can support collaboration, mobility and remote learning need to be part of the technology that we use to drive a healthcare revolution,” says the specialist.
The Augmented Reality revolution in health
GlobalData forecasts that the global AR market will be worth $76bn by 2030, up from $4bn in 2018. As GlobalData analysts write in a recent report on AR in the health industry, “the healthcare sector is undergoing a digital transformation, fuelled by changing healthcare payor and provider need … AR is gradually making its way into the healthcare sector.”
This revolution is coming about for various reasons in more than one area of healthcare. Another application of AR is aiding visualisation during minimally invasive surgery (MIS).
Startups such as Medivis and Proprio are selling MR solutions in the surgical space, with the latter recently talking to Verdict about its AR and artificial intelligence-combined solution. Surgeons can also benefit from the augmented reality “x-ray vision” granted by US brand Augmedix, with an AR navigation tech that allows them to see a patient’s anatomy through skin and tissue whilst operating.
AR is a useful surgeon’s aid for the the visualisation of complex biological processes. EchoPixel’s True 3D software can help healthcare professionals visualise and interact with 3D images that depict human tissue and organs in open space as if they were real objects. Novarad’s OpenSight solution renders 2D, 3D and 4D images of patients interactively, while accurately overlaying them directly onto the patient’s body. Both solutions allow for more accurate surgical planning.
Distant doctors, digital health
The Alder Hey case study, though, shows how AR can be useful when integrated into telemedicine to help healthcare specialists work remotely. The same can apply to medical students: Imperial College London pre-empted Alder Hey somewhat in early 2020 by creating the world’s first virtual ward round for medical students, who watched as a HoloLens-equipped doctor examined a patient.
In times of social distancing, medical students can improve their knowledge and skills by accessing human body models created by AR. Solutions like those from EchoPixel allow medical students to familiarize themselves with the human anatomy and practice surgeries on virtual patients.
Real-life flesh and blood patients can also benefit from an augmented reality when distanced from doctors. A recent GlobalData report on digital health solutions in neurology suggests that virtual reality (VR) and AR are “well placed as a therapy product for neuropsychological conditions because virtual simulations of the outside world are highly realistic and immersive, as well as being non-invasive and non-pharmacological.”
Simulations offer less time-consuming options in comparison to some pharmacological interventions due to their immersive attributes, making for highly engaging forms of therapy.
Indeed, augmented reality technology appears in therapies targeting everything from chronic pain management, mental health and addiction, to behavioural conditions and neurorehabilitation. AR is also used effectively in exposure therapy, overlaying the hands of insect-phobic patients in live video with digital cockroaches and spiders. Cognoa’s Superpower Glass wearable meanwhile, a device based on Google Glass, helps autistic children better understand facial expressions through the use of emojis overlaid on the lens.
The rise of AR in health comes as VR/AR headsets and smart glasses become cheaper, and more AR apps appear on smartphones. Some brands like XRHealth are even sending out headsets to customers to treat ADHD using extended reality (XR). With FDA approval and products covered by insurance for over 50 million people in the US, XRHealth was able to trial a service which also offers a personalised care plan, in-app messaging and video calls with a dedicated clinician. While headsets may be an alien concept in the common household, especially so with regards to healthcare, teleconferencing with a medical professional is perhaps more agreeable to consumers after the pandemic’s video boom.
Medical professionals themselves are also succumbing to AR’s charms. A 2021 poll shared in GlobalData’s report on digital health in neurology found that 18% of 109 respondents in the industry thought AR/VR solutions would be most suitable to treat mental and behavioural health conditions such as depression, anxiety and substance abuse compared to other technologies. Meanwhile 15% of 110 respondents believed AR/VR technologies would be most suitable for neurological conditions such as Alzheimer’s and Parkinson’s.
Big Tech is pushing Augmented Reality in health
Besides the approval of the medical community, AR will also need Big Tech’s money to fully push it through the hospital doors. The adoption of HoloLens devices by Alder Hey and other hospitals shows that Microsoft already has one foot in the door. The tech company’s recent $19.7bn purchase of Nuance Communications will also help.
UK-based Nuance, a speech-recognition company best known for having provided Siri’s speech recognition engine, offers enterprise AI tools that transcribe doctors’ notes and visits during a working day. It isn’t much of a stretch to see that tech mixed with smart glasses: Healthcare professionals regularly require access to information systems, whilst often needing their hands free.
Here AR can reduce the burnout physicians face as it provides easy and quick access to real-time information, thus easing administrative burden. Data from the US suggests that more than 50% of the physician workforce is burned out, with excessive admin a major contributory factor; such burnout can affect safety and the quality of care in hospitals, and also increase healthcare costs.
With smart glasses augmenting reality, staff can work and take notes without the added weight of a physical notepad. This also allows for more accuracy as notes are taken in real-time, rather than the staff trying to recall information after patient interactions.
For Guerrero, this easier way of dealing with information was one reason why the HoloLens experiment was a successful one in Alder Hey.
“Ultimately, the success of any procedure is down to the skills of the surgeon and the team,” he tells Verdict. “Although new technology is used as a tool or aid for the surgeon to use, there are other ways of accessing the same information. It just takes longer and is not as accessible.”
In general, AR in medical devices is led by smaller vendors specialising in software and apps, according to GlobalData analyst Aliyah Farouk. But big tech players are investing in healthcare and working with the industry’s giants. Roche for example partnered with Samsung’s AR/VR subsidiary Harman in 2020.
“It’s difficult to say the extent of investment or impact in augmented healthcare as this is still a largely untapped market,” says Farouk. “You can look at areas where AR is making a big difference in tech, e.g. smart glasses and whether these devices are being used for healthcare purposes. For example, Google relaunched Google Glass Enterprise in 2017 targeting healthcare companies.
“Harman’s collaboration with Roche and Microsoft’s recent purchase of Nuance is reflective of this trend where healthcare companies are increasing their connectivity/digitalization and tech companies are capitalizing on this.”
By Verdict’s Giacomo Lee. Find the GlobalData Augmented Reality in Healthcare – Thematic Research report here.
HoloX advances self-serve augmented reality human holograms
VANCOUVER, British Columbia–(BUSINESS WIRE)–Nextech AR Solutions Corp. (“Nextech” or the “Company”) (OTCQB: NEXCF) (NEO: NTAR) (CSE: NTAR) (FSE: N29), a diversified leading provider of augmented reality (“AR”) experience technologies and services, is pleased to announce an exclusive, early access program for users to create, view and share their human holograms. Program participation will support the development lifecycle of HoloX, leading to its full release expected in the third quarter of 2021. In addition to the smart packaging use case noted below, AR human holograms have uses within dating applications, speaker keynotes, remote support, virtual brand ambassadors for new product launches and episodic content for marketing.
For HoloX early access program – limited to 100 seats – and DEMO VIDEO, CLICK HERE.
“Our goal to allow anyone to create and share their own augmented reality human hologram – using just a smart phone – is the key step towards mass adoption,” commented Evan Gappelberg, Founder and CEO of Nextech AR Solutions. “With the ongoing advancements made to our applications, we continue to tie together our AR offerings into one platform that offers an array of AR solutions. The ability to create the feeling of presence – or being in the same room in real time – with a live-streaming human hologram is one of the core technologies that will lead the transformation towards spatial computing.”
LIKE BEAUTY IN FLAMES is a new work by renowned artist Jenny Holzer that presents her signature text-based art through the medium of augmented reality (AR). This unique artwork, made exclusively for the Guggenheim Museum Bilbao and accessed through a standalone application for mobile devices, provides three distinct AR experiences—two site-specific works that engage the Guggenheim Bilbao’s architecture and a third that can be accessed from anywhere in the world. LIKE BEAUTY IN FLAMES harnesses new technology in a continuation of what Holzer does best: placing thought-provoking texts in the public sphere in a democratic and accessible way.
For over four decades, Holzer has experimented with the written word, constantly innovating strategies for the display of her works. Texts—sometimes the artist’s own, other times quotations by writers she admires—appear in bold fonts chosen for their direct, approachable quality. Since 1996 she has projected words in light onto landscapes and architecture, an act of ephemeral graffiti that recalls her early practice of pasting anonymous Truisms posters on the streets of New York City. The Truisms, comprising over 250 single-sentence declarations, embody a range of voices and viewpoints, eliciting diverse responses. The artist has a long-standing interest in appropriating the modes and media of official communication, presenting compelling messages on news screens, bronze plaques, and stone benches as well as T-shirts, condom wrappers, and pencils. Her work has long occupied an important place in public art, often through temporary offerings such as arresting large-scale projections and roving digital billboard trucks emblazoned with her texts. Her voice remains resonant and courageous, inviting viewer engagement through the simple act of reading.
At the Guggenheim Bilbao, an AR version of one of Holzer’s signature LED signs glides through the central Atrium, the column bending and spiraling as Truisms in English, Spanish, Basque, and French scroll past. Each of the museum’s three floors provides a different experience, as the LED’s interaction with the building’s architecture shifts depending on one’s vantage point. Outside, AR projections appear on the museum’s facade from five locations around the campus in a virtual echo of the artist’s monumental 2019 work For Bilbao, which was presented in conjunction with her retrospective exhibition Thing Indescribable. The texts selected by Holzer for the AR projections are by other authors; the work’s title, LIKE BEAUTY IN FLAMES, is borrowed from Polish poet Anna Świrszczyńska’s poem “Beauty Dies,” one of the many arresting texts that Holzer has made into immaterial spectacle. The artwork’s final component can be accessed from anywhere in the world and enables users to see Holzer’s Truisms take form in space over any live view. This new possibility embodies the spirit that has informed Holzer’s practice throughout her long career of placing language in the public domain to spark reflection and contemplation.
THE IMPACT ON PUBLIC SPACE
The public space surrounding the Guggenheim Museum Bilbao is the setting for one of three augmented reality experiences designed as part of the app LIKE BEAUTY IN FLAMES. Bilbao residents and visitors can look for the QR codes at five points around the museum, open the app and enjoy virtual light projections on the facade.
ABOUT THE TECHNOLOGY
This immersive and dynamic artwork is made possible by a mobile app that uses AR to provide unlimited access to three virtual experiences. Harnessing augmented reality, advanced image-recognition technology, 3D spatial positioning and cutting-edge graphics features, users can watch the AR version of the LED sign move through the museum’s atrium. They can also see a recreation of Holzer’s nighttime projections, facilitated by their devices’ GPS and gyroscope features, which direct them to various points around the outside of the museum. Combining digital technologies with the real-world, this innovative mobile app developed by London-based digital agency Holition brings Holzer’s practice to users’ own mobile devices.
LIKE BEAUTY IN FLAMES demonstrates a commitment on the part of the Guggenheim Museum Bilbao to technology in the arts. This project marks the beginning of a three-year plan to present pioneering projects in which technology is central to the artwork.