SURVEY OF MEDICAL VR APPS with DICOM – KNOW THYSELF AS A VIRTUAL REALITY

The Body VR: Anatomy Viewer displays 3D volumes generated from traditional CT/MRI scans that allows radiologists, medical students, and patients to view anatomical scans in virtual reality. The application includes scans from the brain, abdomen, heart, lungs, spine, and allows users to import their own DICOM files.

Users are able to directly interact with these models using hand controls, allowing a level of engagement on par with high end virtual reality applications. These controllers will allow the user to scale, rotate, and crop scans freely in 3D space. Users can add annotations to specific areas within a scan, to indicate lesions and tumors. The volumes can then be saved and uploaded to be viewed anywhere in the world, all patient health information is removed in the process. Users can select between multiple preset color schemes and even create your own. There are multiple environments built into the Anatomy Viewer to suit your preference.

(Text adapted from https://store.steampowered.com/app/579620/The_Body_VR_Anatomy_Viewer/)

ImmersiveView™ VR

Country US

Developer Immersive Touch

Platform Compatible With Major VR Platforms

Commercial Availability Oculus, VIVE

Links https://www.immersivetouch.com/immersiveview-vr

Keywords CT/MRI, Surgical planning, training, surgical simulations

Features Slicing, opacity tool, cross sectional viewing,

drawing and marking, plan and simulate surgical procedures

ImmersiveView™ VR converts CT, CBCT, 3D angiography and MRI scans into a ‘digital twin’ of the patient and provides a completely unique and unobstructed view of the case in virtual reality. No longer limited to certain angles of view, doctors can easily view the target anatomy, clearly and accurately, just as it exists in reality. Physicians handle the scan as if it were a physical object in the palm of their hand, observing from every angle to get the very best understanding of the situation, even flying into the anatomy to take a closer look. ImmersiveView™ VR provides unprecedented clarity in patient scans and is simple and intuitive to use. The platform is ideal for viewing critical scan information in all areas of the hospital: Radiology labs, ORs, ERs, and doctors’ offices.

Patients are imaged using common modalities like CT and MR. These scans are most frequently in a file format called DICOM. Under current processes, these patient DICOM scans are interpreted one slice at a time on a flat computer screen. The clinician must visualize anatomy and pathology by mentally “stitching” together the slides in their head.

With ImmersiveView™ VR, 2D DICOM scans are merged and put into 3D virtual reality. Every conceivable angle of a patient’s unique anatomy can be engaged, from simple anatomy viewing to overlaying the original scan, creating measurements, placing virtual hardware, and much more.

(Text adapted from

https://www.immersivetouch.com/immersiveview-vr)

Further reading

**Free control**: manipulate translation, rotation and scale of 3D anatomy.

**Measurement system**: Record distance between freely placed points on anatomical data.

**Opacity Control**: Control visualization for multiple layers of 3D anatomical data in realtime.

**Dynamic Cutting**: Slice freely through anatomical data at any angle.

MEDICAL HOLODECK

Country Switzerland

Developer Medicalholodeck.com

Platform Valve Index, HTC Vive, Windows Mixed Reality

Commercial Availability Can be used with VR glasses from all manufacturers

Links https://www.medicalholodeck.com/en/

Keywords CT/MRI, DICOM, XR/VR

Features DICOM segmentation, high-resolution anatomical dissectionsl

Medicalholodeck’s virtual reality applications are used by medical professionals for surgery planning in teams and in medical education for teaching in virtual classrooms.

Medical Imaging XR is the DICOM Viewer for professional teamwork in virtual reality (VR). The software enables the import of medical image data from CT and MRI examinations in DICOM format and is used for surgery preparation and medical education. The app provides tools for editing, filtering, segmenting and measuring DICOM, OBJ and STL data in virtual reality and is ready for teamwork in VR. Groups of users can work together on DICOM and 3D data and discuss patient cases regardless of location. The app can be used with virtual reality glasses from all manufacturers.

Dissection Master XR is the anatomy lab in virtual reality (VR). The app offers 3D data of human dissections in a VR classroom environment. It is ready for group teaching in virtual reality or by online streaming. The professionally dissected bodies are grouped and can be studied layer by layer. They are fully annotated and linked to Wikipedia for further information.

Anatomy Master XR Study the precise 3D male and female anatomical models of Zygote in VR. The high-resolution and carefully textured 3D models can be taken apart. They then convey the structure and organs of the human body as an interactive 3D object. The anatomical data are grouped. They can then be cut, filtered and enlarged in VR with various tools. Anatomy Master XR can be used simultaneously with Dissection Master XR and Medical Imaging XR. This enables the comparison of precise anatomical 3D models, medical patient data in DICOM format and high-resolution anatomical dissections. The software is ready for teamwork in VR. Teams can work on anatomical 3D models in VR at the same time.

(Text adapted from

Medicalholodeck.com)

Further reading

Napa, Sandeep et al. “Advancing Cardiac Surgery Case Planning and Case Review Conferences Using Virtual Reality in Medical Libraries: Evaluation of the Usability of Two Virtual Reality Apps.” JMIR human factors vol. 6,1 e12008. 16 Jan. 2019, doi:10.2196/12008

Medical Imaging VR

Country US

Developer hublab

Platform Valve Index, HTC Vive, Oculus Rift

Commercial Availability N/A

Links https://store.steampowered.com/app/1471980/MedicalImagingVR/

Keywords MRI, medical data, DICOM

Features DICOM import, Rotating, Scaling, Multi-planer visualization

Magnetic resonance imaging (MRI) is a type of scan that uses a strong magnetic field and radio waves to create detailed images of the organs and tissues within the body. An MRI scan uses a large magnet, radio waves, and a computer to create a detailed, cross-sectional image of internal organs and structures. This application allows to display different MRI scans and to browse into the different slices through a virtual reality environment. Users can also import their own MRI data (images from DICOM files) into the application to visualize them in VR.

(Text adapted from

https://store.steampowered.com/app/1471980/MedicalImagingVR/)

Multi-planer visualization: axial, coronal and sagittal.

True 3D

Country US

Developer EchoPixel Inc

Platform

Commercial Availability N/A

Links https://echopixeltech.com/applications.html

Keywords MRI, DICOM

Features Multi-modality of DICOM/MRI, Interoperability with any PACS system

True 3D is a software platform that enables physicians to interact with virtual patient specific anatomy without the need for a bulky VR/AR headset. True 3D is used to assist physicians to plan surgical and interventional procedures to treat Congenital Heart Defects.

Patients with congenital heart defects and previous operations represent a visualization and surgical planning challenge due to a wide variation in morphologic features and clinical presentations. True 3D can provide 3D visualization for surgical interventions for a number of congenital heart defects.

(Text adapted from

https://echopixeltech.com/applications.html)

Further reading

True 3D: Unlocking the full potential of medical imaging

The SuRgical Planner (SRP)

Country US

Developer Surgical theater

Platform

Commercial Availability Oculus, VIVE

Links https://surgicaltheater.net/services/

Keywords CT/MRI scans, surgical planning

Features Patient-specific VR reconstructions, 3D interaction

Surgical Planner accesses files from a patient’s traditional image modalities, such as CT or MRI scans, and processes the information to create patient-specific, VR reconstructions to help a neurosurgeon plan surgery, and educate the patient about their neurological condition.The interactive, immersive reconstruction can be seen on a touchscreen and through the use of a VR headset, such as Oculus Rift® or HTC Vive®. Surgeons can fly through the brain and locate medical issues such as a tumor or an aneurysm and study it from every angle with our medical virtual reality services. What SRP provides is especially important when developing an intricate surgical plan by allowing surgeons to create the best surgical approach based on the patient-specific VR reconstruction.

(Text adapted from

https://surgicaltheater.net/services/)

Further reading

DICOM VR

Country US

Developer Chris Williams (medical physicist)

and Kos Kovtun (radiation oncologist)

Platform

Commercial Availability N/A

Links http://www.dicomvr.com/

Keywords Volumetric medical imaging, DICOM

Features Volume rendering, clipping, different display modes,

contouring using 2D/3D brush, Interactable LINAC model

Chris Williams (medical physicist) and Kos Kovtun (radiation oncologist) work at the Brigham and Women’s Hospital/Dana Farber Cancer Institute. They are developing DICOM VR because of fundamental need to work with the increasing complexity of volumetric medical imaging. This work has recently been funded by a Brigham Research Institute Shark Tank grant.

(Text adapted from

http://www.dicomvr.com/)

Intravision XR

Country US

Developer DICOMDIRECTOR

Platform

Commercial Availability N/A

Links https://www.dicomdirector.com/intravision-xr/

Keywords 3D surgical modeling, DICOM, CT/MRI

Features Segmentation and visualization, scaling, rotation,

and plane on any device

INTRAVISION XR is a cloud-based, 3D model medical software tool that automates the process of creating 3D models and allows you to visualize them. Fully formed 3D models are created from any DICOM dataset such as CT and MRI scans. Models contain complete anatomical detail, including spatial relationships between organs and their internal structure. View organ vasculature, neoplasms and anomalies in-situ within specific structures. Models are viewed in unparalleled visual detail using AR, VR or standard screens (phone, tablet, computer). Enhanced viewing of dimensional models in Augmented or Virtual Reality is the most realistic way to view anatomical structures.

(Text adapted from

https://www.dicomdirector.com/intravision-xr/)

Further reading

CommandEP

Country US

Developer SentiAR

Platform

Commercial Availability N/A

Links https://sentiar.com/commandep/

Keywords AR, 3D electroanatomic mapping

Features Gaze based control, anchored model, clipping

CommandEP uses real-time data from catheters and 3D electroanatomic mapping systems, to recreate an anatomical and electrical map of the particular patient’s heart.

(Text adapted from

https://sentiar.com/commandep/)

Further reading

DICOM AI 3D VR VIEWER

Country US

Developer inlu

Platform PC + HTC Vive

Commercial Availability N/A

Links https://inlu.net/vr-projects/dicom-ai-3d-vr-viewer/

Keywords DICOM

Features 3D visualization, Data access directly from PACS

The initial need for the product appeared from a private medical institution in order to provide their patients with a truly visual and simply understandable way of their health condition. Specifically – DICOM scans. These scans are used widely in medical institutions around the world in order to present doctors with the inner body scan of the patient – a precise organ. This product virtualizes DICOM scans in order to present 3D images to doctors. They can then examine the model of the human body in full 3D mode, not only on a personal desktop but also via virtual reality glasses. Product enables in-depth exploration of organ condition and past changes examination along with patients.

(Text adapted from

https://inlu.net/vr-projects/dicom-ai-3d-vr-viewer/)

Papers

Applications of Virtual and Augmented Reality in Biomedical Imaging

Citation

González Izard, S., Juanes Méndez, J.A., Ruisoto Palomera, P. et al. Applications of Virtual and Augmented Reality in Biomedical Imaging. J Med Syst 43, 102 (2019). https://doi.org/10.1007/s10916-019-1239-z

Abstract

Virtual and Augmented Reality has experienced a steady growth in medicine in recent years. At the same time, the radiological images play a central role in the diagnosis and planification of surgical approaches. The aim of this study is to present the first attempt to enhanced radiological image visualization using virtual and augmented reality for better planification and monitorization of surgeries. This application allows to move beyond traditional two-dimensional images towards three-dimensional models that can be visualized and manipulated with both Augmented Reality and Virtual Reality. We propose possible approaches to automate the segmentation of radiological images, using computer vision techniques and Artificial Intelligence.

Keywords

Automated Segmentation, 3D visualization, AR/VR, DICOM, radiological imaging, CT

General architecture of the designed system

Selecting different options for 3D model manipulation with virtual reality platform

Virtual Reality Educational Tool for Human Anatomy

Citation

Izard, S.G., Juanes Méndez, J.A. & Palomera, P.R. Virtual Reality Educational Tool for Human Anatomy. J Med Syst 41, 76 (2017). https://doi.org/10.1007/s10916-017-0723-6

Abstract

Virtual Reality is becoming widespread in our society within very different areas, from industry to entertainment. It has many advantages in education as well, since it allows visualizing almost any object or going anywhere in a unique way. We will be focusing on medical education, and more specifically anatomy, where its use is especially interesting because it allows studying any structure of the human body by placing the user inside each one. By allowing virtual immersion in a body structure such as the interior of the cranium, stereoscopic vision goggles make these innovative teaching technologies a powerful tool for training in all areas of health sciences. The aim of this study is to illustrate the teaching potential of applying Virtual Reality in the field of human anatomy, where it can be used as a tool for education in medicine. A Virtual Reality Software was developed as an educational tool. This technological procedure is based entirely on software which will run in stereoscopic goggles to give users the sensation of being in a virtual environment, clearly showing the different bones and foramina which make up the cranium, and accompanied by audio explanations. Throughout the results the structure of the cranium is described in detailed from both inside and out. Importance of an exhaustive morphological knowledge of cranial fossae is further discussed. Application for the design of microsurgery is also commented.

Keywords

Educational, Interactive 3D visualization, VR, computed tomography