Know Thyself as a Virtual Reality

Know Thyself as a Virtual Reality is an interdisciplinary project that focuses on the ethics and aesthetics of the use of medical data and virtual reality. In order to explore this field, we have assembled a skilled and diverse team from across the University of Alberta as well as from other Canadian and international universities, bringing together expertise in medical imaging, health law, software development, biomedical engineering, virtual narratives, sound, the history of medicine, and artistic creation. This collaborative team will explore, create, disseminate and share ideas about cutting edge technologies that produce bodies in virtual and augmented reality. These bodies offer striking opportunities for producing new medical knowledge, but also render bodies transparent, accessible and manipulable in potentially exploitative ways, particularly during an era of increasing surveillance. Our project addresses these and other issues, allowing scholars and students to experiment with the technologies, learn from each other, and engage a broader public. This unique fusion of medical diagnostics and fine arts will ultimately help us to better know ourselves and our bodies in the digital age. 

The project unfolds in a series of meetings, workshops, and creative endeavors to produce multiple outcomes. An ethic eSymposium in Summer 2021 brought researchers together to cross-pollinate research across the disciplines of computer science, digital humanities (ethics of AI, and race/gender bias in computer visualization), fine arts (focused on artistic use of medical imaging), nursing (focused on computer literacy and the impact of digitized health records), radiology (full body, 3D/4D MR and ultrasound) and rehabilitation medicine and reflect on best practices for working with sensitive data in creative research. This eSymposium has informed the document Best Practices for Working Artistically with Sensitive Data.

Two research creation projects, My Data Body and Your Data Body, are bring created by the interdisciplinary team to better explore and exemplify the important ethical and aesthetic issues at stake in the field of body scan imaging. These VR projects will be exhibited at a Know Thyself as a Virtual Reality exhibition at FAB Gallery, Edmonton in February 2022 along with five other VR artworks created by invited artists.


Here are examples of virtual reality artworks, medical and educational virtual and augmented reality apps made by the team. As part of the Know Thyself project, the team are currently developing two new virtual reality projects and also new open source tools to allow other students, creatives and healthcare practitioners to easily work with medical scan data in virtual reality.

My Data Body

My Data Body is a collaborative, multimedia installation that include sculptures, video projection, prints and virtual reality. My Data Body brings together different forms of personal data such as medical scans, social media, biometric, banking and health data in an attempt to make visible and manipulable our many intersecting data corpuses so that in VR they can be held, inspected and dissected. In My Data Body, the medically scanned, passive, obedient, semi-transparent body becomes a data processing site that can be pulled apart, de- and re-composed or as Yuval Harari warns ‘surveilled under the skin’. My Data Body is an interdisciplinary and collaborative project made as part of the research project Know Thyself as a Virtual Reality (KTVR). Collaborators include Marilène Oliver (visual art), Scott Smallwood (sound), J.R. Carpenter (poetry) and Stephan Moore (sound). My Data Body was made with the assistance of Daniel Evans, Kirtan Shah, Walter Ostrander, Chelsey Campbell, Alissa Rossi and Jiayi Ye and has been greatly informed by the expertise of the whole KTVR research team.


Your Data Body (Work in Progress)

Your Data Body is a partner project to My Data Body made using a combination of open source and donated datasets. This project focuses on issues of data privacy and ownership, playing on the etymology of the word data meaning ‘given’. The user can pick up and move datasets around. The scanned body parts can resized and re-colored, inviting a playful stacking of the body parts to make a whole Frankenstein-like figure. Attached to each body part is an audio file which is triggered when the user holds and manipulates it. Anonymized open source datasets are accompanied by an automated voice which reads the study data published alongside the dataset, whereas the donated datasets will have different conversational AI characters/chatbot that the user can ‘discuss’ different issues relating to data ownership, privacy and virtuality with.

Deep Connection 2019

Deep Connection is an installation and VR artwork made using a full body MRI scan dataset. When the viewer enters Deep Connection, they see a scanned body lying prone in mid-air. The user can walk around the body and inspect it, lie underneath and walk through it. The user can dive inside and see its inner workings, its lungs, spine, brain. The user can take hold of the figure’s outstretched hand: holding the hand triggers a 4D dataset, making the heart beat and lungs breathe. When the user lets go the hand, the heart stops beating and the lungs stop breathing. MORE

Space Invaders 2019

Space Invaders is a sculpture and VR installation that creates the uncanny experience of having our personal physical space invaded by virtual bodies. When the viewer enters the VR space, they find they are closely surrounded by 3 MR scanned figures which is it possible to move through and into. The physical body of the view becomes the interface with the virtual body. MORE

Je Te Tiens, Tu Me Tiens (work in progress)

A virtual reality art work that uses high resolution MR scan data to create an immersive experience that allows the user to enter the space of two scanned brains and explore them. Inside one brain space will be ‘memory bank’ which will follow the logic of functional neuroanatomy.  The aim is that the user will be able to walk inside a dataset of a brain and with their hand activate different memories. The ‘memories’ will be visible as text paths embedded into axial cross sections through the data that will brighten and become legible when the user touches them with their virtual hand.  For example, when the user enters the occipital lobe, they will see/read and hear memories related to vision. In the other brain space, a white matter tractography map will be the foundation for flow paths inside the brain. Along this these paths text will flow. The user will be able to block, disrupt or catch the text. MORE


Since the discovery of ionizing radiation, clinicians have evaluated X-ray images separately from the patient. The objective of this study by Aaskov, Kawchuk et al was to investigate the accuracy and repeatability of a new technology which seeks to resolve this historic limitation by projecting anatomically correct X-ray images on to a person’s skin using a Hololens Mixed reality system. READ MORE

GaMA (Gaze and Movement Assessment)

Successful hand-object interactions require precise hand-eye coordination with continual movement adjustments. The Gaze and Movement Assessment (GaMA) was developed to provide protocols for simultaneous motion capture and eye track-ing during the administration of two functional tasks, along with data analysis methods to generate standard measures of visuomotor behaviour. Chapman’s team is currently developing GaMA to capture hand-eye coordination data whilst in VR. READ MORE

CogPro VR Projects

CogPro develops practical and novel tools that are complex yet captivating and easy to utilize. Many of the applications are AI powered meaning tools like Natural Language Processing enrich the interactions between user and the virtual environment. Team members Nathanial Maeda and Dr Martin Fergusson-Pell are leaders of the CogPro team and below is a selection of their projects. READ MORE

CORONA Pod 2020

Corona Pod is a virtual reality artwork created using Tilt Brush by Marilene Oliver. In VR, the pod is large enough for an adult human to stand in. Oliver created this pod during the first COVID lockdown as a place to escape, and also as a place to dare imagine the experience of her mother being alive at his time.

Archipelago by Daniel Evans

Archipelago by Daniel Evans is an explorable virtual environment created algorithmically from the contents of the artist’s Google user account. Location tracking data, activity patterns, and email are all used to generate the landscape, weather patterns, and procedural soundscape.

Huldra by Daniel Evans

Huldra by Daniel Evans is a VR experience that juxtaposes themes of resource extraction and data mining. Users connect a personal photo archive, such as Google photos, which is analyzed for monetizable information. The work makes predictions about the user’s behaviour patterns and personal life based on the contents of the images it analyzes.


University of Alberta / Dr. Lianne McTavish’s interdisciplinary research centres on early modern French medical imagery. She has published over 40 single-authored articles in journals such as Social History of Medicine and Medical History, as well as a single-authored monograph titled Childbirth and the Display of Authority in Early Modern France (2005) (and 4 other books). Dr. McTavish has curated several art and science exhibitions, including Perceptions of Promise in 2010 and Flux in 2017.

University of Aberdeen / Dr. Silvia Casini’s work is situated at the crossroad of visual culture and science and technology studies. She is the author of several articles on the aesthetic, epistemological and societal implications of scientific visualisation. She has been recently awarded a Leverhulme Research Fellowship to complete her second book project Bodies of Data: Image-makers, Data and Reinvention in Magnetic Resonance Technology, which is under contract with MIT Press Leonardo book series.

University of Alberta / Dr. Martin Ferguson-Pell is a biomedical engineer and credentialed clinical scientist, dedicated to the study of secondary complications of physical disability. He designed and developed a new Rehabilitation Robotics Laboratory in the Edmonton Clinic Health Academy and uses virtual reality systems to simulate the biomechanical and physiological conditions faced by long-term wheelchair users and elite wheelchair athletes. He recently announced the creation of a not-for-profit called ELIXR Simulations in partnership with Health City and supported by the SMART Network.  ELIXR is intended to bridge the gap between the development of proof of concept XR content and its publication and distribution. ELIXR works in collaboration with YEG post secondaries, Health City and a network of XR industry collaborators.

University of Alberta / Dr. Nathanial Maeda is a Post-Doctoral Fellow conducting research and development in virtual reality technologies as the lead of the Cognitive Projections program: an inter-faculty collaboration that researches, develops, and implements cutting-edge Virtual and Augmented Reality applications for education and healthcare.

University of Alberta / Craig Chapman Associate Professor, Faculty of Kinesiology, Sport, and Recreation, studies how the brain makes decisions that allow people to successfully navigate their world, targeting relevant objects for action and ignoring or avoiding irrelevant ones. These decisions play an essential role in healthy living, but often fall below conscious awareness. Chapman asks what is the added value of consciousness, and what does it mean for non-conscious decision processing to play such a strong role. The question suggests an important blurring of the distinction between the conscious and the non-conscious. In his lab, Chapman uses a detailed analysis of sensorimotor processing that integrates eye-tracking, electroencephalography and motion-tracking. This has shown that a decision about making a movement doesn’t end once the movement is initiated, but continues until the movement is complete. Therefore, movements provide a window into deciding and thinking, and movement recording is a powerful research tool for science and a diagnostic tool for medicine.

University of Alberta / Dr. Scott Smallwood is a sound artist, composer, and performer whose work spans from acoustic concert music to interactive installations. Dr. Smallwood is currently working on Lost Garden, sound-focused VR game for which he is developing ambient compositions to be experienced in virtual reality. He also has expertise in field recording and interactive sound.

University of Alberta / Daniel Evans is an interdisciplinary artist working in the expanded field of printmaking. Incorporating rapid prototyping, microelectronics, and virtual reality alongside traditional printmaking and drawing techniques, he employs the languages of folklore, games, and data visualization as forms of speculative world building. Drawing on current research in technology ethics, data violence, and data dignity, his work explores the material impact of digital infrastructures, the intersection of physical and virtual spaces, and the interconnected relationship between the physical and digital self. He completed undergraduate studies at MacEwan University and the Alberta University of the Arts, and holds an MA in Design from the North Wales School of Art and Design and an MFA in Printmaking from the University of Alberta. His work has been included in exhibitions in Canada, the United States, the United Kingdom, Poland, and Finland.

University of Alberta / Marilène Oliver is Assistant Professor of Visual Art at the University of Alberta in Edmonton. Her work is at the crossroads of new digital technologies, traditional print, and sculpture. It focuses on producing objects that bridge the virtual and the real worlds. She uses medical scanning technologies such as MRI, CT, and PET in works that allow us to materially contemplate our digitized selves. She has exhibited internationally, at the Royal Academy in the UK, the MassMoCA in the US, and the Copernicus Museum in Poland. Her work is also held in a number of private and public collections around the world. She leads LASERAlberta, a series of Arts-Science public talks affiliated with Leonardo ISAST.

University of Alberta / Dr. Ubaka Ogbogu is an Associate Professor cross-appointed to the Faculties of Law and Pharmacy & Pharmaceutical Sciences. Ogbogu’s research interests are in health law, public health law, science and health policy studies, law, bioethics and biomedicine, legal history of public health and health care in Canada. He is particularly interested in the points of confrontation between ethics, morality, economics and law in relation to the governance of novel and controversial health care technologies.

University of Alberta / Fahim Hassan is a Ph.D. student at the School of Public Health at the University of Alberta. His research interest is to study the application of machine learning in public health surveillance, risk communication and health promotion. He is also currently working at the Ministry of Advanced Education at the Government of Alberta. As a civil servant, he has contributed to various government initiatives on open data, social determinants of health, and health informatics. In addition, he is serving as the Vice-chair in an advisory council member for Alberta Health Services in the Greater Edmonton Area. As council member, he works with local communities on health promotion activities and shares their voices to healthcare service providers.

University of Alberta / Katrina Ingram is passionate about art, technology, business and education – all of which are reflected in her diverse career path. A marketer by trade, she spent her early career in the software sector. She segued into media, joining the Canadian Broadcasting Corporation and then landing at CKUA Radio where she held the role of Chief Operating Officer until 2017. She is also a session faculty member with MacEwan University. Katrina holds a Bachelor of Business Administration from Simon Fraser University and recently completed her Master of Arts at the University of Alberta with a focus on Communications & Technology. Her recent work has been in the area of Artificial Intelligence and ethics. Since completing her degree, Katrina has founded a company called Ethically Aligned AI which is an extension of her graduate research and aims to help companies build better AI.

University of Alberta / Dr. Jonathan Cohn researches topics related to digital media and algorithmic culture and is Director of Digital Humanities.

Queen’s University / Dr. Gabor Fichtinger is a Professor and Canada Research Chair in Computer-Integrated Surgery, School of Computing, Queen’s University. His research and teaching specialize in computational imaging and robotic guidance for surgery and medical interventions, focusing on the diagnosis and therapy of cancer and musculoskeletal conditions. He is the director of the Laboratory for Percutaneous Surgery, a core contributor to the 3D Slicer open-source software platform and PI of several open-source translational research platforms dedicated to image-guided surgery, radiation therapy, and interventional ultrasound.

University of Alberta / Dr. Kumaradevan Punithakumar has published more than 85 peer-reviewed scientific papers and holds nine patents. He has developed a number of stereoscopic and VR medical image visualization approaches for pre-operative surgical planning for congenital heart diseases. Dr. Punithakumar collaborated with Marilène Oliver on Deep Connection.

University of Alberta / Ewen Lavoie, a PhD Candidate in Kinesiology and Neuroscience, studies the feeling of embodiment, the sense of being in a body, and how it can be manipulated using virtual reality technologies. Measurement tools like eye and motion tracking along with participant experience surveys allow Ewen to compare differences in users’ behaviour and experience in the real world and under various VR conditions. His work combines theories of visual attention, sensory integration, and consciousness to inform our knowledge about the connection between the mind and the body and increase the efficacy of medical, educational, and mental wellness applications.

University of Alberta / Jonathan Garfinkel‘s written works have been translated into twelve languages, and his plays have been produced throughout Canada, Germany, Russia and Ukraine. He is the author of the books of poems Glass Psalms (Turnstone Press) and Bociany/Storks (KFB, 2017). He has written numerous plays including The Trials of John Demjanjuk: A Holocaust Cabaret and the Governor-General shortlisted House of Many Tongues. His memoir Ambivalence: Crossing the Israel/Palestine Divide was published in five countries to critical acclaim (Penguin Canada and Norton & Norton US). Named by the Toronto Star as “one to watch”, he is currently doing a PhD in MLCS at University of Alberta. His project, A Diabetes Diary: Towards A Daily Phenomenology, is a memoir project about living with Type 1 Diabetes and living with ‘Loop’, a DIY, open source hack that creates a closed loop, artificial pancreas system with an insulin pump, smart phone and CGM.

University of Alberta / Geoffrey Rockwell is a Professor of Philosophy and Humanities Computing at the University of Alberta and is the associate director of University of Alberta’s signature area AI4Soceity. He received a B.A. in philosophy from Haverford College, an M.A. and Ph.D. in Philosophy from the University of Toronto. From 1994 to 2008 he was at McMaster University where he was the Director of the Humanities Media and Computing Centre (1994 – 2004) and he led the development of an undergraduate Multimedia program funded through the Ontario Access to Opportunities Program. He has published and presented papers in the area of philosophical dialogue, textual visualization and analysis, humanities computing, instructional technology, computer games and multimedia. He is the project leader for the CFI (Canada Foundation for Innovation) funded project TAPoR, a Text Analysis Portal for Research, which has developed a text tool portal for researchers who work with electronic texts. He is the author of “Defining Dialogue: From Socrates to the Internet”, published with Humanity Books.

University of Alberta / Gillian Harvey is an Assistant Professor of Visual Communication Design at the University of Alberta. Harvey’s research and practice include evidence-based information design principles and practices. She prioritizes the simplification of complex information in order to make it understandable for people. She advocates for understanding different audiences and the importance of user-centred design. Harvey’s recent work is in the development of communication materials that assist with understanding COVID communication messaging in underrepresented populations and in particular how information design theories and practice can be applied to health care scenarios. She has worked with government, industry and the private sector developing materials that include projects that address systems concerns, and enhance broader healthcare processes. Gillian has served as principal researcher of two public health teams involving over one dozen collaborators from across Canada. Based on the learnings from both of these projects, there is a significant gap in the understanding of how health messages are translated to youth, and to youth in underrepresented populations.

University of Alberta / Dr. Pierre Boulanger is Professor of Computing Science and at the Department of Radiology and Diagnostic Imaging. Dr Boulanger has more than 35 years of experience in 3D computer vision, rapid product development, and the applications of virtual reality systems to medicine and industrial manufacturing. He is currently the Director of the Advanced Man-Machine Interface Laboratory as well as the scientific Director of the Servier Virtual Cardiac Centre.

University of Alberta / Dr. Jacob Jaremko has a combined clinical and academic appointment. Dr. Jaremko’s key research focus is on advanced ultrasound imaging, including 3D ultrasound for diagnosis and management of infant hip dysplasia. He has published over 140 peer-reviewed scientific papers.

University of Alberta / Dr. Manal Kleib’s research aims to improve outcomes of care for patients in acute and long-term care settings in which nurses and nursing students use the electronic health record, through enhancing their informatics competency to use this digital health tool safely and meaningfully. She has completed a number of studies related to the development, integration, and evaluation of informatics competency in nurses, students, and nursing educators. She also collaborates with professional and community-based associations, locally, provincially, and nationally to increase awareness about informatics and digital competency, among healthcare providers and consumers.

University of Alberta / Dr. Greg Kawchuk’s research interests focus on defining the mechanisms that initiate and sustain spinal disorders so that clinically relevant strategies can be developed toward their prevention or resolution. A major component of his research involves developing new technologies to assess spinal structure and function, then using those technologies to evaluate various clinical interventions. 

University of Alberta / Dr. Daniel Laforest is Associate Professor of Cultural Studies and French at the University of Alberta in Edmonton. His work, funded by the Social Sciences and Humanities Research Council of Canada (SSHRC), focuses on the role of literature in Medical Humanities. He examines how medical visualization at the microscopic level of cells and tissues and at the macroscopic level of health data mining and distribution impacts individual life-stories and the sense of self. He has published extensively in Canada, France, the US, and the UK, and has done artist residencies at the Banff Centre Leighton Colony. He was Visiting Scholar at the Stanford University Center for Biomedical Ethics, and is alumnus of the Fulbright Foundation.

Research Assistants

Art & Design / Chelsey Campbell (she/they) is a disabled white settler-Canadian artist and graduate researcher at the University of Alberta in Amiskwacîwâskahikan. Working through the lens of critical disability theory and care ethics, Campbell’s research practice explores the tenuous nature of their queer-crip body in social and medical frameworks fixated on diagnosis and cure. Through a combination of printmaking, 3D modelling, installation, and photography, their work seeks to make space for the body in pain, celebrate disabled narratives as complex and whole, and extend radical care for community through artistic practice. 

Computer Science Multimedia/ Kirtan Shah did an 8 month internship with the Know Thyself team (working on My Data Body) as part of his Multimedia MSc. He is currently a teaching assistant at the Concordia University of Edmonton. His primary interest is in the use of Virtual Reality for medical applications.

Computer Science Multimedia/ Dhara Pancholi did an 8 month internship with the Know Thyself team (working on My Data Body) as part her Multimedia MSc. She has a keen interest in virtual reality, as her passion lies in art and design. She wants to enhance her skills in 3D modelling and animation. Supplementing her interests, in near future she hopes to be a 3D animator and join the animation film industry. She believes that, combination of art and technology is really powerful and can develop most innovative notions.

Engineering / Walter Ostrander is a 3rd year Computer Engineering CO-OP Student from Jasper, Alberta with a love for the outdoors. Walter is currently working on the Know Thyself as a Virtual Reality project as a software developer.


Know Thyself as a Virtual Reality Exhibition

February 2022, FAB Gallery, University of Alberta, Edmonton.

The exhibition will showcase the two VR artworks made by the KTVR Team (My Data Body and Your Data Body), as well as new VR artworks by 5 artists who have been invited to work with the KTVR team to create their own works with sensitive data in VR. The artists include aAron Munson, Nicholas Hertz, Lisa Mayes, Dana dal Bo and Chelsey Campbell.

Know Thyself as a Virtual Reality Ethics e-Symposium / May 17, 2021

9:00AM – 4:30PM MST, hosted on Zoom

In preparation for creating our ethical guidelines for working creatively with medical scan data in VR, we hosted a full-day e-Symposium in order to trouble some of the ethical issues surrounding health data ownership and privacy.