Virtual Reality for Training and Education: A Study of Memory and Trauma in Parkinson’s Disease as an Alternative to Cognitive Physiology and Neurobiology

It’s possible that VR technology could one day replace, or at least improve, conventional methods for studying memory and trauma, he says. People used to be asked to memorize a word list in the lab, but it was obvious that it couldn’t be tested like a memory test. Through VR, we can recreate scenic 3D environments and then test memory alongside additional data” — such as tracking eye movements, heart rate and other vital signs.

He has also been experimenting with VR to study out-of-body experiences — a phenomenon estimated to be experienced by 10% of the global population, the reasons for which are not fully understood3.

Alongside VR’s potential for training and education, many scientists who use the technology hope that it will one day make their practical work more efficient.

These days, he uses the VR environment mostly to train students, who can practise running experiments or complete assignments that are accessible either through a laptop or through VR headsets at home and on campus. Three life-sized, cartoonishly human-like figures that respond when users speak to them can offer support. Each assistant has a dedicated role and backstory, and draws on a programmed knowledge base as well as on the artificial-intelligence large language model ChatGPT to guide users around the virtual space and answer questions about safety, inventory and stock. Private customers, which include companies and research institutes, must pay a fee, bringing in cash for future lab projects. But Hilton’s team also uses the VR environment for educational outreach, hosting visits from secondary-school groups and collaborators from many countries in the global south.

“We have around ten people in the lab with a mix of backgrounds — biomedical, pharmacy and chemistry degrees, but no computing — and we train everyone from the ground up. First was the design of real equipment, followed by the programming of Artificial intelligence and finally to the virtual world. Everyone in the lab runs VR sessions with users so they are used to it. Not all labs have such time and resources.

Virtual Reality: How to Use Virtual Reality to Enhance Science Communication and Public Engagement in Scientists, with a Master’s Student from India

Many regions don’t have a fast Internet connection. In 2021, there were still 2.9 billion people without Internet, according to the United Nations. In South Africa, for instance, even major cities are not always connected to the internet due to energy blackouts and Internet access is not provided in rural areas. Ensuring all of the programs he helps to develop are designed to work with mobile-phone data and don’t rely on broadband Internet is something De Beers wants to make certain all of the programs are designed to do. But even this can be problematic, he says. “We often need to plan ahead to take in our own connectivity sources,” he says, for instance using satellites — although access to aerospace firm Space X’s low-latency satellite internet, called Starlink, has made this easier.

After finishing my PhD at the KEMRI in 2015, I heard about an open call for funding from the AAS for innovative projects designed to encourage science communication and public engagement in researchers. I thought it was a good idea to use Virtual Reality.

My PhD was funded by the DELTAS Africa programme and as part of that, in 2018, I had the opportunity to spend six months at the University of Helsinki. I talked to a master’s student from India at an international science open day. I was introduced to the idea of using new technology in her work as a science teacher by the conversation she had with me.

“VR is essentially a very smart mirror,” he says. When we decide to brush our teeth, it is a reflection of who we are as a person, compared to what we see in the mirror. But once you have an avatar, you can play around with lots of different perspectives. It allows us to ask new questions about the visual self and self-consciousness.”

Robotic chemist Lee Cronin: Virtual reality for drugs discovery in a medicine lab and on the road towards a safer future

According to Cronin, his Glasgow-based spin-out company, Chemify, has raised more than £33 million from international investors who have backed his plan to automate the design of new molecules to speed up drug development.

He believes that using virtual reality in this way could have a positive impact on other scientific simulations, including bomb-disposal research. Productivity benefits can be found, too, and they are that we need to invent more molecules if we are going to solve big societal problems. But chemical space is really hard to research,” he explains. We tried a billion of the small molecule drugs already and only 2,800 were approved for use in the clinic. Cures for cancers are out there, but we don’t physically have the capacity to go through each chemical reaction one by one ourselves.”

The addition of the VR technology allows Cronin to move components and give instructions to the robots, wherever he is. “It also allows us to make shortcuts,” he explains. The virtual conical flask can be held in one hand. If I make a pouring motion, the robot knows to interpret that as moving liquid from hand a to hand b, and so it can turn on a pump and a valve. It isn’t fully deployed yet, but the operating system is there. It is similar to the way a flight simulation works.

In 2020, chemist Lee Cronin and his colleagues at the University of Glasgow, UK, designed a VR programme that allows him to control the robots in his lab remotely. Cronin and his research group use what he calls a “chemputer” — a series of robots that can be programmed to synthesize organic molecules automatically — as part of their quest to discover new molecules and generate artificial life forms. The chemical experiments will be safer since the need for human handling of chemicals is eliminated with this automation.

The laboratory is hands-on and visitors can try their hand at it. No staff passes, qualifications or lab coats are required here; guests are welcome to wander around and prod at the equipment, even spill chemicals on the floor if they fancy it, without repercussions. In fact, he encourages it. Science should be fun. It’s about being able to make a mess and make mistakes, according to the man handing over a 3D shape.

Hilton uses the Oculus Quest 2 headset (now sold as Meta Quest 2) because it is relatively low in cost (around £300, or US$380) and allows him to have multiple headsets on the go at one time. The software runs on the PC, allowing interaction between PC and headset combinations. His virtual lab was built using the 3D design software Unreal Engine from US developer Epic Games, and runs on a local server for single users, switching to a cloud-based one for multiplayer and voice-connectivity programs.

Using XR to teach me how to speak English: How I discovered ten languages in South Africa and how I am going to implement it for my virtual assistant

He says that they have created a tool for the virtual assistants to speak ten foreign languages. “The knowledge base translates in real time, meaning I could speak directly in English to answer a student’s query, and they’ll hear the answer in their local language. There is a big opportunity for this to happen in many countries in the south.

One of the simplest ways to make virtual training more accessible could be by avoiding spoken languages altogether. “XR opens up the ability to use gestures and body language — showing over telling — without having to know the language very well,” says Koos De Beer, an XR solutions consultant and researcher at the University of Pretoria in South Africa.

There are many challenges to producing Virtual Reality videos in Africa, for example the technology is new and we lack the experience to make them. Even though we can make the videos, and even though we have the technology to do them, I want to use virtual-reality to allow students to go into the lab and do the experiments themselves, but it requires more investment than we already have. I hope to bring more interest and collaboration to the people who are using these tools.

De Beer says that the virtual being doesn’t have to speak a certain way. They can simply be shown as robots, for instance, without the need for race or gender — although human biases present challenges when designing the figures. (For now, Hilton says, his program uses robot avatars that have masculine, muscular torsos, for example.)

Virtual Reality as a Tool for Sepsis Recognition: A Case Study of a Research Collaboration Across Medicine, Science, and Science in Africa

The Pandemic made in-person training difficult, but medical dummies are considered the gold standard for training because of how to diagnose and treat sepsis. “It is also very expensive and resource-rich when you take into consideration the cost of the dummies — around £40,000 — the trainer’s time and the fact it is confined to specialist centres. And if you happen to miss a day’s training because you’re sick, it’s tough luck,” she adds.

Sharma was inspired to change this in 2019 after witnessing pregnant people become dangerously ill from sepsis during labour. While a research fellow at Cardiff University, she helped to develop a VR education tool for sepsis recognition as part of Project Sepsis, a UK research collaboration across medicine and science. The project demonstrates the differences between Sepsis symptoms and similar conditions through simulations. Unlike Hilton’s avatar-populated virtual lab, Sharma opted to film actors playing out realistic medical scenes, which students can watch and interact with using VR headset and handsets. The students are tested for knowledge gained during the experience with multiple-choice tests.

I would like to help as many people as I can in Africa understand what a researcher does. I hope that through showing and not just telling, virtual reality will inspire more young people to consider careers in science, technology, engineering and mathematics. And, just as importantly, I hope that it will help them and their families to appreciate the role of science in society. Science is largely funded by the public, but the public doesn’t necessarily know what we do.

Growing up in a small village in Kenya, working in a research lab was a distant dream. Unfortunately, this is still the case for many children in my country, especially for those from remote regions with few classroom resources.

VR Video Production for the KEMRI Research Programme: Biobank, Immunology, Proteomics, Genetics, and Cleaning and Reagent Preparation

The 2022, Eric and Wendy Schwartz Award for excellence in science communications was hosted by the US National Academies of Science, Engineering and Medicine, which is what one of the first films I produced was.

I consider myself a scientist by the day and as a science communication at night. In 2019, I was one of 13 African scientists to be awarded the AAS Public Engagement Fund, and since then I have been funded by the US magazine National Geographic to produce more VR videos, including a programme showcasing the impacts of the pandemic to encourage vaccine uptake.

The videos can be uploaded to the internet and shown to students without the need for a fast internet connection.

I worked with a production company to shoot six of the KEMRI lab areas: biobank, entomology, immunology, proteomics, genetics and the cleaning and reagent-preparation section. We showed our research to the various researchers working at their stations.

The video was supposed to be shot in the Wellcome Trust research programme labs. We invited secondary-school students to a guided tour of the labs as part of the script-writing process in order to see what they understood and what they were learning in school.

The challenges in science, technology, engineering and mathematics (STEM) are similar to those faced by many African countries. And much more needs to be done to encourage uptake of these subjects beyond secondary school.