Video Games controlled by thoughts

By:  Bruno Martín. En: OpenMind BBVA – Marzo de 2020

It is already possible to mentally control the avatar in World of Warcraft.

Crédit dronepicr

You are a child with the power of telekinesis. Using only your mind, you must manipulate various objects to escape the laboratory where you are held captive. This is the premise for Awakening, a virtual reality experience developed by american neurotech startup Neurable. In order to play, a headband must be worn with six electrodes that register the brain’s activity — they allow the user to control the first-person video game via thoughts, emulating the protagonists’ super powers. Brain wave sensors of the sort used to play Awakening were developed for research and medical applications, but they have already demonstrated their potential in video game control.

In general, neurotechnology aims to connect the mind with machines via brain-computer interfaces (BCIs), which can be as complex and delicate as the neural implants used by some amputees to control bionic prosthetics. In video games developed for healthy users, non-invasive electroencephalography (EEG) devices are the technology of choice. These instruments use electrodes on the scalp to measure voltage fluctuations produced by neurons firing beneath the skull.

After a calibration phase in which the program familiarizes itself with the users’ brain, the BCI can link specific brain waves — identified by their frequency and location — with predetermined actions. Usually, the software is coded to associate specific thoughts with movement intent: if the user concentrates on moving forward, the program executes an advance, if the user concentrates on turning right, the program executes a right turn. In this way, mind-control has been used to steer wheelchairs, toy vehicles and even telepresence robots before finally making it to video games.

THE POSSIBILITIES OF BCI TECHNOLOGY

With an EEG interface and some tweaks to the program, it’s already possible to mentally control the avatar in World of Warcraft, a popular online role-playing game.

In Awakening, the experience developed by Neurable, the available mental controls are even more varied, as there is no option to use a gamepad, keyboard or mouse. First, the player must focus his or her attention to levitate one of the objects strewn across the room: a wooden cube, a toy plane, a ball… When the ball is mentally hurled against a mirror, it shatters to reveal several numbers scrawled upon the wall. Using only thoughts, the protagonist types this code into a numeric keypad in order to open a door leading out of the lab. According to Adam Molnar, Neurable cofounder, “Awakening is not so much a game as it is an application to help understand what can be made possible with powerful BCI technology.”

Normally, data collected by EEG is noisy — it lacks sensitivity — because the electrodes must pick up neural signals through several layers of bone, skin and hair. In research laboratories, conductive gels are used with wet electrodes to improve the signal resolution.

However, wet electrodes aren’t an option for hardware companies trying to sell sleek EEG headsets to casual consumers for gaming and wellness applications. Neurable has developed proprietary algorithms that increase the signal-to-noise ratio in data collected by EEG sensors — that’s how they have addressed the technology’s low sensitivity.

Using this type of device, third-party developers have a chance to incorporate mind control in their game design, or to forgo traditional controls altogether. For example, a group of students from the University of Alberta, Canada, used an EEG headset from a different manufacturer, NeuroTechX, to create a 2D shooter game operated solely by mind control. Already, dozens of simple video games have been developed incorporating complete or partial EEG-controls, but there aren’t any fully-featured, commercially successful video games outside of the medical sphere.

A GAME TO TREAT ADHD

A study from the University of Masaryk, in Chech Republic, found BCIs demand high concentration from the user and can cause fatigue, so they are not yet suitable for long gaming sessions. The authors also claim that “the research community is primarily concerned with proofs of concept rather than innovative and functional design” of video games. Besides all this, Molnar points out that current high-quality EEG systems are priced in the range of several thousand dollars and are not manufactured at scale, as they are made primarily for clinical and research laboratories.

He thinks that “it’s still early days in terms of ideation, since most BCI systems are either too expensive to justify mass market video game production or their capabilities are so limited that it does not justify their incorporation”. Having said that, the evolution of entertainment neurotechnology is closely linked to the rapid medical and scientific progress of the field. Given that video games demand an intrinsic motivation from the user to progress, they are ideal to encourage compliance with BCI-based treatments. According to Will Rosellini, CEO of neurotechnology company Nexeon, this phenomenon is leading to the “gamification of rehab”, as he told Factor magazine.

A clear example is that of BrainCo, a Harvard startup. They have developed a video game, FocusOasis, to treat attention deficit hyperactivity disorder (ADHD) in children and teens using neurofeedback technology. In the game, users are transported to a universe where concentration — as measured by an EEG sensor — is rewarded with visible and pleasing changes in the environment. The idea is to motivate children to improve without turning their therapy into a chore.

This synergy between the video game industry and medical neurotechnology is pushing great changes in the development of BCIs. While hardware manufacturers aim to make sleek, efficient and cheap headsets for laboratories and lay users alike, neuroscientists are learning how to make the most of this EEG technology. There is likely a limit, imposed by physical constraints, to the information that can be gleaned from brain waves without penetrating the skull, but nevertheless, experts do believe that mind control can feasibly become a staple of video game functionality.

https://www.bbvaopenmind.com/en/technology/innovation/video-games-controlled-by-thoughts/