Summary: Researchers developed a groundbreaking simulator that offers a glimpse into artificial visual observations, crucial for advancing visual prosthesis research.
This simulator, designed to mimic the potential vision provided by stimulating the brain’s visual cortex with electrodes, is a step toward restoring sight in individuals with severe visual impairments. By simulating how tiny points of light, or ‘phosphenes’, could form images, the team aims to bridge the gap between current technological limitations and the functional vision needed for tasks such as navigation and reading.
Open to researchers worldwide, this tool is pivotal in exploring the future of vision restoration.
Key Facts:
- Global Impact of Blindness: Approximately 40 million people worldwide are blind, with numbers expected to rise, highlighting the urgent need for solutions like visual prostheses.
- Innovative Approach: The simulator mimics vision through phosphenes generated by electrical stimulation of the visual cortex, offering insights into how a prosthesis could enable sight.
- Publicly Accessible Tool: This open-source simulator is available for global research use, encouraging advancements in visual prosthesis through collaborative innovation and AI optimization.
Source: KNAW
In collaboration with their colleagues at the Donders Institute, researchers at the Netherlands Institute for Neuroscience have developed a simulator that enables artificial visual observations for research into the visual prosthesis.
This open source tool is available to researchers and offers those who are interested insight into the future application.
Blindness affects approximately forty million people worldwide and is expected to become increasingly common in the coming years.
Patients with a damaged visual system can be broadly divided into two groups: those in whom the damage is located in front of or in the photoreceptors of the retina; and those in whom the damage is further along in the visual system.
Various retinal prostheses have been developed for the first group of patients in recent years and clinical tests are underway. The problems for the second group are more difficult to tackle.
A potential solution for these patients is to stimulate the cerebral cortex. By implanting electrodes in the brain’s visual cortex and stimulating the surrounding tissue with weak electrical currents, tiny points of light known as ‘phosphenes’ can be generated.
This prosthesis converts camera input into electrical stimulation of the cerebral cortex. In doing so, it bypasses part of the affected visual system and thus allow some form of vision. You could compare it with a matrix sign along the highway, where individual lights form a combined image.
How we can ensure that such an implant can actually be used to navigate the street or read texts remains an important question. Maureen van der Grinten and Antonia Lozano, from Pieter Roelfsema’s group, along with colleagues from the Donder’s Institute, are members of a large European consortium. This consortium is working on a prosthesis that focuses on the visual cerebral cortex.
Maureen van der Grinten emphasizes: “At the moment there is a discrepancy between the amount of electrodes we can implant in people and the functionalities we would like to test. The hardware is simply not far enough yet. To bridge this gap, the process is often imitated through a simulation.”
Simulated Phosphene Vision
“Instead of waiting until blind people have received implants, we’re trying to simulate the situation based on the knowledge we have. We can use that as a basis to see how many points of light people need to find a door for example. We call this ‘simulated phosphene vision’.
“So far this has only been tested with simple shapes: 200 light points that are neatly-orientated, rectangular pixels of equal size on a screen. People can test this with VR glasses, which is very useful, but does not correspond to the actual vision of blind people with a prosthesis.”
“To make our simulation more realistic, we collected a whole load of literature, created and validated models and looked at the extent to which the results correspond to the effects that people reported. It turns out that the dots vary greatly in shape and size depending on the parameters used in the stimulation.
“You can imagine that if you increase the current, the stimulation in the brain will spread further, hit more neurons and therefore provide a larger bright spot. The location of the electrode also determines the size of the dots. By influencing the various parameters, we looked at how this actually changes what people see.”
Publicly Accessible
“The simulator is currently being used for research in Nijmegen, where they are investigating the impact of eye movements. With this article we hope to offer other researchers the opportunity to use our simulation as well.
“We would like to emphasize that the simulator is publicly accessible to everyone, with the flexibility to make adjustments where necessary. It is even possible to optimize the simulation using AI, which can assist you in identifying the necessary stimulation for a specific image.”
“We are now also using the simulator to give people an idea of where this research could go and what to expect when the first treatments are carried out in a few years. Using VR glasses we can simulate the current situation with 100 electrodes, which also highlights how limited vision through a prosthesis is: they may be able to find a door, but won’t have the ability to recognize facial expressions.
“Alternatively, we can show a situation with tens of thousands electrodes and what that will bring us when this technology is developed far enough.”
About this neurotech and visual neuroscience research news
Author: Eline Feenstra
Source: KNAW
Contact: Eline Feenstra – KNAW
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses” by Maureen van der Grinten et al. eLife
Abstract
Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses
Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system.
The artificially induced visual percept (a pattern of localized light flashes, or ‘phosphenes’) has limited resolution, and a great portion of the field’s research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information.
A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines.
An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision.
We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates.
The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated.
Our results demonstrate the simulator’s suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments.
The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.
