Reaching for rewards: how spatial targets are learned
An interview with Dr Alice Mosberger, Columbia University, conducted by April Cashin-Garbutt
From reaching to pick up a cup of coffee to performing the latest TikTok dance move, we know the human brain is capable of refining movements into precise skills. To achieve this the brain is thought to learn which movements led to success, such as hitting a tennis ball just right, by assigning credit to the activity in the brain that generated that movement. But how do neural networks in the brain assign credit to such movements?
In our first Emerging Neuroscientists Seminar Series (ENSS) talk of 2023/24, Dr Alice Mosberger, a postdoctoral researcher at Columbia University, described a behavioural task she developed to dissect the sensorimotor areas that control different aspects of skilled movements. In this Q&A, she shares her ENSS experience at SWC, what drives her research and her hopes for future clinical applications.
What prompted you to apply for the SWC ENSS?
I heard from a colleague that the Sainsbury Wellcome Centre was running the Emerging Neuroscientists Seminar Series and I thought it sounded like a great opportunity. I had previously met researchers from SWC at conferences around the world and I was always very impressed by their work.
From speaking with students, postdocs, and faculty at SWC, there seemed to be a lot of interaction between trainees and faculty and really good training programs, so I was eager to visit and was very excited when I was selected to give an ENSS seminar.
How useful did you find the experience?
Very useful! As postdocs, you often only give short presentations at conferences, but ENSS gave me the opportunity to give a full-length seminar. I found this very valuable for training purposes and I enjoyed having more time to talk about my research at length.
Interacting with faculty at SWC was extremely fun. I found it very interesting to see how people from different fields think about similar concepts but from different angles. Every time I speak to someone a little outside my field, it enriches my research and helps me think about new ideas.
I really enjoyed the scientific interaction both with the faculty and also the trainees at SWC. It was exciting to watch the first-year student presentations -- they are a very impressive cohort of graduate students!
What advice would you give to those considering applying to ENSS 2024/25?
My main advice would be to talk to people who have previously applied. There is an interesting section in the application that asks about your future research plans and where you see the field going in the next 20 years. That question is quite different from other postdoc seminar series that I have seen, and I enjoyed thinking about the future of neuroscience, and my research.
What first sparked your interest in sensorimotor circuits?
I had a biology teacher at high school that taught us about the brain and encouraged me to read Oliver Sacks’ work. I found it mind blowing how well he described the way the brain misfunctioning can make you experience the world very differently. There was a section about hemineglect that particularly interested me and that’s when I decided to study neuroscience.
Later, I became very interested in how the brain controls our movements, and so I did my PhD in spinal cord injury. I found that extremely motivating because I had a lot of exchanges with clinicians and patients who I learned a lot from. One topic I became more interested in was proprioception. I got to meet Ian Waterman, a famous patient who had an infection when he was around 19 years old that led to the degeneration of his sensory nerves, particularly proprioception and touch below the neck. As a result, he could initially not move at all and had to re-learn moving his body by relying on other senses. That really exemplifies how movement and sensation are so closely linked together.
The motor system continuously integrates sensory feedback and that is very important for movement control. Ian relearned how to move by teaching himself how to control his muscles consciously and that really opened up my thinking about how motor control involves different strategies, some conscious, others subconscious. How that is implemented in the brain is still very much a mystery.
Movement is one of these things that we often don’t think about in our daily lives, but it is crucial. We only realise how much is done by the brain automatically when that ability gets lost and we can’t do things, then it becomes very apparent how important movement is for our daily lives.
What is the goal of your research?
My current research studies how we learn new movements. The goal is to understand how different parts of the brain are relevant for learning different aspects of a specific skill. I am also studying how the way the brain explores different movements can bias what it learns about an action, and lead to different strategies being used.
I have developed a new behavioural task in which mice learn to reach to spatial targets with their forelimbs and found that individual animals learn different reach strategies. Such sophisticated behaviour is still very novel in mice. Because mice have an amazing toolbox that allows us to dissect the circuitry, I can now use this behaviour task to understand how these different strategies are algorithmically and physically implemented in the brain.
Different reach strategies are learned by individual mice related to how they explore during learning.
Prior to your work, how much was known about the way the brain assigns credit to movements that lead to desired outcomes?
We have had very strong models for a long time based around the concept of reinforcement learning. Algorithmically, we also have a lot of models for how credit assignment might work.
There has also been some mapping onto brain structures, in particular how dopamine acts as a reward prediction error signal. More recently, we’ve learned that dopamine’s role in the brain is much more complex and multi-faceted. And previous work has shown that plasticity in the striatum is crucial for learning.
But there are still a lot of open questions about how the brain implements this learning from reinforcement and credit assignment. How does the brain know which part of a multi-dimensional, complex movement led to the desired outcome? One of the ideas is that if different circuits relay different types of information, then that could become a structural solution to the credit assignment problem. Also this idea gives us a very strong testable hypothesis, and we have found some specific roles for certain sensorimotor subcircuits. But there are still a lot of open questions about the details of how new movements are learned.
Two populations of neurons in the motor cortex that relay information to the striatum.
Do we know if there is individual variability? Can different individuals have different movement strategies?
My research shows that on a behavioural basis, different animals seem to be biased towards different strategies depending on how they explore during early training. We think that the variability in their exploratory movements early on gives them different experiences that can be reinforced.
What we don’t know yet is why they explore differently. Further experiments are needed to find the brain structure that generates the variability. This could be in cortex, and we have seen from studies in songbirds that there is a very specific area in the brains of birds that regulates the variability of the song a bird produces. In the rodent and human brain, it is less clear.
There could be innate differences and also different experiences in life, for example hierarches and social standings in rodents that affect their tendencies to explore.
Were you surprised to find the strategy an individual animal learned (direction-based or endpoint-based) correlated with the degree of spatial directional variability during exploration?
I think I was surprised to see that individual animals had learned different strategies, as they were all trained in the same way. But that the different strategies could be explained by differences in exploration and variability early during training confirmed the hypothesis that we had, and it made sense. For example, if an animal had experienced entering the target (and receiving reward) from variable directions, then that made them more biased towards learning an endpoint-based strategy. This makes sense from a credit-assignment perspective.
Do your findings have any clinical implications?
Yes, I definitely hope that will be the case long-term. In my future research, I want to explore whether we can use this behavioural task as a rehabilitation tool. I would like to be able to identify the different brain circuits that are required for different movement strategies and then develop training regimes, and stimulation approaches of these circuits, where we can activate one strategy over another.
If a person has deficits from stroke or peripheral injuries resulting in pain or other sensory issues, I’d like to be able to develop a training regime or therapeutic intervention that can either trigger the damaged circuit and allow regeneration or help stimulate alternate circuits to allow recovery through compensation and reorganisation.
My PhD work was related to this and showed that rehabilitative training and reactivating circuits that have been damaged can allow them to reorganise above the lesion and regain control over movements through alternate pathways through reorganisation of the axon itself.
I want to combine my old research and my current research towards that goal with these new behavioural and circuit dissection tools.
What is the next piece of the puzzle your research is going to focus on?
I am very excited about looking at sensory pathways and how they are integrated to generate these two different movement strategies in mice. I think this work could bridge the gap between the type of human and non-human primate motor research we can do in the field and mouse research, where we have access to the neural circuits and recording tools.
I also want to explore how the brain uses different descending pathways because that also could be very relevant for a translational approach where selective pathways are damaged after spinal cord injury.
About Dr Mosberger
Dr Alice Mosberger’s work dissects the interplay of the sensorimotor circuits that govern the skilled movements of our arms, particularly reaching movements. She has earned her PhD from ETH Zurich in Switzerland for her work in the field of spinal cord injury, under the supervision of Dr. Martin Schwab. Her main PhD work discovered that recovery of forelimb movements after corticospinal lesions is possible because axotomized corticospinal neurons re-route their control via the rubrospinal tract. Her work was supported by the Christopher and Dana Reeve Foundation. For her postdoctoral training, Alice moved to New York to join the group of Dr. Rui Costa at Columbia University where she continued her investigation of the sensorimotor circuitry underlying reaching movements in mice. She developed a novel spatial reaching task with which she discovered that individual mice learn a direction-based or endpoint-based reach strategy and has started to dissect the role of different sensorimotor areas in these target reaches using different neural recording, decoding, and manipulation techniques. Her postdoctoral work was supported by the Swiss National Science Foundation (SNSF), and she has been awarded a K99/R00 career transition award by the NIH BRAIN Initiative.