Pawan Sinha on how brains learn to see

If you are a blind child in India, you will very likely have to contend with at least two big pieces of bad news. The first bad news is that the chances of getting treatment are extremely slim to none, and that's because most of the blindness alleviation programs in the country are focused on adults, and there are very, very few hospitals that are actually equipped to treat children. In fact, if you were to be treated, you might well end up being treated by a person who has no medical credentials as this case from Rajasthan illustrates. This is a three-year-old orphan girl who had cataracts. So, her caretakers took her to the village medicine man, and instead of suggesting to the caretakers that the girl be taken to a hospital, the person decided to burn her abdomen with red-hot iron bars to drive out the demons. The second piece of bad news will be delivered to you by neuroscientists, who will tell you that if you are older than four or five years of age, that even if you have your eye corrected, the chances of your brain learning how to see are very, very slim -- again, slim or none.

So when I heard these two things, it troubled me deeply, both because of personal reasons and scientific reasons. So let me first start with the personal reason. It'll sound corny, but it's sincere. That's my son, Darius. As a new father, I have a qualitatively different sense of just how delicate babies are, what our obligations are towards them and how much love we can feel towards a child. I would move heaven and earth in order to get treatment for Darius, and for me to be told that there might be other Dariuses who are not getting treatment, that's just viscerally wrong. So that's the personal reason.

Scientific reason is that this notion from neuroscience of critical periods -- that if the brain is older than four or five years of age, it loses its ability to learn -- that doesn't sit well with me, because I don't think that idea has been tested adequately. The birth of the idea is from David Hubel and Torsten Wiesel's work, two researchers who were at Harvard, and they got the Nobel Prize in 1981 for their studies of visual physiology, which are remarkably beautiful studies, but I believe some of their work has been extrapolated into the human domain prematurely. So, they did their work with kittens, with different kinds of deprivation regiments, and those studies, which date back to the '60s, are now being applied to human children.

So I felt that I needed to do two things. One: provide care to children who are currently being deprived of treatment. That's the humanitarian mission. And the scientific mission would be to test the limits of visual plasticity. And these two missions, as you can tell, thread together perfectly. One adds to the other; in fact, one would be impossible without the other. So, to implement these twin missions, a few years ago, I launched Project Prakash. Prakash, as many of you know, is the Sanskrit word for light, and the idea is that in bringing light into the lives of children, we also have a chance of shedding light on some of the deepest mysteries of neuroscience. And the logo -- even though it looks extremely Irish, it's actually derived from the Indian symbol of Diya, an earthen lamp. The Prakash, the overall effort has three components: outreach, to identify children in need of care; medical treatment; and in subsequent study. And I want to show you a short video clip that illustrates the first two components of this work.

This is an outreach station conducted at a school for the blind.

(Text: Most of the children are profoundly and permanently blind ...)

Pawan Sinha: So, because this is a school for the blind, many children have permanent conditions. That's a case of microphthalmos, which is malformed eyes, and that's a permanent condition; it cannot be treated. That's an extreme of micropthalmos called enophthalmos. But, every so often, we come across children who show some residual vision, and that is a very good sign that the condition might actually be treatable. So, after that screening, we bring the children to the hospital. That's the hospital we're working with in Delhi, the Schroff Charity Eye Hospital. It has a very well-equipped pediatric ophthalmic center, which was made possible in part by a gift from the Ronald McDonald charity. So, eating burgers actually helps.

(Text: Such examinations allow us to improve eye-health in many children, and ... ... help us find children who can participate in Project Prakash.)

PS: So, as I zoom in to the eyes of this child, you will see the cause of his blindness. The whites that you see in the middle of his pupils are congenital cataracts, so opacities of the lens. In our eyes, the lens is clear, but in this child, the lens has become opaque, and therefore he can't see the world. So, the child is given treatment. You'll see shots of the eye. Here's the eye with the opaque lens, the opaque lens extracted and an acrylic lens inserted. And here's the same child three weeks post-operation, with the right eye open.

(Applause)

Thank you.

So, even from that little clip, you can begin to get the sense that recovery is possible, and we have now provided treatment to over 200 children, and the story repeats itself. After treatment, the child gains significant functionality. In fact, the story holds true even if you have a person who got sight after several years of deprivation. We did a paper a few years ago about this woman that you see on the right, SRD, and she got her sight late in life, and her vision is remarkable at this age. I should add a tragic postscript to this -- she died two years ago in a bus accident. So, hers is just a truly inspiring story -- unknown, but inspiring story. So when we started finding these results, as you might imagine, it created quite a bit of stir in the scientific and the popular press. Here's an article in Nature that profiled this work, and another one in Time. So, we were fairly convinced -- we are convinced -- that recovery is feasible, despite extended visual deprivation.

The next obvious question to ask: What is the process of recovery? So, the way we study that is, let's say we find a child who has light sensitivity. The child is provided treatment, and I want to stress that the treatment is completely unconditional; there is no quid pro quo. We treat many more children then we actually work with. Every child who needs treatment is treated. After treatment, about every week, we run the child on a battery of simple visual tests in order to see how their visual skills are coming on line. And we try to do this for as long as possible. This arc of development gives us unprecedented and extremely valuable information about how the scaffolding of vision gets set up. What might be the causal connections between the early developing skills and the later developing ones?

And we've used this general approach to study many different visual proficiencies, but I want to highlight one particular one, and that is image parsing into objects. So, any image of the kind that you see on the left, be it a real image or a synthetic image, it's made up of little regions that you see in the middle column, regions of different colors, different luminances. The brain has this complex task of putting together, integrating, subsets of these regions into something that's more meaningful, into what we would consider to be objects, as you see on the right. And nobody knows how this integration happens, and that's the question we asked with Project Prakash.

So, here's what happens very soon after the onset of sight. Here's a person who had gained sight just a couple of weeks ago, and you see Ethan Myers, a graduate student from MIT, running the experiment with him. His visual-motor coordination is quite poor, but you get a general sense of what are the regions that he's trying to trace out. If you show him real world images, if you show others like him real world images, they are unable to recognize most of the objects because the world to them is over-fragmented; it's made up of a collage, a patchwork, of regions of different colors and luminances. And that's what's indicated in the green outlines. When you ask them, "Even if you can't name the objects, just point to where the objects are," these are the regions that they point to. So the world is this complex patchwork of regions. Even the shadow on the ball becomes its own object. Interestingly enough, you give them a few months, and this is what happens.

Doctor: How many are these?

Patient: These are two things.

Doctor: What are their shapes?

Patient: Their shapes ... This one is a circle, and this is a square.

PS: A very dramatic transformation has come about. And the question is: What underlies this transformation? It's a profound question, and what's even more amazing is how simple the answer is. The answer lies in motion and that's what I want to show you in the next clip.

Doctor: What shape do you see here?

Patient: I can't make it out.

Doctor: Now?

Patient: Triangle.

Doctor: How many things are these? Now, how many things are these?

Patient: Two.

Doctor: What are these things?

Patient: A square and a circle.

PS: And we see this pattern over and over again. The one thing the visual system needs in order to begin parsing the world is dynamic information. So the inference we are deriving from this, and several such experiments, is that dynamic information processing, or motion processing, serves as the bedrock for building the rest of the complexity of visual processing; it leads to visual integration and eventually to recognition.

This simple idea has far reaching implications. And let me just quickly mention two, one, drawing from the domain of engineering, and one from the clinic. So, from the perspective of engineering, we can ask: Goven that we know that motion is so important for the human visual system, can we use this as a recipe for constructing machine-based vision systems that can learn on their own, that don't need to be programmed by a human programmer? And that's what we're trying to do.

I'm at MIT, at MIT you need to apply whatever basic knowledge you gain. So we are creating Dylan, which is a computational system with an ambitious goal of taking in visual inputs of the same kind that a human child would receive, and autonomously discovering: What are the objects in this visual input? So, don't worry about the internals of Dylan. Here, I'm just going to talk about how we test Dylan. The way we test Dylan is by giving it inputs, as I said, of the same kind that a baby, or a child in Project Prakash would get. But for a long time we couldn't quite figure out: Wow can we get these kinds of video inputs? So, I thought, could we have Darius serve as our babycam carrier, and that way get the inputs that we feed into Dylan? So that's what we did. (Laughter) I had to have long conversations with my wife. (Laughter) In fact, Pam, if you're watching this, please forgive me.

So, we modified the optics of the camera in order to mimic the baby's visual acuity. As some of you might know, babyies are born pretty much legally blind. Their acuity -- our acuity is 20/20; babies' acuity is like 20/800, so they are looking at the world in a very, very blurry fashion. Here's what a baby-cam video looks like.

(Laughter) (Applause)

Thankfully, there isn't any audio to go with this. What's amazing is that working with such highly degraded input, the baby, very quickly, is able to discover meaning in such input. But then two or three days afterward, babies begin to pay attention to their mother's or their father's face. How does that happen? We want Dylan to be able to do that, and using this mantra of motion, Dylan actually can do that. So, given that kind of video input, with just about six or seven minutes worth of video, Dylan can begin to extract patterns that include faces. So, it's an important demonstration of the power of motion.

The clinical implication, it comes from the domain of autism. Visual integration has been associated with autism by several researchers. When we saw that, we asked: Could the impairment in visual integration be the manifestation of something underneath, of dynamic information processing deficiencies in autism? Because, if that hypothesis were to be true, it would have massive repercussions in our understanding of what's causing the many different aspects of the autism phenotype.

What you're going to see are video clips of two children -- one neurotypical, one with autism, playing Pong. So, while the child is playing Pong, we are tracking where they're looking. In red are the eye movement traces. This is the neurotypical child, and what you see is that the child is able to make cues of the dynamic information to predict where the ball is going to go. Even before the ball gets to a place, the child is already looking there. Contrast this with a child with autism playing the same game. Instead of anticipating, the child always follows where the ball has been. The efficiency of the use of dynamic information seems to be significantly compromised in autism. So we are pursuing this line of work and hopefully we'll have more results to report soon.

Looking ahead, if you think of this disk as representing all of the children we've treated so far, this is the magnitude of the problem. The red dots are the children we have not treated. So, there are many, many more children who need to be treated, and in order to expand the scope of the project, we are planning on launching The Prakash Center for Children, which will have a dedicated pediatric hospital, a school for the children we are treating and also a cutting-edge research facility. The Prakash Center will integrate health care, education and research in a way that truly creates the whole to be greater than the sum of the parts.

So, to summarize: Prakash, in its five years of existence, it's had an impact in multiple areas, ranging from basic neuroscience plasticity and learning in the brain, to clinically relevant hypotheses like in autism, the development of autonomous machine vision systems, education of the undergraduate and graduate students, and most importantly in the alleviation of childhood blindness. And for my students and I, it's been just a phenomenal experience because we have gotten to do interesting research, while at the same time helping the many children that we have worked with.

Thank you very much.

(Applause)