Luce Professor of Computer Science Stella Yu: "Computer science has always been stimulated by our desire to understand ourselves by asking 'What is intelligence?'"
A New Vision for Computer Science
BC faculty member sees role for art, visual perception in field
By Stephen Gawlik
Spend five minutes in the company of Luce Professor of Computer Science Stella Yu and you won't believe your eyes.
Yu will show a visitor a photo on her computer screen of a line of soldiers boarding a large passenger jet. Look at it closely, she says.
Next, she'll offer another photo of the exact same thing - or so it may appear.
"See any difference?" Yu asks.
She flips the photos back and forth, again and again, a little quicker each time.
"Can you see any difference in the photos?" she asks again.
After several minutes of this, the frustrated visitor gives up, and Yu reveals the seemingly impossible-to-miss difference in the photos.
"Notice the plane's engine," she says, continuing to flip between the photos.
There, in the dead center of the first photo, is the large aircraft engine, tucked under the plane's wing in the shadows. In the second photo the engine is absent.
"When we look at a scene like this, our eyes tend to be drawn to salient features or familiar objects, and in this case you are often looking for differences in the people and nothing else," says Yu.
"Shadows are neither salient, nor of practical consequence to any cognitive task. They are often ignored during the eye's active exploration," she says.
The lesson is just one of a host of examples Yu uses to explain the psychology behind vision, and how that understanding is crucial to researchers at Boston College and around the world who are trying to grasp new means of teaching computers to understand images.
"Vision is very difficult to understand. And it's also very difficult to teach a computer to understand it as well."
This semester Yu began teaching a new interdisciplinary course called Art and Visual Perception for undergraduates that is pushing the field of computer vision in a new direction. Yu says she came up with the idea for the course while reading an art book and wondering how she could apply what she was seeing in the book to a difficult problem she was facing in her research.
"The study of computer vision has only been around for four decades, but art has been around for thousands of years. There is a lot we can learn from art. Combining this with a study of psychology, we can learn a lot about vision.""
Art and Visual Perception combines a study of neuroscience, psychology, computer science and visual art together to examine how people perceive light, color, motion, shape, material, depth and distance. The course focuses on the contribution of visual perception to the generation and viewing of pictorial art, and the role of artistic rendering to the understanding of inner workings of visual sense.
The course is listed in psychology, fine arts, and computer science departments, and this fall will be team-taught with Assoc. Prof. Michael Mulhern (Fine Arts), who served as a guest lecturer this semester.
In the long tradition of line drawing, which the earliest known humans did on cave walls, certain peculiarities in shapes trigger brain cells to process images in certain ways, she said.
"Artists discovered a long time ago that line drawings are a universally recognized and effective means to convey a shape," said Yu, explaining that an object's edges are defined by lines in this art form.
"Vision neuroscience has concluded that the brain's visual system is extracting those same lines out of edges."
Lines, of course, are only one aspect of art, she notes: Shading, color, depth and other concepts which are basic to artists must be understood by computer vision researchers to develop a means of teaching computers to see.
"It is incredibly difficult," she says. "There is so much to learn."
But understanding the details of an image is only half of the battle for researchers in her field, says Yu. Next comes the challenge of asking a computer to recognize an image it is processing.
For example, she says, "It takes an instant for a person to recognize a face, but how can this be done by a computer given an image that contains that face?"
Yu says the current state-of-art computer vision algorithms answer that question with a number indicating the result of a segment-by-segment image comparison.
"That does not seem to be what the brain is doing," laments Yu. "Computers only understand numbers. The challenge is to render a human-like ability to parse the image and remember objects entirely in terms of numbers."
For Yu, the concept of drawing these new influences into her research speaks to the nature of computer science.
"Many people don't realize that the progress of computer science has always been stimulated by our desire to understand ourselves by asking 'What is intelligence?'" says Yu.
"I am hopeful that a course like this will help people see that the questions we explore in computer science have as much to do with human intelligence as computers." •