CMOS Imager with Embedded Analog Early Image Processor
Christophe Basset, Bedabrata Pain (JPL), Pietro Perona

Abstract. We are developing a computational CMOS imager with integrated early image processing general-purpose filter. The goal of this collaborative work with the Jet Propulsion Laboratory is to produce a single chip serving as a camera able to pre-process the image in real-time through a filter chosen by the user, allowing an efficient implementation of a variety of computationally intensive applications such as autonomous navigation, object avoidance or intercept, real-time target tracking and recognition. (full report)

Spike Based Saliency Detection
Ulrik Beierholm, Pietro Perona

Abstract. Trying to quickly ascertain which parts of a visual scene is most relevant for a recognition task and then focusing on each of these areas, is an economical use of processing power known to be employed in the human visual system. Most models for saliency detection however are too slow to explain the performance of the biological system. We are currently working on implementing a fast neuronal spike based saliency detector model based on rank order coding. (full report)

Fly Flight Simulator to Study Visual and Rotational Stimuli
John Bender, Michael Dickinson, Pietro Perona

The fly flight arena was designed (not by me!) to explore the connections between the different sensory modalities that fruit flies use to control their flight. The fly is glued to a metal post mounted in the center of a cylindrical arena. The walls of this cylinder are made out of 11,340 LEDs which are controlled in real time by a computer. (Flies have poor spatial resolution, estimated at 5°, but very fast temporal resolution - around 200 Hz. Human vision has spatial resolution of about 1/30th degree and temporal resolution around 20 Hz.) (full report)

Decomposition of Human Motion into Dynamics Based
Primitives with Application to Drawing Tasks
Domatilla Del Vecchio, Richard Murray, Pietro Perona

Abstract. Using tools from dynamical systems and systems identification we develop a framework for the study of primitives for human motion, which we refer to as movemes. The objective is understanding human motion by decomposing it into a sequence of elementary building blocks that belong to a known alphabet of dynamical systems. We develop a segmentation and classification algorithm in order to reduce a complex activity into the sequence of movemes that have generated it. We test our ideas on data sampled from five human subjects who were drawing figures using a computer mouse. Our experiments show that we are able to distinguish between movemes and recognize them even when they take place in activities containing an unspecified number of movemes. (full report)

Dynamic Recurrent Neural Networks for Pattern Recognition
Alex Holub, Gilles Laurent, Pietro Perona

We are investigating the computational properties of recurrent neural networks of binary artificial neurons. Our investigations are guided by recent work performed in the laboratory of Gilles Laurent which involves elucidating the underlying processing mechanisms in early olfactory processing. These physiological investigations indicate that the initial olfactory processing layer (in the locust the antennal lobe) consists of a dynamic recurrent neural network of excitatory and inhibitory units. The presentation of stimuli to the network results in stereotyped spatio-temporal neural firing patterns, with each unique stimulus presentation invoking a unique temporally-varying pattern of activity within the population of neurons. We have approximated the biological networks using recurrent networks with discrete binary neural elements. These non-linear networks exhibit chaotic behavior such that similar input patterns obtain very dissimilar network representations through the network dynamics. Similar pattern spreading characteristics have been observed in the initial processing networks of fish by members of the Laurent laboratory and it has been hypothesized that pattern spreading may be one computational benefit which the initial processing layer provides. (full report)

Human Motion Detection and Classification
Claudio Fanti, Pietro Perona

Abstract. We foresee a future in which machines autonomously interact with Humans in the surrounding environment. So far, very good results have been achieved in detecting the presence of Humans and labeling their body parts by means of graphical-models based algorithms. We unavoidably have to deal with uncertainty and reasoning in absence of complete information. To that extent, we explore and enhance the state of the art in probabilistic inference and sampling techniques having the machines understanding human actions as a primary application. (full report)

Rapid Natural Scene Categorization without Attention
Fei Fei Li, Rufin VanRullen, Christof Koch, Pietro Perona

Abstract. What can we see when we do not pay attention? While attention is not necessary for some detection tasks on simple synthetic stimuli, without it we are “blind” even to major aspects of a natural complex scene. It would thus appear that only visual tasks that have an explanation in the early stages of the visual system may be carried out without attention. We report on a complex visual task that requires no attention. Our subjects can rapidly detect animals in briefly presented natural scenes while simultaneously performing another visual task that demands full attention. By comparison, they are unable to discriminate large ‘T’s from ‘L’s in the same conditions. We conclude that attention may not be necessary for some visual tasks that are associated with ‘high level’ cortical areas. (full report)

Object Categorization: Unsupervised One-Shot Learning
Fei-Fei Li, Rob Fergus, Pietro Perona

Abstract. Learning visual models of object categories notoriously requires thousands of training examples; this is due to the diversity and richness of object appearance which requires models containing hundreds of parameters. We present a method for learning object categories from just a few images (1 - 5). It is based on incorporating "generic'' knowledge which may be obtained from previously learnt models of unrelated categories. We operate in a variational Bayesian framework: object categories are represented by probabilistic models, and "prior'' knowledge is represented as a probability density function on the parameters of these models. The "posterior'' model for an object category is obtained by updating the prior in the light of one or more observations. Our ideas are demonstrated on four diverse categories (human faces, airplanes, motorcycles, spotted cats). Initially three categories are learnt from hundreds of training examples, and a "prior'' is estimated from these. Then the model of the fourth category is learnt from 1 to 5 training examples, and is used for detecting new exemplars a set of test images. (full report)

Object Recognition by Probabilistic Hypothesis Construction
Pierre Moreels, Michael Maire, Pietro Perona

Abstract. We present a probabilistic framework for recognizing objects in images of cluttered scenes. Hundreds of objects may be considered and searched in parallel. Each object is learned from a single training image and is modeled by the visual appearance of a set of features, as well as their position with respect to a common reference frame. The recognition process computes both the identity and position of objects in the scene by computing the best interpretation (or hypothesis) of the scene in the light of a database of known objects. A hypothesis pairs features in an input image either with features in the database or marks them as clutters. Each hypothesis may be scored in a principled way using a generative model of the image which is defined using the learned objects as well as a model for clutter. While the space of all possible hypotheses is enormously large, one may find the best hypothesis efficiently—we explore a couple of heuristics to do so. In our initial experiments our algorithm compares favorably with state-of-the-art recognition systems. (full report)

Monotonic Bernoulli Trials
Amrit Pratap, Yaser Abu-Mostafa, Pietro Perona

Abstract. When estimating a number of bernoulli variables which have a certain monotonicity constraint, if the number of samples for each variable is small, then the estimates will not satisfy the monotonicity constraint. Better performance is achieved by endorcing the monotonicity constraint on the estimation procedure. (full report)

Attentional Selection for Learning and Recognition of Objects in Cluttered Scenes
Ueli Rutishauser, Dirk Walther, Christof Koch, and Pietro Perona

The problem of serial processing of highly complex visual stimuli containing multiple objects is not only faced by humans and other primates, but also by machine vision systems. Advanced object recognition algorithms are capable of achieving very good recognition performance with objects learned from a single image (one-shot learning). These algorithms perform well as long as they are trained on images in which a major part of the image is occupied by the object to be learned and recognized. As soon as major parts of an image are occupied by clutter it becomes impossible to learn from such images without manual pre-labeling. These approaches are thus not suitable in an unsupervised environment, as they would mainly learn background clutter instead of the actual objects. (full report)

Perception of Mirror Surfaces
Silvio Savarese, Fei Fei Li, Pietro Perona

Abstract. The aim of our work is to investigate how the human visual system perceives specular surfaces and which cues can be used to recover the shape of such class of objects. Our experiments show that mirror reflections are a weak cue for most human observers when additional information is not available. (full report)

3D Reconstruction of Specular Surfaces
Silvio Savarese, Min Chen and Pietro Perona

Abstract. Specular reflections carry valuable information on surface shapes. A curved mirror surface produces "distorted" images of the surrounding world. For example a straight line reflected by a curved mirror is in general a curve. It is clear that such distortions are systematically related to the shape of the surface. Our goal is to explore the geometry linking the shape of a curved mirror surface to the distortions produced on a scene it reflects. To this effect, we assume a simple known (calibrated) scene composed of lines passing through a point. We demonstrate that local shape geometry of the surface may be recovered from local deformation of the reflected images of at least three intersecting lines. (full report)