| Content |
|---|
| Playing 20 questions against Nature. |
| Review of calculus: functions, directional deritives, gradients, vector fields, gradient ascent.
Vignette #1: Euglena and E-coli. Run+Twiddle behavior of E-coli. Tangent Plane is the best "linear" approximation to surface near a point. Gradient points in the direction of maximum increase. Gradient Ascent (Hill Climing) algorithm. The Tangent and Normal (T, N) local frame attached to curves. Twiddle frees E-coli from local maxima. |
| Review of neuroscience.
Vignette #2: Barnacle. Anatomy of Barnacle's nervous system. From neural circuitry to elementary visual behavior. Shadow reflex: if dI/dt>threshold, then withdraw. Shadow reflex helps Barnacle avoid being food. Handout: Neuroscience Overview - Chapter 2 from Sejnowski and Churchland. |
| The Limulus visual behavior and possible roles of the two vi
sual systems.
Introduction of Linear Systems: (1) Additive; (2) Homogeneous. The Principle of Superposition. Unit step function, step response; Dirac Delta function, impulse response . Representation of signals as a linear combination of step functions. Convolution operator, the output of a linear system via convolution. Check out Delta Function -- from Mathworld and Representations of Delta Function. "Premier Courseware of 2001": Demonstrations in Signals, Systems and Control by Wilson J. Rugh, et. al at Johns Hopkins Unive rsity |
| The Fourier transform - Frequency domain representation of c
ontinuous signals.
Sine and Cosine as eigen functions of linear systems. Fourier basis are the eigen functions of linear systems. Modulation Transfer Function (MTF) as a characterization of linear system s in the frequency domain. Fourier transformation of convolution operation on functions. Low pass filter, band pass fileter, and high pass filter. Linear systems being represented in frequency domain. Sampling theory in a nutshell. |
| Nonlinearities in the human visual system.
The human visual system is band pass. Vignette #4: Frog. "What Does the Frog's Eye Tell the Frog's Brain". Image Blurring vs Image Sharpening. The primate retina: rods, cones, fovea, the optical nerve, the blind spot. |
| Introduction to the primate visual physiology.
Retinal receptive fields : On-center and Off-center. Visual Areas V1, six cortical layers. Lateral Geniculate Nucleus (LGN, dLGN). Receptive field types: simple cells, complex cells, hypercomplex cells. Functional architecture of V1 ("Ice Cube Model"): orientation columns, ocular dominance bands, hypercolumns, processes running within a column. Models for composing simple features into more complex ones. |
| Hierarchies of feature detectors.
First Paradigm Computer Vision. Introduction to computer vision: histograms, differential operators in edge detection, composition of smoothing and differential operators (regularized edge detectors) followed by histogram peak selection. Handout: Functional architecture of macaque monkey visual cortex" by D.H. Hubel and T.N. Wiesel. |
| Second Paradigm Computer Vision: Running backwards the physics of image formation.
Physics Based Vision. Models for image formation, relation between image irradiance and scene radiance. The BRDF (Bidirectional Reflectance Distribution Function). Lambertian and specular surfaces. Raise the Shape from Shading problem. Handout: Chapter 8 Edges & Edge Finding -- from "Robot Vision", by Berthold K. P. Horn. The hough transform. |
| Introduction to Projective Geometry: homogeneous coordinates, projective transformation.
Pinhole Camera Model: image coordinate system, camera coordinate system and world coordinate system. Projection matrix, intrinsic/internal parameters, extrinsic/external parameters. |
| The Reflectance Map and the Image Irradiance Equation.
The foundation of Shape From Shading and its corresponding PDE. |
| Other inverse problems: Strucure from Motion, optical flow constraint equation, Stereoscopic Vision.
The Laplace equation and the heat equation, equilibrium distribution. Image deblurring as solving the heat equation backwards in time. Handout: Chapter 11 Reflectance Map: Shape from Shading -- from "Robot Vision", by Berthold K. P. Horn. |
| Wrap up of Second Paradigm Computer Vision.
Inverse problems visited: Shape-from-Shading, Structure-from-Motion, and Stereo. Functions and connections of different areas: V1, V2, V4. "What" pathway and "Where" pathway. |
| Local to global transition via simple parametrized models - The hough transform.
Local to global transition for general shapes as equilibrium problem. Equilibrium configurations. Solving Laplace's equation: the notions of harmonic functions, the minimal surfaces, and the Maximum Principle. Southwill relaxation, and its relation to lateral inhibition. Handout: On the Foundations of Relaxation Labeling Processes, by Robert A. Hummel and Steven W. Zucker. |
| Spring Break. |
| Discrete Constraint Satisfaction Problem.
Discrete Relaxation Labeling. Relaxation Labeling: nodes, labels, compatibilities, Label Discarding Rule, Average Local Consistency. |
| Relaxation Labeling continued.
Handout: On the Foundations of Relaxation Labeling Processes, by Robert A. Hummel and Steven W. Zucker. |
| Term Projects.
Constraint Satisfaction Problem. |
| Introduction to differential geometry:
The notions of curve, tangent, curvature and tortion, The Frenet frame and the Frenet equations.
Handout: Chapter 1 CALCULUS ON EUCLIDEAN SPACE, Chapter 11 FRAME FIELDS - from "Elementary Differential Geometry", by Barrett O'Neil. |
| Canny edge detector and its weakness. Regular curves, piecewise regular curves.
Discontinuities in image: Fold, Cusp. The principle of building reliable non-linear operators via logical combination of simple linear filters: the Logical/Linear Operators. |
| Endstopping. The connection between V1, and the differential geometry of curves.
Introduction to cocicularity, and how to put it in a relaxation labeling network. |
| Brief discussion of stereo.
Introduction to Stereo Vision: 3D point reconstruction. Correspondence pr oblem. Intensity-based area correlation technique (Frontol parallel planes assumption in local neigh borhood) and its weakness. Expand cocircularity to 3D space, connecting the differential structures within images with 3D curves. |
| Texture Flow: tangential curvature and normal curvature, covariant derivative, 1-form, the minimal surfaces by Euler-Lagrance equation: plane, left helicoid, right helicoid.
Right helicoid has constant ratio of tangential curvature and normal curvature. Choose right helicoid as the compatibility structure. |
| Wrapping up - predictions for V1 and extensions to texture f low, shading and stereo. |
| Color. Cytochrome oxidize blobs, V4(color). The comparison between geometric aspect of early vision and color/contrast +other scalar measurements. Experiments on Apparent Motion and Texture.
Bridging information processing, computational modeling, physiology, neur obiology...... |