Lesson 08 · Book chapter pointer
Vision
Read the chapter source: vision.tex — do not treat this note as the book.
Learning goals
- Treat an image as a sampled 2D signal, not as a picture you “just look at.”
- Write the pinhole projection and say what a pixel ray is.
- Explain convolution as a local weighted sum and use it for smoothing and edges.
- Separate photometric facts from geometric facts.
- Know why vision is late, wide, and lying — and still worth it.
Teaching note
Vision is the sensor students think they already understand because they have eyes. An image is a grid of numbers produced by optics, exposure, and a Bayer story you may choose to ignore. Each pixel is a measurement along a ray, not a 3D point. Until you have depth, stereo, or a planar assumption, you do not know how far the ray went. That one sentence prevents a semester of “why is my cube the wrong size.”
The pinhole model is the workhorse: a 3D point maps to \(u = f X/Z\) (plus a principal point and maybe distortion). Intrinsics live in the camera; extrinsics live in the world. Calibrate once, then stop treating focal length as a vibe. If the class has no calibration rig, print a chessboard and spend an hour. It pays for every later feature and SLAM lecture.
Convolution is how you talk about neighborhood computation without waving at neural nets yet. A box or Gaussian kernel smooths. A derivative kernel finds edges. Students should compute a 3×3 by hand on a tiny patch, then watch a Sobel image light up a doorway. Frequency language can wait; the local-sum picture is enough to make “filter” mean something other than Instagram.
Photometry (brightness, color, exposure) and geometry (rays, pose, 3D) get mixed constantly. A shadow is not a new object. A specular highlight is not a feature you want to match next week. Teach one demos where you change only the lights and watch a brittle detector fail. That failure is the door to invariant features and, later, learned descriptors.
Vision is late (a frame is milliseconds you may not have), wide (megabytes), and lying (aliasing, motion blur, rolling shutter). Still: it is the cheapest way to get a thousand range-adjacent measurements if you are willing to compute. Pair this lesson with the next two. Then read the book chapter — images as signals, and the algorithms this note is only allowed to name.
Pinhole: \(\begin{bmatrix}u\\v\end{bmatrix} = \frac{f}{Z}\begin{bmatrix}X\\Y\end{bmatrix} + \begin{bmatrix}c_x\\c_y\end{bmatrix}\). Depth \(Z\) is the unmeasured coordinate that makes vision hard.