Computer Vision Introduction

By the end of this lesson you will know what computer vision is, which problems it solves, and how a photo becomes a grid of numbers that a program can work with. You will also build your first tiny image in Python and display it in the browser.

What is computer vision?

Computer vision is the field of teaching computers to get useful information from images and videos. A person looks at a photo and says "that is a cow on a road". A computer only receives a long list of numbers. Computer vision is the set of methods that turns those numbers into answers.

You already use it every day:

• A FASTag-style toll plaza camera reads a vehicle number plate.
• Your phone camera finds faces and focuses on them.
• A banking app reads the details from a photo of a cheque or a PAN card.
• A farmer photographs a leaf, and an app suggests which disease it may have.
• A factory camera in Pune rejects bottles with a missing cap.

Think of a railway ticket checker. He looks at a ticket, finds the date, the train number and the seat, and decides "valid" or "not valid". A computer vision system does the same kind of job: look, find the important parts, decide.

The main tasks

Most computer vision work falls into a few task types. You will meet each one in this course.

There are two broad ways to solve these tasks:

1. Classical computer vision. You write the rules yourself: blur the image, find edges, find shapes, measure them. It is fast, needs no training data, and is easy to explain.
2. Deep learning. You show a neural network thousands of labelled examples, and it learns its own rules. It is far more accurate on hard tasks such as recognising 1,000 kinds of objects.

Real systems often mix both. A document scanner app, for example, uses classical edge detection to find the page and may use deep learning to read the text. This course teaches the classical tools first, because they explain what deep networks learn later.

How a computer sees a photo

A digital photo is a grid of tiny squares called pixels (short for "picture elements"). Each pixel stores a number for its brightness. In the most common format, 0 means black, 255 means white, and the values in between are shades of grey.

So a photo is just a table of numbers. Python stores such tables in NumPy arrays, which you may know from the NumPy arrays lesson. Let us build a tiny 5 by 5 image by hand.

Step 1: create the grid of numbers. Step 2: print it to see the numbers. Step 3: display it as a picture.

import numpy as np
import matplotlib.pyplot as plt

# Step 1: a 5x5 grey image, 0 = black, 255 = white
img = np.array([
    [  0,   0,   0,   0,   0],
    [  0, 255, 255, 255,   0],
    [  0, 255, 128, 255,   0],
    [  0, 255, 255, 255,   0],
    [  0,   0,   0,   0,   0],
], dtype=np.uint8)

# Step 2: look at the numbers
print(img)
print("Shape:", img.shape, " Data type:", img.dtype)

# Step 3: look at the picture
plt.imshow(img, cmap="gray", vmin=0, vmax=255)
plt.title("A 5x5 image: white square, grey centre")
plt.show()

[[  0   0   0   0   0]
 [  0 255 255 255   0]
 [  0 255 128 255   0]
 [  0 255 255 255   0]
 [  0   0   0   0   0]]
Shape: (5, 5)  Data type: uint8

After Step 2 you see the same table you typed. After Step 3 you see a picture: a black border, a white square and a grey dot in the middle. Nothing magic happened. The numbers are the image.

Two details matter here:

• dtype=np.uint8 means "unsigned 8-bit integer". It holds whole numbers from 0 to 255, which is exactly the range of a normal photo pixel.
• cmap="gray" tells matplotlib to draw the numbers as shades of grey. vmin and vmax fix 0 as black and 255 as white.

Your first "vision" program

Now let us answer a real question with numbers: is a photo too dark? A camera app does this before it switches on night mode. We make two fake photos and compare their average brightness.

import numpy as np

rng = np.random.default_rng(7)
# Step 1: a dark room photo and a sunny street photo, 60x80 pixels each
dark_room = rng.integers(10, 70, size=(60, 80), dtype=np.uint8)
sunny_street = rng.integers(120, 250, size=(60, 80), dtype=np.uint8)

# Step 2: a simple rule written by us (classical computer vision)
def needs_night_mode(img, limit=80):
    return img.mean() < limit

# Step 3: test the rule on both photos
for name, img in [("dark room", dark_room), ("sunny street", sunny_street)]:
    print(f"{name:12} mean brightness {img.mean():6.1f}  night mode: {needs_night_mode(img)}")

dark room    mean brightness   39.6  night mode: True
sunny street mean brightness  183.0  night mode: False

This is classical computer vision in its simplest form: a measurement (the mean) and a rule (below 80). Later lessons replace this rule with much smarter ones, and finally with networks that learn the rules from data.

Tools you will use

The browser examples teach the ideas step by step. The OpenCV and PyTorch examples show how professionals do the same job in a few lines. For neural network basics, the Deep Learning tutorial teaches neurons, training and convolution from scratch, so this course links to it instead of repeating it.

Common mistakes

• Thinking the computer "sees" like you. It only has numbers. A small change to the numbers, such as lighting, can change the answer completely.
• Forgetting the data type. A uint8 pixel cannot go above 255 or below 0. You will see in later lessons what happens when maths overflows.
• Jumping straight to deep learning. Many real problems, such as checking if a scanned page is too dark, need only a few lines of classical code.

Key points

• Computer vision gets useful information from images and video.
• The main tasks are classification, detection, segmentation, OCR and tracking.
• A grey image is a grid of pixels, each a number from 0 (black) to 255 (white).
• In Python, images are NumPy arrays, usually with dtype=np.uint8.
• Classical methods use hand-written rules; deep learning learns rules from labelled examples.

import numpy as np
import matplotlib.pyplot as plt

img = np.zeros((7, 7), dtype=np.uint8)
img[3, :] = 255   # the middle row
img[:, 3] = 255   # the middle column
print(img.shape)
print(round(img.mean(), 1))
plt.imshow(img, cmap="gray", vmin=0, vmax=255)
plt.show()