Convolution Step by Step

By the end of this lesson you will understand convolution, the single most important operation in computer vision. You will code it with loops, then make it fast with NumPy, and you will know how padding changes the output size. Every filter in the next lessons, and every layer of a CNN, is built on this idea.

The idea: a small window that slides

A kernel (also called a filter) is a small grid of numbers, often 3 by 3. Convolution slides the kernel over the image, one pixel at a time. At each position it multiplies each kernel number by the pixel under it, adds the nine products, and writes the sum into the output.

Picture a teacher checking a big answer sheet with a small cardboard stencil that has 9 holes. She places the stencil, looks at the 9 numbers through the holes, combines them into one score, writes it down, and moves the stencil one step to the right. When she reaches the end of the row, she moves down one step and starts again.

The numbers in the kernel decide what the filter does:

• All values equal to 1/9: the output is the average of the 9 pixels, which blurs.
• A 1 in the centre and 0 elsewhere: the output equals the input (the identity kernel).
• Positive on one side and negative on the other: the output is large where brightness changes, which finds edges.

Step by step with loops

Let us do it the slow, clear way first. Step 1: make a tiny image with one bright pixel. Step 2: make a 3 by 3 averaging kernel. Step 3: slide it with two loops.

import numpy as np

# Step 1: a 5x5 image, all 10 except one bright pixel of 90
img = np.full((5, 5), 10.0)
img[2, 2] = 90

# Step 2: an averaging (box) kernel
kernel = np.ones((3, 3)) / 9

# Step 3: slide the kernel over every position where it fits completely
kh, kw = kernel.shape
out_h, out_w = img.shape[0] - kh + 1, img.shape[1] - kw + 1
out = np.zeros((out_h, out_w))
for r in range(out_h):
    for c in range(out_w):
        window = img[r:r + kh, c:c + kw]          # the 9 pixels under the stencil
        out[r, c] = np.sum(window * kernel)       # multiply and add
print("output shape:", out.shape)
print(np.round(out, 1))

output shape: (3, 3)
[[18.9 18.9 18.9]
 [18.9 18.9 18.9]
 [18.9 18.9 18.9]]

Every 3 by 3 window here contains the bright pixel, so every output is (8 × 10 + 90) / 9 = 18.9. The single bright spot has been spread out over its neighbours. That is blurring.

Notice the output is 3 by 3, smaller than the 5 by 5 input. The kernel can only sit where it fits completely, so we lose a 1-pixel border on each side. For a k by k kernel, the output shrinks by k minus 1 in each direction. This is called valid convolution.

Padding: keeping the same size

Usually we want the output to be the same size as the input. The fix is padding: add a border of extra pixels around the image before sliding. For a 3 by 3 kernel, a 1-pixel border is enough. The question is what values to put in the border:

Zero padding darkens the borders of a blurred photo, because black pixels get averaged in. Replicate and reflect avoid that. OpenCV's default border mode is the reflect type BORDER_REFLECT_101.

import numpy as np

img = np.array([[1, 2, 3],
                [4, 5, 6]])
for mode in ["constant", "edge", "reflect"]:
    print(mode)
    print(np.pad(img, 1, mode=mode))

constant
[[0 0 0 0 0]
 [0 1 2 3 0]
 [0 4 5 6 0]
 [0 0 0 0 0]]
edge
[[1 1 2 3 3]
 [1 1 2 3 3]
 [4 4 5 6 6]
 [4 4 5 6 6]]
reflect
[[5 4 5 6 5]
 [2 1 2 3 2]
 [5 4 5 6 5]
 [2 1 2 3 2]]

Look at the corners: zero padding puts 0s there, edge copies the nearest pixel, and reflect mirrors around the edge pixel without repeating it.

Making it fast with NumPy

Python loops are slow: a 1000 by 1000 photo needs a million window positions. NumPy can build all windows at once with sliding_window_view, then multiply and sum them in one call. The result is identical; only the speed changes.

import numpy as np
from numpy.lib.stride_tricks import sliding_window_view

def conv2d(img, kernel, pad_mode="edge"):
    """Same-size 2D filtering (correlation, like cv2.filter2D)."""
    k = kernel.shape[0]
    padded = np.pad(img.astype(np.float64), k // 2, mode=pad_mode)
    windows = sliding_window_view(padded, kernel.shape)    # shape (H, W, k, k)
    return np.einsum("ijkl,kl->ij", windows, kernel)       # multiply and add per window

img = np.full((5, 5), 10.0)
img[2, 2] = 90
print(np.round(conv2d(img, np.ones((3, 3)) / 9), 1))

big = np.random.default_rng(0).integers(0, 256, (400, 600))
out = conv2d(big, np.ones((5, 5)) / 25)
print("big image:", big.shape, "->", out.shape)

[[10.  10.  10.  10.  10. ]
 [10.  18.9 18.9 18.9 10. ]
 [10.  18.9 18.9 18.9 10. ]
 [10.  18.9 18.9 18.9 10. ]
 [10.  10.  10.  10.  10. ]]
big image: (400, 600) -> (400, 600)

sliding_window_view does not copy any pixels; it creates a clever view where each output position "sees" its window. np.einsum("ijkl,kl->ij", ...) reads as: for each window (i, j), multiply its k by l values by the kernel and sum over k and l. We will reuse this conv2d function in the next lessons.

Convolution or correlation?

Strictly speaking, what we coded is correlation. True mathematical convolution first flips the kernel upside down and left to right, then slides it. For symmetric kernels, such as the box blur, the two are identical. For lopsided kernels the result is mirrored.

import numpy as np
from numpy.lib.stride_tricks import sliding_window_view

def correlate(img, kernel):
    padded = np.pad(img, 1)
    return np.einsum("ijkl,kl->ij", sliding_window_view(padded, kernel.shape), kernel)

def convolve(img, kernel):
    return correlate(img, kernel[::-1, ::-1])       # flip, then slide

img = np.zeros((3, 5)); img[1, 2] = 1               # one dot
k = np.array([[0, 0, 0],
              [1, 2, 3],                            # a lopsided kernel
              [0, 0, 0]])
print("correlation:", correlate(img, k)[1])
print("convolution:", convolve(img, k)[1])

correlation: [0. 3. 2. 1. 0.]
convolution: [0. 1. 2. 3. 0.]

Applying a kernel to a single dot "stamps" the kernel into the output. Convolution stamps it the right way round; correlation stamps it mirrored. OpenCV's cv2.filter2D and the "convolution" layers in PyTorch both actually compute correlation. For learned CNN kernels it does not matter, because the network simply learns the flipped numbers.

Convolution in OpenCV

cv2.filter2D applies any kernel you give it. The second argument, -1, means "output has the same data type as the input".

import cv2
import numpy as np

img = np.array([[10, 10, 10, 10, 10],
                [10, 10, 10, 10, 10],
                [10, 10, 90, 10, 10],
                [10, 10, 10, 10, 10],
                [10, 10, 10, 10, 10]], dtype=np.float32)
kernel = np.ones((3, 3), np.float32) / 9

out = cv2.filter2D(img, -1, kernel, borderType=cv2.BORDER_REPLICATE)   # -1: same depth as input
print(np.round(out, 1))

[[10.  10.  10.  10.  10. ]
 [10.  18.9 18.9 18.9 10. ]
 [10.  18.9 18.9 18.9 10. ]
 [10.  18.9 18.9 18.9 10. ]
 [10.  10.  10.  10.  10. ]]

The centre 3 by 3 matches our loop version exactly, and padding has kept the size at 5 by 5.

Common mistakes

• Filtering in uint8. Sums overflow or get rounded; convert to float first.
• Forgetting that valid convolution shrinks the image. Pad by k // 2 for same-size output.
• Zero-padding photos. Dark borders appear; use replicate or reflect padding.
• Assuming filter2D flips the kernel. It computes correlation; flip the kernel yourself if you need true convolution.

Key points

• Convolution slides a small kernel over the image; each output is a weighted sum of the pixels under it.
• The kernel's numbers decide the effect: averaging blurs, opposite signs detect edges.
• Without padding, a k by k kernel shrinks the output by k minus 1; pad by k // 2 to keep the size.
• sliding_window_view plus np.einsum gives a fast, loop-free NumPy version.
• cv2.filter2D and CNN layers compute correlation; for symmetric kernels it equals convolution.

import numpy as np
from numpy.lib.stride_tricks import sliding_window_view

def conv2d(img, kernel, pad_mode="edge"):
    k = kernel.shape[0]
    padded = np.pad(img.astype(np.float64), k // 2, mode=pad_mode)
    return np.einsum("ijkl,kl->ij", sliding_window_view(padded, kernel.shape), kernel)

img = np.zeros((5, 5)); img[2, 2] = 1
k = np.array([[0, 0, 0], [0, 0, 1], [0, 0, 0]])
print(conv2d(img, k))