Detecting colors in a webcam feed

A friend asked me to help debug an app that tried to detect three different colors. It was written in Python and used three precreated image files as test subjects, and it seemed like a fairly easy thing to get his app finished.

But I was wrong.

I knew only the basic things about colors, namely that RGB  means read-green-blue and that those made up the actual color that you saw. 255-0-0 is red you know. Easy!

However I quickly discovered that not only does some frameworks (OpenCV) use BGR instead of RGB, there are also a bunch of other “color spaces” as they are called.

HSV (Hue Saturation Value) seemed like a good choice, however there was also a similar competitor HSL (Hue Light Saturation). They are almost the same but I chose HSV. more on that later.

HSV has the property that you can keep the color (hue) but change the saturation and lighting.  That way you can look for a certain color/hue, but ignore the saturation and lighting.

Also, for historical reasons OpenCV uses BGR format internally, so one must remember that when converting to HSL. However OpenCV has many conversion methods so that is easy.

Anyway, to filter out for example Blue from an image, use this code:

image = cv2.imread("test.jpg")
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

lower_blue = np.array([100, 0, 0])
upper_blue = np.array([125, 255, 255])

# Threshold the HSV image to get only blue colors
mask = cv2.inRange(hsv, lower_blue, upper_blue)

# mask is 2-dimensional image, where each byte acts as a mask

# if there are "set" pixels in this mask, the color we searched for existed!
print(f"{cv2.countNonZero(mask)} nonzero pixels in mask")

# Use the mask to show the filtered image
res = cv2.bitwise_and(image, image, mask=mask)

Now all this is fine and dandy, but there are a few gotchas.

If you for some reason have RGB images (if you are not using OpenCV for loading/capturing), you must convert from RGB to HSV somehow. OpenCV has the cvtColor(img, cv2.COLOR_RGB2HSV) method for that.

Remember that even though the image may be stored on disk as RGB,  OpenCV will convert in cv2.imread() to the internal BGR format.

Another problem is that there are stray pixels of the color that you search for. A web camera is not exact, and depending on the quality the colours can vary.

You can use this table as a guide: https://stackoverflow.com/questions/12357732/hsv-color-ranges-table

So you need to calibrate the detection according to your situation.

Specifically, you need to know how many detected pixels from the cv2.countNonZero() call means that you have a positive detection of this color. Whether this is 100 or 1000 you have to decide.

Possibly you also have to crop your image to skip parts that may give a false positive.

The following little program was VERY useful when trying to understand which HSV values should be used when trying to filter the exact color I was out for.

import cv2
import numpy as np


def nothing(x):
    pass


# Open the camera
cap = cv2.VideoCapture(0)

# Create a window
cv2.namedWindow('image')

# create trackbars for color change
cv2.createTrackbar('lowH', 'image', 0, 179, nothing)
cv2.createTrackbar('highH', 'image', 179, 179, nothing)

cv2.createTrackbar('lowS', 'image', 0, 255, nothing)
cv2.createTrackbar('highS', 'image', 255, 255, nothing)

cv2.createTrackbar('lowV', 'image', 0, 255, nothing)
cv2.createTrackbar('highV', 'image', 255, 255, nothing)

while True:
    ret, frame = cap.read()
    # BGR format
    cv2.imshow('source', frame)

    # get current positions of the trackbars
    ilowH = cv2.getTrackbarPos('lowH', 'image')
    ihighH = cv2.getTrackbarPos('highH', 'image')
    ilowS = cv2.getTrackbarPos('lowS', 'image')
    ihighS = cv2.getTrackbarPos('highS', 'image')
    ilowV = cv2.getTrackbarPos('lowV', 'image')
    ihighV = cv2.getTrackbarPos('highV', 'image')

    # convert color to hsv because it is easy to track colors 
    # in this color model
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    lower_hsv = np.array([ilowH, ilowS, ilowV])
    higher_hsv = np.array([ihighH, ihighS, ihighV])
    # Apply the cv2.inrange method to create a mask
    mask = cv2.inRange(hsv, lower_hsv, higher_hsv)

    # Apply the mask on the BGR image to extract the original color
    frame = cv2.bitwise_and(frame, frame, mask=mask)

    pixels = cv2.countNonZero(mask)
    print("{} pixels are matching color {} to {} ".format(pixels, 
        lower_hsv, higher_hsv))

    cv2.imshow('image', frame)

    # Press q to exit
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Just run this and play with the settings until you see only the color you are after!