Of all the senses a robot can have, vision gets the most attention. Always has. Probably always will.
And honestly? Fair enough. There’s something deeply satisfying about watching a machine see something and respond to it — no buttons pressed, no keyboard involved, just the robot making a decision on its own based on what’s in front of it.
We’re not talking about fully autonomous vehicles today. But line following with a webcam — where a robot locks onto a colored line and tracks it automatically — might be the single easiest way to see that idea come to life. Draw a line on paper, run some code, watch Python chase it around the desk like a very focused, very confused puppy.
Let’s do it.
How It Actually Works (A Scenic Ride Through the Theory)
Some of this will feel abstract. That’s fine — we’re not going deep today, just getting the lay of the land. Think of it as a quick sightseeing tour: you don’t need to know the history of every building to enjoy the view.
The system runs on a closed loop: detect → calculate → decide. Here’s what that looks like in practice:
Capture: The webcam continuously grabs frames from whatever it’s pointing at.
Image Processing: Raw color video gets converted from BGR to HSV — a color space that makes it much easier to isolate specific colors regardless of lighting. Then the system applies a color mask: everything that matches the target color stays white, everything else goes black. Clean signal, no noise.
Find the line: OpenCV’s contour detection scans the masked image and finds the edges of the colored region. We grab the largest contour — that’s our line — and calculate its center point using image moments.
Decide: Compare the center of the line to the center of the frame. Is the line on the left? Turn left. On the right? Turn right. Dead center? Go straight. That’s the entire decision tree. Three outcomes. Surprisingly effective.
Act: In a real robot, this decision gets sent to the motor controller. Today, we’re printing it on screen instead. Same logic, no wheels.
Real-World Applications (Why This Actually Matters)
This isn’t just a fun webcam trick. Line following with computer vision is the foundation of some surprisingly serious technology:
AGV/AMR systems in factories: Smart warehouses use camera-equipped robots that follow floor markings to transport materials between workstations — no physical rails, no human drivers.
Prototype autonomous vehicles: Entry-level self-driving research often starts here — detect the lane, stay in it, handle exceptions. The fancy stuff comes later.
Hospital and care facility robots: Some medical robots navigate by following marked corridors to deliver medication or lab samples.
STEM education: Line following is practically a rite of passage in robotics courses worldwide — it teaches perception, decision-making, and closed-loop control all in one project.
One Honest Warning Before We Get Excited
Camera-based line following has real advantages: cheap, flexible, easy to experiment with, and genuinely “impressive”-looking results. But let’s be honest about the limitations too.
Imagine a power outage. Everything goes dark. No lights, no visibility. Our “impressive” robot would just… sit there. Completely blind. Utterly useless. Meanwhile, a robot with an infrared sensor would keep rolling along without missing a beat — because infrared doesn’t need visible light.
Also, processing a continuous video stream is computationally hungry. On a lightweight board, a robot that looks great on a laptop webcam might start struggling when you try to move the same code onto embedded hardware. We’ll cross that bridge when we get there.
For now: webcam on a laptop, colored line on paper, Python doing the thinking. Controlled conditions, clear results, zero excuses.
What You Need
One sheet of white A4 paper and a red Sharpie. That’s it. Draw a thick, bold line — curves, angles, whatever you like. The bolder the line, the easier it is for the color mask to pick it up cleanly. Red works great against white. Other bright colors work too — we’ll cover how to switch in the code.
The Code
Create a new Python file called LineFollow.py:
import cv2
import numpy as np
# Open webcam (change 0 if you have multiple cameras)
cap = cv2.VideoCapture(0)
# Define HSV color range for the line we're tracking (red, range 1)
# Note: red in HSV spans two ranges — this covers the lower one
lower_color = np.array([0, 120, 70])
upper_color = np.array([10, 255, 255])
while True:
ret, frame = cap.read()
if not ret:
break
# Flip the frame horizontally (mirror mode — easier to work with)
frame = cv2.flip(frame, 1)
# Convert from BGR to HSV color space
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
# Create a mask — keep only pixels that fall within our color range
mask = cv2.inRange(hsv, lower_color, upper_color)
# Clean up the mask — remove small noise specks
mask = cv2.erode(mask, None, iterations=2)
mask = cv2.dilate(mask, None, iterations=2)
# Find contours of the colored region
contours, _ = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
if len(contours) > 0:
# Take the largest contour — that's our line
c = max(contours, key=cv2.contourArea)
# Calculate the center point of the contour
M = cv2.moments(c)
if M["m00"] > 0:
cX = int(M["m10"] / M["m00"])
cY = int(M["m01"] / M["m00"])
# Draw a dot at the detected center point
cv2.circle(frame, (cX, cY), 7, (0, 0, 255), -1)
# Decide direction based on where the center falls
h, w, _ = frame.shape
if cX < w // 2 - 50:
cv2.putText(frame, "Action: Turn LEFT", (20, 30),
cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 0, 0), 2)
elif cX > w // 2 + 50:
cv2.putText(frame, "Action: Turn RIGHT", (20, 30),
cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
else:
cv2.putText(frame, "Action: Go STRAIGHT", (20, 30),
cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
# Show both windows — the live feed and the color mask
cv2.imshow("Line Follow Simulation", frame)
cv2.imshow("Color Mask", mask)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Hit Run. Two windows will appear: the live camera feed and a black-and-white mask showing exactly what the algorithm “sees.” Hold your paper with the red line in front of the camera and move it slowly left and right. You’ll see the text on screen switch between “Turn LEFT”, “Turn RIGHT”, and “Go STRAIGHT” as the line shifts across the frame.
Code Walkthrough — The Three Parts That Matter
1. Finding the line (Contours)
contours, _ = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
c = max(contours, key=cv2.contourArea)
findContours scans the black-and-white mask and traces the edges of every white region it finds. max(..., key=cv2.contourArea) picks the largest one — we’re assuming that’s our line, not a stray red pixel on your shirt.
2. Calculating the center (Image Moments)
M = cv2.moments(c)
cX = int(M["m10"] / M["m00"])
cY = int(M["m01"] / M["m00"])
cv2.moments computes geometric properties of the contour — think of it as finding the “center of gravity” of the colored region. cX and cY are the coordinates of that center point. That red dot you see on screen? That’s it.
3. Simulating navigation
if cX < w // 2 - 50: # Line is left of center
→ Turn LEFT
elif cX > w // 2 + 50: # Line is right of center
→ Turn RIGHT
else: # Line is roughly centered
→ Go STRAIGHT
The frame gets divided into three zones based on horizontal position. The 50-pixel buffer on each side creates a “tolerance zone” in the middle — without it, the robot would jitter left-right constantly trying to hit an exact pixel. In a real robot, these three text outputs would be replaced by motor commands. The logic stays identical.
Want a Different Color?
Red was convenient for this demo, but any bright color on a white background works. Here are the HSV ranges for common choices:
| Color | Lower | Upper |
|---|---|---|
| Red (range 1) | [0, 120, 70] | [10, 255, 255] |
| Red (range 2) | [170, 120, 70] | [180, 255, 255] |
| Blue | [100, 150, 50] | [140, 255, 255] |
| Green | [40, 70, 70] | [80, 255, 255] |
| Yellow | [20, 100, 100] | [30, 255, 255] |
Swap the lower_color and upper_color values in the code and you’re good to go.
What We Just Built
A Python program that watches a live video feed, isolates a specific color, finds its center of mass, and makes a directional decision — all in real time, on a laptop, with a webcam and a piece of paper.
That’s the full perception-decision loop. The same loop that lives inside every line-following robot, every lane-keeping system, and every autonomous warehouse vehicle — just at a different scale and with better hardware.
When we’re ready to put this on a real robot, the change is minimal: swap the putText commands for motor control signals. The vision logic doesn’t change at all.
Next up: Part 10 — Phase 1 Recap. Everything we’ve built, everything we’ve learned, and a proper send-off before we start buying actual hardware.