My First OpenCV Assignment

Wednesday, 08 September 2010 06:00

Attention: open in a new window. PDFPrintE-mail

In my quest to become a computer vision guru, I discovered a library called OpenCV, which has a lot of optimized tools to allow any programer to begin computer vision research.  Unfortunately, the internet documentation is sparse, so I bought the book on the subject: Learning OpenCV: Computer Vision with the OpenCV library by Gary Bradski and Adrian Kaebler.  I read through the entire book, and understood only about 75% of it.  I figured the OpenCV Forums at Yahoo! Groups would be able to answer any questions later.  So, I charged forward with my first project: Track something in a video.
This turned out to be a very ambiguous task, so I refined it a bit: Track a ball in a video.
By tracking, I mean draw a rectangle or circle around the ball, no matter where it moves on the screen.  After Googling for videos of orange balls, I found an excellent video of a cute dog balancing on a ball.  I immediately ripped the video from YouTube, and tried to wrap my brain around how to track something in a video.  I was able to use the OpenCV library to play the video frame-by-frame, but trying to track something is much more difficult than I initially imagined.  So I took a small step backward, and changed my assignment:  Draw a rectangle around an orange ball in a picture.
I ran into brick walls trying to isolate the orange ball from the rest of the image.  The ball is the only orange object in the picture, so it should be easy enough, right?  Well, the image is an RGB image, but the color of the ball is orange, which is a weird combination of all three colors.  So I tried (with little success) to "filter" out all the pixels that were not within a small distance from each channel of each pixel.  This gave me unsatisfactory results, and after 2 days of no results, I asked the OpenCV forums for help.  The next morning, I got a suggestion from Tõnu Samuel to convert the image to HSV. With an HSV image, the color aspect of each pixel is just one value, which is faster and easier to compare to what I want.  After that, I was able to isolate the ball, and draw a rectangle in the original image.  Joy!  So I broadened my assignment again to Tracking a ball in a video, and within minutes, I was able to use the same methods on each individual frame, then play them back in sequence.  Keep in mind that my algorithm computes the balls location for each frame in real time.  Pretty cool, huh?  I think so.
To summarize how I made it work:
1) Load the video.
The original frame grabbed from the video
2) Create 5 IplImages as follows:
-one for the current frame in the video
-one as a temporary scratch image
-one for each channel of the HSV image you convert your temporary image to
3) Convert the frame to HSV, and store the HSV image in your temporary image.
The original frame, converted to HSV
4) Split the temporary image into it's components, H, S, and V.
5) Scan through each pixel of the H-channel image, and set the pixel to black if it's not within 10 units of the hue you want. (In our case, we were looking for the hue 122, and colors near that.)
The HSV image, filtered by hue
6) Use cvMorphologyEx() to remove the noise from the previous filter, leaving you with only the large blob that is our ball.
The original frame grabbed from the video
7) Use cvFindContour() to find the edges of our ball.
8) Use cvBoundingRect() to draw the bounding box of the blob around the ball in the original frame.
The original frame with the balls bounding box
9) Display the original frame, with the new rectangle drawn on it.
10) Repeat.
Now wasn't that fun?! Here's the source code and files.  Talk to you soon.
-Jay