How to detect the color of a person’s shirt

The task of “Face Recognition”, where you want a computer to figure out who a person is from a photo of them, is a very difficult function to do well. But in some cases such as robotics, it may not be so important to figure out exactly who it is, you might just want to get an idea of where the people are, or to notice when the camera sees the same person again after the camera has moved around in the room. These are cases when Shirt Detection can be used as a simple method to keep track of who is in the room. For example, a robot could just keep track that there is a person wearing a red shirt to its left, a person wearing a blue shirt in front, and a person wearing a yellow shirt on its right side, so that it can track where these people are when they move around or when the robot moves around.

Shirt Detection can be performed quite easily compared to Face Recognition, by using OpenCV’s very reliable Face Detection. Once the program knows where a person’s face is, it can look in a small region below the face (where a person’s shirt would be), and determine the approximate color of their shirt in that shirt region. This can be performed even without a complex method of determining the exact contour region of a person’s shirt, since all you need to know is the color of a region below their face, as opposed to the color of their entire shirt or body.

Steps to detect color of shirt

• First convert each pixel into an approximate color type (eg: Green or Orange or Purple or Black, etc), based on its HSV color components.
• Then perform Face Detection to find where the faces are within the original image. For each face that it finds, it should look at the approximate color types in a small rectangle region below the detected face. It should then see if there is a certain color type that is most dominant in that small region, such as if 60% of the pixels in the shirt rectangle are detected as purple pixels, then it should classify the person’s shirt as Purple, with a 60% confidence rating.
• In most photos, it is reliable enough for simple uses such as Human-Robot-Interaction, where a false detection is not such a problem.
• Note that if the person is so close to the camera that the shirt region goes below the bottom of the image, you should either ignore that face or try a smaller region (with less confidence) in the hope of atleast seeing the top of the shirt. This method works on the assumption that most of their shirt is in the photo, so don’t expect good results if only a tiny part of their shirt is visible, and since the shirt color is converted to just a few possible colors, don’t expect good results with more than 3 or 4 people!