November 30, 2019 by Reginald Watson
Wanna learn about image recognition systems? Start with this License Plate Recognition project on Raspberry Pi!
In this tutorial, I’ll show you how to build a license plate recognition (LPR) system (also sometimes referred to as automatic license plate recognition using OpenCV on Raspberry Pi.
We’ll begin by setting up the actual license plate detection, which is the most important stage of the system. At this stage, the system determines the position of the license plate, inputs an image of the vehicle, and outputs the license plate.
From there, we move onto character segmentation, where the characters on the license plate get mapped out and segmented into individual images.
Finally, the system performs character recognition, where the characters we segmented in the second step are identified. This is where machine learning comes in.
Begin by ensuring that your Raspberry Pi is up to date by typing the following commands. These commands will update the installed packages on your Raspberry pi to the latest versions.
sudo apt-get update
sudo apt-get upgradeThese commands update the installed packages on your Raspberry pi to the latest versions.
Then type:
sudo apt install libatlas3-base libsz2 libharfbuzz0b libtiff5 libjasper1 libilmbase12 libopenexr22 libilmbase12 libgstreamer1.0-0 libavcodec57 libavformat57 libavutil55 libswscale4 libqtgui4 libqt4-test libqtcore4to install required packages for OpenCV on your Raspberry pi.
After that, type:
sudo pip3 install opencv-contrib-python libwebp6to install OpenCV 3 for python 3 on your Raspberry pi. Pip3 means that OpenCV will get installed for python 3.
The last library we need to install is NumPy. NumPy is a library that makes it very easy to perform array operations in Python. Type the following command to install it.
Pip3 install python-numpyNow we are going to clone the License Plate Recognition GitHub repository by Chris Dahms. Thank you, Chris!
This repository contains all the files that we need to recognize license plates. His code can only recognize number plates from an image, so later in the tutorial, we are going to update this code to work in real-time.
First, the code from GitHub reads out the training classifications and will train the KNN object that recognizes characters.
try:
npaClassifications = np.loadtxt("classifications.txt", np.float32) # read in training classifications
except: # if file could not be opened
print("error, unable to open classifications.txt, exiting program\n") # show error message
os.system("pause")
return False # and return False
# end try
try:
npaFlattenedImages = np.loadtxt("flattened_images.txt", np.float32) # read in training images
except: # if file could not be opened
print("error, unable to open flattened_images.txt, exiting program\n") # show error message
os.system("pause")
return False # and return False
# end try
npaClassifications = npaClassifications.reshape((npaClassifications.size, 1)) # reshape numpy array to 1d, necessary to pass to call to train
kNearest.setDefaultK(1) # set default K to 1
kNearest.train(npaFlattenedImages, cv2.ml.ROW_SAMPLE, npaClassifications)After that, we check whether training was successful or not. If training isn’t successful, it will show an error.
if blnKNNTrainingSuccessful == False: # if KNN training was not successful
print("\nerror: KNN traning was not successful\n") # show error message
returnThen it reads out the image and checks whether it was properly read or not. If image read isn’t successful, it will show an error.
imgOriginalScene = frame #cv2.imread("LicPlateImages/car.jpg") # open image
if imgOriginalScene is None: # if image was not read successfully
print("\nerror: image not read from file \n\n") # print error message to std out
os.system("pause") # pause so user can see error message
return
In the next step, the system tries to detect the region of possible plates in the image. Detecting the plates consists of following steps.
1. Downsampling the image.
imgGrayscale = extractValue(imgOriginal)
def extractValue(imgOriginal):
height, width, numChannels = imgOriginal.shape
imgHSV = np.zeros((height, width, 3), np.uint8)
imgHSV = cv2.cvtColor(imgOriginal, cv2.COLOR_BGR2HSV)
imgHue, imgSaturation, imgValue = cv2.split(imgHSV)
return imgValue
2. Applying a threshold.
imgThresh = cv2.adaptiveThreshold(imgBlurred, 255.0, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, ADAPTIVE_THRESH_BLOCK_SIZE, ADAPTIVE_THRESH_WEIGHT)
3. Finding and drawing contours.
imgContours, contours, npaHierarchy = cv2.findContours(imgThreshCopy, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
height, width = imgThresh.shape
imgContours = np.zeros((height, width, 3), np.uint8)
for i in range(0, len(contours)):
if Main.showSteps == True: # show steps
cv2.drawContours(imgContours, contours, i, Main.SCALAR_WHITE)
4. Finding possible characters in the image.
for possibleChar in listOfPossibleCharsInScene:
contours.append(possibleChar.contour)
# end for
cv2.drawContours(imgContours, contours, -1, Main.SCALAR_WHITE)
cv2.imshow("2b", imgContours)
5. Matching characters in the image.
for listOfMatchingChars in listOfListsOfMatchingCharsInScene:
intRandomBlue = random.randint(0, 255)
intRandomGreen = random.randint(0, 255)
intRandomRed = random.randint(0, 255)
contours = []
for matchingChar in listOfMatchingChars:
contours.append(matchingChar.contour)
# end for
cv2.drawContours(imgContours, contours, -1, (intRandomBlue, intRandomGreen, intRandomRed))
6. Extracting plate area.
for i in range(0, len(listOfPossiblePlates)):
p2fRectPoints = cv2.boxPoints(listOfPossiblePlates[i].rrLocationOfPlateInScene)
cv2.imshow("4b", listOfPossiblePlates[i].imgPlate)
1. Downsampling license plate region.
imgGrayscale = extractValue(imgOriginal)
def extractValue(imgOriginal):
height, width, numChannels = imgOriginal.shape
imgHSV = np.zeros((height, width, 3), np.uint8)
imgHSV = cv2.cvtColor(imgOriginal, cv2.COLOR_BGR2HSV)
imgHue, imgSaturation, imgValue = cv2.split(imgHSV)
return imgValue
2. Applying a threshold to license plate region.
imgThresh = cv2.adaptiveThreshold(imgBlurred, 255.0, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, ADAPTIVE_THRESH_BLOCK_SIZE, ADAPTIVE_THRESH_WEIGHT)
3. Resizing for easier viewing and character detection and applying the threshold again to eliminate any gray areas.
possiblePlate.imgThresh = cv2.resize(possiblePlate.imgThresh, (0, 0), fx = 1.6, fy = 1.6)
4. Finding contours.
imgContours = np.zeros((height, width, 3), np.uint8)
del contours[:] # clear the contours list
for possibleChar in listOfPossibleCharsInPlate:
contours.append(possibleChar.contour)
# end for
cv2.drawContours(imgContours, contours, -1, Main.SCALAR_WHITE)
5. Recognizing the characters using the trained KNN object.
retval, npaResults, neigh_resp, dists = kNearest.findNearest(npaROIResized, k = 1)
Then we draw a rectangle in the original image and write the numbers below in the license plate.
drawRedRectangleAroundPlate(imgOriginalScene, licPlate)
writeLicensePlateCharsOnImage(imgOriginalScene, licPlate)
You can recognize license plates from other images by adding the images in the LicPlateImages directory.
Type the following command to run the code.
python3 Main.pyNow we are going to recognize a license plate in real-time using the camera with the Raspberry Pi to capture the frames. When any key will be pressed, it will take that frame and should recognize the license plate from the captured image.
We need to make a few changes in the code we used in the previous example in order to make it work in real-time.
Instead of reading the images, we will add the camera object to help us access the camera on the Raspberry Pi.
cap = cv2.VideoCapture(0)Next, we will use this camera object to capture frames one by one and show these on the output window.
_, frame = capture.read()
cv2.imshow('frame', frame)After that, we need to wait for some time to set up what will happen if any keys get pressed. If any key is pressed, it will stop to take new frames and will start to recognize license plates in the current frame.
key = cv2.waitKey(1)
if key == 27:
breakThe full Python code for detecting license plates in real-time can be found at the following link:
Real-time License Plate Detection Python Code
Paste it in the Main.py and run the file by typing:
python3 Main.pyNote that the real-time output depends a lot on the camera quality.
I love challenging myself by creating new projects using different microcontrollers to see what I can come up with.
