Processing engine

Vehicles detection

Applying the stereo method provides the possibility of detecting the location and 3-D dimensions of the vehicles regardless of the plate. This results in not being affected by reflections, shadows, different plate heights, and the other items that cause inefficiencies or mistakes in plate recognition, in cases which the LPR is accomplished only based on the plate. The design of vehicles identification software in this system is based on one of the most advanced processing techniques for identifying, tracking and categorizing objects. This technique, which is based on Stereo Vision, provides the ability to understand the surrounding environment in three dimensions for the system. Application of this technique is very beneficial and it is immune from the other algorithm’s weaknesses. In the stereo method, every single point on the 2-dimensional picture, including Pixcel_x and Pixcel_y, would be shown in 3 X, Y and Z dimensions, through two cameras and special processing software.After this stage, the system will be able to precisely separate 3D moving objects from the street surface and from each other by subsequent processing. The main advantages of this method include the capability of recognizing all of the moving objects higher than the street surface (motorcycle, truck, etc.), regardless of the appearance, movement direction, and also the capability of measuring dimensions.

Also, because the images of these cameras are wide and cover the whole width of the intersection, it is possible to track the direction of the offender vehicle in order to determine the type of its violation.

In addition, due to the processing power of this system, all of the vehicles in the intersection will be detected, tracked, and their speed will be measured by the stereo system. Even if the all passing vehicles commit an offense simultaneously, the system records and stores a separate violation for each one. So, before green light or after red light when several vehicles commit some offenses simultaneously, the system’s performance will not be disturbed.

The placement of cameras and intersection coverage by wide images

On the other hand, other similar systems apply the primary methods based on background separation or plate tracking, which background separation is very sensitive to the light condition, shadow direction and traffic rate. For example, in this method, during the hours of the day when the vehicle’s shadow is next to it and a bit big, the system will face difficulty in identifying the exact location of the vehicles and separating them.

Another problem with the single-camera systems is the lack of recognizing the vehicles’ height, so the location of vehicles will not be precisely detected by the system in special circumstances. For example, if the system is installed before the intersection, vehicles’ locations with more height will be estimated farther.

The above pictures show that different height of plate installation results in disability in recognition of vehicle’s location through recognition of plate’s location

Besides, single-camera systems have difficulty at nights, because of the light reflections on the street surface; lots of articles can be found which are about the deficiencies of these systems. Also, plate identification based systems, are not capable of detecting violations related to vehicles without license plate or motorcycles and, moreover, they are highly sensitive to the location of the plate installation place on the vehicle. For example, imagine a bus which has stopped on the crosswalk, considering that this vehicle’s plate is located on a few meters distance in front of the bus, the system is unable to identify vehicle’s location correctly. Installation of the plates at different heights will cause some errors in these systems (see figure above).

Also, there is no possibility to recognize the vehicle’s license plate, if there is overlap. And the license plate will be recognized and recorded only when it is visible. Since the main vision of this system is 3-D based, and all of the image information will be processed in a 3-D way, the exact shape and size of vehicles can be extracted from the system, thus the vehicles can be classified without the requirement of license plate reading. And all of the heavy and light vehicles can be detected containing valuable information for law enforcement or statistical surveys.

3D tracking

Having the Continuous vehicle location in 3-D world, will enable us to track every single vehicle in 3-D way, before entering the intersection and crosswalk till exiting from the intersection and in result, we have the possibility of movement direction recognition and separation and recording the left, right, and direct circumvention violations based on 3-D target tracking. It is also possible to record stop on the crosswalk violation. Through using the location to time 3-D curves for each vehicle, this system is also capable of instantaneous speedometer for all of the vehicles.

Imaging from detection phase and tracking of vehicles is “wide” in this system and covers the whole area of intersection. This imaging method enables the system to cover multiple lines with a single stereo camera simultaneously, so it can be applied in wide intersections. In this way, two major functionalities are implemented in 3-D detecting and tracking of the vehicles in this system: 1, 3-D detection and 2, imaging, in such a way that the path tracking of vehicles when they are entering or passing the intersection could be explicit. Due to this type of design, it is possible to record and categorize various types of violations in this system. Therefore, the direct passage from red light, right or left turn can be distinguished and recorded even in cases which one of these is allowed and the other is a violation.

Also, this system has the unique capability of stopping on crosswalk violation recording. Because, on one hand, it can correctly detect the exact and 3-D location of the vehicle at the intersection (despite the plate-based systems which cannot perform correctly when dealing with high installed plates or long-length vehicles) and, on the other hand, even in overlapping license plate situation, according to vehicle pursuing, when it is in the frame, extracts the license plate when it is out of overlapping situation.

Plate reading

One of the main parts of red light violation enforcement system is designation and implementation of license plate reader software, which can completely be compatible to the specific features of various types of standard plates around the world. The taken image from the offender vehicle at day and night is completely colored in this system and the license plate and the type of vehicle are clear in this picture. In the case of unconventional plates, their violation will also be recorded and their speed will be detected, and a high-quality picture would also be provided and finally, their plate would be read by the user at the monitoring center and the violation will be recorded.

The measured accuracy of this system, in the real environment and for all of the passing vehicles up to 35 meters distance, is 96.5%. This plate reader is capable of reading all types of license plates including private, governmental and public plates. The kernel of this system has also been applied in various projects and has been tested in different situations.

It should be noted that despite the high precision of the system compared to other existing systems, regarding to the special design of the system and vehicles with special plates (such as political, transit, military, etc.), as well as vehicles with manipulated plate or without a license plate, if a vehicle commits a violation and its plate would not be extracted, this vehicle will be displayed on a separate list to the user in order to observe and making decision, then the user enforce the law for that vehicle by observing the high quality taken image.

Considering that human mind is more powerful than all LPR software, however, this system is immune from any failure, even because of the special type of plates, and it can be easily implemented in other countries. Independence of the system from plate characters detection results in high reliability of the system so that if the detection of vehicles were dependent to plate characters, those vehicles which are not read out will not definitely be detected in the system and will be lost.

For example, some of the other systems claim the accuracy of more than 96% for each character in character recognition and according to the standard plates which usually have 8 characters, the total accuracy of these systems would be 72%. This issue becomes more serious when violators can feel free to commit the red light violation by simple plate manipulations, such as cleaning a part of the plate or placing talc on that which could be considered allowed by the police.

Calibration

Despite other similar systems, no specific environmental calibration is required for this system, and the accuracy of the system’s performance is related to the internal calibration of the cameras.

One of the important points in applying the stereo-based systems, is the calibration of these systems, and since the main calibration part of these systems is dependent on the position of the cameras related to each other, their calibration will be almost done without pre-suppose about the installation place and even any pre-suppose in production time, this makes it easy to install the system in different locations. Also, these systems do not require marking the street surface because the location of the objects is not closely related to the distance and the angle of cameras and street. So the resistance of these systems to vibration, expansion and contraction is much higher than the other similar systems which also makes the check-up periods even longer for probable calibration.

Traffic light status detection

The traffic light status can be detected as a wireless electronic signal or by image processing.

The software of this system is able to detect the traffic light status based on image processing without the requirement of connection to the red light. This capability facilitates the installation of this system at almost all of intersections and eliminates the requirement of extra wiring.

It is also possible to detect the status of the light through the intersection controller, for which a transmitter will be located beside the intersection controller and transfers the traffic light status to the system wirelessly and thus, it is tried to use the least possible wiring requirement, in system structure. Also, if several systems are installed in a single intersection, it is possible to apply only one transmitter in the intersection box and send all traffic light signals to the systems.

Comparison with other technologies