Mono-cam Image Processing

Mono-cam image processing based on trigonometric approach

In this method the first thing to do is to recognize the body in the picture, so its location will be clearly defined. Since this picture is a two dimensional identity, detecting the object’s location needs finding the connecting line between camera and body in the 3-D space, so that the junction point of this line and ground line shows the objects location. Generally, we have:

Thus, knowing h and measuring β, by processing the camera’s image and also knowing the camera’s setup angle, d can be calculated which is body's distance on the ground from the pole and its changes over time indicates body's Speed.

But as simple as this problem is, there are some hidden assumptions in it which not satisfying any of them will affect the output. We have supposed that:

• The exact camera’s setup angle is known
• The accurate height of the body from the ground is known
• The street is flat

Camera setup angle changing problem:

The camera can be fixed rigidly enough and if we utilize enough knowledge in instrumentation and image processing we would be able to measure the exact fixed setup angle, but this angle is likely to change after some time due to mechanical reasons such as:

• Foundation subsidence
• Expansions and contractions of the pole
• Joints wearing out
• The pole vibration because of strong winds and crossing heavy vehicles

And that's where the difference between assumed angle and the real angle (that is changed over time) would cause inaccuracies in system. In fact, any vertical deviation in camera's angle directly affects the amount of β, and changes it. To calculate the amount of this error we suppose that we've got 2 pictures of a car, in a 0.3 sec time period: in the first one β=80°, the body is placed at a distance of 34 meters then the car approaches the camera (that is set on a 6-meter pole) so in the second picture β=75°, and it's placed at a distance of 22 meters. Therefore, the body's movement over this period is equivalent to 11.6 meters car travel down the road in our 3-D world. And if It lasted for 0.3 sec, the car's speed will be calculated about 38.7 m/s or 139 km/h. (Notice: in this example the numbers are chosen in a way close to the reality of speedometer cameras)

But let's imagine that our primary fixed angle has been changed or even it was measured incorrectly from the beginning, so the junction point of the ground line and the connecting line between camera and body is showing another point in 3-D space which is not its real location.

Now we suppose that the difference in angles is only about 0.5°, let's see how much the error would be in the calculated speed of the car:

It can be seen that only a 0.5 degree of displacement in camera’s angle caused an increase from 139 km/h to 151 km/h in the vehicle’s speed which equals to an error more than 8% in speedometer. It is interesting that the error grows nonlinearly with angular displacement of the camera such that the error will be more than 18% due to only 1 degree of displacement.

Object’s height problem:

A parameter with length dimension was needed in order to create the triangle to determine the location of the body; hence the height of triangle was used. However, the important issue is that which point on the vehicle is chosen to be the characteristic point of the car and also what the value of h for that point is.

Different height of calculating point, if the license plate or the common border between car and the ground be considered as the characteristic point

In order to find the characteristic point of the vehicle we do not have more than two options. The First one is to detect the car itself and then using the border of its body and the ground. The second is to use the license plate coordinates as the characteristic point for the whole car. Although in the first choice, h (the height of the camera pole from the ground) can be used as length parameter but specifying precisely the common border between car and ground and having it constant in different frames taken as the vehicle moves, is an important issue which is not always feasible in every light conditions and all day long. For example, during nights, especially on wet asphalt, a wave of light comes in before the car enters to the frame; so when the car gets in, only the license plate and headlights of the car are clear in the infra-red camera, and consequently there is no border detectable between car and the ground. This problem is not specifically for nights, this effect is also noticeable in many other light conditions such as shadows and cloudy weather. As a result, this method cannot be utilized as a practical approach on which a stable algorithm could be based on it. (It is worth mentioning that a minimum level of appropriate performance of a system must be necessarily tested and applicable in all different conditions.)

The border between earth and the vehicle is not clear in nights and the light condition is inappropriate

Second choice will be using the license plate coordinates as characteristic point of the vehicle. Here we face the fact that not all the license plates are installed at the same height. So difference between plates’ height will be a source of error. Also in this method it is necessary for the system to find the correct position of the license plate. And thus if the plate were mistaken (for example a writing on the car can be seen as the license plate) or system was not able to locate it (no software is as powerful as human eyes in reading license plates in every condition) then there will possibility be an error in performance or even no output.

Roads flatness problem:

It is fundamentally supposed that the road is flat and without curve In formation of the triangle. Obviously the system designed on this basis can have appropriate performance on totally flat roads only. In other words, the junction point of connecting line between the object and the curved road will be lower or higher than the balance line, which in turn causes error in locating the object and consequently in the speedometer.

Three problems mentioned for mono-cam image processing using triangle method are the major problems specified to this method. While it is possible to suggest some solutions like marking up the streets, but these solutions cannot be helpful in different times of the day or all kinds of weather, light and traffic conditions. Also they will increase the installing and calibration cost of the system such that the final system won’t be applicable in large scale projects and no extra daily repairing and resetting cost can be devoted. Nothing is more dangerous than the fact that all driving fines sent out by it cannot be absolutely accurate and reliable.

Mono-cam image processing Image size variation approach

As we mentioned before, we need a parameter with length dimension (m) in image processing and in this method the parameter will be Iran standard license plate dimensions (51 centimeters length and 11 centimeters height). So when vehicle approaches from far distance toward the camera, the size of plate will increase. Since the actual size of the plate is constant in real 3-D world, this increase in image size is due to changes in distance of the car which can be calculated by knowing the characteristics of the camera and the license plate (image processing of the license plate size in depth image of the object). Relationships describing this issue can be written as follows:

The role of the camera is to image all the bodies in the outer world into the picture’s plane. The picture below simply shows what a camera does:

As we can see, a 51 centimeter license plate which is at a distance of z from the camera with f as focal length is imaged into a number of pixels. As shown in the above picture one can write the following equation for the camera:

In the above equation f is the focal length of the camera lens, which is a constant number, and it is calculated upon the resolution and the type of lens itself. The Plate Width parameter is Iran's standard license plate width, which is 51 cm. Pixel variable is the number of width of the plate in the digital picture which comes from the image processing phase. Finally, Z is the distance of the vehicle from the camera which its changes determine the speed of the car.

As a numerical example, let’s consider there are two frames of a car where plate width is 150 pixels in the first one and 100 pixels in the second one. The time difference between these two is 0.3 seconds, thus we will have:

The main source of error in this method is the amount of plate width parameter coming from image processing in which we need to have the car’s license plate read in two frames and the plate’s width must be calculated in pixels for both images, whereas image processing is a fuzzy science in which there are always some errors. For example, suppose two last digits in the plate’s zone were 22 but they were misread as 23. Consequently, there will be some error in plate’s width. Even if the plate were read correctly still detecting the borders of the license plate and counting the number of pixels can be different from the actual one.

As a practical approximation it is absolutely usual to have up to 2 pixels error or even more errors in special conditions.

(Example of a real license plate from a speedometer camera in which the border of the plate is not as accurate as every single pixel.)

In order to calculate the error, suppose in one frame we have a 2-pixel decrease in plate’s width and a 2-pixel increase in the other frame. Thus the calculations are as follows:

As we can see, a 2-pixel error in width of the license plate results the speed to change from 142 km/h to 155 km/h which is equivalent to more than 8% error in speedometer. It is worth to mention that the final error grows non-linearly with any increase in the number of error pixels. And 3-pixel error in width of the plate is equivalent to 13% error in total.