## OpenCV - Rotation (Deskewing)

In a previous article I presented how to compute the skew angle of a digitized text document by using the Probabilistic Hough Transform. In the last article I presented how to compute a bounding box using OpenCV, this method was also used to compute the skew angle but with a reduced accuracy compared to the first method.

### Test Set

We will be using the same small test set as before:

The naming convention for those images is simple, the first letter stands for the sign of the angle (p for plus, m for minus) and the following number is the value of the angle.
m8.jpg has therefore been rotated by an angle of -8 degrees.

### Bounding Box

In this article I will assume we have computed the skew angle of each image with a good accuracy and we now want to rotate the text by this angle value. We therefore declare a function called deskew that takes as parameters the path to the image to process and the skew angle.

```void deskew(const char* filename, double angle) { cv::Mat img = cv::imread(filename, 0);   cv::bitwise_not(img, img);   std::vector<cv::Point> points; cv::Mat_<uchar>::iterator it = img.begin<uchar>(); cv::Mat_<uchar>::iterator end = img.end<uchar>(); for (; it != end; ++it) if (*it) points.push_back(it.pos());   cv::RotatedRect box = cv::minAreaRect(cv::Mat(points));```

This code is similar to the previous article: we load the image, invert black and white and compute the minimum bounding box. However this time there is no preprocessing stage because we want the bounding box of the whole text.

### Rotation

We compute the rotation matrix using the corresponding OpenCV function, we specify the center of the rotation (the center of our bounding box), the rotation angle (the skew angle) and the scale factor (none here).

` cv::Mat rot_mat = cv::getRotationMatrix2D(box.center, angle, 1);`

Now that we have the rotation matrix, we can apply the geometric transformation using the function warpAffine:

``` cv::Mat rotated; cv::warpAffine(img, rotated, rot_mat, img.size(), cv::INTER_CUBIC);```

The 4th argument is the interpolation method. Interpolation is important in this situation, when applying the transformation matrix, some pixels in the destination image might have no predecessor from the source image (think of scaling with a factor 2). Those pixels have no defined value, and the role of interpolation is to fill those gaps by computing a value using the local neighborhood of this pixel.
The quality of the output and the execution speed depends on the method chosen.

The simplest (and fastest) interpolation method is INTER_NEAREST, but it yields awful results:
.

There are four other interpolation methods: INTER_NEAREST, INTER_AREA, INTER_CUBIC and INTER_LANCSOZ4.
For our example those 4 methods yielded visually similar results.
The rotated image using INTER_CUBIC (bicubic interpolation):

### Cropping

We should now crop the image in order to remove borders:

``` cv::Size box_size = box.size; if (box.angle < -45.) std::swap(box_size.width, box_size.height); cv::Mat cropped; cv::getRectSubPix(rotated, box_size, box.center, cropped);```

As mentioned in the previous article, if the skew angle is positive, the angle of the bounding box is below -45 degrees because the angle is given by taking as a reference a "vertical" rectangle, i.e. with the height greater than the width.
Therefore, if the angle is positive, we swap height and width before calling the cropping function.

Cropping is made using getRectSubPix, you must specify the input image, the size of the output image, the center of the rectangle and finally the output image.
We use the original center because the center of a rotation is invariant through this transformation.
This function works at a sub-pixel accuracy (hence its name): the center of the rectangle can be a floating point value.

The cropped image:

To better understand the problem we have with positive angles, here what you would get without the correction:

We can immediately see that we just need to swap the height and the width of the rectangle.

### Display

This is a small demo so let's display the original image, the rotated image and the cropped image:

``` cv::imshow("Original", img); cv::imshow("Rotated", rotated); cv::imshow("Cropped", cropped); cv::waitKey(0); }```

That's it ! It's really simple to rotate an image with OpenCV !

• Your posts on skew/deskew are amazing. Thank you!

• Anonymous

I'm glad I could help !

• Murat Ozmanav

Hey Felix!
In the most recent version of OpenCV, the atomic variables that are CvArr and IPlImage , are mostly used in newer functions. When I tried to convert these data structures between them , I got runtime errors.
For example, in the computing skew function, I had to do this

cv::Mat MATT;
MATT = cvarrToMat(IMG,0,1,0); // which IMG is an IPlImage*

everything looks fine , but just after the execution of this function I receive error. And the application crashes. Then I changed the whole thing like that :

double Compute_Skew(IplImage* IMG)
{

double angle = 0.0;

IplImage* dst = cvCreateImage( cvGetSize(IMG), 8, 1 );
IplImage* color_dst = cvCreateImage( cvGetSize(IMG), 8, 3 );
CvMemStorage* storage = cvCreateMemStorage(0);
CvSeq* lines = 0;
int i;

cvErode(dst,dst,0,1);
cvCanny( IMG, dst, THRES0-25, THRES0+25, 3 );
cann = cvCloneImage(dst);

lines = cvHoughLines2( dst,
storage,
CV_HOUGH_STANDARD,
1,
CV_PI/180,
100,
30,
10 );

for( i = 0; i total,100); i++ )
{
float* line = (float*)cvGetSeqElem(lines,i);
float rho = line[0];
float theta = line[1];
CvPoint pt1, pt2;
double a = cos(theta), b = sin(theta);
double x0 = a*rho, y0 = b*rho;
pt1.x = cvRound(x0 + 1000*(-b));
pt1.y = cvRound(y0 + 1000*(a));
pt2.x = cvRound(x0 - 1000*(-b));
pt2.y = cvRound(y0 - 1000*(a));
cvLine( dst, pt1, pt2, CV_RGB(255,0,0), 3, 8 );

angle += atan2((double)(pt2.y - pt1.y),
(double)(pt2.x - pt1.x));
}

angle /= lines->total;

return angle;
}

Do you have these kind of newer functions to do the deskewing or rotating ?
or
Do you suggest using the cvMat structs?

Thank you,

• Anonymous

Hello,

Unfortunately I have never used the C API of OpenCV so I cannot help you with the error you are encountering. But are you sure of the parameter you have used for 'cvarrToMat' ? Have you tried using the default parameters ? i.e. just 'cvarrToMat(IMG)'.

Personally I prefer to use the C++ interface, I think it's easier to write code with it.

• Alice Carli

I am trying to find a program to deskew music scores, and specifically one that will do so without altering image size. I do not want recropping and am not worried about borders. I am very frustrated because since 2010 all deskew programs seem to want to crop. Can you help, or even write one? We would pay for one...

• Edii

Are you still looking for one? I have might have an online one ready.

• laconcombremasque

dear Edii, I have the same needs as Alice : "I am trying to find a program to deskew music scores" automatically. Such scores come in multi-page pdfs, or more rarely in separate image files. So I am very interested in your online possibility ! Thanks

• Edii

@laconcombremasque:disqus I wrote a program to deskew pdf text documents and 2D CAD drawings. Deskewing music scores should be between those two. Email me @ admin@s3zipper.com with a few samples(3-5) and specifications . I couldn't find enough samples to test my program but the accuracy was about 98%.

• Edii

Just in case you are wondering about that online possibility. The website is http://www.autodeskew.com. We are working on a few kinks here and there but we should be able to deskew text, CAD drawings and music sheets accurately now. Drop by there and send us an email

• Pingback: propecia()

This is a good article. Very helpful. Is this method based on literature books or your own experiments?

• Bart Kappenburg

There is a mistake in your code.

In this expression:

cv::Mat rot_mat = cv::getRotationMatrix2D(box.center, angle, 1);

angle is defined in degrees, your previous code (detect skew angle) returns the angle in radians.

Took me an hour to find out. 😉

• elham

hi . i want to transform to matlab code. can you send matlab code above for me.

• Eddie Bole

Is there anyway to make an active X of this CV deskew code? I would like to make use of it in a VB6 program.

• Robel Alazar

anybody with C# code for the above source code

• Mayank

It helped me alot in understanding the concepts. But the only problem is i am not getting the deskewed images. Though i am getting the right values of decesion angles. Thanks again.

• Moussa Kecibi

Very good and helpfull it thx ... I need your help again how to get ride of noise