OpenCV - Rotation (Deskewing)

In a previous article I presented how to compute the skew angle of a digitized text document by using the Probabilistic Hough Transform. In the last article I presented how to compute a bounding box using OpenCV, this method was also used to compute the skew angle but with a reduced accuracy compared to the first method.

Test Set

We will be using the same small test set as before:

The naming convention for those images is simple, the first letter stands for the sign of the angle (p for plus, m for minus) and the following number is the value of the angle.
m8.jpg has therefore been rotated by an angle of -8 degrees.

Bounding Box

In this article I will assume we have computed the skew angle of each image with a good accuracy and we now want to rotate the text by this angle value. We therefore declare a function called deskew that takes as parameters the path to the image to process and the skew angle.

void deskew(const char* filename, double angle)
{
  cv::Mat img = cv::imread(filename, 0);
 
  cv::bitwise_not(img, img);
 
  std::vector<cv::Point> points;
  cv::Mat_<uchar>::iterator it = img.begin<uchar>();
  cv::Mat_<uchar>::iterator end = img.end<uchar>();
  for (; it != end; ++it)
    if (*it)
      points.push_back(it.pos());
 
  cv::RotatedRect box = cv::minAreaRect(cv::Mat(points));

This code is similar to the previous article: we load the image, invert black and white and compute the minimum bounding box. However this time there is no preprocessing stage because we want the bounding box of the whole text.

Rotation

We compute the rotation matrix using the corresponding OpenCV function, we specify the center of the rotation (the center of our bounding box), the rotation angle (the skew angle) and the scale factor (none here).

  cv::Mat rot_mat = cv::getRotationMatrix2D(box.center, angle, 1);

Now that we have the rotation matrix, we can apply the geometric transformation using the function warpAffine:

  cv::Mat rotated;
  cv::warpAffine(img, rotated, rot_mat, img.size(), cv::INTER_CUBIC);

The 4th argument is the interpolation method. Interpolation is important in this situation, when applying the transformation matrix, some pixels in the destination image might have no predecessor from the source image (think of scaling with a factor 2). Those pixels have no defined value, and the role of interpolation is to fill those gaps by computing a value using the local neighborhood of this pixel.
The quality of the output and the execution speed depends on the method chosen.

The simplest (and fastest) interpolation method is INTER_NEAREST, but it yields awful results:
.

There are four other interpolation methods: INTER_NEAREST, INTER_AREA, INTER_CUBIC and INTER_LANCSOZ4.
For our example those 4 methods yielded visually similar results.
The rotated image using INTER_CUBIC (bicubic interpolation):

Cropping

We should now crop the image in order to remove borders:

  cv::Size box_size = box.size;
  if (box.angle < -45.)
    std::swap(box_size.width, box_size.height);
  cv::Mat cropped;
  cv::getRectSubPix(rotated, box_size, box.center, cropped);

As mentioned in the previous article, if the skew angle is positive, the angle of the bounding box is below -45 degrees because the angle is given by taking as a reference a "vertical" rectangle, i.e. with the height greater than the width.
Therefore, if the angle is positive, we swap height and width before calling the cropping function.

Cropping is made using getRectSubPix, you must specify the input image, the size of the output image, the center of the rectangle and finally the output image.
We use the original center because the center of a rotation is invariant through this transformation.
This function works at a sub-pixel accuracy (hence its name): the center of the rectangle can be a floating point value.

The cropped image:
Cropped
To better understand the problem we have with positive angles, here what you would get without the correction:


We can immediately see that we just need to swap the height and the width of the rectangle.

Display

This is a small demo so let's display the original image, the rotated image and the cropped image:

  cv::imshow("Original", img);
  cv::imshow("Rotated", rotated);
  cv::imshow("Cropped", cropped);
  cv::waitKey(0);
}

That's it ! It's really simple to rotate an image with OpenCV !

  • http://twitter.com/karlphillip Karl Phillip Buhr

    Your posts on skew/deskew are amazing. Thank you!

    • Anonymous

      I'm glad I could help !

  • Murat Ozmanav

    Hey Felix!
    In the most recent version of OpenCV, the atomic variables that are CvArr and IPlImage , are mostly used in newer functions. When I tried to convert these data structures between them , I got runtime errors.
    For example, in the computing skew function, I had to do this

    cv::Mat MATT;
    MATT = cvarrToMat(IMG,0,1,0); // which IMG is an IPlImage*

    everything looks fine , but just after the execution of this function I receive error. And the application crashes. Then I changed the whole thing like that :

    double Compute_Skew(IplImage* IMG)
    {

    double angle = 0.0;

    IplImage* dst = cvCreateImage( cvGetSize(IMG), 8, 1 );
    IplImage* color_dst = cvCreateImage( cvGetSize(IMG), 8, 3 );
    CvMemStorage* storage = cvCreateMemStorage(0);
    CvSeq* lines = 0;
    int i;

    cvErode(dst,dst,0,1);
    cvCanny( IMG, dst, THRES0-25, THRES0+25, 3 );
    cann = cvCloneImage(dst);

    lines = cvHoughLines2( dst,
    storage,
    CV_HOUGH_STANDARD,
    1,
    CV_PI/180,
    100,
    30,
    10 );

    for( i = 0; i total,100); i++ )
    {
    float* line = (float*)cvGetSeqElem(lines,i);
    float rho = line[0];
    float theta = line[1];
    CvPoint pt1, pt2;
    double a = cos(theta), b = sin(theta);
    double x0 = a*rho, y0 = b*rho;
    pt1.x = cvRound(x0 + 1000*(-b));
    pt1.y = cvRound(y0 + 1000*(a));
    pt2.x = cvRound(x0 - 1000*(-b));
    pt2.y = cvRound(y0 - 1000*(a));
    cvLine( dst, pt1, pt2, CV_RGB(255,0,0), 3, 8 );

    angle += atan2((double)(pt2.y - pt1.y),
    (double)(pt2.x - pt1.x));
    }

    angle /= lines->total;

    return angle;
    }

    Do you have these kind of newer functions to do the deskewing or rotating ?
    or
    Do you suggest using the cvMat structs?

    Thank you,

    • Anonymous

      Hello,

      Unfortunately I have never used the C API of OpenCV so I cannot help you with the error you are encountering. But are you sure of the parameter you have used for 'cvarrToMat' ? Have you tried using the default parameters ? i.e. just 'cvarrToMat(IMG)'.

      Personally I prefer to use the C++ interface, I think it's easier to write code with it.

  • Pingback: Executing cv::warpPerspective for a fake deskewing on a set of cv::Point | Question and answer

  • Pingback: Executing cv::warpPerspective for a fake deskewing on a set of cv::Point

  • Alice Carli

    I am trying to find a program to deskew music scores, and specifically one that will do so without altering image size. I do not want recropping and am not worried about borders. I am very frustrated because since 2010 all deskew programs seem to want to crop. Can you help, or even write one? We would pay for one...

  • Pingback: propecia

  • Pingback: Executing cv::warpPerspective for a fake deskewing on a set of cv::Point | BlogoSfera

  • Muhammad Akbar Yasin

    This is a good article. Very helpful. Is this method based on literature books or your own experiments?

  • Bart Kappenburg

    There is a mistake in your code.

    In this expression:

    cv::Mat rot_mat = cv::getRotationMatrix2D(box.center, angle, 1);

    angle is defined in degrees, your previous code (detect skew angle) returns the angle in radians.

    Took me an hour to find out. ;-)

  • Pingback: android – Rotate image in opencv depending on skew angle…! | TechwikiHow

  • Pingback: Executing cv::warpPerspective for a fake deskewing on a set of cv::Point | Ask Programming & Technology

  • Pingback: OpenCV Rotation (Deskewing) in Android – C++ to Java Conversion