Image Stitching

Sorry for not give any update on this blog. Yeah, I am still busy with courses, homework and project and also exams. Let’s discuss about Image Stitching. This is very common Computer Vision Project, which is my CV homework too. But, it is okay to keep discuss about it because the popularity has risen recently. Especially after it is implemented in most smartphones.

What is Image Stitching?

Image Stitching is the process of combining multiple images with overlapping views to produce a wide-angle panorama. Although this combines many images into one, it combines two images at a time. For gives the clear definition I would like to give an example.

Suppose we are given 3 images

Building 1

Building 1

Building 2

Building 2

Building 3

Building 3

those 3 images are look same, just a bit different. Yes, it indeed captures the same object. I am going to jump to the result. The result, roughly like the following

Result Building

Result Building

This result is produced by my own code. It’s very rough and far from perfect. But, I wish it helps to point out what I mean.

How to do Image Stitching?

In my perspective, it is not easy doing Image Stitching. But, it can be easy if we use library (like opencv). Although we use library we need to know the steps.

Feature Extraction

The first step in order to stitch images is Feature Extraction. What is a feature? the rough definition is to identify an interesting point in an image. There is more than one algorithm to do this, like Harris Corners Detector and SIFT. In my implementation I used SIFT with OpenCV.

Feature Matching

After we get some feature between two images. We need to correlate them. How do we do that? It is called feature matching. There is more than one algorithm. In example maximum correlation rule, KNN and Flann. In my experiment this step takes time, of course, depend on the images. If images have many features, it will take more time. In my implementation I used KNN (k=1) without library (by my hand).

Homography matrix estimation

Now, we have two sets correlated points. All we have to do is define a model which is can translate a point from one image to another. In implementation level it simply just calculates a matrix transformation. But, behind that, it’s contents complicated geometry mathematic concept. Here is one of the best explanations about homography matrix.

RANSAC

“Random sample consensus (RANSAC) is an iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers” from Wikipedia. It is the iterative method to select four match points that fulfill the threshold, which is those four match points use for estimate homography matrix. For more detail about RANSAC please refer to the following articles.

RANSAC Algorithm in Stackoverflow
RANSAC Algorithm
RANSAC for Dummies

Blending

After we have the homography matrix, it’s almost done. We simply map the point from one image to another. In implementation phase just simply multiply the point with the matrix homography.

Result

Here, the result using my own code.

Mansion 1

Mansion 1

Mansion 2

Mansion 2

Feature 1

Feature 1

Feature 2

Feature 2

Mansion Result

Mansion Result

Some summary about my code.

– Feature Extraction: SIFT
– Feature Matching: KNN
– Homography Estimation
– RANSAC (params: 30 % match & 0.3 pixel error.)

Sorry I have not been able to publish my code for now.

Leave a Reply

Your email address will not be published. Required fields are marked *