图像失真与透视变换

失真(distortion)

​ 图像失真通常发生在摄像机将3D转为2D时。失真改变了3D物体实际的大小形状。

失真分类
  • 由于镜头边缘产生的弯曲distorts the edges of images,称为径向失真(radial distortion)

  • 另一种称为切向失真(tangential distortion) 产生于摄像头为与成像平面平行,使得图像中的部分之间的距离变得异常的进或远。

失真的系数和校正

​ ————(Coefficients and Correction)

校正径向失真图片的系数 $k_1,k_2,k_3$ , $(x,y)$是失真图中的点,$r=dis( (x_{corrected},y_{corrected}),(x_c, y_c))$ ,前者是校正后图中的点,后者是校正后图片的中心也是相机的光学中心(distortion center)

  • 在失真图像的系数中$k_3$ 反映了主要的径向失真,但通常相机的镜头产生较小的径向失真,故忽略不计该参数

下面是两种失真的校正表达式

radial distortion :$\left\{\begin{matrix}
x_{corrected}=x(1+k_1r^2+k_2r^4+k_3r^6) \\
y_{corrected}=y(1+k_1r^2+k_2r^4+k_3r^6)\end{matrix}\right.
$

tangential distortion: $\left\{\begin{matrix} x_{corrected}=x+[2p_1xy+p_2(r^2+2x^2)] \\ y_{corrected}=y+[2p_2xy+p_1(r^2+2y^2)]\end{matrix}\right.$

Finding Corners 探测内角

​ use OpenCV functions findChessboardCorners() and drawChessboardCorners() to automatically find and draw corners in an image of a chessboard pattern.

#-*- coding: utf-8 -*-
import cv2
import pylab
import matplotlib.pyplot as plt

Size = (row,col) # 内点(黑白相间的点)占据的Size
fname = 'calibration_test.img' 
#if any error,use the full path of image with forward slash.
img = cv2.imread(fname) # read colorful img
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # corlorful img to Gray img
ret, corners = cv2.findChessboardCorners(gray, Size, None)
if(ret == True): # 返回是否在灰度图中中检测到内点尺寸为Size的棋盘
  cv2.drawChessboardCorners(img, Size, corners, ret)#在棋盘上(原图)标注corners
cv2.imwrite('ans.jpg', img)# 覆盖保存原图
plt.imshow(img)
pylab.show()

Camera Calibration 相机校准

相机内参

1. cameraMatrix

​ 焦距$(f_x,f_y)$ ,光学中心$(c_x,c_y)$ 表示成摄像机Matrix =$\begin{bmatrix} f_x&0 &c_x \\0 &f_y &c_y \\ 0&0 &1\end{bmatrix}$

2. distCoeffs 失真系数向量$(k_1,k_2,p_1,p_2[,k_3[,k_4,k_5,k_6]])$

​ 通过cameraMatrix和distCoeffs将世界坐标系转换到摄像机坐标系

相机外参

​ 外参与旋转和平移矩阵(rotation-translation matrix)相对应,用来将点的坐标固定成相机坐标也就是在相机中确定坐标轴,将3D点的坐标转换到坐标系统中

​ 3D点称为对象点(世界坐标系),2D点称为图像点(平面坐标系)

透视变换

使用透视变换将3D点投影到图形平面中形成场景视图:

​ $$s{m}’=A[R|t]{M}’$$

or

​ $s\begin{bmatrix} u \\ v \\ 1 \end{bmatrix}=\begin{bmatrix} f_x&0 &c_x \\0 &f_y &c_y \\ 0&0 &1\end{bmatrix}\begin{bmatrix} r_{11}&r_{12} &r_{13}&t_1 \\r_{21} &r_{22} &r_{23}&t_2 \\ r_{31}&r_{32} &r_{33}&t_3\end{bmatrix} \begin{bmatrix} X \\ Y \\ Z \\ 1 \end{bmatrix}$

  • $(X,Y,Z)^T$是3D坐标
  • $(u,v)^T$是以像素为单位的投影点的坐标,2D 坐标
  • $A$是相机内在参数矩阵
  • $(c_x,c_y)$通常是在相机成像中心
  • $f_x,f_y$是以像素为单位的镜头焦距
  • $[R|t]是$旋转平移外部参数矩阵

calibrateCamera参数解释

IN: objectPoints, imagePoints, imageSize, cameraMatrix, distCoeffs

OUT: retval, cameraMatrix, distCoeffs, rvecs, tvecs

  • distCoeffs 失真系数向量
  • rvecs 每一个视图的旋转向量(3*1)
  • tvecs 每一个视图的平移向量(3*1)
  • retval 误差均方根root mean square(RMS)

获取2D坐标code:

#prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((6<em>8,3), np.float32) # 世界坐标系48</em>3矩阵
objp[:,:2] = np.mgrid[0:8, 0:6].T.reshape(-1,2) # 前两列插入世界坐标系(x,y,z), all Z=0
#Arrays to store object points and image points from all the images.
objpoints = [] # 3d points in real world space
imgpoints = [] # 2d points in image plane.
#Make a list of calibration images
images = glob.glob('calibration_wide/GO*.jpg')
#Step through the list and search for chessboard corners
for idx, fname in enumerate(images): #enimerate 返回索引和值
    img = cv2.imread(fname)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #转为灰度图
    #Find the chessboard corners #找到内角
    ret, corners = cv2.findChessboardCorners(gray, (8,6), None)
    #If found, add object points, image points
    if ret == True:
        objpoints.append(objp)
        imgpoints.append(corners)
       #Draw and display the corners
        cv2.drawChessboardCorners(img, (8,6), corners, ret)
        #write_name = 'corners_found'+str(idx)+'.jpg'
        #cv2.imwrite(write_name, img)
        cv2.imshow('img', img)
        cv2.waitKey(500)
#print len(imgpoints)
cv2.destroyAllWindows()

畸变校正

fname = './calibration_wide/GOPR0054.jpg'
objp = np.zeros((6 * 8, 3), np.float32)
objp[:,:2] = np.mgrid[0:8, 0:6].T.reshape(-1,2)
objpoints = [] # 3D
imgpoints = [] # 2D

img = cv2.imread(fname)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_size = (img.shape[0], img.shape[1]) # ret height and width
# img_size = img.shape[:2]

ret, corners = cv2.findChessboardCorners(gray, (8, 6), None)
if ret == True:
        objpoints.append(objp)
        imgpoints.append(corners)
        cv2.drawChessboardCorners(img, (8,6), corners, ret)
        #write_name = 'corners_found'+str(idx)+'.jpg'
        #cv2.imwrite(write_name, img)
        #cv2.imshow('img', img)
        #cv2.waitKey(50)

# 相机校准
ret,mtx,dist,rvecs,tvecs=cv2.calibrateCamera(objpoints,imgpoints,img_size, None, None) 


dst = cv2.undistort(img, mtx, dist, None, mtx)
cv2.imwrite('calibration_wide/test_undist.jpg', dst)

dist_pickle = {}
dist_pickle["mtx"] = mtx
dist_pickle["dist"] = dist
pickle.dump( dist_pickle, open( "calibration_wide/wide_dist_pickle.p", "wb" ) )
# 将获取到的相机内参dist_pickle永久保存到wide_dist_pickle.p中
#dst = cv2.cvtColor(dst, cv2.COLOR_BGR2RGB)

# Visualize undistortion
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
ax1.imshow(img)
ax1.set_title('Original Image', fontsize=30)
ax2.imshow(dst)
ax2.set_title('Undistorted Image', fontsize=30)

pylab.show()

效果图:TIM截图20171113162351

扭曲与校正图像

#Read in the saved camera matrix and distortion coefficients 使用上文已经提取的相机内参

#These are the arrays you calculated using cv2.calibrateCamera()
dist_pickle = pickle.load( open( "wide_dist_pickle.p", "rb" ) )
mtx = dist_pickle["mtx"]
dist = dist_pickle["dist"]

#Read in an image
img = cv2.imread('GOPR0038.jpg')
nx = 8 # the number of inside corners in x
ny = 6 # the number of inside corners in y

#MODIFY THIS FUNCTION TO GENERATE OUTPUT
#THAT LOOKS LIKE THE IMAGE ABOVE
def corners_unwarp(img, nx, ny, mtx, dist):
    # 1) Undistort using mtx and dist
    udst = cv2.undistort(img, mtx, dist, None, mtx)
    #2) Convert to grayscale
    gray = cv2.cvtColor(udst,cv2.COLOR_BGR2GRAY)
    #3) Find the chessboard corners
    ret, corners = cv2.findChessboardCorners(gray, (nx,ny), None)
    #4) If corners found:
    warped, M = None, None
    if ret == True:
        # a) draw corners
        udst = cv2.drawChessboardCorners(udst, (nx,ny), corners, ret)
        #b) define 4 source points src = np.float32([[,],[,],[,],[,]])
        #定义源点来自棋盘四个角
        src = np.float32([corners[0],
                          corners[nx - 1],
                          corners[ny*nx - nx],
                          corners[ny*nx -1]])
        #c) define 4 destination points dest = np.float32([[,],[,],[,],[,]])
        #定义目标点
        #来源真实图像的四个点
        h, w = img.shape[0], img.shape[1]
        dest = np.float32([[100,100],[w-100,100],[100,h-100],[w-100,h-100]])
        #d) use cv2.getPerspectiveTransform() to get M, the transform matrix
        M = cv2.getPerspectiveTransform(src, dest)
        #e) use cv2.warpPerspective() to warp your image to a top-down view
        warped = cv2.warpPerspective(udst, M, (w,h), flags=cv2.INTER_LINEAR)
    return warped, M

top_down, perspective_M = corners_unwarp(img, nx, ny, mtx, dist)
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(24, 9))
f.tight_layout()
ax1.imshow(img)
ax1.set_title('Original Image', fontsize=50)
ax2.imshow(top_down)
ax2.set_title('Undistorted and Warped Image', fontsize=50)
plt.subplots_adjust(left=0., right=1, top=0.9, bottom=0.)

pylab.show()

效果图:TIM截图20171113162351


发表评论

电子邮件地址不会被公开。 必填项已用*标注

This site uses Akismet to reduce spam. Learn how your comment data is processed.