Computer Vision Practice Sample Paper | Calculating the Parameters Of 3D Rotation And Translation

Problem Statement: In the class, you had seen how to estimate 2D transformations given a set of 2D point correspondences. In this assignment, you will work out how to calculate the parameters of 3D rotation and translation between two 3D point-sets given a subset of point correspondences.

This assignment has two parts.

In the first part, work out the mathematics of the estimation process. You could adopt any of the approaches outlined in Section 6.1.5 on 3D alignment in your textbook. Or you can generalize the 2D-2D matching approach we studied in class but focusing on 3D rotation transformation. Recall that this approach involves two steps. In the first step, you will form 3D affine estimates using linear least squares and QR decomposition. Using these affine estimates, you will develop an initial estimate of the 3D rotation parameters. In the second step, you will refine this initial estimate using Gauss-Newton (or Newton) non-linear optimization.

In the second part, you have to show that your approach works by coding it up (python is preferable) and demonstrating your estimation approach's accuracy using synthetic data. The steps for this empirical demonstration.

(i) Grab some 3D point cloud data from

https://github.com/intel-isl/Open3D/tree/master/examples/test_data

You can use Open3D for I/O - http://www.open3d.org/docs/latest/tutorial/geometry/file_io.html#Point-cloud

(ii) Randomly select a sparse set of 100 to 200 points spread across the scene of the chosen data. This will form the "train" set that you will use to estimate the 3D rotation and translation.

(iii) Pick a 3D rotation and 3D translation and apply it to the point cloud to create a second point cloud data. Randomly add noise to the 3D data. Note you know the correspondences between the two point-clouds.

(iv) Use the "train subset" of points to estimate the rotation and translation parameters. Compute the residual on the rest of the data points.

(v) Repeat steps (ii) - (iii) - (iv) with five different random train sets and random 3D rotation and translation parameters. Report the mean and the standard deviation of the residual error of the estimation.

(vi)Repeat step (v) 10 times with different amounts of noise in step (iii).

(vii) Report the results from step (vi) in a table.

Implementation

#import libraries
import open3d as o3d
import numpy as np

print("Testing IO for meshes ...")
mesh = o3d.io.read_triangle_mesh("color.ply")

print('Vertices:')
print(len(np.asarray(mesh.vertices)))

print('Triangles:')
print(len(np.asarray(mesh.triangles)))

len(mesh.triangles)

pcd = o3d.geometry.PointCloud()
xyz= np.array(mesh.vertices)[:100]
pcd.points = o3d.utility.Vector3dVector(xyz)
o3d.io.write_point_cloud("t100.ply", pcd)

pcd = o3d.geometry.PointCloud()
xyz= np.array(mesh.vertices)[100:200]
pcd.points = o3d.utility.Vector3dVector(xyz)
o3d.io.write_point_cloud("t200.ply", pcd)

pcd = o3d.geometry.PointCloud()
xyz= np.array(mesh.vertices)[200:300]
pcd.points = o3d.utility.Vector3dVector(xyz)
o3d.io.write_point_cloud("t300.ply", pcd)

pcd = o3d.geometry.PointCloud()
xyz= np.array(mesh.vertices)[300:400]
pcd.points = o3d.utility.Vector3dVector(xyz)
o3d.io.write_point_cloud("t400.ply", pcd)

pcd = o3d.geometry.PointCloud()
xyz= np.array(mesh.vertices)[400:500]
pcd.points = o3d.utility.Vector3dVector(xyz)
o3d.io.write_point_cloud("t500.ply", pcd)

import plotly.graph_objects as go
import copy
location1 = "t100.ply"

location2 = "t200.ply"
location3 = "t300.ply"
location4 = "t300.ply"
location5 = "t300.ply"

def run(location):
  pcd = o3d.io.read_point_cloud(location)
  alpha = 0.03
  colors=None
  print(f"alpha={alpha:.3f}")
  mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_alpha_shape(pcd, alpha)
  points = np.array(mesh.vertices)

  

  fig = go.Figure(
      data=[
          go.Scatter3d(
              x=points[:,0], y=points[:,1], z=points[:,2], 
              mode='markers',
              marker=dict(size=1, color=colors)
          )
      ],
      layout=dict(
          scene=dict(
              xaxis=dict(visible=False),
              yaxis=dict(visible=False),
              zaxis=dict(visible=False)
          )
      )
  )
  fig.show()

  print('translate')

  mesh_tx = copy.deepcopy(mesh).translate((0,0,0.1))
  mesh_ty = copy.deepcopy(mesh).translate((0,0,0.1))

  print(f'Center of mesh: {mesh.get_center()}')
  print(f'Center of mesh tx: {mesh_tx.get_center()}')
  print(f'Center of mesh ty: {mesh_ty.get_center()}')

  o3d.visualization.draw_geometries([mesh, mesh_tx, mesh_ty])

  mesh_r = copy.deepcopy(mesh) # change values belo for change in param
  R = mesh.get_rotation_matrix_from_xyz((0,0,0))
  print('rotate')
  print(R)
  mesh_r.rotate(R, center=(0,0,0))
  o3d.visualization.draw_geometries([mesh, mesh_r])

  print('create noisy mesh')
  mesh_in = mesh
  vertices = np.asarray(mesh_in.vertices)
  noise = 1
  vertices += np.random.uniform(0, noise, size=vertices.shape)
  mesh_in.vertices = o3d.utility.Vector3dVector(vertices)
  mesh_in.compute_vertex_normals()
  o3d.visualization.draw_geometries([mesh_in])

  print('filter with average with 1 iteration')
  mesh_out = mesh_in.filter_smooth_simple(number_of_iterations=1)
  mesh_out.compute_vertex_normals()
  o3d.visualization.draw_geometries([mesh_out])

  print('filter with average with 5 iterations')
  mesh_out = mesh_in.filter_smooth_simple(number_of_iterations=5)
  mesh_out.compute_vertex_normals()
  o3d.visualization.draw_geometries([mesh_out])

#run two three four multiple times
run(location1)
run(location1)
run(location1)
run(location1)
run(location1)

points = np.asarray(mesh.vertices)

import plotly.graph_objects as go

fig = go.Figure(
    data=[
        go.Scatter3d(
            x=points[:,0], y=points[:,1], z=points[:,2], 
            mode='markers',
            marker=dict(size=1, color=None)
        )
    ],
    layout=dict(
        scene=dict(
            xaxis=dict(visible=False),
            yaxis=dict(visible=False),
            zaxis=dict(visible=False)
        )
    )
)
fig.show()



def noisy(noise):
  print('create noisy mesh')
  print('amount of noise : ', noise)
  mesh = o3d.io.read_triangle_mesh("color.ply")
  mesh_in = mesh
  vertices = np.asarray(mesh_in.vertices)

  print('with noise = ',noise)
  vertices += np.random.uniform(0, noise, size=vertices.shape)
  mesh_in.vertices = o3d.utility.Vector3dVector(vertices)
  mesh_in.compute_vertex_normals()
  o3d.visualization.draw_geometries([mesh_in])

  print('filter with average with 1 iteration')
  mesh_out = mesh_in.filter_smooth_simple(number_of_iterations=1)
  mesh_out.compute_vertex_normals()
  o3d.visualization.draw_geometries([mesh_out])

  print('filter with average with 5 iterations')
  mesh_out = mesh_in.filter_smooth_simple(number_of_iterations=5)
  mesh_out.compute_vertex_normals()
  o3d.visualization.draw_geometries([mesh_out])

#Repeat step (v) 10 times with different amounts of noise in step (iii).

for n in [0,1,2,3,4,5,6,7,8,9,10]:
  noisy(n)

You can run above code in jupyter notebook to get result as per given requirement.

If you need any any other computer vision related assignment help then Contact Us at:

realcode4you@gmail.com

And get instant help with an affordable price.