Three-dimensional vision

Objective

The student will review and analyze the state-of-the-art techniques in three-dimensional computer vision, the objective is to understand the functioning and the scope of the subject in order to apply the best one for real-world problems related to computer engineering.

[Español] El estudiante revisará y analizará las técnicas más recientes en la visión computacional tridimensional con el objetivo de comprender el funcionamiento y alcance de las mismas en problemas reclacionados con la tecnología de cómputo.

Requirements

• Obligatory
  • Linear algebra
  • Calculus
  • Object oriented programming,
  • Statistics
  • Pattern recognition
• Desirable:
  • Introduction to computer vision
  • Deep learning

Content

0. Background Recall
   1. Linear algebra
      1. Basic operations
      2. Partial derivatives
   2. 3D Solid object transformations
      1. Rotation matrices
      2. Rotation conventions
      3. Homogeneous transformations
      4. Quaternions
   3. Deep learning
      1. Perceptron
      2. Gradient descent
      3. Multi-layer perceptron (MLP)
      4. Back propagation
      5. Multinomial Logistic Classsifier
      6. Convolutional neural networks
1. Introduction
   1. The 3d world
2. Camera calibration
   1. Extrinsic and intrinsic parameters
3. Depth estimation
   1. Depth image
   2. Shape from X
   3. Stereo vision
   4. Multi-view geometry
   5. Data driven depth estimation
4. 3D representation
   1. Point clouds
   2. Polyhedrons
   3. Uniform grids
   4. Hierarchical grids
   5. Probabilistic grids
5. Model integration
   1. Registration
   2. Voxelization
   3. Bayes filter
6. Camera localization
   1. Feature points
   2. Kalman Filter
   3. Graph SLAM
   4. ORB SLAM
7. View Planning
   1. Model based view planning
      1. Global information
      2. TSP problem
   2. Next-best-view planning
      1. Greedy approaches
      2. Information gain
      3. Data driven next-best-view
8. Model completion
   1. 3D Autoencoder
   2. Surface inference
9. Semantic segmentation
   1. Unet-based segmentation
   2. Surfaces segmentation
   3. Tissue segmentation
10. 3D object recognition
    1. 3D-CNN recognition
    2. Dissease detection
    3. Next-best-view for recognition
11. Image based rendering
    1. Neural rendering

Bibliography

• Richard Hartley and Andrew Zisserman.Multiple View Geometry in Com-puter Vision. Cambridge University Press, 2 edition, 2004.
• Jeff Heaton. Ian goodfellow, yoshua bengio, and aaron courville: Deep learning, 2018.
• J. Irving Vasquez-Gomez, Planificación de Vsitas para Reconstrucción Tridimensional de Objetos con Robots Móviles, Tesis de doctorado, INAOE, 2014.
• Richard Szeliski. Computer vision: algorithms and applications. SpringerScience & Business Media, 2010.
• Sebastian Thrun. Probabilistic robotics.Communications of the ACM,45(3):52–57, 2002.
• Telea, A. C. (2014). Data visualization: principles and practice. CRC Press.

Papers

1. Niklaus, S., Mai, L., Yang, J., & Liu, F. (2019). 3D Ken Burns effect from a single image. ACM Transactions on Graphics (TOG), 38(6), 1-15.
2. Chen, R., Mahmood, F., Yuille, A., & Durr, N. J. (2018). Rethinking monocular depth estimation with adversarial training. arXiv preprint arXiv:1808.07528.
3. Besl, P. J., & McKay, N. D. (1992, April). Method for registration of 3-D shapes. In Sensor fusion IV: control paradigms and data structures (Vol. 1611, pp. 586-606). Spie.
4. Mendoza, Miguel and Vasquez-Gomez, J Irving and Taud, Hind and Sucar, Luis Enrique and Reta, Carolina, Supervised Learning of the Next-Best-View for 3D Object Reconstruction, Pattern Recognition Letters, (2020), https://doi.org/10.1016/j.patrec.2020.02.024, I.F. 2.810