OpenCV/Minutes 2009-07-02

Agenda

• Status, code/freeze. go through summer release list below
  • Beta release
• FLANN from/by Marius
  • Marius's Chamfer matching
• High GUI interface
• New descriptors from bazAR tracking package
• Bundle adjustment status?
• HOG training
• One way descriptor -- Victor
• User contribution

• (Post Release)

  • Alex Teichman's feature framework
    • His "kernel based" boosting framework
  • Optimized pipelines

OpenCV Summer Release:

• Feature freeze end of ... sometime in July for test and doc.
• Release in August, 2009
• General Structure:

  • ? Provide way to build OpenCV statically, then the footprint will be variable and usually much smaller than now

• New Features

  • C++ interface

    • cxcore

    • cv

    • HighGUI (except for video I/O

  • Better python interface

    • In progress thanks to (James Bowman)

    • Use this to expand unit test functions for OpenCV

  • 2D features

    • HOG

      • Chamfer matching with orientation In progress thanks to Marius Muja

    • Keypoints (mostly thanks to LePetit group's great work in this area)

      • LePetit keypoint detector YAPE (Yet Another Point Extractor). We have found this to be the most stable detector.

      • C-descriptor ... (Vadim)

      • LePetit one way descriptor (Victor) (to Anatoly)

    • MSER

    • Star

    • Self-similar detector

  • Alpha channel, alternatives to contours RLE

    • New alpha transparency (per pixel) added

    • Introduce functions with advance ways of blending

    • Mask channels

  • 3D features

    • Bundle Adjustment (probable contribution by Phase Space)

    • Plane fit (delayed for integration with Radu's point cloud library -- more comprehensive approach)

    • PFH, FPFH from Radu,

    • 3D Delaunay,

    • 3D convex hull,

    • mesh generation,

    • normals in grid (Gary)

    • normal based 3D patch (Gary)

    • SPIN

    • 3D capture http://www.3dsom.com/index.html

    • VSLAM

    • Image stitching

  • Optical flow

    • Dense (Farnerback)

    • Variational optical flow

  • MLL

    • Extreme trees

    • Reduce memory footprint

    • More convenient training via a data class

    • Nister trees (Patrick)

    • FLANN or appropriate parts (Marius)

    • LePetit style random trees

    • Generalized (boosted) cascade (generalizing Haar cascade)

• Misc:

  • Focus detector,

  • adaptive skin tracking

• Optimizations

  • Various functions, ongoing ...

• Documentation

  • Move to one Latex source

    • Use to generate HTML, PDF

  • Add search function

  • Add user feedback/documentation bug system

  • Proof read

  • Add content

  • Create C++ documentation

  • Create Python documenation

• High GUI

  • In memory .jpg, .png

  • move to C++ interface

  • move to using QT as the HighGUI interface look and feel

• Testing functions

  • Regression tests for all functions at least in python->OpenCv to make these easier to write

• Define user contribution process:

Minutes

Vadim

1. c-descriptor has been debugged (thanks to Rainer for the code review and useful recommendations on what could be wrong). It now shows comparable with SURF localization accuracy, provided that the object scale in the scene is specified more or less correctly. Much better robustness can be achieved by using Lepetit's YAPE (Yet Another Point Extractor) detector - see item 2 below. Perhaps, it makes sense to make more universal class than just C-descriptor. The class can easily combine randomized trees-based classifier (as in the original paper by Lepetit from 2006), Michael Calonder's c-descriptor, ferns-based (a.k.a. semi-naive Bayesian) classifier and the corresponding descriptor. The techniques used are very similar, and in the papers ferns are claimed to provide better accuracy than randomized trees.
2. We did some experiments with open-source (GPL) bazAR tracking package from cvlab.epfl.ch site and the results are very promising - the keypoint detector constantly finds the same points on the quite different views of the same object, i.e. the stability is great! Unfortunately, we could not make the multi-scale variant of the detector work, but the idea is clear, and, I think, we can do independent implementation pretty quickly.
3. Maria added LBP features to the new "traincascade" application (the substitute for haartraining). It worked well and now we are adding the corresponding detection function to OpenCV (for now it will be sort of reference code w/o any serious optimization).
4. Generation of normally distributed random numbers is now ~3x faster with Ziggurat algorithm (it can be useful for faster keypoint detection and classifier training using Lepetit method, as they add uniform or gaussian noise to the generated patches (and seem to prefer the gaussian noise))
5. Several bugs have been fixed: 1) build issues on Linux when using autotools, 2) precompiled headers did not work well on Linux, 3) crash in calcHist (reported by Alex Teichman), 4) crash in mltest, found and fixed by Maria

Discussion Minutes

• Stable keypoint detection. Lepetit's group
  • Run keyboard detector, used pyramid, maximal Laplacian detection with edge suppression
    • Repeat procedure 100-1000 times using affine distortion (know where the correspondences are)
    • Find most stable keypoints that are stable to affine distortion
    • Train classifiers using forest or ferns
      • Can be GPU'd (Anatoly knows for rendering) OpenCL Intel/NVidia, CPI, AMD
      • Suitable for SSE for image warping
    • One way descriptor uses multiple points in kd-tree, others use patches in random forests/ferns
    • In bazAR there are several demos, keypoint detection and training in randomized trees
  • We have c-descriptor code but bazAR performs better, maybe it will work the same with the better key points above
  • Will finish and commit to SVN this week
  • C descriptors are in
• FLANN
  • Marius have commit privilege
  • In 3rd party directory
    • Documentation to be melded in ...
• Alex Tiechman sent in feature pipeline for architectural feedback
  • We may consider GPU of high payoff features found with this
• HighGUI
  • Probably use QT (LGPL) ~GTK
  • Do it after release
• Bundle Adjustment
  • Sergey said that it will be ready this Friday. Debugging
  • Initialization might not be included, just pure optimization
    • Will include unit tests
  • To run, you need to provide initial estimates for camera intrinsic and code
    • Documentation will describe
  • For complete 3D pipeline we are missing
    • Initialization
    • Bundle adjustment
    • Verification and
    • Mesh reconstruction
    • Probably only bundle adjustment will go into Summer release. Anatoly working on

      • Waits on the robust keypoints from LePetit's group

      • LePetit's group uses larger solution ImageModeler (Autodesk product)

• HOG
  • Put the training code for now to OpenCV extra
  • Coefficients for human detection are shipped
• Python
  • If interface is C++
  • We can adjust James scripts to handle new functions
    • New functions create output images will be created or resized if haven't been created before

• For BETA, we want

  • C-descriptor

  • LePetit's affine robust YAPE keypoint detector

  • improved training cascade classifier (training cascade supporting haar and other featuers
  • Self-similarity feature
  • Brief documentation for C++ interface
  • ~ FLANN Integration
  • Bundle adjustment

• For Release

  • Make Python interface for C++ functionality
  • FLANN
  • Finish documentation
  • Write python tests
  • Chamfer matching
  • One way descriptor

Action Items

Gary

• Ask Marius about chamfer matching with orientation

  • Documentation ? LaTex into OpenCV

• Look into purchasing ImageModeler

• Talk with James about interface to C++ code

PREVIOUSLY

Gary

• Ask about Nistor Trees

• Have Marius contribute Chamfer+orientation

• Talk to Marius about contributing FLANN

• PUT IN CONTRIBUTORS DIRECTORY INTO WIKI

• Talk with Ian Goodfellow about bugs in HOG training code

• See if we can get Balan's matlab code for symmetric object 3D capture

• Ask Scott about voting site for user contribution

Vadim

• Find out about user contribution website similar to Matlab style

Not done:

• 3D features
• Nistor Tree
• Figure out code contribution process
• Fast Chamfer with orientation

• OpenCV

ProjectMeetingCategory