OpenCVMeetingNotes/Minutes 2009-12-08

Agenda

Twitter needs to tweet more
- Every week, let's say Tuesday, the "plans" should go there with names
- Then people should say what they are working on every day, what they plan to do
  - Should I have an account on the opencv twitter?
A function is complete when:
- It is coded (obviously)
- It has documentation
- It has good conditional test coverage
- It has sample code, which might be the same as test coverage if it is well explained, but might be better if it is kept very simple.
  - A non-author should try to use the function from the documentation
  - For priority Willow code, we might want a "user test" here.
- We need to collect a priority list of functions that should be at full compliance
  - I sent out a post to Willow
    - Functions Willow is using/wants to/plans to use (Calibration, Stereo, Homography, Affine, GrabCut, Calonder, FAST, background, things in support of visual odemetry ... even Focus ...)
  - Core image processing functions (arithmetic, logical, matrix ops, matrix reference counting,
  - Basic cv functions (Canny, Optical Flow, flood fill, convolution, edge detect, blur, threshold, morphology etc)
  - Basic learning: (Cascade, Main techique in FLANN, K-NN, k-means)
  - Others?
- We will then make priority tiers and some basic time lines
All out focus this on the calibration tickets and documentation (about half the tickets might be documentation there?)
- Vadim, take what resources you need including myself and Victor if we can be useful in this.
Adding circular element calibration patterns (What Victor is working on here)
Need everyone's email ... so I can remind them if they don't twitter enough. I will create an internal willow opencv mailing list so that willow people can mail the team directly.
NVidia statement of work
Some comments about future strategy

Minutes

People will tweet more often (1-2x/day at least) and tell them to use it informally ... just what are you doing.
New functions should have a non-author use the documentation before the documentation is considered complete
Next release of code
- May sometime (for new functions and hardened code)
  - Put out intermediate release in Feb sometime that has mostly fixes, docs, tests
Priority functions
- Obvious simple ones such as
  - Image arithmetic, logic, creation/destruction
  - Cascade, Main techique in FLANN, K-NN, k-means
  - Canny, Optical Flow, flood fill, convolution, edge detect, blur, threshold, morphology etc
  - Willow ones: Calonder, one way, stereo, calibration ...
  - Come up with candidate lists of priority functions by Monday
Calibration:
- Victor to help with documentation
- Change function to allow control of principle and focal point
- Simple of way of getting min and max
  - 0: only good section, 1: all, ROI returned of entire and can pass values in between

Calibration pattern with circular elements

Victor wants to change template matching in OpenCV in general
- Feature detection and template matching as separate functions
  1. Separate the topology mapper from the blob detector
    - Make this modular
    - Submit point matcher to OpenCV
    - Separate topology matcher
    - Generalized Hough transform
    - Geometric hashing
  2. Make a very stable detector
    - Look at the SRI detector

Draft a research plan going forward after Calibration

NVidia

Can we get large speed up for HOG on GPU?
Stereo is a clear win on GPU however, that should probably be first
- Correlation based
- Stereo (Stereo Processing by Semi-Global Matching and Mutual Information Hirschmuller, improving correlation-based algorithm with projected light). Might want to switch to stereo correlation in opencv
Then put this together in an object segmentation/recognition framework to be inivestitgate

Report

several new tests have been written, several old tests have been extended:
- new sparse optical flow tests, extended Farneback test [Anatoly]
  new SURF & StarDetector tests - the repeatability is checked [Anatoly] new tests for LSH (locally sensitive hash), PCA, reduce [Maria] 2 new MLL tests: knearest, EM [Maria]
  new simple regression test of stereo calibration & rectification, based on samples/c/stereo_calib.cpp sample [Vadim] extended tests to cover C++ API: Canny, floodfill, Rodrigues, template matching, color conversion [Vadim]

The current coverage chart is attached.

Fixed 6 bugs in the code and the documentation [Vadim]:
- trac ## 7, 8, 11.
  other bugs have been reported in opencvlibrary-devel or e-mailed to me (build on Win64, potential crash in SparseMat::convertTo, uninitialized memory read in bitwise_not).
Calonder descriptor is now in cvaux [Alexey]

Implemented draft version of the function cv::getOptimalNewCameraMatrix that should solve all the mentioned difficulties with OpenCV undistortion functions [Vadim] The spec is:

 Mat getOptimalNewCameraMatrix( const Mat& cameraMatrix, const Mat& distCoeffs,
                                          const Mat& R, Size imgSize,
                                          double minFScale, Size maxImgSize,
                                          Size& newImgSize);

(A similar function will be added to C API as well)

The function takes the estimated camera matrix, distortion parameters and the optional rectification transform R, as well as the original image size. It computes the optimal new camera matrix and the new image size, so that the whole undistorted image fits the new image rectangle, or some part of it.
Some pictures from single-head camera are attached.
- wide_angle_calibration_orig.jpg - original frame from wide-angle camera (thanks to Victor for the image and the associated camera data)
- undistort_small.jpg - the image resolution is kept the same, and the new focal length is computed so that the image fits entirely.
- undistort_large.jpg - the image resolution is increased by 20% at max, while the new focal lengths fx, fy are chosen close to the original ones.
- any other reasonable variant is possible with the function too (e.g. keep the focal length intact, make the image as big as necessary to fit the undistortion result).
One stereo camera the function is being debugged.

Plans:

continue to increase cxcore & cv coverage.
add James' Python tests into the regression testing loop
close urgent other tickets at Trac and SF bug tracker.

Original View:

Large View:

Narrow View:

Code Coverage:

Outlets

Outlet detection:
- pose estimation for outlet detection has been stabilized. The distance between the tip of the plug and the ground hole that the plug touches is within a range of 0.1-3mm.
- Outlet and plug nodes have been changed from working with prosilica poll service to forearm messages processing.
Texas parking:
- A color template detection method has been implemented for texas parking. It is based on the code that was used for outlet detection (hole detection + geometric hashing).
- A video from texas camera has been obtained by hacking a gstreamer plugin.
- Now working on a ros node that will actually drive texas to the parking.
Other: A lot of talking to opencv stakeholers, trying to boost the communication between opencv and willow teams.