AulaRACornersLucas.ppt
-
Upload
jared56 -
Category
Technology
-
view
492 -
download
0
Transcript of AulaRACornersLucas.ppt
![Page 1: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/1.jpg)
Extração de Características
cap 4 – Trucco e Verri
![Page 2: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/2.jpg)
Características de uma imagem
• Globais: histograma, conteúdo de freqüências, etc...
• Locais: regiões com determinada propriedade, arestas, cantos, curvas, etc...
![Page 3: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/3.jpg)
Arestas e cantos
• Locais de mudanças significativas na intensidade da imagem
![Page 4: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/4.jpg)
Edgedels = edge elements
![Page 5: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/5.jpg)
Tipos de arestasdegrau(step) rampa(ramp)
cume(roof) impulso(spike)
![Page 6: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/6.jpg)
Gráfico sem e com ruído
![Page 7: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/7.jpg)
Derivadas e arestas
f(x) f(x)+n(x) | f'(x)+n'(x) | f"(x)+n"(x)
![Page 8: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/8.jpg)
Série de Taylor
)()(2
)()()()()( 3"
2' xOxf
xxfxxfxxf
iiii ffff "'1 2
1
Com x=1, f(x)=fi e f(x+x)=fi+1
Com x=-1, f(x)=fi e f(x+x)=fi-1
iiii ffff "'1 2
1
(a)
(b)
![Page 9: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/9.jpg)
Aproximações para derivadas
(a-b) 2/)( 11'
iii fff 2/)( 11'
iii fff
(a+b) )2( 11"
iiii ffff )2( 11"
iiii ffff
f(x)
x
fi-1
fi fi+1
i+1ii-1
![Page 10: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/10.jpg)
Em 2D
y
fx
f
yxf ),(
Gradiente
Laplaciano
2
2
2
22 ),(
y
f
x
fyxf
x
yxfyxf
x
yxf mnmn
,,, 1
x
yxfyxf
y
yxf mnmn
,,, 1
11
1
1
Convolution Kernels
141
4204
141
![Page 11: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/11.jpg)
Laplaciano
)2( 11"
iiii ffff
•Sometimes we are interested only in changing magnitude without regard to the changing orientation.
•A linear differential operator called the Laplacian may be used.
•The Laplacian has the same properties in all directions and is therefore invariant to rotation in the image.
http://www.cee.hw.ac.uk/hipr/html/log.html
http://ct.radiology.uiowa.edu/~jiangm/courses/dip/html/node83.html
![Page 12: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/12.jpg)
Finite differences
11* II x
1
1*II y
IKhurram Hassan-Shafique
![Page 13: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/13.jpg)
Classical Operators
Prewitt’s Operator
11
11
11
1
1
11
Smooth Differentiate
101
101
101
111
000
111
111
111
Khurram Hassan-Shafique
![Page 14: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/14.jpg)
Classical Operators
Sobel’s Operator
11
22
11
1
1
11
SmoothDifferentiate
101
202
101
121
000
121
121
121
Khurram Hassan-Shafique
![Page 15: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/15.jpg)
• Sobel Edge Detector
Detecting Edges in Image
Image I
101
202
101
121
000
121
*
*
Idx
d
Idy
d
22
Idy
dI
dx
d
ThresholdEdges
Khurram Hassan-Shafique
![Page 16: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/16.jpg)
Sobel Edge Detector
Idx
d
Idy
d
I
Khurram Hassan-Shafique
![Page 17: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/17.jpg)
Sobel Edge Detector
I
22
I
dy
dI
dx
d
100 ThresholdKhurram Hassan-Shafique
![Page 18: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/18.jpg)
Marr and Hildreth Edge Operator
• Smooth by Gaussian
• Use Laplacian to find derivatives
IGS * 2
22
2
2
1
yx
eG
Sy
Sx
S2
2
2
22
Khurram Hassan-Shafique
![Page 19: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/19.jpg)
Marr and Hildreth Edge Operator
IGIGS ** 222
2
22
22
22
3
2 22
1
yx
eyx
G
Khurram Hassan-Shafique
![Page 20: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/20.jpg)
Marr and Hildreth Edge Operator
2
22
22
22
3
2 22
1
yx
eyx
G
0.0008 0.0066 0.0215 0.031 0.0215 0.0066 0.00080.0066 0.0438 0.0982 0.108 0.0982 0.0438 0.00660.0215 0.0982 0 -0.242 0 0.0982 0.0215
0.031 0.108 -0.242 -0.7979 -0.242 0.108 0.0310.0215 0.0982 0 -0.242 0 0.0982 0.02150.0066 0.0438 0.0982 0.108 0.0982 0.0438 0.00660.0008 0.0066 0.0215 0.031 0.0215 0.0066 0.0008
X
Y
Khurram Hassan-Shafique
![Page 21: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/21.jpg)
Marr and Hildreth Edge Operator
Zero CrossingsDetection
I ImageG2*
IG *2 Edge Image
IG *2 Zero Crossings
Khurram Hassan-Shafique
![Page 22: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/22.jpg)
1
6
3
Khurram Hassan-Shafique
![Page 23: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/23.jpg)
Quality of an Edge Detector
• Robustness to Noise
• Localization
• Too Many/Too less Responses
Poor robustness to noise Poor localization Too many responses
True Edge
Khurram Hassan-Shafique
![Page 24: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/24.jpg)
Canny Edge Detector
• Criterion 1: Good Detection: The optimal detector must minimize the probability of false positives as well as false negatives.
• Criterion 2: Good Localization: The edges detected must be as close as possible to the true edges.
• Single Response Constraint: The detector must return one point only for each edge point.
Khurram Hassan-Shafique
![Page 25: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/25.jpg)
Hai Tao
![Page 26: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/26.jpg)
The result– General form of the filter (N.B. the filter is odd so h(x) = -h(-x) the following expression
is for x < 0 only)
h x e a x a x e a x a xx x( ) ( sin cos ) ( sin cos ) / 1 2 3 4 1 2
2 05220
2 91540
156939
.
.
.
a
a
a
a
1
2
3
4
1
01486768717
0 2087553476
1244653939
0 7912446531
2
.
.
.
.
Camillo J. Taylor
![Page 27: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/27.jpg)
Approximation– Canny’s filter can be approximated by the derivative of a Gaussian
h xd
dxe
xe
x x
( ) ( )
2
2
2
222
2
Camillo J. Taylor
Derivative of GaussianCanny
![Page 28: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/28.jpg)
Canny Edge Detector
• Convolution with derivative of Gaussian
• Non-maximum Suppression
• Hysteresis Thresholding
Khurram Hassan-Shafique
![Page 29: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/29.jpg)
Algorithm Canny_Enhancer• Smooth by Gaussian
IGS * 2
22
2
2
1
yx
eG
Tyx
T
SSSy
Sx
S
22yx SSS
x
y
S
S1tan Khurram Hassan-Shafique
• Compute x and y derivatives
• Compute gradient magnitude and orientation
![Page 30: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/30.jpg)
Canny Edge Operator
IGIGS ** T
y
G
x
GG
T
Iy
GI
x
GS
**
Khurram Hassan-Shafique
![Page 31: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/31.jpg)
Canny Edge Detector
xS
yS
I
Khurram Hassan-Shafique
![Page 32: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/32.jpg)
Canny Edge Detector
I
22yx SSS
25 ThresholdS
Khurram Hassan-Shafique
![Page 33: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/33.jpg)
We wish to mark points along the curve where the magnitude is biggest.We can do this by looking for a maximum along a slice normal to the curve(non-maximum suppression). These points should form a curve. There arethen two algorithmic issues: at which point is the maximum, and where is thenext one?
Algorithm Non-Maximum Suppression
Khurram Hassan-Shafique
![Page 34: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/34.jpg)
Non-Maximum Suppression
• Suppress the pixels in ‘Gradient Magnitude Image’ which are not local maximum
edgean tonormaldirection thealong
in of neighbors theare and Sx,yy,xy,x
otherwise0,,&
,, if,
, yxSyxS
yxSyxSyxS
yxM
yx ,
yx,
yx ,
Khurram Hassan-Shafique
![Page 35: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/35.jpg)
Non-Maximum Suppression
0
12
3
41420tan41422- :3
41422tan :2
41422tan41420 :1
41420tan41420 :0
.θ.
.θ
.θ.
.θ.-
x
y
S
Sθ tan
Khurram Hassan-Shafique
![Page 36: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/36.jpg)
Non-Maximum Suppression
22yx SSS M
25ThresholdM
Khurram Hassan-Shafique
![Page 37: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/37.jpg)
Hysteresis Thresholding
Khurram Hassan-Shafique
![Page 38: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/38.jpg)
Hysteresis Thresholding
• If the gradient at a pixel is above ‘High’, declare it an ‘edge pixel’
• If the gradient at a pixel is below ‘Low’, declare it a ‘non-edge-pixel’
• If the gradient at a pixel is between ‘Low’ and ‘High’ then declare it an ‘edge pixel’ if and only if it is connected to an ‘edge pixel’ directly or via pixels between ‘Low’ and ‘ High’
Khurram Hassan-Shafique
![Page 39: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/39.jpg)
Hysteresis Thresholding
M 25ThresholdM
15
35
Low
High
Khurram Hassan-Shafique
![Page 40: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/40.jpg)
Resultado de algoritmo de histerese
![Page 41: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/41.jpg)
Subpixel Localization– One can try to further localize the position of the edge within a pixel by analyzing the response to the
edge enhancement filter
– One common approach is to fit a quadratic polynomial to the filter response in the region of a maxima and compute the true maximum.
a
bx
yyya
yyb
cbay
cbay
cy
cbxaxxy
2
);0())1()1((2
1
));1()1((2
1
)1(
;)1(
;)0(
)(
max
2
0 1-1
![Page 42: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/42.jpg)
Derivadas direcionais
y
n
y
f
x
n
x
f
n
f
x
y
f(x,y)
y
x
n
nn
h
pfnhpf
n
pfh
)()(lim
)(0
yx ny
fn
x
f
n
f
nfn
f
y
xyx n
n
y
f
x
f
y
fx
f
nnn
f
n
f
n
f
2
![Page 43: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/43.jpg)
Detecting corners
– If Ex and Ey denote the gradients of the intensity image, E(x,y), in the x and y directions then the behavior of the gradients in a region around a point can be obtained by considering the following matrix
2
2
yyx
yxxyx
y
x
EEE
EEEEE
E
EC
Camillo J. Taylor
![Page 44: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/44.jpg)
Examining the matrix
– One way to decide on the presence of a corner is to look at the eigenvalues of the 2 by 2 matrix C.
• If the area is a region of constant intensity we would expect both eigenvalues to be small
• If it contains a edge we expect one large eigenvalue and one small one
• If it contains edges at two or more orientations we expect 2 large eigenvalues
Camillo J. Taylor
![Page 45: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/45.jpg)
Finding corners
– One approach to finding corners is to find locations where the smaller eigenvalue is greater than some threshold
– We could also imagine considering the ratio of the two eigenvalues
![Page 46: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/46.jpg)
Computing Image Gradients
![Page 47: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/47.jpg)
Corner Analysis– The ellipses indicate the eignvalues and eigenvectors of the C matrices
![Page 48: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/48.jpg)
Corner Detection
• Basic idea: Find points where two edges meet—i.e., high gradient in two directions
• “Cornerness” is undefined at a single pixel, because there’s only one gradient per point– Look at the gradient behavior over a small window
• Categories image windows based on gradient statistics– Constant: Little or no brightness change– Edge: Strong brightness change in single direction– Flow: Parallel stripes– Corner/spot: Strong brightness changes in orthogonal directions
![Page 49: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/49.jpg)
Corner Detection: Analyzing Gradient Covariance
• Intuitively, in corner windows both Ix and Iy should be high– Can’t just set a threshold on them directly, because we want rotational invariance
• Analyze distribution of gradient components over a window to differentiate between types from previous slide:
• The two eigenvectors and eigenvalues ¸1, ¸2 of C (Matlab: eig(C)) encode the predominant directions and magnitudes of the gradient, respectively, within the window
• Corners are thus where min(¸1, ¸2) is over a threshold courtesy of Wolfram
![Page 50: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/50.jpg)
Contents
• Harris Corner Detector
– Description
– Analysis
• Detectors
– Rotation invariant
– Scale invariant
– Affine invariant
• Descriptors
– Rotation invariant
– Scale invariant
– Affine invariant
![Page 51: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/51.jpg)
Harris Detector: Mathematics
2,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
Change of intensity for the shift [u,v]:
IntensityShifted intensity
Window function
orWindow function w(x,y) =
Gaussian1 in window, 0 outside
![Page 52: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/52.jpg)
Harris Detector: Mathematics
( , ) ,u
E u v u v Mv
For small shifts [u,v] we have a bilinear approximation:
2
2,
( , ) x x y
x y x y y
I I IM w x y
I I I
where M is a 22 matrix computed from image derivatives:
![Page 53: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/53.jpg)
Harris Detector: Geometric Interpretation
( , ) ,u
E u v u v Mv
constv
uvu
v
uRRvuE T
'
'
0
0''
0
0
2
1
2
1
1, 2 – eigenvalues of M
(max)-1/2
(min)-1/2
constvu 22
21 )'()'(
![Page 54: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/54.jpg)
Harris Detector: Mathematics
1
2
“Corner”1 and 2 are large,
1 ~ 2;
E increases in all directions
1 and 2 are small;
E is almost constant in all directions
“Edge” 1 >> 2
“Edge” 2 >> 1
“Flat” region
Classification of image points using eigenvalues of M:
![Page 55: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/55.jpg)
Harris Detector: Mathematics
Measure of corner response:
2det traceR M k M
1 2
1 2
det
trace
M
M
(k – empirical constant, k = 0.04-0.06)
![Page 56: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/56.jpg)
Harris Detector: Mathematics
2
• R depends only on eigenvalues of M
• R is large for a corner
• R is negative with large magnitude for an edge
• |R| is small for a flat region
1
“Corner”
“Edge”
“Edge”
“Flat”
R > 0
R < 0
R < 0|R| small
![Page 57: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/57.jpg)
Algoritmo
• Comparação dos gráficos
1
“Corner”
“Edge”
“Edge”
“Flat”
R > 0
R < 0
R < 0|R| small
2
2det traceR M k M
![Page 58: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/58.jpg)
Algoritmo
• Comparação dos gráficos
1
“Corner”
“Edge”
“Edge”
“Flat”
R > 0
R < 0
R < 0|R| small
2
2det traceR M k M
![Page 59: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/59.jpg)
Algoritmo
• Comparação dos gráficos
1
“Corner”
“Edge”
“Edge”
“Flat”
R > 0
R < 0
R < 0|R| small
2
2det traceR M k M
![Page 60: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/60.jpg)
Harris Detector
• The Algorithm:– Find points with large corner response function R (R >
threshold)
– Take the points of local maxima of R
![Page 61: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/61.jpg)
Harris Detector: Workflow
![Page 62: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/62.jpg)
Harris Detector: Workflow
Compute corner response R
![Page 63: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/63.jpg)
Harris Detector: Workflow
Find points with large corner response: R>threshold
![Page 64: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/64.jpg)
Harris Detector: Workflow
Take only the points of local maxima of R
![Page 65: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/65.jpg)
Harris Detector: Workflow
![Page 66: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/66.jpg)
Example: Gradient Covariances
Full imageDetail of image with gradient covar-
iance ellipses for 3 x 3 windows
from Forsyth & Ponce
Corners are where both eigenvalues are big
![Page 67: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/67.jpg)
Example: Corner Detection (for camera calibration)
courtesy of B. Wilburn
![Page 68: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/68.jpg)
Example: Corner Detection
courtesy of S. Smith
SUSAN corners
![Page 69: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/69.jpg)
Harris Detector: Summary
• Average intensity change in direction [u,v] can be expressed as a bilinear form:
• Describe a point in terms of eigenvalues of M:measure of corner response
• A good (corner) point should have a large intensity change in all directions, i.e. R should be large positive
( , ) ,u
E u v u v Mv
2
1 2 1 2R k
![Page 70: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/70.jpg)
Contents
• Harris Corner Detector
– Description
– Analysis
• Detectors
– Rotation invariant
– Scale invariant
– Affine invariant
• Descriptors
– Rotation invariant
– Scale invariant
– Affine invariant
![Page 71: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/71.jpg)
Tracking: compression of video information
• Harris response (uses criss-cross gradients)• Dinosaur tracking (using features)• Dinosaur Motion tracking (using correlation)• Final Tracking (superimposed)Courtesy: (http://www.toulouse.ca/index.php4?/CamTracker/index.php4?/CamTracker/FeatureTracking.html)
This figure displays results of feature detection over the dinosaur test sequence with the algorithm set to extract the 6 most "interesting" features at every image frame.
It is interesting to note that although no attempt to extract frame-to-frame feature correspondences was made, the algorithm still extracts the same set of features at every frame.
This will be useful very much in feature tracking.
![Page 72: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/72.jpg)
One More..
• Office sequence
• Office Tracking
![Page 73: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/73.jpg)
Harris Detector: Some Properties
• Rotation invariance
Ellipse rotates but its shape (i.e. eigenvalues) remains the same
Corner response R is invariant to image rotation
![Page 74: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/74.jpg)
Harris Detector: Some Properties
• Partial invariance to affine intensity change
Only derivatives are used => invariance to intensity shift I I + b
Intensity scale: I a I
R
x (image coordinate)
threshold
R
x (image coordinate)
![Page 75: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/75.jpg)
Harris Detector: Some Properties
• But: non-invariant to image scale!
All points will be classified as edges
Corner !
![Page 76: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/76.jpg)
Harris Detector: Some Properties• Quality of Harris detector for different scale changes-- Correspondences calculated using distance (and threshold)
-- Improved Harris is proposed by Schmid et al
-- repeatability rate is defined as the number of points
repeated between two images w.r.t the total number of
detected points. Repeatability rate:
# correspondences# possible correspondences
C.Schmid et.al. “Evaluation of Interest Point Detectors”. IJCV 2000
Imp.Harris uses derivative of Gaussianinstead of standard template used byHarris et al.
![Page 77: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/77.jpg)
Contents
• Harris Corner Detector
– Description
– Analysis
• Detectors
– Rotation invariant
– Scale invariant
– Affine invariant
• Descriptors
– Rotation invariant
– Scale invariant
– Affine invariant
![Page 78: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/78.jpg)
We want to:
detect the same interest points regardless of image changes
![Page 79: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/79.jpg)
Models of Image Change
• Geometry– Rotation
– Similarity (rotation + uniform scale)
– Affine (scale dependent on direction)valid for: orthographic camera, locally planar object
• Photometry– Affine intensity change (I a I + b)
![Page 80: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/80.jpg)
Contents
• Harris Corner Detector
– Description
– Analysis
• Detectors
– Rotation invariant
– Scale invariant
– Affine invariant
• Descriptors
– Rotation invariant
– Scale invariant
– Affine invariant
![Page 81: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/81.jpg)
Rotation Invariant Detection
• Harris Corner Detector
C.Schmid et.al. “Evaluation of Interest Point Detectors”. IJCV 2000
![Page 82: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/82.jpg)
Contents
• Harris Corner Detector
– Description
– Analysis
• Detectors
– Rotation invariant
– Scale invariant
– Affine invariant
• Descriptors
– Rotation invariant
– Scale invariant
– Affine invariant
![Page 83: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/83.jpg)
Scale Invariant Detection• Consider regions (e.g. circles) of different sizes around a point
• Regions of corresponding sizes (at different scales) will look the same in both images
Fine/Low Coarse/High
![Page 84: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/84.jpg)
Scale Invariant Detection• The problem: how do we choose corresponding circles independently
in each image?
![Page 85: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/85.jpg)
Scale Invariant Detection
• Solution:
– Design a function on the region (circle), which is “scale invariant” (the same for corresponding regions, even if they are at different scales)
Example: average intensity. For corresponding regions (even of different sizes) it will be the same.
scale = 1/2
– For a point in one image, we can consider it as a function of region size (circle radius)
f
region size
Image 1 f
region size
Image 2
![Page 86: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/86.jpg)
Scale Invariant Detection
• Common approach:
scale = 1/2
f
region size/scale
Image 1 f
region size/scale
Image 2
Take a local maximum of this function
Observation: region size (scale), for which the maximum is achieved, should be invariant to image scale.
s1 s2
Important: this scale invariant region size is found in each image independently!
Max. is called characteristic scale
![Page 87: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/87.jpg)
Characteristic Scale
• The ratio of the scales, at which the extrema were found for corresponding points in two rescaled images, is equal to the scale factor between the images.
• Characteristic Scale: Given a point in an image, compute the function responses for several factors sn The characteristic scale is the local max. of the function (can be more than one).
• Easy to look for zero-crossings of 2nd derivative than maxima.
scale = 1/2
f
region size/scale
Image 1 f
region size/scale
Image 2
s1 s2
Max. is called characteristic scale
![Page 88: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/88.jpg)
Scale Invariant Detection
• A “good” function for scale detection: has one stable sharp peak
f
region size
bad
f
region size
bad
f
region size
Good !
• For usual images: a good function would be the one with contrast (sharp local intensity change)
![Page 89: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/89.jpg)
Scale Invariant Detection
• Functions for determining scale
2 2
21 22
( , , )x y
G x y e
2 ( , , ) ( , , )xx yyL G x y G x y
( , , ) ( , , )DoG G x y k G x y
Kernel Imagef Kernels:
where Gaussian
Note: both kernels are invariant to scale and rotation
(Laplacian)
(Difference of Gaussians)
![Page 90: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/90.jpg)
Build Scale-Space Pyramid
• All scales must be examined to identify scale-invariant features
• An efficient function is to compute the Difference of Gaussian (DOG) pyramid (Burt & Adelson, 1983) (or Laplacian)
B l u r
R e s a m p l e
S u b t r a c t
B l u r
R e s a m p l e
S u b t r a c t
B lu r
R esam p le
S u b trac t
![Page 91: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/91.jpg)
Key point localization
• Detect maxima and minima of difference-of-Gaussian in scale space
B l u r
R e s a m p l e
S u b t r a c t
![Page 92: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/92.jpg)
Scale Invariant Detectors• Harris-Laplacian1
Find local maximum of:– Harris corner detector in space
(image coordinates)– Laplacian in scale
1 K.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 20012 D.Lowe. “Distinctive Image Features from Scale-Invariant Keypoints”. Accepted to IJCV 2004
scale
x
y
Harris
L
apla
cian
• SIFT (Lowe)2
Find local maximum of:– Difference of Gaussians in
space and scale
scale
x
y
DoG
D
oG
![Page 93: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/93.jpg)
Normal, Gaussian..
A normal distribution in a variate with mean and variance 2 is a statistic distribution with probability function
![Page 94: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/94.jpg)
Harris-Laplacian
• Existing methods search for maxima in the 3D representation of an image (x,y,scale). A feature point represents a local maxima in the surrounding 3D cube and its value is higher than a threshold.
• THIS (Harris-Laplacian) method uses Harris function first, then selects points for which Laplacian attains maximum over scales.
• First, prepare scale-space representation for the Harris function. At each level, detect interest points as local maxima in the image plane (of that scale) – do this by comparing 8-neighborhood. (different from plain Harris corner detection)
• Second, use Laplacian to judge if each of the candidate points found on different levels, if it forms a maximum in the scale direction. (check with n-1 and n+1)
![Page 95: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/95.jpg)
Scale Invariant Detectors
K.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 2001
• Experimental evaluation of detectors w.r.t. scale change
Repeatability rate:
# correspondences# possible correspondences
(points present)
![Page 96: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/96.jpg)
Scale Invariant Detection: Summary
• Given: two images of the same scene with a large scale difference between them
• Goal: find the same interest points independently in each image• Solution: search for maxima of suitable functions in scale and in space
(over the image)
Methods:
1. Harris-Laplacian [Mikolajczyk, Schmid]: maximize Laplacian over scale, Harris’ measure of corner response over the image
2. SIFT [Lowe]: maximize Difference of Gaussians over scale and space
![Page 97: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/97.jpg)
Contents
• Harris Corner Detector
– Description
– Analysis
• Detectors
– Rotation invariant
– Scale invariant
– Affine invariant (maybe later)
• Descriptors
– Rotation invariant
– Scale invariant
– Affine invariant
![Page 98: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/98.jpg)
Affine Invariant Detection
• Above we considered:Similarity transform (rotation + uniform scale)
• Now we go on to:Affine transform (rotation + non-uniform scale)
![Page 99: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/99.jpg)
Affine Invariant Detection
• Take a local intensity extremum as initial point
• Go along every ray starting from this point and stop when extremum of function f is reached
T.Tuytelaars, L.V.Gool. “Wide Baseline Stereo Matching Based on Local, Affinely Invariant Regions”. BMVC 2000.
f
points along the ray
• We will obtain approximately corresponding regions
Remark: we search for scale in every direction
![Page 100: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/100.jpg)
Affine Invariant Detection
• Algorithm summary (detection of affine invariant region):– Start from a local intensity extremum point– Go in every direction until the point of extremum of some function f– Curve connecting the points is the region boundary– Compute geometric moments of orders up to 2 for this region– Replace the region with ellipse
T.Tuytelaars, L.V.Gool. “Wide Baseline Stereo Matching Based on Local, Affinely Invariant Regions”. BMVC 2000.
![Page 101: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/101.jpg)
Affine Invariant Detection : Summary
• Under affine transformation, we do not know in advance shapes of the corresponding regions
• Ellipse given by geometric covariance matrix of a region robustly approximates this region
• For corresponding regions ellipses also correspond
Methods:
1. Search for extremum along rays [Tuytelaars, Van Gool]:
2. Maximally Stable Extremal Regions [Matas et.al.]
![Page 102: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/102.jpg)
SIFT
• Introdução
• Motivação
• Visão Geral do Algoritmo
![Page 103: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/103.jpg)
Introdução
• O que é o SIFT?– Scale Invariant Feature Transform
David Lowe
![Page 104: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/104.jpg)
Motivação
• Como ele ajuda em Markless?– Keypoints – Detecção de Texturas
by lowe
![Page 105: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/105.jpg)
Visão Geral do Algoritmo
• Detecção dos máximos locais no espaço de escala– Usando a função diferença de Gaussianos (DoG)
• Localização dos Keypoint– Calculo do subpixel do keypoint
• Calculo da Orientação– 1 ou mais por Keypoint
• Keypoint descriptor– Apartir do gradiente da imagem
![Page 106: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/106.jpg)
Detecção dos máximos
• Construção da piramide de DoG
by Brandon Green
![Page 107: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/107.jpg)
Detecção dos máximos• Usa o DoG para calcular os keypoints
),(),,(),,( yxIyxGyxL (def. espaço de Escala)
![Page 108: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/108.jpg)
Detecção dos máximos
• Procura nos 26 vizinhos o máximo local
![Page 109: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/109.jpg)
Filtrando os Keypoints
• Eliminação dos máximos que tem pouco contraste com sua vizinhança
• Eliminação dos pontos fracos mal localizados (sobre linhas)
1. Imagem inicial2. Maximos3. Eliminando por contraste4. Eliminando por localização ruim
![Page 110: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/110.jpg)
Cálculo da Orientação
• Cálculos dos Keypoints relativos a orientação.
• Permite a invariança na rotação
• Pre-calculados
![Page 111: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/111.jpg)
Keypoint descriptor
• Calculo referente a orientação
• Histograma com 8 classes
• Melhor descritor 4x4x8 = 128 classes
![Page 112: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/112.jpg)
Keypoint matching
• Calculo dos pares usando distancia Euclidiana– Brute force
– kd – tree
– Best-Bin-First
• Calculo da transformação para alinhamento das imagens– RANSAC
– Hough TransformNormalmente uma imagem tem ~2,000 keypoints
![Page 113: AulaRACornersLucas.ppt](https://reader035.fdocumentos.com/reader035/viewer/2022070316/5560b259d8b42a033c8b4714/html5/thumbnails/113.jpg)
Resultados