Английская грамматика
<<  По русскому междометие как часть речи Content-Based Image Retrieval using the Bag-of-Words Concept  >>
Part 1: Bag-of-words models
Part 1: Bag-of-words models
Related works
Related works
Part 1: Bag-of-words models
Part 1: Bag-of-words models
Analogy to documents
Analogy to documents
A clarification: definition of “BoW”
A clarification: definition of “BoW”
A clarification: definition of “BoW”
A clarification: definition of “BoW”
Part 1: Bag-of-words models
Part 1: Bag-of-words models
2.
2.
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
1.Feature detection and representation
2. Codewords dictionary formation
2. Codewords dictionary formation
2. Codewords dictionary formation
2. Codewords dictionary formation
2. Codewords dictionary formation
2. Codewords dictionary formation
Image patch examples of codewords
Image patch examples of codewords
3. Image representation
3. Image representation
2.
2.
Learning and Recognition
Learning and Recognition
Learning and Recognition
Learning and Recognition
2 generative models
2 generative models
First, some notations
First, some notations
Case #1: the Na
Case #1: the Na
Csurka et al
Csurka et al
Csurka et al
Csurka et al
Case #2: Hierarchical Bayesian text models
Case #2: Hierarchical Bayesian text models
Case #2: Hierarchical Bayesian text models
Case #2: Hierarchical Bayesian text models
Case #2: Hierarchical Bayesian text models
Case #2: Hierarchical Bayesian text models
Case #2: the pLSA model
Case #2: the pLSA model
Case #2: the pLSA model
Case #2: the pLSA model
Case #2: Recognition using pLSA
Case #2: Recognition using pLSA
Case #2: Learning the pLSA parameters
Case #2: Learning the pLSA parameters
Demo
Demo
task: face detection – no labeling
task: face detection – no labeling
Demo: feature detection
Demo: feature detection
Demo: learnt parameters
Demo: learnt parameters
Demo: recognition examples
Demo: recognition examples
Demo: categorization results
Demo: categorization results
Learning and Recognition
Learning and Recognition
Discriminative methods based on ‘bag of words’ representation
Discriminative methods based on ‘bag of words’ representation
Discriminative methods based on ‘bag of words’ representation
Discriminative methods based on ‘bag of words’ representation
Summary: Pyramid match kernel
Summary: Pyramid match kernel
Pyramid Match (Grauman & Darrell 2005)
Pyramid Match (Grauman & Darrell 2005)
Pyramid Match (Grauman & Darrell 2005)
Pyramid Match (Grauman & Darrell 2005)
Pyramid match kernel
Pyramid match kernel
Example pyramid match
Example pyramid match
Example pyramid match
Example pyramid match
Example pyramid match
Example pyramid match
Example pyramid match
Example pyramid match
Summary: Pyramid match kernel
Summary: Pyramid match kernel
Object recognition results
Object recognition results
Object recognition results
Object recognition results
Part 1: Bag-of-words models
Part 1: Bag-of-words models
?
?
What about spatial info
What about spatial info
What about spatial info
What about spatial info
What about spatial info
What about spatial info
What about spatial info
What about spatial info
Invariance issues
Invariance issues
Invariance issues
Invariance issues
Invariance issues
Invariance issues
Invariance issues
Invariance issues
Model properties
Model properties
Model properties
Model properties
Model properties
Model properties
Model properties
Model properties
Model properties
Model properties
Weakness of the model
Weakness of the model

Презентация на тему: «Part 1: Bag-of-words models». Автор: Rob Fergus. Файл: «Part 1: Bag-of-words models.ppt». Размер zip-архива: 10265 КБ.

Part 1: Bag-of-words models

содержание презентации «Part 1: Bag-of-words models.ppt»
СлайдТекст
1 Part 1: Bag-of-words models

Part 1: Bag-of-words models

by Li Fei-Fei (Princeton)

2 Related works

Related works

Early “bag of words” models: mostly texture recognition Cula & Dana, 2001; Leung & Malik 2001; Mori, Belongie & Malik, 2001; Schmid 2001; Varma & Zisserman, 2002, 2003; Lazebnik, Schmid & Ponce, 2003; Hierarchical Bayesian models for documents (pLSA, LDA, etc.) Hoffman 1999; Blei, Ng & Jordan, 2004; Teh, Jordan, Beal & Blei, 2004 Object categorization Csurka, Bray, Dance & Fan, 2004; Sivic, Russell, Efros, Freeman & Zisserman, 2005; Sudderth, Torralba, Freeman & Willsky, 2005; Natural scene categorization Vogel & Schiele, 2004; Fei-Fei & Perona, 2005; Bosch, Zisserman & Munoz, 2006

3 Part 1: Bag-of-words models
4 Analogy to documents

Analogy to documents

Of all the sensory impressions proceeding to the brain, the visual experiences are the dominant ones. Our perception of the world around us is based essentially on the messages that reach the brain from our eyes. For a long time it was thought that the retinal image was transmitted point by point to visual centers in the brain; the cerebral cortex was a movie screen, so to speak, upon which the image in the eye was projected. Through the discoveries of Hubel and Wiesel we now know that behind the origin of the visual perception in the brain there is a considerably more complicated course of events. By following the visual impulses along their path to the various cell layers of the optical cortex, Hubel and Wiesel have been able to demonstrate that the message about the image falling on the retina undergoes a step-wise analysis in a system of nerve cells stored in columns. In this system each cell has its specific function and is responsible for a specific detail in the pattern of the retinal image.

5 A clarification: definition of “BoW”

A clarification: definition of “BoW”

Looser definition Independent features

6 A clarification: definition of “BoW”

A clarification: definition of “BoW”

Looser definition Independent features Stricter definition Independent features histogram representation

7 Part 1: Bag-of-words models
8 2.

2.

1.

3.

Representation

9 1.Feature detection and representation

1.Feature detection and representation

10 1.Feature detection and representation

1.Feature detection and representation

Regular grid Vogel & Schiele, 2003 Fei-Fei & Perona, 2005

11 1.Feature detection and representation

1.Feature detection and representation

Regular grid Vogel & Schiele, 2003 Fei-Fei & Perona, 2005 Interest point detector Csurka, et al. 2004 Fei-Fei & Perona, 2005 Sivic, et al. 2005

12 1.Feature detection and representation

1.Feature detection and representation

Regular grid Vogel & Schiele, 2003 Fei-Fei & Perona, 2005 Interest point detector Csurka, Bray, Dance & Fan, 2004 Fei-Fei & Perona, 2005 Sivic, Russell, Efros, Freeman & Zisserman, 2005 Other methods Random sampling (Vidal-Naquet & Ullman, 2002) Segmentation based patches (Barnard, Duygulu, Forsyth, de Freitas, Blei, Jordan, 2003)

13 1.Feature detection and representation

1.Feature detection and representation

Detect patches [Mikojaczyk and Schmid ’02] [Mata, Chum, Urban & Pajdla, ’02] [Sivic & Zisserman, ’03]

Compute SIFT descriptor [Lowe’99]

Normalize patch

Slide credit: Josef Sivic

14 1.Feature detection and representation

1.Feature detection and representation

15 2. Codewords dictionary formation

2. Codewords dictionary formation

16 2. Codewords dictionary formation

2. Codewords dictionary formation

Vector quantization

Slide credit: Josef Sivic

17 2. Codewords dictionary formation

2. Codewords dictionary formation

Fei-Fei et al. 2005

18 Image patch examples of codewords

Image patch examples of codewords

Sivic et al. 2005

19 3. Image representation

3. Image representation

codewords

frequency

20 2.

2.

1.

3.

Representation

21 Learning and Recognition

Learning and Recognition

category models (and/or) classifiers

22 Learning and Recognition

Learning and Recognition

Generative method: - graphical models Discriminative method: - SVM

category models (and/or) classifiers

23 2 generative models

2 generative models

Na?ve Bayes classifier Csurka Bray, Dance & Fan, 2004 Hierarchical Bayesian text models (pLSA and LDA) Background: Hoffman 2001, Blei, Ng & Jordan, 2004 Object categorization: Sivic et al. 2005, Sudderth et al. 2005 Natural scene categorization: Fei-Fei et al. 2005

24 First, some notations

First, some notations

wn: each patch in an image wn = [0,0,…1,…,0,0]T w: a collection of all N patches in an image w = [w1,w2,…,wN] dj: the jth image in an image collection c: category of the image z: theme or topic of the patch

25 Case #1: the Na

Case #1: the Na

ve Bayes model

c

w

N

Csurka et al. 2004

26 Csurka et al

Csurka et al

2004

27 Csurka et al

Csurka et al

2004

28 Case #2: Hierarchical Bayesian text models

Case #2: Hierarchical Bayesian text models

Probabilistic Latent Semantic Analysis (pLSA)

Latent Dirichlet Allocation (LDA)

Hoffman, 2001

Blei et al., 2001

29 Case #2: Hierarchical Bayesian text models

Case #2: Hierarchical Bayesian text models

Probabilistic Latent Semantic Analysis (pLSA)

Sivic et al. ICCV 2005

30 Case #2: Hierarchical Bayesian text models

Case #2: Hierarchical Bayesian text models

Latent Dirichlet Allocation (LDA)

Fei-Fei et al. ICCV 2005

31 Case #2: the pLSA model

Case #2: the pLSA model

32 Case #2: the pLSA model

Case #2: the pLSA model

Slide credit: Josef Sivic

33 Case #2: Recognition using pLSA

Case #2: Recognition using pLSA

Slide credit: Josef Sivic

34 Case #2: Learning the pLSA parameters

Case #2: Learning the pLSA parameters

Maximize likelihood of data using EM

Observed counts of word i in document j

M … number of codewords N … number of images

Slide credit: Josef Sivic

35 Demo

Demo

Course website

36 task: face detection – no labeling

task: face detection – no labeling

37 Demo: feature detection

Demo: feature detection

Output of crude feature detector Find edges Draw points randomly from edge set Draw from uniform distribution to get scale

38 Demo: learnt parameters

Demo: learnt parameters

Codeword distributions per theme (topic)

Theme distributions per image

Learning the model: do_plsa(‘config_file_1’) Evaluate and visualize the model: do_plsa_evaluation(‘config_file_1’)

39 Demo: recognition examples

Demo: recognition examples

40 Demo: categorization results

Demo: categorization results

Performance of each theme

41 Learning and Recognition

Learning and Recognition

Generative method: - graphical models Discriminative method: - SVM

category models (and/or) classifiers

42 Discriminative methods based on ‘bag of words’ representation

Discriminative methods based on ‘bag of words’ representation

Decision boundary

Zebra

Non-zebra

43 Discriminative methods based on ‘bag of words’ representation

Discriminative methods based on ‘bag of words’ representation

Grauman & Darrell, 2005, 2006: SVM w/ Pyramid Match kernels Others Csurka, Bray, Dance & Fan, 2004 Serre & Poggio, 2005

44 Summary: Pyramid match kernel

Summary: Pyramid match kernel

optimal partial matching between sets of features

Grauman & Darrell, 2005, Slide credit: Kristen Grauman

45 Pyramid Match (Grauman & Darrell 2005)

Pyramid Match (Grauman & Darrell 2005)

Histogram intersection

Slide credit: Kristen Grauman

46 Pyramid Match (Grauman & Darrell 2005)

Pyramid Match (Grauman & Darrell 2005)

Histogram intersection

Slide credit: Kristen Grauman

47 Pyramid match kernel

Pyramid match kernel

Weights inversely proportional to bin size Normalize kernel values to avoid favoring large sets

Slide credit: Kristen Grauman

48 Example pyramid match

Example pyramid match

Level 0

Slide credit: Kristen Grauman

49 Example pyramid match

Example pyramid match

Level 1

Slide credit: Kristen Grauman

50 Example pyramid match

Example pyramid match

Level 2

Slide credit: Kristen Grauman

51 Example pyramid match

Example pyramid match

pyramid match

optimal match

Slide credit: Kristen Grauman

52 Summary: Pyramid match kernel

Summary: Pyramid match kernel

optimal partial matching between sets of features

difficulty of a match at level i

number of new matches at level i

Slide credit: Kristen Grauman

53 Object recognition results

Object recognition results

ETH-80 database 8 object classes (Eichhorn and Chapelle 2004) Features: Harris detector PCA-SIFT descriptor, d=10

84%

85%

84%

Kernel

Complexity

Recognition rate

Match [Wallraven et al.]

Bhattacharyya affinity [Kondor & Jebara]

Pyramid match

Slide credit: Kristen Grauman

54 Object recognition results

Object recognition results

Caltech objects database 101 object classes Features: SIFT detector PCA-SIFT descriptor, d=10 30 training images / class 43% recognition rate (1% chance performance) 0.002 seconds per match

Slide credit: Kristen Grauman

55 Part 1: Bag-of-words models
56 ?

?

What about spatial info?

57 What about spatial info

What about spatial info

Feature level Spatial influence through correlogram features: Savarese, Winn and Criminisi, CVPR 2006

58 What about spatial info

What about spatial info

Feature level Generative models Sudderth, Torralba, Freeman & Willsky, 2005, 2006 Niebles & Fei-Fei, CVPR 2007

59 What about spatial info

What about spatial info

Feature level Generative models Sudderth, Torralba, Freeman & Willsky, 2005, 2006 Niebles & Fei-Fei, CVPR 2007

60 What about spatial info

What about spatial info

Feature level Generative models Discriminative methods Lazebnik, Schmid & Ponce, 2006

61 Invariance issues

Invariance issues

Scale and rotation Implicit Detectors and descriptors

Kadir and Brady. 2003

62 Invariance issues

Invariance issues

Scale and rotation Occlusion Implicit in the models Codeword distribution: small variations (In theory) Theme (z) distribution: different occlusion patterns

63 Invariance issues

Invariance issues

Scale and rotation Occlusion Translation Encode (relative) location information Sudderth, Torralba, Freeman & Willsky, 2005, 2006 Niebles & Fei-Fei, 2007

64 Invariance issues

Invariance issues

Scale and rotation Occlusion Translation View point (in theory) Codewords: detector and descriptor Theme distributions: different view points

Fergus, Fei-Fei, Perona & Zisserman, 2005

65 Model properties

Model properties

Intuitive Analogy to documents

66 Model properties

Model properties

Intuitive Analogy to documents Analogy to human vision

Olshausen and Field, 2004, Fei-Fei and Perona, 2005

67 Model properties

Model properties

Intuitive generative models Convenient for weakly- or un-supervised, incremental training Prior information Flexibility (e.g. HDP)

Li, Wang & Fei-Fei, CVPR 2007

Sivic, Russell, Efros, Freeman, Zisserman, 2005

68 Model properties

Model properties

Intuitive generative models Discriminative method Computationally efficient

Grauman et al. CVPR 2005

69 Model properties

Model properties

Intuitive generative models Discriminative method Learning and recognition relatively fast Compare to other methods

70 Weakness of the model

Weakness of the model

No rigorous geometric information of the object components It’s intuitive to most of us that objects are made of parts – no such information Not extensively tested yet for View point invariance Scale invariance Segmentation and localization unclear

«Part 1: Bag-of-words models»
http://900igr.net/prezentacija/anglijskij-jazyk/part-1-bag-of-words-models-252492.html
cсылка на страницу

Английская грамматика

28 презентаций об английской грамматике
Урок

Английский язык

29 тем
Слайды