Inhibition in V1 1. Non-classical receptive field 2. Push pull inhibition

Transcription

1 BICV Summer School, Shenyang, China, Inhibition in V1 1. Non-classical receptive field 2. Push pull inhibition George Azzopardi Nicolai Petkov

2 BICV Summer School, Shenyang, China, Non-classical receptive field inhibition

3 BICV Summer School, Shenyang, China,

4 BICV Summer School, Shenyang, China,

5 BICV Summer School, Shenyang, China,

6 BICV Summer School, Shenyang, China,

7 BICV Summer School, Shenyang, China,

8 BICV Summer School, Shenyang, China,

9 BICV Summer School, Shenyang, China,

10 BICV Summer School, Shenyang, China, Visual perception [Solomon & Pelli, 1994]

11 BICV Summer School, Shenyang, China, 2015 Visual perception [Petkov & Westenberg, 2003] 11

12 BICV Summer School, Shenyang, China,

13 BICV Summer School, Shenyang, China,

14 BICV Summer School, Shenyang, China,

15 BICV Summer School, Shenyang, China, Visual perception [Galli & Zama, 1931]

16 BICV Summer School, Shenyang, China, Visual perception Orientation-contrast pop-out

17 BICV Summer School, Shenyang, China, Visual perception

18 BICV Summer School, Shenyang, China, Neurophysiology Receptive field and response of an orientation [Hubel & Wiesel, 1962] selective neuron

19 BICV Summer School, Shenyang, China, Neurophysiology Impulse response of an orientation selective neuron modelled by a 2D Gabor function [Daugman, 1985]

20 BICV Summer School, Shenyang, China, What is the neurophysiology of this visual perception?

21 BICV Summer School, Shenyang, China, Neurophysiology CRF Inhibition surround

22 BICV Summer School, Shenyang, China, Neurophysiology [Blakemore and Tobin, 1972]

23 BICV Summer School, Shenyang, China, Link to perception

24 BICV Summer School, Shenyang, China, Neurophysiology Mean firing rate [spikes/s] Orientation-contrast cell [Nothdurft et. al, 1999]

25 BICV Summer School, Shenyang, China, Link to perception

26 BICV Summer School, Shenyang, China, Neurophysiology Mean firing rate [spikes/s] (a) (b) (c) (d) General suppression cell [Nothdurft et. al, 1999]

27 BICV Summer School, Shenyang, China, Link to perception

28 BICV Summer School, Shenyang, China, Computational models

29 BICV Summer School, Shenyang, China, Gabor filter

30 BICV Summer School, Shenyang, China, Gabor filter Kernel: 27

31 BICV Summer School, Shenyang, China, Gabor filter Kernel: Filter:

32 BICV Summer School, Shenyang, China, Gabor energy filter

33 BICV Summer School, Shenyang, China, Gabor energy filter Gabor energy:

34 BICV Summer School, Shenyang, China, Gabor energy filter input I output E 0 o output E 90 o max(e 0 o,e 90 o )

35 BICV Summer School, Shenyang, China, Surround suppression CRF Inhibition surround

36 BICV Summer School, Shenyang, China, Surround suppression Different effects of the surround

37 BICV Summer School, Shenyang, China, Surround weighting function

38 BICV Summer School, Shenyang, China, Surround weighting function

39 BICV Summer School, Shenyang, China, Surround weighting function

40 BICV Summer School, Shenyang, China, Orientation-contrast suppression Mean firing rate [spikes/s]

41 BICV Summer School, Shenyang, China, Orientation-contrast suppression term

42 BICV Summer School, Shenyang, China, Orientation-contrast suppression term Input I Gabor energy E 0 o Suppression S 0 o term

43 BICV Summer School, Shenyang, China, Orientation-contrast suppression filter

44 BICV Summer School, Shenyang, China, Orientation-contrast suppression filter

45 BICV Summer School, Shenyang, China, Orientation-contrast suppression

46 BICV Summer School, Shenyang, China, Back to visual perception

47 BICV Summer School, Shenyang, China, Back to visual perception 38

48 BICV Summer School, Shenyang, China, Computer vision application

49 BICV Summer School, Shenyang, China, Texture suppression Input Gabor energy With suppression

50 BICV Summer School, Shenyang, China, Contour enhancement by texture suppression Input Gabor energy With suppression

51 BICV Summer School, Shenyang, China, Binary contour maps Input Edge strength Binarized response

52 BICV Summer School, Shenyang, China, Desired output Input image and associated desired output Dataset: imaging

53 BICV Summer School, Shenyang, China, Performance measure Desired output Operator output

54 BICV Summer School, Shenyang, China, Performance measure

55 BICV Summer School, Shenyang, China, Canny performance Input Desired output Best Canny P = 0.23

56 BICV Summer School, Shenyang, China, Suppressed Gabor performance Input Desired output Suppressed Gabor P = 0.42

57 BICV Summer School, Shenyang, China, Performance comparison Input Best Canny Suppressed Gabor P = 0.23 P = 0.42

58 BICV Summer School, Shenyang, China, Performance comparison Input Best Canny Suppressed Gabor P = 0.14 P = 0.34 [Grigorescu et al., 2003]

59 BICV Summer School, Shenyang, China, Performance comparison Goat 3 Elephant 2 Hyena Gazelle Performance C G B1 B2 C G B1 B2 C G B1 B2 C G B1 B2 C G B1 B2 (C) Canny, (G) Gabor, (B1) orientation contrast suppression, (B2) general suppression

60 BICV Summer School, Shenyang, China, Beyond receptive fields and Gabor functions

61 BICV Summer School, Shenyang, China, Beyond receptive fields and Gabor functions Canny edge detector

62 BICV Summer School, Shenyang, China, Beyond receptive fields and Gabor functions Canny edge detector with surround suppression

63 BICV Summer School, Shenyang, China, Canny with surround suppression Input Canny With suppression [Grigorescu et al., 2004]

64 BICV Summer School, Shenyang, China, Split surround

65 BICV Summer School, Shenyang, China, Input Best Canny Suppressed Gabor

66 BICV Summer School, Shenyang, China, Multi-scale Canny with split surround suppression Input Suppressed Canny (multi-scale) Desired output [Papari et al., 2006]

67 BICV Summer School, Shenyang, China, Push-pull inhibition

68 BICV Summer School, Shenyang, China, Simple Cells receive push-pull inhibition (Anderson et al., 2000) Stimuli of the reverse contrast evoke responses of the opposite sign With push-pull inhibition linear models achieve contrast invariant orientation tuning CORF model (without inhibition) Already achieves contrast invariant orientation tuning Only responds to preferred orientation and contrast We explore further properties

69 BICV Summer School, Shenyang, China, Push-Pull inhibition RFs arrangement of LGN cells with preferred polarity RFs arrangement of LGN cells with opposite polarity push (excitation) pull (inhibition) Preferred stimulus

70 BICV Summer School, Shenyang, China, Overlap index (Martinez et al., 2005)

71 BICV Summer School, Shenyang, China, Overlap Index 1 β = 0 Normalized response to preferred orientation β = 2 β = 4 β = Orientation bandwidth (degrees)

72 BICV Summer School, Shenyang, China, Response of CORF with push pull inhibition

73 BICV Summer School, Shenyang, China, Response of CORF with push pull inhibition r S (x, y )

74 BICV Summer School, Shenyang, China, Response of CORF with push pull inhibition r S (x, y ) r Sβ (x, y ) + β

75 BICV Summer School, Shenyang, China, Response of CORF with push pull inhibition r S (x, y ) kr Sβ (x, y ) + β

76 BICV Summer School, Shenyang, China, Signal-to-noise Ratio Edge stimulus Band-limited texture

77 BICV Summer School, Shenyang, China, Signal-to-noise Ratio + = Edge stimulus Band-limited texture Test image

78 BICV Summer School, Shenyang, China, Signal-to-noise ratio Input Output of models CORF CORF+PP SNR= SNR= SNR= 9.45 SNR= 23.83

79 BICV Summer School, Shenyang, China, Signal-to-noise ratio Input Output of models CORF CORF+PP SNR= SNR= SNR= 2.83 SNR= 14.73

80 BICV Summer School, Shenyang, China, Signal-to-noise ratio Input Output of models CORF CORF+PP SNR= SNR= SNR= 5.85 SNR= 8.18

81 BICV Summer School, Shenyang, China, Separability of spatial frequency and orientation Real simple cell (Webster, De Valois, et al., 1985). CORF without inhibition Fundamental amplitude Orientation ( ) Spatial Frequency (cycles/degree) Normalized response 1 Orientation ( ) Spatial Frequency

82 BICV Summer School, Shenyang, China, Separability of spatial frequency and orientation Real simple cell (Webster, De Valois, et al., 1985) CORF with push-pull inhibition Fundamental amplitude Orientation ( ) Spatial Frequency (cycles/degree) Normalized response 1 Orientation ( ) Spatial Frequency

83 BICV Summer School, Shenyang, China, Spatial frequency tuning sensitive to contrast (Sceniak et al., 2002) Normalized response CORF without inhibition Spatial Frequency CORF with push-pull inhibition Contrast 80% 20% Spatial Frequency

84 BICV Summer School, Shenyang, China, Contour detection Input image CORF with push-pull inhibition CORF without inhibition

85 BICV Summer School, Shenyang, China, Contour detection Input image CORF with push-pull inhibition Gabor with isotropic surround suppression

86 BICV Summer School, Shenyang, China, Contour detection Input image CORF with push-pull inhibition Canny

87 BICV Summer School, Shenyang, China, Contour detection Input image CORF with push-pull inhibition CORF without inhibition

88 BICV Summer School, Shenyang, China, Contour detection Input image CORF with push-pull inhibition Gabor with isotropic surround suppression

89 BICV Summer School, Shenyang, China, Contour detection Input image CORF with push-pull inhibition Canny

90 BICV Summer School, Shenyang, China, Results: CORF with push-pull inhibition vs others Right-tailed paired t -test RuG: 40 images CORF without inhibition: p < 0.01 Gabor with surround suppression: p < 0.01 Canny: p < 0.01 Berkeley: 500 images CORF without inhibition: p < 0.01 Gabor with surround suppression: p < 0.01 Canny: p < 0.01

91 BICV Summer School, Shenyang, China, Conclusions Computational modelling of two types of inhibition Gabor à non-classical receptive field CORF à push-pull inhibition CORF+PP improves SNR CORF+PP is more effective than Gabor with surround suppression in contour detection

92 BICV Summer School, Shenyang, China, 2015 Outlook Combine non-classical receptive field and push-pull inhibition in one model? Contour detection is expected to be improved even further Incorporate this model for object segmentation in real images The state-of-the-art for segmentation algorithms use orientation- selective filters without inhibition 92

93 BICV Summer School, Shenyang, China, References G. Azzopardi and N. Petkov, A CORF computational model that relies on LGN input outperforms the Gabor function model, Biological Cybernetics, 106(3), , G. Azzopardi, A. Rodriguez-Sanchez, J. Piater, N. Petkov, A computational CORF model of a simple cell with push-pull inhibition exhibits improved SNR and outperforms state-of-the-art contour detection operators, to be submitted.

94 BICV Summer School, Shenyang, China, Matlab Code

95 BICV Summer School, Shenyang, China, Matlab Code Thank you! 谢谢 Grazzi! Dank jullie well!

96 Webster, Michael A, Russell L De Valois, et al. (1985). Relationship between spatial-frequency and orientation tuning of striate-cortex cells.in: JOSA A 2.7, pp References BICV Summer School, Shenyang, China, Anderson, JS et al. (2000). Orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex.in: Journal of Neurophysiology 84.2, Blakemore, C and EA Tobin (1972). Lateral inhibition between orientation detectors in cats visual-cortex.in: Experimental Brain Research 15.4, 439 &. ISSN: Daugman, J. G. (1985). Uncertainty Relation for Resolution in Space, Spatial-Frequency, and Orientation Optimized by Two-Dimensional Visual Cortical Filters.In: Journal of the Optical Society of America a-optics Image Science and Vision 2.7, pp Hubel, D. H. and T. N. Wiesel (1962). Receptive Fields, binocular interaction and functional architecture in cats visual cortex.english. In: J.Physiol.(Lond.) 160.1, pp Jones, J. P. and L. A. Palmer (1987). An Evaluation of the Two-Dimensional Gabor Filter Model of Simple Receptive-Fields in Cat Striate Cortex.In: Journal of Neurophysiology 58.6, pp Marr, D. and E. Hildreth (1980). Theory of Edge-Detection.In: B-Biological Sciences , pp Proceedings of the Royal Society of London Series Martinez, LM et al. (2005). Receptive field structure varies with layer in the primary visual cortex.in: Neuroscience 8.3, Nature Nothdurft, HC et al. (1999). Response modulation by texture surround in primate area V1: Correlates of popout under anesthesia.in: Visual Neuroscience 16.1, Petkov, N and MA Westenberg (2003). Suppression of contour perception by band-limited noise and its relation to nonclassical receptive field inhibition.in: Biological Cybernetics 88.3, Sceniak, Michael P et al. (2002). Contrast-dependent changes in spatial frequency tuning of macaque V1 neurons: effects of a changing receptive field size.in: Journal of Neurophysiology 88.3, pp

97 BICV Summer School, Shenyang, China, Hysteresis Thresholding