BICV Summer School, Shenyang, China, 2015 1 Inhibition in V1 1. Non-classical receptive field 2. Push pull inhibition George Azzopardi Nicolai Petkov
BICV Summer School, Shenyang, China, 2015 2 1.Non-classical receptive field inhibition
BICV Summer School, Shenyang, China, 2015 3
BICV Summer School, Shenyang, China, 2015 4
BICV Summer School, Shenyang, China, 2015 5
BICV Summer School, Shenyang, China, 2015 6
BICV Summer School, Shenyang, China, 2015 7
BICV Summer School, Shenyang, China, 2015 8
BICV Summer School, Shenyang, China, 2015 9
BICV Summer School, Shenyang, China, 2015 10 Visual perception [Solomon & Pelli, 1994]
BICV Summer School, Shenyang, China, 2015 Visual perception [Petkov & Westenberg, 2003] 11
BICV Summer School, Shenyang, China, 2015 12
BICV Summer School, Shenyang, China, 2015 13
BICV Summer School, Shenyang, China, 2015 14
BICV Summer School, Shenyang, China, 2015 15 Visual perception [Galli & Zama, 1931]
BICV Summer School, Shenyang, China, 2015 16 Visual perception Orientation-contrast pop-out
BICV Summer School, Shenyang, China, 2015 17 Visual perception
BICV Summer School, Shenyang, China, 2015 18 Neurophysiology Receptive field and response of an orientation [Hubel & Wiesel, 1962] selective neuron
BICV Summer School, Shenyang, China, 2015 19 Neurophysiology Impulse response of an orientation selective neuron modelled by a 2D Gabor function [Daugman, 1985]
BICV Summer School, Shenyang, China, 2015 20 What is the neurophysiology of this visual perception?
BICV Summer School, Shenyang, China, 2015 21 Neurophysiology CRF Inhibition surround
BICV Summer School, Shenyang, China, 2015 22 Neurophysiology [Blakemore and Tobin, 1972]
BICV Summer School, Shenyang, China, 2015 23 Link to perception
BICV Summer School, Shenyang, China, 2015 24 Neurophysiology Mean firing rate [spikes/s] 60 40 20 Orientation-contrast cell [Nothdurft et. al, 1999]
BICV Summer School, Shenyang, China, 2015 25 Link to perception
BICV Summer School, Shenyang, China, 2015 26 Neurophysiology Mean firing rate [spikes/s] 60 40 20 (a) (b) (c) (d) General suppression cell [Nothdurft et. al, 1999]
BICV Summer School, Shenyang, China, 2015 27 Link to perception
BICV Summer School, Shenyang, China, 2015 28 Computational models
BICV Summer School, Shenyang, China, 2015 29 Gabor filter
BICV Summer School, Shenyang, China, 2015 30 Gabor filter Kernel: 27
BICV Summer School, Shenyang, China, 2015 31 Gabor filter Kernel: Filter:
BICV Summer School, Shenyang, China, 2015 32 Gabor energy filter
BICV Summer School, Shenyang, China, 2015 33 Gabor energy filter Gabor energy:
BICV Summer School, Shenyang, China, 2015 34 Gabor energy filter input I output E 0 o output E 90 o max(e 0 o,e 90 o )
BICV Summer School, Shenyang, China, 2015 35 Surround suppression CRF Inhibition surround
BICV Summer School, Shenyang, China, 2015 36 Surround suppression Different effects of the surround
BICV Summer School, Shenyang, China, 2015 37 Surround weighting function
BICV Summer School, Shenyang, China, 2015 38 Surround weighting function
BICV Summer School, Shenyang, China, 2015 39 Surround weighting function
BICV Summer School, Shenyang, China, 2015 40 Orientation-contrast suppression Mean firing rate [spikes/s] 60 40 20
BICV Summer School, Shenyang, China, 2015 41 Orientation-contrast suppression term
BICV Summer School, Shenyang, China, 2015 42 Orientation-contrast suppression term Input I Gabor energy E 0 o Suppression S 0 o term
BICV Summer School, Shenyang, China, 2015 43 Orientation-contrast suppression filter
BICV Summer School, Shenyang, China, 2015 44 Orientation-contrast suppression filter
BICV Summer School, Shenyang, China, 2015 45 Orientation-contrast suppression
BICV Summer School, Shenyang, China, 2015 46 Back to visual perception
BICV Summer School, Shenyang, China, 2015 47 Back to visual perception 38
BICV Summer School, Shenyang, China, 2015 48 Computer vision application
BICV Summer School, Shenyang, China, 2015 49 Texture suppression Input Gabor energy With suppression
BICV Summer School, Shenyang, China, 2015 50 Contour enhancement by texture suppression Input Gabor energy With suppression
BICV Summer School, Shenyang, China, 2015 51 Binary contour maps Input Edge strength Binarized response
BICV Summer School, Shenyang, China, 2015 52 Desired output Input image and associated desired output Dataset: www.cs.rug.nl/ imaging
BICV Summer School, Shenyang, China, 2015 53 Performance measure Desired output Operator output
BICV Summer School, Shenyang, China, 2015 54 Performance measure
BICV Summer School, Shenyang, China, 2015 55 Canny performance Input Desired output Best Canny P = 0.23
BICV Summer School, Shenyang, China, 2015 56 Suppressed Gabor performance Input Desired output Suppressed Gabor P = 0.42
BICV Summer School, Shenyang, China, 2015 57 Performance comparison Input Best Canny Suppressed Gabor P = 0.23 P = 0.42
BICV Summer School, Shenyang, China, 2015 58 Performance comparison Input Best Canny Suppressed Gabor P = 0.14 P = 0.34 [Grigorescu et al., 2003]
BICV Summer School, Shenyang, China, 2015 59 Performance comparison 1 0.8 Goat 3 Elephant 2 Hyena Gazelle 2 0.6 Performance 0.4 0.2 0 C G B1 B2 C G B1 B2 C G B1 B2 C G B1 B2 C G B1 B2 (C) Canny, (G) Gabor, (B1) orientation contrast suppression, (B2) general suppression
BICV Summer School, Shenyang, China, 2015 60 Beyond receptive fields and Gabor functions
BICV Summer School, Shenyang, China, 2015 61 Beyond receptive fields and Gabor functions Canny edge detector
BICV Summer School, Shenyang, China, 2015 62 Beyond receptive fields and Gabor functions Canny edge detector with surround suppression
BICV Summer School, Shenyang, China, 2015 63 Canny with surround suppression Input Canny With suppression [Grigorescu et al., 2004]
BICV Summer School, Shenyang, China, 2015 64 Split surround
BICV Summer School, Shenyang, China, 2015 65 Input Best Canny Suppressed Gabor
BICV Summer School, Shenyang, China, 2015 66 Multi-scale Canny with split surround suppression Input Suppressed Canny (multi-scale) Desired output [Papari et al., 2006]
BICV Summer School, Shenyang, China, 2015 67 2. Push-pull inhibition
BICV Summer School, Shenyang, China, 2015 68 Simple Cells receive push-pull inhibition (Anderson et al., 2000) Stimuli of the reverse contrast evoke responses of the opposite sign With push-pull inhibition linear models achieve contrast invariant orientation tuning CORF model (without inhibition) Already achieves contrast invariant orientation tuning Only responds to preferred orientation and contrast We explore further properties
BICV Summer School, Shenyang, China, 2015 69 Push-Pull inhibition RFs arrangement of LGN cells with preferred polarity RFs arrangement of LGN cells with opposite polarity push (excitation) pull (inhibition) Preferred stimulus
BICV Summer School, Shenyang, China, 2015 70 Overlap index (Martinez et al., 2005)
BICV Summer School, Shenyang, China, 2015 71 Overlap Index 1 β = 0 Normalized response to preferred orientation 0.57 0.19 0.04 β = 2 β = 4 β = 6 90 100 106 120 Orientation bandwidth (degrees)
BICV Summer School, Shenyang, China, 2015 72 Response of CORF with push pull inhibition
BICV Summer School, Shenyang, China, 2015 73 Response of CORF with push pull inhibition r S (x, y )
BICV Summer School, Shenyang, China, 2015 74 Response of CORF with push pull inhibition r S (x, y ) r Sβ (x, y ) + β
BICV Summer School, Shenyang, China, 2015 75 Response of CORF with push pull inhibition r S (x, y ) kr Sβ (x, y ) + β
BICV Summer School, Shenyang, China, 2015 76 Signal-to-noise Ratio Edge stimulus Band-limited texture
BICV Summer School, Shenyang, China, 2015 77 Signal-to-noise Ratio + = Edge stimulus Band-limited texture Test image
BICV Summer School, Shenyang, China, 2015 78 Signal-to-noise ratio Input Output of models CORF CORF+PP SNR= 20.35 SNR= 31.54 SNR= 9.45 SNR= 23.83
BICV Summer School, Shenyang, China, 2015 79 Signal-to-noise ratio Input Output of models CORF CORF+PP SNR= 12.06 SNR= 28.68 SNR= 2.83 SNR= 14.73
BICV Summer School, Shenyang, China, 2015 80 Signal-to-noise ratio Input Output of models CORF CORF+PP SNR= 17.57 SNR= 21.52 SNR= 5.85 SNR= 8.18
BICV Summer School, Shenyang, China, 2015 81 Separability of spatial frequency and orientation Real simple cell (Webster, De Valois, et al., 1985). CORF without inhibition Fundamental amplitude 50 40 30 20 10 Orientation ( ) 5 20 35 50 65 0 0.17 0.33 0.67 1.33 2.67 5.34 Spatial Frequency (cycles/degree) Normalized response 1 Orientation ( ) 0 10 25 30 35 0 2 4 8 16 32 Spatial Frequency
BICV Summer School, Shenyang, China, 2015 82 Separability of spatial frequency and orientation Real simple cell (Webster, De Valois, et al., 1985) CORF with push-pull inhibition Fundamental amplitude 50 40 30 20 10 Orientation ( ) 190 200 215 220 225 0 0.17 0.33 0.67 1.33 2.67 5.34 Spatial Frequency (cycles/degree) Normalized response 1 Orientation ( ) 0 10 25 30 35 0 2 4 8 16 32 Spatial Frequency
BICV Summer School, Shenyang, China, 2015 83 Spatial frequency tuning sensitive to contrast (Sceniak et al., 2002) Normalized response 1 0.5 0 CORF without inhibition 1 2 4 8 16 32 Spatial Frequency CORF with push-pull inhibition Contrast 80% 20% 1 2 4 8 16 32 Spatial Frequency
BICV Summer School, Shenyang, China, 2015 84 Contour detection Input image CORF with push-pull inhibition CORF without inhibition
BICV Summer School, Shenyang, China, 2015 85 Contour detection Input image CORF with push-pull inhibition Gabor with isotropic surround suppression
BICV Summer School, Shenyang, China, 2015 86 Contour detection Input image CORF with push-pull inhibition Canny
BICV Summer School, Shenyang, China, 2015 87 Contour detection Input image CORF with push-pull inhibition CORF without inhibition
BICV Summer School, Shenyang, China, 2015 88 Contour detection Input image CORF with push-pull inhibition Gabor with isotropic surround suppression
BICV Summer School, Shenyang, China, 2015 89 Contour detection Input image CORF with push-pull inhibition Canny
BICV Summer School, Shenyang, China, 2015 90 Results: CORF with push-pull inhibition vs others Right-tailed paired t -test RuG: 40 images CORF without inhibition: p < 0.01 Gabor with surround suppression: p < 0.01 Canny: p < 0.01 Berkeley: 500 images CORF without inhibition: p < 0.01 Gabor with surround suppression: p < 0.01 Canny: p < 0.01
BICV Summer School, Shenyang, China, 2015 91 Conclusions Computational modelling of two types of inhibition Gabor à non-classical receptive field CORF à push-pull inhibition CORF+PP improves SNR CORF+PP is more effective than Gabor with surround suppression in contour detection
BICV Summer School, Shenyang, China, 2015 Outlook Combine non-classical receptive field and push-pull inhibition in one model? Contour detection is expected to be improved even further Incorporate this model for object segmentation in real images The state-of-the-art for segmentation algorithms use orientation- selective filters without inhibition 92
BICV Summer School, Shenyang, China, 2015 93 References G. Azzopardi and N. Petkov, A CORF computational model that relies on LGN input outperforms the Gabor function model, Biological Cybernetics, 106(3), 177-189, 2012. G. Azzopardi, A. Rodriguez-Sanchez, J. Piater, N. Petkov, A computational CORF model of a simple cell with push-pull inhibition exhibits improved SNR and outperforms state-of-the-art contour detection operators, to be submitted.
BICV Summer School, Shenyang, China, 2015 94 Matlab Code http://matlabserver.cs.rug.nl/
BICV Summer School, Shenyang, China, 2015 95 Matlab Code http://matlabserver.cs.rug.nl/ Thank you! 谢谢 Grazzi! Dank jullie well!
Webster, Michael A, Russell L De Valois, et al. (1985). Relationship between spatial-frequency and orientation tuning of striate-cortex cells.in: JOSA A 2.7, pp. 1124 1132. References BICV Summer School, Shenyang, China, 2015 96 Anderson, JS et al. (2000). Orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex.in: Journal of Neurophysiology 84.2, 909 926. Blakemore, C and EA Tobin (1972). Lateral inhibition between orientation detectors in cats visual-cortex.in: Experimental Brain Research 15.4, 439 &. ISSN: 0014-4819. Daugman, J. G. (1985). Uncertainty Relation for Resolution in Space, Spatial-Frequency, and Orientation Optimized by Two-Dimensional Visual Cortical Filters.In: Journal of the Optical Society of America a-optics Image Science and Vision 2.7, pp. 1160 1169. Hubel, D. H. and T. N. Wiesel (1962). Receptive Fields, binocular interaction and functional architecture in cats visual cortex.english. In: J.Physiol.(Lond.) 160.1, pp. 106 154. Jones, J. P. and L. A. Palmer (1987). An Evaluation of the Two-Dimensional Gabor Filter Model of Simple Receptive-Fields in Cat Striate Cortex.In: Journal of Neurophysiology 58.6, pp. 1233 1258. Marr, D. and E. Hildreth (1980). Theory of Edge-Detection.In: B-Biological Sciences 207.1167, pp. 187 217. Proceedings of the Royal Society of London Series Martinez, LM et al. (2005). Receptive field structure varies with layer in the primary visual cortex.in: Neuroscience 8.3, 372 379. Nature Nothdurft, HC et al. (1999). Response modulation by texture surround in primate area V1: Correlates of popout under anesthesia.in: Visual Neuroscience 16.1, 15 34. Petkov, N and MA Westenberg (2003). Suppression of contour perception by band-limited noise and its relation to nonclassical receptive field inhibition.in: Biological Cybernetics 88.3, 236 246. Sceniak, Michael P et al. (2002). Contrast-dependent changes in spatial frequency tuning of macaque V1 neurons: effects of a changing receptive field size.in: Journal of Neurophysiology 88.3, pp. 1363 1373.
BICV Summer School, Shenyang, China, 2015 97 Hysteresis Thresholding