S. D. V. Stephens

notes / Basic Neuroscience and Geometry of Neuroscience / chapter 10: The Geometry of Vision

G10 Conformal Geometry of the Visual Cortex

This lecture is about the functional architecture of the primary visual cortex V1. Visual neurons filter optical signals and so extract local differential data; these local data are then integrated into a global geometry, through the global coherence of the functional architecture of the visual areas. The lecture is based on Petitot’s Neurogéométrie de la vision [Pe], on Ta, Shi, Barton, Brewer, Lu and Wang [TSBBLW] on human retinotopic mapping and conformal geometry, and on Wang, Gu, Chan, Thompson and Yau [WGCTY] on conformal mapping of brain surfaces (see the References at the end).

The primary visual cortex V1

Areas of the visual cortex

  • V1 is the first of the primary visual areas. There are numerous feedback connections from the successive areas, such as V2 and V4; here we focus only on the geometry of V1.
  • The high-resolution buffer hypothesis of Lee–Mumford (Lee, Mumford, Romero, Lamme, Vision Research, 1998): V1 is more than a bottom-up early module, and takes part in all the visual processes that require fine resolution.

The slide “Location of the Visual Areas” is a labelled drawing of the brain showing the frontal, parietal, temporal and occipital lobes, the cerebellum, the lateral geniculate nucleus (LGN), the inferotemporal cortex (ITC), area MT/V5 and area LO, and in the occipital lobe the nested visual areas V1, V2, V3, V3A, VP, V4v, V7 and V8, with V1 innermost at the occipital pole.

Structures in V1

Neurophysiology identifies three types of structure in primate V1:

  1. laminar;
  2. retinotopic (retinal mapping);
  3. (hyper)columnar.

Laminar structure. V1 is organized in 6 distinct horizontal layers, parallel to the surface of the cortex. We look in particular at layer 4 (and its sublayer 4C), which is the main target of the thalamocortical afferents and of the intra-hemispheric corticocortical afferents. It contains different types of stellate and pyramidal neurons. The slide shows a stained section of striate cortex with the sublayers 4A, 4B, 4C marked, the line of Gennari (a band of myelinated fibres in layer 4B, visible to the naked eye, which gives the striate cortex its name), layer 4 running along the fold, and the V1–V2 transition where this layering changes.

Retinotopy.

  • Adjacent neurons have receptive fields covering overlapping portions of the visual field.
  • The mappings of the visual input from the retina to the visual cortex are conformal maps: they preserve local shape and local angles, though not distances and sizes.
  • There is a logarithmic conformal mapping from the retina to the sublayer 4C of layer 4 of the laminar structure.
  • Note: in cortical areas other than V1, adjacent points of the visual field may be mapped to non-adjacent regions.

The slide reproduces from [TSBBLW] the two kinds of retinotopic data. A ring-shaped checkerboard stimulus, colour-coded by eccentricity (distance from the centre of gaze), produces a colour gradient across V1 on the inflated occipital cortex; a rotating wedge stimulus, colour-coded by polar angle, produces a second colour gradient running across the first. A small medial view of the brain marks the occipital region shown.

Conformal maps

Definition G10.1 (Conformal map). A conformal map is a biholomorphic map , where . Conformal maps are angle preserving.

The slide shows the images of the coordinate grid of the -plane under six such maps: , , , , and . Three of them are redrawn below. In each case the images of the horizontal lines (red) and of the vertical lines (blue) still cross at right angles (away from the points where , such as for and for ).

Three panels showing the images of the square coordinate grid under the conformal maps z^2/2 (two families of parabolas), e^z (concentric circles and rays) and sin z (confocal ellipses and hyperbolas with foci at plus and minus 1); in each panel the image curves cross at right angles.

A further picture from [WGCTY] shows a genus zero surface conformally mapped to : a closed triangulated surface in the shape of a hand, next to the sphere onto which it is conformally mapped, with the mesh carried onto the sphere.

Harmonic maps and minimal immersions

The reference for the Riemannian geometry in this part is Jürgen Jost, Riemannian Geometry and Geometric Analysis (Springer, 2002).

Isometric and minimal immersions

Let and be Riemannian manifolds of dimensions and , locally diffeomorphic to and , with metric tensors and respectively. Repeated indices are summed.

Definition G10.2 (Isometric immersion). A map is an isometric immersion if near each point of there is an open neighborhood on which is an isometry:

In general need not be an embedded submanifold of : an immersion can have self-intersections. Here we are interested in the submanifold case.

Definition G10.3 (Minimal isometric immersion). A local variation of is a family of immersions with . An isometric immersion is minimal if for all such local variations that is, if is a critical point of the volume functional (and one also wants an actual minimum).

The variational problem can be reformulated: being a critical point of the volume is equivalent to the vanishing of the mean curvature, which is in turn equivalent to the differential formulation with the Laplace–Beltrami operator on , where and is the inverse matrix of . (Beware of sign conventions: with the sign used here, on with the flat metric is .) The are the Christoffel symbols of the metric on , which give the covariant derivatives:

Remark (Geodesic equation). The geodesic equation is the special case of this variational problem (for length) for immersions of a real 1-dimensional curve in a Riemannian manifold .

Harmonic maps and energy

Definition G10.4 (Harmonic map). Harmonic maps are the solutions of the variational problem Here is not required to be an immersion: can have the same dimension as , or higher.

Heuristics: if a copy of made of elastic material is mapped to , the stable configurations are those of minimal elastic energy. So harmonic maps should be the minimizers of a suitable energy functional, as follows.

Definition G10.5 (Energy). The local energy of a differentiable map of Riemannian manifolds is where is the pullback to of the tangent bundle of . The energy of is

The variational problem gives the harmonic maps:

Harmonic maps in two dimensions

Riemann surfaces and conformal metrics

Definition G10.6 (Riemann surface). A Riemann surface is complex 1-dimensional (real 2-dimensional), with local coordinates and biholomorphic changes of variables

Let be a local coordinate on the Riemann surface , and a holomorphic change of coordinates. The Cauchy–Riemann equations give, on substituting, These identities say that a holomorphic change of coordinates preserves metrics of the following form.

Definition G10.7 (Conformal metric). A conformal metric (conformal structure) on a Riemann surface is given in local coordinates by with positive real valued, where and . In other words, conformal metrics satisfy

Definition G10.8 (Conformal map from a Riemann surface). Let be a Riemann surface and a Riemannian manifold with metric . A map is conformal if or equivalently

The equivalence holds because, with extended complex-bilinearly and ,

Holomorphic and antiholomorphic maps between Riemann surfaces with conformal metrics are conformal maps.

Harmonic maps from Riemann surfaces

The Laplace–Beltrami operator for a conformal metric on is where and .

Proposition G10.9 (Harmonic maps from a Riemann surface). A smooth map from a Riemann surface with a conformal metric to a Riemannian manifold is harmonic if and only if In particular the harmonic condition is independent of the conformal metric: it only depends on the complex structure of .

(Writing the harmonic map equation with , the whole left-hand side acquires the factor , which drops out.)

Proposition G10.10.

  1. Holomorphic and antiholomorphic maps between Riemann surfaces are harmonic.
  2. Any holomorphic or antiholomorphic function composed with a harmonic map gives another harmonic map .
  3. The energy of is conformally invariant (independent of the conformal metric, dependent only on the complex structure) and invariant under holomorphic or antiholomorphic changes of coordinates on .

Holomorphic quadratic differentials

Definition G10.11 (Holomorphic quadratic differential). A holomorphic quadratic differential on is with holomorphic: given two holomorphic vector fields on , is a holomorphic function on .

Proposition G10.12. A harmonic map determines a holomorphic quadratic differential

Proof. The harmonic map condition implies holomorphicity: differentiating, and absorbing the derivatives of into the Christoffel symbols, ∎

The case of genus zero:

Proposition G10.13. Every holomorphic quadratic differential on vanishes.

Proof. is covered by two charts , with coordinates and and change of coordinates . Then and this must be bounded for (so that the differential extends to all of ); hence as . But a holomorphic function on with as is identically zero (Liouville theorem). ∎

Corollary G10.14. Every harmonic map is a conformal map.

Proof. Since is harmonic, is a holomorphic quadratic differential, hence zero on ; and is exactly the condition for to be a conformal map. ∎

Retinotopic maps

The mapping of the visual field on V1

A picture of the unfolded striate cortex with the mapping of the visual field shows V1 flattened into a teardrop shape. The fovea (the cone density peak) sits at the narrow end; bands of increasing eccentricity, marked , , (near the peak of rod density) and the far periphery, follow one another towards the broad end, and lines of constant polar angle (the horizon, , , , ) cross these bands. The central few degrees of the visual field take up a large part of the cortex.

Retinotopic mapping (Tootell, Silverman, Switkes, De Valois, Science, 1982): a stimulus made of concentric rings and radial rays is recorded by deoxyglucose labelling in the flattened primate striate cortex, where the rings and rays appear as an approximately rectangular grid. The central area of the visual field is represented by a larger area in V1.

Distortions in the mapping (Connolly–Van Essen, Journal of Comparative Neurology, 1984): the slide shows the mapping of the visual field (A, a hemifield with circles of eccentricity to and radial lines) to the lateral geniculate nucleus (B) and to the striate cortex (C) in monkeys, with matching black patches in the three pictures. The representation of the central degrees of the visual field (shaded) occupies about of the cortex area.

Models of the retinotopic conformal map

Example G10.15 (The monopole model). The model, also referred to as the monopole model, maps the visual field coordinate to the cortical coordinate

Left: a polar grid on the right visual hemifield, circles of eccentricity 1 to 10 and rays every 15 degrees of polar angle. Right: its image under w = log(z + a) with a = 1, a D-shaped region where the rays fan out from the point log a and become horizontal lines, and the circles become nearly vertical lines that crowd together at large eccentricity.

In the figure (drawn for ), the circles of constant eccentricity are mapped to curves that become nearly vertical lines , and the rays of constant polar angle to curves that fan out from the point and become horizontal lines . Equal steps of eccentricity crowd together at the periphery, so the centre of the visual field gets most of the cortical area, while the grid still meets at right angles.

Example G10.16 (The wedge-dipole model). The more general model is also known as the wedge-dipole model.

Conformal mapping of human retinotopy [TSBBLW]

Ta, Shi, Barton, Brewer, Lu and Wang [TSBBLW] use a two-step procedure to model retinotopy by conformal mapping:

  1. a conformal map from the brain visual cortex to the unit disk;
  2. a conformal map from the visual field to the unit disk.

From the visual cortex to the unit disk: conformal flattening

The flattening goes through five stages, illustrated in [TSBBLW] on a cortical surface coloured by the retinotopic data:

  • (a) slice along a plane to isolate the visual cortex regions;
  • (b) the visual regions after slicing;
  • (c) the double covering (two copies of the sliced region glued along their boundary, a closed surface);
  • (d) projection of the double covering to a sphere;
  • (e) stereographic projection to the unit disk.

Mesh and -coordinates

Data collection provides:

  • a simplicial complex (mesh triangulation) of the cortical area;
  • colour gradient data for eccentricity and polar angle, that is, a parametrization of the visual stimulus in the visual field as

The figure from [TSBBLW] shows the mesh with the colour gradient data for eccentricity and for polar angle, which determine the -coordinates at each vertex of the mesh: the eccentricity and polar angle read off at a vertex give the point in the -plane.

The general technique for constructing the conformal mapping comes from Wang, Gu, Chan, Thompson and Yau [WGCTY].

Constructing the conformal maps by energy minimization [WGCTY]

Discrete energy and the discrete Laplacian

Definition G10.17 (Discrete energy). Let be the space of piecewise linear functions on the mesh (determined by their values at the vertices). Consider the quadratic form where are the edges, are the source and target vertices of , and are parameters. The energy functional is When all this is the Tutte energy.

Definition G10.18 (Discrete Laplacian). The discrete Laplacian of at a vertex is An energy minimizing satisfies . For vector valued functions, is applied componentwise.

Now let be a map between two meshes, embedded in Euclidean space . The normal component of is with the normal vector to at .

Definition G10.19 (Discrete harmonic map). A map is a harmonic map if and only if for every vertex (only a normal component, no tangential component), that is, if and only if the absolute derivative vanishes.

Conformal maps to by steepest descent

The solutions are not unique: if is a conformal map to , so is for every Möbius transformation acting on , Constraints are imposed to obtain a unique solution:

  • the zero-mass constraint: with
  • landmark constraints: manually labelled sets of curves or points, and the optimal Möbius transformation that reduces the distance between the images of the landmarks in the sphere .

Algorithm 1 [WGCTY]: steepest descent with the Tutte energy

  1. Start with the mesh and the Gauss map , the normal to at .
  2. Compute the Tutte energy .
  3. Compute the absolute derivative .
  4. Update by (fixed increment length ).
  5. Compute the Tutte energy . If it differs from by less than a fixed threshold , output ; otherwise update to and repeat.

The minimum is unique, and the algorithm converges to the Tutte embedding of the graph (the 1-skeleton of ) in the sphere .

Algorithm 2 [WGCTY]: from the Tutte embedding to a conformal map

  1. Compute the Tutte embedding as before, and its Tutte energy .
  2. Compute the absolute derivative and update .
  3. Compute the Möbius transformation that minimizes the norm of the mass center:
  4. Compute the harmonic energy , where the coefficients are for an edge in the boundary of the faces and , with where is the third vertex of the triangle face (and similarly for ).
  5. If differs from by less than , output the current function; otherwise update to and repeat.
Two triangles F_alpha and F_beta sharing the oriented edge e from s(e) to t(e), with opposite vertices v_alpha and v_beta and the angles theta_alpha and theta_beta at those vertices marked.

The ratio of the dot product to the length of the cross product is the cotangent of the angle at , so , where is the angle of opposite to .

Comparing parametrizations with landmarks

Algorithm 2 uses the minimization of the norm of the mass center over Möbius transformations. One also wants to evaluate how good a conformal parametrization is with respect to some given landmarks. Suppose we have obtained two parametrizations () and want to compare them in terms of the landmarks. This is again formulated with an energy functional, and we look for the Möbius transformation that minimizes this energy:

Landmarks are used to compare only over a finite set of points (or over some assigned curves). Say the landmarks are finite sets of points and , with a bijection , , between their preimages on . We look for the Möbius transformation that minimizes This is a non-linear problem; but assuming , so that is affine, , stereographic projection transforms it into a least squares problem.

The slide’s illustration (from [WGCTY]) shows the landmark constraints at work: two brain surfaces from different subjects with landmark curves (sulci) drawn on them, and their conformal images on the sphere, matched along the preassigned curves so as to minimize the landmark mismatch.

Spherical harmonics

Definition G10.20 (Spherical harmonics). For and with (degree and order), the spherical harmonics are with normalization constants and the associated Legendre polynomials, solutions of The spherical harmonics form an orthonormal basis of .

The slide plots the normalized associated Legendre functions on for and . The case , where is the Legendre polynomial, is redrawn here.

Graphs of the Legendre polynomials P_0 to P_4 on the interval from -1 to 1; all equal 1 at x = 1, and P_ell has ell zeros in the open interval.

A further picture shows the real spherical harmonics for as surfaces whose distance from the origin in each direction is the value in that angular direction, coloured blue where the function is positive and yellow where it is negative: one sphere for , three two-lobed shapes for , five for and seven for , with more lobes as grows.

Expansion of the cortical surface in spherical harmonics

Every has an expansion in spherical harmonics Suppose we have constructed a conformal mapping of the visual cortex to . Then we have coordinates on the cortex surface (embedded in ) with the angle coordinates on . These are functions the coefficients of their expansion in spherical harmonics.

  • The Fast Spherical Harmonic Transform computes the .
  • Compression, denoising, feature detection and shape analysis are performed more efficiently on the Fourier modes .

Two illustrations from [WGCTY]: a conformal map from to the brain surface, shown by carrying a fine checkerboard on the sphere onto the cortex, where the squares stay square though their sizes vary; and geometric compression, a row of four brain surfaces reconstructed using only the low spherical harmonics and rescaling to smaller low frequency coefficients, going from the full folded cortex to a smooth blob.

How good is modeling by conformal maps?

Deviation from conformality is measured by the deviation from right angles, through the inverse mapping from to the cortex surface. The figure from [WGCTY] shows the latitude–longitude grid of the sphere carried onto the folded cortex surface, and a histogram of the angles between the grid lines there: the distribution is sharply peaked at .

Quasiconformal maps and the Beltrami equation

The Beltrami coefficient

Definition G10.21 (Complex dilatation). A conformal structure at a point is determined by a complex dilatation with . Intuitively, a conformal structure picks an ellipse centered at the origin as the new circle.

Notation: for ,

Definition G10.22 (Beltrami equation). The Beltrami equation with coefficient is

Example G10.23 (Constant dilatation). If is constant, the function satisfies the Beltrami equation, since and . A round circle in the -plane corresponds to the ellipse in the -plane: the direction of its axes comes from the argument of , and its eccentricity from . (Writing , one has , so the axes point in the directions and , with ratio .)

Left: in the z-plane, the ellipse |z + mu z-bar| = 1 for mu = 0.4 e^{i pi/3}, with semi-minor axis 1/(1+|mu|) in the direction arg(mu)/2 and semi-major axis 1/(1-|mu|) perpendicular to it. Right: the unit circle in the h-plane, its image under h(z) = z + mu z-bar, with the two axis directions preserved.

Equivalent formulation

Start with complex coordinates and construct a new complex structure with and smooth -valued and . For this to be a complex structure we need to satisfy , which gives Write and . The change of variables corresponds to which gives with

Definition G10.24 (Beltrami differential). In this form the Beltrami equation reads and is called the Beltrami differential.

Existence of solutions and quasiconformal maps

Theorem G10.25 (Local solutions of the Beltrami equation). For real analytic there is a local solution of the Beltrami equation (Gauss, isothermal coordinates). For measurable (with ) local solutions also exist (Morrey).

Definition G10.26 (Quasiconformal mapping). A solution of the Beltrami equation on a local open set is a quasi-conformal mapping with complex dilatation . Its angular distortions are bounded: the Beltrami differential measures how close it is to being conformal.

Conformal structures on Riemann surfaces

Definition G10.27 (Conformal structure on a Riemann surface). A conformal structure on a Riemann surface is a section of a disk bundle over , with fiber the unit disk : local functions glued on the overlaps by

The local solutions of the Beltrami equation determine conformal coordinates for a Riemann surface , topologically equivalent to but with a new conformal structure.

Theorem G10.28 (Genus zero). In the genus zero case, by the Uniformization Theorem is conformally equivalent to , with a unique conformal isomorphism that fixes . We view as a quasiconformal isomorphism with dilatation .

Why conformal maps?

The conclusion from [TSBBLW]: computing the Beltrami coefficient for the regions of V1 and V2 where the eccentricity and polar angle data are reasonably smooth shows that the conformal map is a very good approximation.

From the neuroscience point of view one may ask why conformal maps. It seems the key is to achieve a mapping with energy minimizing properties while dilating the central part of the visual field without affecting angular relations.

Next: the columnar structure of V1

The next aspect of V1 to discuss is the columnar structure, another type of geometric structure present in the visual cortex V1. Hubel–Wiesel found columnar structures in V1: neurons sensitive to orientation record data , where

  • is a position on the retina;
  • is a line in the plane.

This gives a local product structure The slide’s picture is the “ice cube” model of a block of cortex: a row of orientation columns, each labelled by a short line segment whose orientation turns steadily from one column to the next, crossed by alternating ocular dominance slabs labelled L and R (left and right eye).

Definition G10.29 (Fiber bundle). A fiber bundle is a topological space (or smooth differentiable manifold) with base and fiber , together with

  • a surjection ;
  • fibers for all ;
  • an open covering of such that , with restricted to the projection on .

A section is a map with ; locally on ,

Model of V1: a bundle with base the retinal surface and fiber the set of lines in the plane. Topologically (a circle), so locally V1 is a product . We will see that this leads to geometric models of V1 based on contact geometry.

References

  • [Pe] Jean Petitot, Neurogéométrie de la vision, Les Éditions de l’École Polytechnique, 2008.
  • [TSBBLW] Duyan Ta, Jie Shi, Brian Barton, Alyssa Brewer, Zhong-Lin Lu, Yalin Wang, Characterizing human retinotopic mapping with conformal geometry: A preliminary study, 2014.
  • [WGCTY] Yalin Wang, Xianfeng Gu, Tony F. Chan, Paul M. Thompson, Shing-Tung Yau, Intrinsic Brain Surface Conformal Mapping using a Variational Method, Proceedings of SPIE Vol. 5370, 2004.
  • T.S. Lee, D. Mumford, R. Romero, V.A.F. Lamme, The role of primary visual cortex in higher level vision, Vision Research 38 (1998) 2429–2454.
  • Jürgen Jost, Riemannian Geometry and Geometric Analysis, Springer, 2002.
  • R.B. Tootell, M.S. Silverman, E. Switkes, R.L. De Valois, Deoxyglucose analysis of retinotopic organization in primate striate cortex, Science 218 (1982) no. 4575, 902–904.
  • Michael Connolly, David Van Essen, The representation of the visual field in parvicellular and magnocellular layers of the lateral geniculate nucleus in the macaque monkey, Journal of Comparative Neurology 226 (1984) no. 4, 544–564.