Cerebral Cortex, Vol. 10, No. 2, 127-141,
February 2000
© 2000 Oxford University Press
Theoretical Neuroanatomy: Relating Anatomical and Functional Connectivity in Graphs and Cortical Connection Matrices
The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 92121, USA
 |
Abstract |
|---|
 Top Abstract IntroductionTheory and Computer...ResultsDiscussionReferences |
|---|
Neuroanatomy places critical constraints on the functional connectivity of the cerebral cortex. To analyze these constraints we have examined the relationship between structural features of networks (expressed as graphs) and the patterns of functional connectivity to which they give rise when implemented as dynamical systems. We selected among structurally varying graphs using as selective criteria a number of global information-theoretical measures that characterize functional connectivity. We selected graphs separately for increases in measures of entropy (capturing statistical independence of graph elements), integration (capturing their statistical dependence) and complexity (capturing the interplay between their functional segregation and integration). We found that dynamics with high complexity were supported by graphs whose units were organized into densely linked groups that were sparsely and reciprocally interconnected. Connection matrices based on actual neuroanatomical data describing areas and pathways of the macaque visual cortex and the cat cortex showed structural characteristics that coincided best with those of such complex graphs, revealing the presence of distinct but interconnected anatomical groupings of areas. Moreover, when implemented as dynamical systems, these cortical connection matrices generated functional connectivity with high complexity, characterized by the presence of highly coherent functional clusters. We also found that selection of graphs as they responded to input or produced output led to increases in the complexity of their dynamics. We hypothesize that adaptation to rich sensory environments and motor demands requires complex dynamics and that these dynamics are supported by neuroanatomical motifs that are characteristic of the cerebral cortex.
|
Introduction |
|---|
|
TopAbstractIntroductionTheory and Computer...ResultsDiscussionReferences |
|---|
There is an intricate relationship between the anatomical connectivity and the functional connectivity of the cerebral cortex. Anatomical connections among cortical areas and groups of cortical neurons are a major determinant of their functional connectivity, i.e. of the patterns of temporal correlations generated by the dynamics of their interactions. Conversely, functional connectivity can shape anatomical connectivity through plastic changes driven by selective events that occur during development and evolution.
As a result of efforts to compile databases of corticocortical and corticothalamic pathways in various species (Scannell et al., 1999
), a wealth of neuroanatomical data has become available in recent years. The growing need for an in-depth analysis of this information has prompted a variety of approaches. These approaches have focused mainly on structural aspects such as organization into streams and hierarchies (Young, 1992; Scannell et al., 1995; Hilgetag et al., 2000), wiring length (Mitchison, 1991), total volume occupied by connectivity (Ringo, 1991) and spatial placement of components (Cherniak, 1995). Only rarely, however, have structural analyses been placed in the context of functional connectivity and dynamics. In this paper, we aim at a theoretical analysis of neuroanatomical structures, in particular those of the cerebral cortex, in relation to the specific modes of functional connectivity that they can support.
To characterize the anatomical connectivity of a network, we utilize concepts and measures provided by graph theory (Harary et al., 1975; Bollobás, 1985; Palmer, 1985). Representing networks as graphs has the important advantage of a parsimonious structural description that allows comparisons of different connection patterns within a common theoretical framework. It is important to note that nervous systems exhibit distinct neuroanatomical patterns at several levels of scale, ranging from the fine structure of single neurons to local circuitry, and finally to pathways linking distinct cortical areas. In this paper, we focus our analysis on the level of pathways that link different areas of the cerebral cortex. This permits direct application of our theoretical analysis to published cortical connection matrices based on actual neuroanatomy (Felleman and Van Essen, 1991; Scannell et al., 1995).
To characterize the functional connectivity (Gerstein et al., 1978; Boven and Aertsen, 1990; Friston, 1994) of a network it must be implemented as a dynamical system. The neural activity that emerges can be described as a multidimensional stochastic process whose joint probability density function can be characterized by measures of entropy and mutual information (Papoulis, 1991). We concentrate here on a set of global functional measures that describe the functional connectivity of a given network, such as its entropy, integration and complexity (Tononi et al., 1998b). Entropy and integration capture, respectively, the overall degree of statistical independence and dependence. Of particular interest for the present study is the measure of complexity, which characterizes the interplay of functional segregation and functional integration, a central organizing principle of the cerebral cortex. More recently, we have also defined information-theoretical measures of matching complexity and degeneracy (Tononi et al., 1996, 1999). Matching captures how well a system's functional connectivity matches the statistical structure of input patterns. Degeneracy captures the ability of structurally different subsets of the system to produce similar effects on output patterns. Both measures characterize the interactions of a neural system with an environment on the basis of the network's functional connectivity.
In our analysis of the relationship between structural properties of neuronal networks and their functional dynamics, we proceed in three steps. First, we use artificially produced graphs in a search for distinct anatomical motifs that emerge after selection for different measures of functional connectivity such as entropy, integration and complexity. Second, we examine actual cortical connection matrices and analyze them both structurally using graph theory and functionally by implementing them as dynamical systems. Third, we investigate how interactions with an environment might drive the emergence of actual patterns of anatomical and functional connectivity that are found in the cerebral cortex.
To carry out this program, we have developed an iterative quasi-evolutionary procedure (referred to as ‘graph selection’) to search for networks that are of possible biological interest. This procedure, which is based on structural variation and selection, provides a fast and robust way of generating graphs that give rise to specific patterns of functional connectivity, e.g. graphs characterized by high entropy, high integration or high complexity. Selection for each of these different functional measures was found to be correlated with distinct classes of anatomical patterns. Networks obtained by selecting for high complexity were characterized by the formation of densely linked groups (or clusters) that are sparsely and reciprocally interconnected one to another. Graph-theoretical analysis revealed that graphs giving rise to high values of complexity contain a large number of short reciprocal paths or cycles, and that their connections can be arranged with very short overall wiring lengths. A graph-theoretical analysis of connection matrices of the macaque visual cortex and of the cat cortex yielded very similar structural characteristics. Furthermore, we found that when such cortical connection matrices were ‘run’ as dynamical systems they gave rise to functional connectivity with high complexity values. Using functional cluster analysis we identified dynamic groupings of cortical areas corresponding to the dorsal and ventral streams of the primate visual system. Finally, in selecting for graphs that match specific input patterns or produce specific output patterns, we observed an increase in complexity that was sustained by the distinctive neuroanatomical motifs exhibited by the cerebral cortex.
|
Theory and Computer Implementation |
|---|
|
TopAbstractIntroductionTheory and Computer...ResultsDiscussionReferences |
|---|
General Nomenclature
Throughout this paper, we distinguish between structural and dynamic aspects of neuronal networks that are based, respectively, on their anatomical connectivity and their functional connectivity.
Structural aspects are captured using concepts from graph theory. For the purpose of structural analysis, we describe neuronal systems X comprising n units and k connections as order-n digraphs Gn,k with n vertices and k edges (excluding self-connections, k 
n2 – n). The adjacency matrix Aij of a digraph corresponds to the system's connection matrix Cij, which describes its anatomical connectivity. Dynamic aspects are captured by implementing networks as dynamical systems. Assuming a multi-dimensional Gaussian stochastic process, global statistical measures of activity patterns can be derived from the system's covariance matrix COV, which describes the system's functional connectivity.
Graph Selection
In searching for relationships between patterns of functional connectivity and the underlying anatomy of networks, we will follow two different strategies. First, we investigate a large set of possible patterns of anatomical and functional connectivity, an approach we have adopted initially because it provides a relatively unbiased view of structure–function relationships. Second, we consider specific examples of actual neuroanatomical structures and investigate the functional dynamics to which they give rise. In pursuing both strategies, we need to bridge the gap between structure and function, by implementing anatomical networks as dynamical systems. This is accomplished by endowing units and connections with dynamical properties (such as synaptic weights, or the ability to sum up inputs and produce outputs, either in a linear or nonlinear fashion), and by injecting inputs or uncorrelated noise. Neural activity patterns can then be analyzed in terms of the deviations from statistical independence produced by their underlying connection patterns.
First, we turn to the problem of evaluating large numbers of networks and their dynamics. We consider global measures of dynamics as functions over a space of networks (graphs) of n units and k connections whose arrangement is described by the connection matrix Cij. The vastness of this space of possible graphs (‘graph space’), even for modest n and k (~10700 individual graphs, for n = 32, k = 256), precludes the use of exhaustive search techniques. To confront this problem, we employ a quasi-evolutionary strategy based on concurrent variation and selection of graphs. Variation is structural and is implemented as the continuous generation of varying anatomical patterns. Selection occurs according to a global measure of dynamics, which acts as a fitness function. To obtain dynamics, graphs are implemented as dynamical systems, i.e. as networks with defined properties of units and connections.
Figure 1 shows a schematic diagram of the computational procedure used. Graph selection takes place in h discrete generations, in which a global functional measure F (e.g. complexity; see below) is computed for each of u individual variant graphs. After the generation of an initial random set, all subsequent generations are created by copying the single graph with the highest selection score Fmax (‘parent’) and deriving u – 1 copies (‘offspring’) for which r edges are rewired (‘mutated’). This process is repeated for a total of hmax generations. The generation size u and the rewiring rate r define, respectively, the grain and the volume of the region of graph space around the parent that is explored in each generation. For example, choosing a small value for u and a large value for r will produce offspring that vary widely in comparison to their parent, sparsely occupying a large volume of graph space. Conversely, large values for u and small values for r result in offspring that closely resemble the parent and densely occupy a relatively small volume of graph space.
|
Throughout graph selection, all graphs maintain the original number of n vertices and k edges. We decided to use values for n large enough to allow comparison of graphs with real neuroanatomical matrices (Felleman and Van Essen, 1991
). It was important to avoid basing our conclusions on very small networks, the connection patterns and system dynamics of which may differ radically from those of large networks. Values of k were chosen to reflect mean levels of known connection densities (~25–35%) within extended cortical networks (Felleman and Van Essen, 1991; Young, 1993; Scannell et al., 1995, 1999).
In graph selection, two structural constraints are applied. We placed a constraint on the in-degree (i.e. the number of incoming connections) of each vertex of the graph population. The in-degree was initialized to k/n for each vertex and this value was maintained throughout graph selection (the ‘constant input constraint’). In addition, in each generation a strong bias was placed on maintaining offspring graphs as strongly connected (the ‘connectedness constraint’). A graph is strongly connected if at least one path exists between all pairs {i, j} of vertices, such that i can be reached from j and j can be reached from i. For all values of n and k used in this study, random graphs were strongly connected with very high probability (k/n > ln(n)) (Bollobás, 1985). Some mutations, however, may result in the formation of additional graph components (i.e. they disconnect the original graph into individual components), and such graphs were deleted from the procedure. While the connectedness constraint is a necessary prerequisite for a valid analysis of structure and function of individual graphs, we found that the constant input constraint could be relaxed without altering the major conclusions of this study.
Graph-theoretical Measures
The ijth entry of a graph's adjacency matrix Aij is equal to 1 if a connection is present between vertex i and vertex j, and 0 otherwise. A pair of vertices can be connected by zero, one or two (reciprocal) connections. We designate the fraction of edges for which a directly reciprocal connection exists as frecip. To describe the possible ways to traverse a graph Gn,k we assemble its path matrix Pqij. Paths are any sequence of adjacent edges from a source vertex vj to a target vertex vi in which all vertices (and therefore all edges) are distinct. The ijth entry of Pqij corresponds to the total number of unique paths containing q edges that exist between vertices vj and vi. A graph with n vertices may contain noncyclic paths composed of up to n – 1 edges. If vj = vi, paths are called cycles, with a maximal length of n edges. A strongly connected digraph has bidirectional paths between all pairs {i, j} of its vertices. The matrix of shortest paths Sij has as its elements the lengths of the shortest path linking vertices vj and vi. The global maximum of Sij is also called the diameter diamG of the graph; it is equivalent to the maximal number of steps needed to reach any vi from any vj. We define the global mean of Sij, also called the ‘characteristic path length’ lpath (Watts and Strogatz, 1998). The ‘cluster index’ fclust (Watts and Strogatz, 1998) evaluates how many connections out of all possible ones exist between the immediate neighbors of a vertex, averaged over all vertices.
We propose another graph-theoretical measure that expresses the probability that paths of length q – 1 can be completed as cycles of length q, terminating on the source vertex vj. This measure, which we call cycle probability pcyc(q), captures the relative abundance of cycles of length q within a given graph. Note that 0 pcyc(q) 1 and that pcyc(2) = frecip.
To visualize connection matrices as actual graphs plotted in two dimensions, we use a simple search algorithm that rearranges columns and rows of Cij (preserving graph-isomorphism) such that a cost function is minimized. The cost function is given by
 |
Dynamics and Global Functional Measures
In order to derive global statistical measures of functional connectivity, graphs are implemented as dynamical systems, i.e. neuronal networks with defined properties of units and connections. We characterize the dynamics of a system through measurements of deviations from statistical independence. We do not consider time delays or activity-dependent effects on connections, such as those that result from voltage dependency or synaptic modification.
Linear System Dynamics
Linear systems were employed in all graph selection computations conducted in the present study, as they allow analytic derivation of their covariance matrices (see below) and are thus computationally inexpensive. (For a discussion of nonlinear dynamics, see Discussion.) As in previous work (Tononi et al., 1994, 1996), all connections between units i and j (i 
j) are excitatory, with uniform and constant strengths wij (0 <wij << 1). In keeping with the ‘constant input’ constraint (see above), for each unit the sum of the afferent synaptic weights is set to a constant value w (w = wij • k/n = 0.8 in all runs shown in this paper). All systems were run by injecting uncorrelated noise into each of the units; the amount of noise was set such that most of the unit's variance was due to their mutual interactions. To assure that observed changes in global functional measures are due to increases in covariance and not due to changes in variance, a small self-inhibitory weight (0 < wii < wij) is added for each unit of the network before global functional measures are computed. A separate constraint optimization is applied such that these self-inhibitory weights limit the variance (as given by the diagonal terms of COV) of each unit to a fixed value 
( = 0.015 throughout this study).
Covariance Matrices
All global measures of dynamics (functional interactions) used in this study are derived from the covariance matrix COV of each system, under the assumption that the activity of its n units can be described as a Gaussian multidimensional stationary stochastic process (see Discussion). Under Gaussian assumptions, all deviations from statistical independence among the units can be described by COV. For linear systems, the matrix COV, reflecting the system's dynamics, can be obtained analytically from the connection matrix Cij. Considering the vector A of random variables that represents the activity of the units of a given system X subject to uncorrelated noise R, we obtain A = Cij * A + R, when the elements settle under stationary conditions. Substituting Q = [1 – Cij]–1 and averaging over the states produced by successive values of R, we obtain the covariance matrix
 |
Global Functional Measures
In this study, we use five global dynamic measures to characterize and also to select for a given system X: entropy H(X), integration I(X), complexity C(X), matching CM(X;S) between system X and input S, and degeneracy D(X;O) of system X producing output O. We note that all of these global measures can be applied to linear as well as nonlinear dynamical systems. Detailed derivations and discussions of these measures are given elsewhere (Tononi et al., 1994, 1996, 1999).
The entropy H(X) of a system measures its overall degree of statistical independence; it is the multivariate generalization of variance. Assuming stationarity, the entropy of a system X composed of n units is computed as (Papoulis, 1991)
 | (1) |
The integration I(X) measures the overall degree to which a system deviates from statistical independence. This measure is derived as the difference between the entropies of the individual components of X, considered independently, and the entropy H(X) of the entire system:
 | (2) |
Complexity expresses the extent to which a system X is both functionally segregated (small subsets of the system tend to behave independently) and functionally integrated (large subsets tend to behave coherently). Originally, neural complexity was defined as the sum of the average mutual information across all bipartitions of the system (Tononi et al., 1994). The mutual information MI between a subset Xk of size k of a system X and its complement X – Xk, defined as MI(Xk;X – Xk) = H(Xk) + H(X – Xk) – H(X), measures the portion of entropy shared by Xk and X – Xk. We have recently introduced a related mathematical definition of complexity which does not require the full distribution of the average mutual information over all subset sizes (Tononi et al., 1998b). It expresses the portion of the entropy of a system that is accounted for by the interactions among its elements. This measure is computed as
 | (3) |
An input pattern on a sensory sheet S connected to a neural system X produces changes in the system's functional connectivity. Some covariances between units of the system may be increased, while others may be decreased. If these changes match the intrinsic pattern of covariances produced by the system in the absence of input, the mutual information between many subsets of the system X will increase, resulting in an increase in its overall complexity (Tononi et al., 1996). We have expressed the matching, or ‘fit’, between a stimulus present on the sensory sheet S and a system X as the total complexity C(X)T when the input is present minus the intrinsic complexity C(X)I and the complexity C(X)S due to direct input from the sensory sheet.
 | (4) |
When an output pattern is produced by a system X connected to an output sheet O, many different subsets of the system may have effects on this output sheet. We refer to this ability to affect a set of output units in many different ways (i.e. through different subsets of the system) as the system's degeneracy (Tononi et al., 1999). Degeneracy can be expressed in terms of mutual information between the system's subsets and the output. In the same way as for complexity (equation 3), we have introduced a measure of degeneracy that does not require averaging among subsets of all sizes. This measure expresses the portion of the entropy of the output that is jointly accounted for by different elements of the system. It is computed as
 | (5) |
). Degeneracy is low (i) for systems in which each element affects the output sheet independently; and (ii) for redundant systems in which many elements can affect the output sheet in a similar way but these elements do not have independent effects. Instead, we found degeneracy to be high for systems in which many different elements can affect the output sheet in a similar way and at the same time can have independent effects. Functional Cluster Analysis
When analyzing a neural system X, an important first step in the analysis is to attempt to identify strongly interactive subsets of elements. Such subsets are characterized by a high degree of statistical dependence within the subset as well as a simultaneous low degree of statistical dependence with elements outside of the subset. Such functional clusters can be identified by measuring for many different subsets the cluster index CI(Xik) defined as (Tononi et al., 1998a)
 | (6) |
 |
k n – 2), we employed an evolutionary search algorithm. For each subset size k, this algorithm examines a large number of randomly chosen subsets and iteratively varies copies of subsets with high cluster index values for a fixed number of generations. For systems of moderate size (n = 32), convergence is achieved reliably. |
Results |
|---|
|
TopAbstractIntroductionTheory and Computer...ResultsDiscussionReferences |
|---|
Graph Selection for Entropy, Integration and Complexity
Selection of graphs for high entropy, integration and complexity yields fast convergence for moderate population sizes u and low rewiring rates r (u = 10, r = 1). Figure 2A shows graphs (n = 32, k = 256) and their corresponding COV matrices obtained at several stages (generations h = 1, 100, 500, 2000) during graph selection for high complexity. Plots on the extreme left show a graph drawn from an initial population of u = 10 random graphs; the complexity of its functional dynamics is low (C(X) = 1.92). Subsequent stages show graphs at generations 100 and 500, with gradually increasing complexity. Plots on the extreme right of Figure 2A show a graph obtained after h = 2000 generations of graph selection; its complexity is significantly increased (C(X) = 2.76). Note that graph selection does not alter n or k, but merely redistributes connectivity while satisfying the additional constraints of ‘constant input' and ‘connectedness' (see above). All observed changes in complexity are due to changes in covariances, as the variances of individual units are fixed ( = 0.015 throughout). Random graphs tend to produce a COV with uniformly low values, while networks with high C(X) have many strong and many weak covariances. Figure 2B plots the temporal evolution of C(X) over 10 individual graph selection runs for which C(X) was the selection criterion. On average, good asymptotic convergence of C(X) is achieved within 500–1000 generations. Figure 2C shows plots comparing the relative changes of H(X), I(X) and C(X) in the course of graph selection for high H(X), I(X) and C(X), respectively, each for a total of N = 10 individual simulations. In the plots, random graphs occupy a roughly central region from which selected graphs rapidly diverge, settling into distinct subregions. Graphs selected for C(X) and I(X) tend to dissociate. Graphs selected for high C(X) tend to decrease their corresponding I(X), while graphs selected for I(X) decrease their C(X). Graph selection for H(X) initially yields a concomitant increase in C(X). This is to be expected given the requirement to achieve maximal statistical independence under the ‘connectedness' constraint. Note, however, that the complexity of graphs selected for H(X) always remains below the complexity of graphs directly selected for C(X).
|
Graph selection is relatively insensitive to variations in parameters specific to the procedure (u, r) as well as to the populations of graphs (n, k,
). While we did not attempt to systematically explore all of these parameters, we found that the structural motifs of the final graphs were rather robust. Some differences emerged for graphs that were either very sparse (k 
n) or nearly fully connected (k n2 – n); however, these regimes were considered to be less relevant for analyses of cortical connectivity. We investigated different rewiring rates by determining r randomly from a Poisson distribution Pr = e–
r/r!, with a mean rewiring rate . With n = 32, k = 256 and = 0.015, low values for (ranging from = 0.001•k to = 0.01•k, i.e. from r = 1 to r 3) produced virtually identical and near optimal outcomes. At higher rates ( = 0.1•k), convergence was slow and suboptimal. Structural Analysis and Graph Theory
Each simulation of graph selection yielded graphs whose fine structure differed in detail, i.e. the graphs were structurally non-isomorphic (as revealed by comparing their out-degree distribution). Nonetheless, for any given global selection criterion (e.g. H(X), equation 1; I(X), equation 2; or C(X), equation 3) all solutions obtained from individual simulations shared common structural motifs. This consistency is in marked contrast to the fact that the different global measures yield graphs with radically different structural motifs. Figure 3 shows examples of a random graph and of graphs obtained after 2000 generations of graph selection for H(X), I(X) and C(X), respectively. While all the graphs maintain constant n, k and , and are strongly connected, even casual visual inspection shows that their connection patterns differ considerably, as do their patterns of covariances. Random graphs (Fig. 3A) exhibit no topological ordering, with relatively few reciprocal connections present. Their COV exhibits rather uniform values of covariance. Graphs with high H(X) (Fig. 3B) contain mostly reciprocal connections linking pairs of units that appear spread out without any apparent local clustering. This pattern is consistent with dynamics characterized by a very high degree of statistical independence. Graphs with high I(X) (Fig. 3C) consist of a central group of densely and reciprocally interconnected units, surrounded by a loose mesh of units that receive input from the central group and interact only weakly with each other. This pattern is consistent with maximal deviation from statistical independence of the entire system. Increasing k generally increases the size of the central group (not shown), without changing the overall pattern. Graphs with high C(X) (Fig. 3D) contain a very high proportion of reciprocal connections, and appropriate reordering of their Cij indicates the presence of dense groups of vertices. These groups appear to be linked by a relatively small number of reciprocal bridges. This pattern is consistent with the emergence of a functionally segregated and functionally integrated system.
|
Graphs obtained by graph selection can be structurally analyzed using tools from graph theory. Table 1
summarizes the results of an analysis of populations of random graphs, as well as graphs selected for high H(X), I(X) and C(X), all with n = 32, k = 256 and = 0.015. (To interpret Table 1, the reader should refer back to Fig. 3
for examples.) Random graphs are characterized by small lpath and diamG, as well as low frecip and fclust. Their small diameter and short path length indicate that no two vertices are separated by more than a few edges. Low values for frecip and fclust indicate that no distinct groups of vertices exist. Graphs with high H(X) have a very high frecip but give low values for fclust, indicating that grouping of vertices is absent, even though direct reciprocal connections dominate. Graphs with high I(X) have a large lpath and diamG, a low frecip and a very high fclust. These graphs contain numerous pairs of vertices that only interact over large distances; these vertices are found surrounding the densely interconnected and highly interactive central group. The existence of a single large group is responsible for the very high values for fclust. As k approaches n2 – n, graphs with high integration become more and more uniformly interconnected, i.e. the size of the central group approaches n. Of particular interest for the present study are graphs that give rise to highly complex functional dynamics (high C(X)). These graphs have small lpath and diamG, and high frecip, as well as high fclust. This combination indicates close proximity (and thus potential functional interaction) of all vertices in an architecture consisting of distinct groups that are reciprocally linked. Variations in the number of connections k relative to n produce essentially similar connectivities with distinct groups of different average size.
|
An efficient way of plotting graphs in two dimensions is to place their vertices on a circle. Using a simple search algorithm we rearranged the graphs such that vertices that shared many of their connections were placed next to each other. Such a reordering scheme (reminiscent of nonmetric multidimensional scaling, or seriation) (Goodhill et al., 1995
; Young et al., 1995) not only provides good graphical representations (see e.g. Fig. 2), but also, through a cost function, gives an indication of the total cost of wiring minimally needed to physically implement a given graph. Significant differences between graphs with high H(X) or I(X) on one side and high C(X) on the other side emerge in the potential wiring length lwire. Essentially, graphs that produce functional dynamics showing high entropy or high integration result in values for lwire that are close to or even exceed the wiring lengths of random graphs. This means that their connections cannot be arranged in any economical way. Only graphs producing highly complex dynamics can be ‘wired’ using a total wiring length that approaches the theoretical minimum. While we systematically tested only the potential wiring length for a ring-like arrangement of vertices, similar results are obtained for other embedding schemes (e.g. two-dimensional lattices).
While the index fclust (Watts and Strogatz, 1998) gives an indication of shared local neighborhoods or ‘cliques' in graphs, it fails to discriminate between graphs that consist of a single large group and those that are partitioned into multiple interconnected groups of vertices (see Table 1). In contrast, the distribution of cycle probabilities pcyc(q) allows one to distinguish between these cases and provides an indication of the group size. Figure 4A shows the distribution of the cycle probability for cycles of length q, pcyc(q), for each of the cases considered. Clearly, complex graphs are characterized not only by a high proportion of cycles of length q = 2 (equivalent to directly reciprocal connections, frecip) but also by significantly higher proportions of cycles of length q = 3 and q = 4. The eventual drop in cycle probabilities for longer cycles provides an indication of the average size of the individual groups. In random graphs, cycles of all lengths occur with approximately equal probability (which is mainly a function only of the overall connection density, i.e. pcyc(q) k/(n2 – n)). In graphs giving rise to high H(X), despite a very high probability for cycles of length q = 2, the probability of cycles of length q = 3 is close to zero. This is consistent with the requirement for the production of functional connectivity with high entropy (disorder) and reflects the apparent (Fig. 3B) lack of groups of vertices. Graphs giving rise to high I(X) have pcyc(q) distributions very similar to random graphs.
|
), as well as graphs selected for high H(X), I(X) and C(X) ( 
, 
and 
, respectively). (B) Cycle probability for the macaque visual cortex (‘Ctx’,•) and random (‘Rnd’, Analysis of Cortical Connection Matrices
A number of cerebral cortical connection matrices have become available for analysis. Of these, we chose the matrix of inter-connections between segregated areas of the macaque monkey visual cortex (Felleman and Van Essen, 1991) (n = 32, k = 315) and of interconnections between cat cortical areas (Scannell and Young, 1993; Scannell et al., 1995) (n = 65, k = 1136). We have examined these matrices in two ways. First, we analyzed them structurally (Table 2), in particular for the abundance of cycles of varying length pcyc(q) (Fig. 4B,C). Second, we have ‘run’ them as dynamical systems to determine the degree of complexity of their functional connectivity (Fig. 5) and the existence of functional clusters of highly interactive areas (Fig. 6).
|
|
|
Structural Analysis
To facilitate the structural analysis we converted the original connection matrices of the macaque visual cortex (Felleman and Van Essen, 1991) and the cat cortex (Scannell et al., 1995) to a binary format. In this format, elements of the resulting matrices signify the absence (0) or presence (1) of a pathway, without regard to its density or strength. In addition, following other authors (e.g. Young, 1992), the original macaque matrix was modified to eliminate areas PIT, CIT and STP; their connections were reassigned to areas PITd/PITv, CITd/CITv and STPp/STPa, respectively.
We compare structural data obtained from cortical matrices with structural data obtained from random graphs and from graphs selected for high C(X), both of equivalent size and degree of connectivity. Table 2 shows that graph-theoretical measures for the macaque visual cortex and the cat cortex produce values similar to those for complex graphs obtained by graph selection. In particular, cortical networks have small diameters and characteristic path lengths, but high frecip and fclust. Actual values for frecip may turn out to be somewhat higher as more anatomical information about reciprocal connections becomes available. Plotting the distribution of pcyc(q) for cortical matrices (Fig. 4B,C) reveals values for pcyc(q) for cycles of lengths at least up to q = 5 that are significantly above those obtained for random graphs, but that fall within the same range as those obtained for graphs selected for high C(X). The slow decline in pcyc(q) indicates a relatively large group size, probably in excess of five cortical areas. These results confirm the finding (Young, 1992; Hilgetag et al., 2000) that cortical areas can be partitioned into distinct streams or clusters based on their anatomical connectivity.
Functional Analysis
We implemented the macaque visual cortex (Felleman and Van Essen, 1991) and the cat cortex (Scannell et al., 1995) as dynamical systems, to determine whether the resulting functional connectivity has high or low complexity. In attempting to approximate the large-scale dynamics of cortical systems, we chose to implement them as linear systems, noting that not much is known at present about the extent of nonlinear functional interactions between areas of the macaque visual system. Cortical matrices ( = 0.01, wij = 0.04) were ‘run’ by injecting uncorrelated noise and deriving the system's covariance matrix. The results are shown in Figure 5. Both cortical matrices gave rise to functional connectivity (Fig. 5A,B) with high complexity (Fig. 5C,D). The values for complexity were significantly above those obtained from a number of random graphs of equivalent n and k. We produced structural variants of cortical connection matrices by randomly rewiring a fraction of their pathways. In virtually all cases, the complexity of the functional connectivity was decreased, suggesting that actual cortical connection matrices represent configurations of pathways that are ‘near-optimal’ with respect to complexity. This observation was robust with respect to changes in dynamical parameters (i.e. , wij) of cortical matrices.
The high complexity of cortical matrices is due to their anatomical organization into distinct groupings of areas, the presence of which was revealed by structural analysis. Next, we attempted to identify coherent sets of cortical areas by examining patterns of functional connectivity and performing functional cluster analysis (Tononi et al., 1998). Figure 6 shows the results of such an analysis performed on the macaque visual cortex. Figure 6A lists values for the cluster index CI(Xik) (equation 6) for a given subset size k, ranked by their statistical significance. Figure 6B displays the corresponding subsets i (corresponding to functional clusters). Reading Figure 6B from top to bottom provides a cross-section of the hierarchical organization of functional clusters in the macaque visual cortex. The first areas to be excluded from a cluster of size k = 30 were areas MIP and MDP; this may be explained by the lack of anatomical information for these areas. Next, area VOT dissociates away from the functional cluster (k = 29). VOT is an area with uncertain attachment in anatomical clustering schemes (Hilgetag et al., 1999). Next, over a series of steps, a set of six areas (PITd, PITv, CITd, CITv, AITd, AITv), all components of the inferior temporal cortex, dissociated from the main functional cluster (k = 23). These areas are all considered components of the ventral stream of visual cortex (Hilgetag et al., 1998). Then a second set of areas (STPp, STPa, TF, TH, 46 and 7a), also considered ventral, is found to dissociate (k = 17). These areas appear to form a subcomponent of the ventral stream that maintains stronger functional interactions with the dorsal stream (see also Fig. 7C). Given the definition of cluster boundaries, elements of each cluster can be ordered according to the amount of their mutual information with the other members of the cluster (here, k = 17; Fig. 7A). The original cortical connection and covariance matrices can then be rearranged (Fig. 7B,C; compare Fig. 7C with Fig. 5A) according to this ordering scheme. This reveals that areas within the dorsal and the ventral streams, respectively, are highly interactive, while functional interactions between the streams are more limited. A subset of ventral areas (STPp, STPa, TF, TH, 46 and 7a) interact more strongly with dorsal areas, rendering them strong candidates for mediating cross-stream interactions.
|
Overall, we note that functional cluster analysis yields information that is largely consistent with the anatomically based ‘optimal set analysis' of Hilgetag et al. (Hilgetag et al., 1998
, 2000). In particular, functional cluster analysis reproduces the fundamental dichotomy of the primate visual systems as expressed in the subdivision into dorsal and ventral streams. In addition, our analysis is sensitive to subgroupings of areas that cannot be subsumed under the dorsal/ventral ordering scheme. Graph Selection for Matching and Degeneracy
The analysis and simulations presented so far suggest that graphs giving rise to complex functional dynamics share distinct structural motifs, characterized by local groups of vertices and economical wiring lengths. In the last part of this study we consider graph selection for networks that efficiently match an input stimulus or produce an output stimulus with high degeneracy. This step is motivated by the notion that selectional processes shaping real neuronal networks during development and evolution occur while inputs are received and outputs are produced, i.e. as networks interact with an external environment (Edelman, 1987).
To produce graphs with high matching M(X;S) (equation 4), graph selection is conducted as described previously, but while an external input pattern is presented. This pattern is described by a covariance matrix COVS impinging on a sensory sheet S that is connected to a subset of units of the system X. An example of a network that has been selected to match a particular input pattern is shown in Figure 8A. After reordering (Fig. 8A, middle and right), it is evident that the internal connectivity of the network has arranged itself to form distinct groups of units. Arrows in Figure 8A (right) indicate the positions of the units receiving input from S.
|
To produce graphs that generate an output pattern with high degeneracy, we use a global functional measure that is given as a combination of degeneracy D(X;O) (equation 5) and a distance
, evaluating the closeness of the actually produced output pattern to the desired one:
 |
shows an example of a network that produces a particular output pattern with high degeneracy. Reordering of the vertices (Fig. 8B, middle and right) shows that the units that directly produce the output pattern (arrows) have become part of larger groups that are weakly interconnected.
In all cases, selection for matching or degeneracy resulted in graphs that gave rise to highly complex intrinsic functional dynamics (Fig. 9). These graphs show characteristic structural motifs similar to those of graphs selected for high complexity (Table 3; compare with Table 1). Thus, our simulations suggest that driving networks towards increased matching or increased degeneracy produces intrinsic dynamics of increasing complexity. This is accompanied by the emergence of neuroanatomical motifs that, as we have shown earlier, are capable of generating complex dynamics.
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionTheory and Computer...ResultsDiscussionReferences |
|---|
Neuroanatomy, in particular the patterns of interconnectivity found in networks, is crucial in determining the functioning of nervous systems. Can neuroanatomical structures be usefully analyzed from a theoretical perspective? A useful theoretical framework is provided by graph theory (Harary et al., 1975
; Bollobás, 1985; Palmer, 1985). All neuronal networks can be represented as graphs (or, more specifically, digraphs), ordered collections of vertices and edges. To date, most mathematical approaches have been limited to the study of random graphs (Erdös and Rényi, 1960; Cohen, 1988); however, given evolutionary selection, biological networks have distinctly nonrandom characteristics. So far, only a few studies have applied graph theory to neuroanatomical problems (Changeux et al., 1973; Nicolelis et al., 1990; Bienenstock, 1996; Jouve et al., 1998). In the present study, we use graphs as simple structural representations of extended brain networks, in particular as sets of brain areas (vertices) connected by pathways (edges). All such graphs, for a given combination of n and k, occupy a ‘graph space’, with each position of the space occupied by a graph with a different (nonisomorphic) connection pattern. We are interested in identifying which subregions of this space are associated with particular kinds of functional dynamics. To examine this vast space efficiently, we used graph selection, a fast and efficient way of searching for graphs that produce dynamics with certain statistical characteristics. The procedure itself resembles an evolutionary or genetic algorithm (Holland, 1992). The translation of structural neuroanatomy into functional dynamics is analogous to a mapping from a geno-typic to a phenotypic space. Unlike genetic algorithms, graph selection is directly based on specific functional (phenotypic) characteristics of the graphs, in particular the dynamical activity patterns they produce.
The relationship between structural and functional characteristics of a set of networks depends to some degree on the kind of dynamical system that is implemented. We chose a linear systems approach, in order to allow a computationally efficient analytic derivation of the system's covariance matrix. We have, however, tested other nonlinear dynamics, in particular implementations of spiking neurons linked by excitatory AMPA-ergic synaptic connections, following previous work (Lumer et al., 1997). Architectures characterized by interconnected groups of units produced high values for complexity, indicating that relationships similar to the ones sketched out for linear systems in this paper might also hold for at least one class of nonlinear systems (unpublished observations).
Consistent Structural Motifs and Functional Dynamics
A key question at the outset of this study was: are there consistent and coherent regions of ‘graph space’ with distinctive structural motifs such that graphs within these regions produce dynamics characterized by high entropy, integration or complexity? Our simulations show that this is indeed the case (see Fig. 3). Two points are worth noting. First, for a given selection criterion, irrespective of the initial condition (always a sample of random graphs with given n and k), all simulations converge on specific sets of solutions. Second, for different selection criteria, these specific sets of solutions have very different structural motifs. After appropriate reordering of vertices, even a simple visual inspection of the resulting matrices revealed striking differences between networks that were selected to produce different kinds of dynamics (Fig. 3).
The anatomical pattern associated with high complexity is particularly striking (Figs 2, 3 etc.). It consists of groups of densely interconnected vertices that are more sparsely interconnected among each other. Clearly, this anatomical pattern mirrors the dual requirements for functional segregation and integration, requirements that are met by the functional organization of the cerebral cortex, as reviewed elsewhere (Zeki, 1993; Mountcastle, 1998). It is important to note that over wide ranges of key parameters (n, k, , r, u, wij) graph selection for high complexity never produced graphs that substantially deviated from this general pattern.
To quantify specific structural motifs, we used several graph-theoretical measures. We employed measures introduced by Watts and Strogatz (Watts and Strogatz, 1998), the characteristic path length lpath and cluster index fclust, as well as associated measures, such as the diameter diamG and the fraction of reciprocal connections, frecip. Networks that give rise to complex dynamics are associated with values for lpath and fclust that are typical of so-called ‘small-world’ networks. We noted, however, that fclust does not distinguish between graphs containing a single large group of vertices and others containing multiple smaller ones that are interconnected; also, it gives no indication of the group size. To overcome these limitations, we introduced another measure, the cycle probability pcyc(q), which expresses the likelihood that any path of length q – 1 can be completed as a cycle of length q. Unlike fclust, which evaluates local connectivity only within a vertex's immediate neighborhood, pcyc(q) expresses the probability of finding cycles over all lengths q = {1, . . ., n}. In particular, pcyc(q) can distinguish between networks that are composed only of one central group (see Fig. 3, ‘Integration’) and others composed of many relatively dense, but globally interconnected groups (see Fig. 3, ‘Complexity’). While fclust is high in both cases (Table 1), the distribution of pcyc(q) shows very significant differences (Fig. 4A). Furthermore, the value of q at which pcyc(q) falls below corresponding values for random networks (see Fig. 4A) corresponds to the average group size within the network.
Cycle probability is a measure of the relative proportion of reentrant loops (Edelman, 1987, 1989) of length q within the network. Short reentrant loops (e.g. q = 2, 3, 4), which couple sets of units tightly into groups, are clearly prevalent in networks that give rise to complex dynamics (Tononi et al., 1994). In addition, such networks maintain long reentrant loops that link individual groups and ensure global coherency. This distribution of reentrant loops is consistent with their role in integrating neuronal activity within and between distributed and segregated brain areas (Sporns et al., 1991; Tononi et al., 1992). All other classes of networks we examined contain a much smaller proportion of reentrant loops (Fig. 4A).