Convergent Evolutionary Paths in Biological and Technological Networks

In order to illustrate our ideas, we use the protein–protein interaction map (or proteome) as our case study. Some of the structures found in these networks will reappear when we consider technological webs. In this context, proteome maps will help us understand the origins of a deep similarity between networks generated through evolution and their man-made counterparts. An example of protein network is shown in Fig. 1, where we display the human transcription factor (TF) network (Rodriguez-Caso et al. 2005). Here nodes represent TFs and edges linking two nodes indicate a physical interaction between them. TFs are proteins that directly interact with DNA, and to some extent what we display here is the map of the cell “hardware” that drives DNA dynamics. The links between different proteins indicate that two given TFs interact physically at some point, somewhere within the cell. They thus provide a picture of the cell’s complex machinery. In this context, the so-called degreek of a given protein is the number of proteins with which it interacts.

The pattern displayed by this web is fairly typical and is shared by many other biological, social, and technological graphs (Buchanan 2003). This particular network contains N = 230 elements (nodes) and L = 850 interactions (links), thus corresponding to an average connectivity (average number of links emanating from a node) . This is a low value, indicating that the network is sparse: the average number of interactions is much smaller than the maximum possible (k_max = N − 1). But from Fig. 1, we can clearly appreciate that the number of links of different proteins is very diverse: Many TFs have just one or two links, whereas a few—the so-called hubs—display many connections. The network also displays correlated subsets of proteins (Fig. 1a) indicated with different colors associated with the presence of modules (see below). Hubs and their connections with other proteins are highlighted in Fig. 1a. Hubs include the basal transcription initiator, several tumor suppressors (p53, P300) and proto-oncogenes (c-jun, c-myc, or c-fos). Their topological importance is consistent with an important role in terms of essential cellular functions. This heterogeneity is captured by the degree distribution P(k) (Fig. 2a) which gives the probability that a given TF has k links (Amaral et al. 2000).

These networks are also small worlds (SW). This interesting behavior relates two apparently antagonistic properties at the local and global scale in sparse networks. At the local level, it is shown that nodes in a SW graph are connected to a small number of neighbors (on average) and they tend to be connected among them too (i.e., many triangles are observed). On the other hand, most nodes can be reached from all others by a small number of hops, also known as the number of degrees of separation (d). In other words, the local structure is compatible with very efficient communication (Watts and Strogatz 1998).

Protein networks display modularity. One way of detecting this is to look at the topology of interactions among proteins. A module in this context would be a subset of elements having more connections among them than with other parts of the web (Wagner et al. 2007). Modularity can be detected and measured in different ways (Kepes 2007) and allows the detection of communities of preferentially related nodes. An example is shown in Fig. 1a, where proteins belonging to the same module share a common color. In many of these networks, modules appear to be organized around hubs, as can be seen by comparing a and b of Fig. 1.

Beyond modular patterns, a second level of analysis considers the frequency of subgraphs of a given size, also known as subgraph census (see Wasserman and Faust 1994 and references therein). In Fig. 2b, we display the subgraph census for n = 4 subgraphs for the human TF web. In this plot, we show the frequency of subgraphs ordered by rank, from the most common to the least. As we can see, their abundance decays quickly as the graphs become more dense (Solé and Valverde 2007).

These subgraphs have received considerable attention in relation to the so-called network motifs (Milo et al. 2002, 2004; Wolf and Arkin 2003). These are patterns of interconnections occurring in complex networks at numbers that are significantly higher than those expected from a randomized graph having the same number of nodes and links (Milo et al. 2002, 2004; Valverde and Solé 2005a, b). Some subgraphs (motifs) would be much more common while others (anti-motifs) would be much less common than expected. The analysis of their statistical distribution reveals that each class of natural and artificial network seems to display common patterns of motif abundances.

The statistical pattern has been interpreted as functionally meaningful: the higher relevance of a motif would be tied to a potentially important functional role. In this perspective, motifs have been proposed as the minimal building blocks of network complexity (Milo et al. 2002, 2004). It seems thus reasonable to expect their abundances to reflect some type of adaptive trait (Kahstan and Alon 2005). But other analyses do not support this view (Solé and Valverde 2007). In this context, a recent study (Mazurie et al. 2005) shows that network motifs are not subject to any particular evolutionary pressure to be preserved. The reason is that most motifs are not found in isolation and are part of larger aggregates. This is consistent with other studies indicating that a more appropriate, functionally meaningful approach requires considering higher-order interconnection patterns (Dobrin et al. 2004). Actually, improved understanding of cellular webs has led to the identification of much larger superstructures, some of them dubbed with new labels, such as themes and thematic maps (Zhang et al. 2005).

These studies show that reducing biological complexity to small subsystems of fixed size might not be possible. On the other hand, cellular networks are not generated through random processes, and we can ask how much of the deviations from random expectations are associated with the rules of network growth. As shown below, non-adaptive processes might pervade the origins of both modules and motifs.

How can we incorporate tinkered evolution into a model of proteome evolution? The simplest approach is using a duplication–divergence (DD) model (Ohno 1970; Patthy 1999; Wagner 2001; Hogeweg 2002) where growth takes place through random duplications followed by divergence in the redundant genes. Instead of taking into account many of the underlying complexities of the process, we will restrict ourselves to a graphic theoretic description of protein–protein interactions, as previously followed by several authors (Solé et al. 2002b; Dokholyan et al. 2006; Vázquez et al. 2003; Pastor-Satorras et al. 2003, Teichmann and Babu 2004; van Noort et al. 2004; Goh et al. 2005; Colizza et al. 2005; Ispolatov et al. 2005; Cordero and Hogeweg 2006; Foster et al. 2006; see also Koonin et al. 2006)

This model (and its variants) has been shown to successfully capture most statistical features displayed by real protein networks (Vázquez et al. 2003; Rice et al. 2005; Maere et al. 2005). The model has two parameters which can be tuned. Moreover, it has been shown (Wagner 2001) that the rates of link deletion are large, while new links have a much smaller probability. An important point to be made is that for large deletion rates, the system can become disconnected, whereas for small ones it becomes fully and densely connected. At intermediate values (consistent with estimations), sparse networks with most nodes forming a single large graph are observed. These networks are actually very similar to those found in their real counterparts (see Fig. 3a, b). They are small worlds, exhibit the same heterogeneity, and display hubs. These are not just qualitative observations: the simulated nets fit measurable properties found in protein nets very well.

The surprising similarity between non-functional toy models and the real protein maps is remarkable. This supports the idea that the “shapes” of protein maps result from non-adaptive processes. Such a view is fully confirmed by the analysis of modularity: Although modular organization is a desirable and functionally key property of cellular maps, it can arise as a by-product of the rules driving network growth. Modularity would be obtained “for free” without a small-scale tuning of protein–protein interactions. These results are confirmed by the analysis of motif abundances.

It has been shown (Solé and Valverde 2006, 2007) that the distribution of subgraphs is also a consequence of tinkering (see also Banzhaf and Kuo 2004; Kuo et al. 2006; Knabe et al. 2008). Specifically, if we count the number of different subgraphs using our previous model, it can be shown that the same census plot can be obtained (Fig. 3c). For most parameter combinations, the shape of this census is the same: an exponential function with less dense subgraphs being more common than denser ones. Interestingly, it was observed that a very good match for four-node subgraphs appears to occur at parameter values where proteome network distributions are recovered (Solé and Valverde 2007). Thus, provided that we tune deletion and addition rates so that a heterogeneous net is obtained, subgraph abundances are also fitted.

However, this pattern does not rule out an active role of selection at the lower scale: links need to be introduced and removed at appropriate rates so that the network integrity is maintained. In other words, although modules, motifs, and robustness might be, to some extent, a by-product of the duplication–divergence scenario, network connectivity needs to be properly tuned.

The results reported here with a non-directed model have been shown to be robust when more detailed implementations are used. When regulatory interactions are considered using protein interactions matching binding sites, it has been shown that the abundance of some well-known motifs (such as feed-forward loops) is largely a consequence of duplication and divergence (Cordero and Hogeweg 2006). These in silico results are consistent with data from transcription networks.

Let us go back to our comparison between design and evolution. Apparently, technological artifacts are expected to result from a tinkering-free process. It is actually interesting to see that engineering has been a source of inspiration for some system biologists who see biological designs as closely related to engineered structures (see Lazebnik 2002). Although all of them agree that there is no intentional design, there is some belief that structures closely relate to optimal (or nearly optimal) functions. If the proteome resembles the Internet (for example), the implication would be that they share common optimality principles of organization. Some examples support this view. A recent study by Moses et al. (2008) on the structure of microprocessors suggests that they follow similar scaling laws relating energy dissipation and spatial organization. But if we seek insight from technology, we should first ask ourselves if technology is really free from constraints and tinkered evolution. In this context, it is worth mentioning that electronic circuits have been shown to display small-world and scale-free patterns of organization (Ferrer-Cancho et al. 2001). This is a surprising result, given the well-known wiring cost problem affecting circuit designs and the almost two-dimensional packing of components on a surface. Although the rules of construction are different, the common pattern of topological organization suggests (once again) that strong constraints canalize network architecture. As will be shown below, this seems the case in large-scale technological networks (Fig. 4).

Computational models of technological innovation offer a promising approach to the evolution of artifacts. An example of this approach is the work of Brian Arthur and Wolfgang Polak. These authors used a simple evolutionary algorithm to evolve complex electronic circuits by combination (Arthur and Polak 2006). Specifically, this work starts using a minimal logic gate, the so-called NAND gate (Fig. 5a) as the elementary building block. A previous list of goals is given, defined as desired computations to be performed. Such goals allow the definition of a fitness function and selection of some given designs. The gates are randomly wired and the resulting circuits tested, so that those that work better are selected. Selected circuits are then preserved as new building blocks (Fig. 5b, c); thus the evolutionary process is carried out by combining more and more complex subsystems.

The model is successful in generating all desired computations, although the exact hardware that implements them is path-dependent: different runs generate different solutions. In spite of the enormous combinatorial space to be—in principle—explored, it is remarkable that complex functions (such as eight-bit adders) are obtained. Moreover, the final circuits involve some amount of junk pieces resembling some of the non-functional structures found in biological systems.

Further, the length of time designs take to evolve (how fitness changes) indicates that punctuated equilibrium seems to be an essential ingredient of change. It is also worth noting that this combinatorial origination of technological complexity leads to a timeline of design evolution that cannot be collapsed into a true evolutionary tree. If we try to draw such a thing, we end up defining a phylogenetic network, similar to the ones associated with prokaryotic evolution (Dagan and Martin 2009). Due to the widespread role of horizontal gene transfer, branches in these trees are inevitably connected through genetic exchange events. In a sentence: if something can be combined, it gets combined.

The previous model illustrates one approach to evolved systems that is based in simulation. Using a given set of basic rules and an appropriate definition of fitness, we can generate complex structures. Brian Arthur’s view of technological evolution strongly advocates for combination as the key source of change (Arthur 2010). In this view, technological designs would be closer to chemistry than to selection-based evolutionary forces. The latter would be involved as soon as key ideas are born and applied. And yet, as we turn our attention to the most important piece of technology used in our daily lives, the fingerprints of evolutionary dynamics and in particular tinkering reveal unexpected similarities between engineering and biology.

The Arthur–Polak model deals with a simplified view of technological change. But it certainly provides proof of our concept picture. The model generates new structures by combining preexisting components, somehow as technology creates new artifacts by means of using existing ones. But we can also analyze real designs by looking at the network structure of some designs and its evolution through time.

Most previous studies on cultural and technological evolution and their similarities have concentrated mostly on old forms of technology. This includes writing, steam engines, computers, and other artifacts. In some cases, it is possible to follow not only how structures change over time but also how they are linked through phylogenetic relations (Temkin and Eldredge 2007). However, the current, invisible but most widely used technological advance is software. Soon after the first computers were designed, software also emerges and, after a delay, starts becoming the key actor for the computer revolution along with smaller machines. Software programs perform controlling tasks, which are executed by means of the underlying hardware. Thus, software defines the set of logical constraints acting on a given hardware, seemingly as what happens between proteins and DNA. In this sense, software might be a better approach to biological networks and pathways than hardware (Cardelli 2007). Actually, it is interesting to see that software and programming languages have evolved through time—experiencing life-like patterns of change. These include both gradual changes and horizontal evolution but also sharp innovations (Sammet 1969). Interestingly, one of the greatest changes was the transition toward modular structures, the so-called object-oriented language. In such a language, the building blocks involve well-defined, reusable units of programming logic. In this way, it became possible to build programs composed of self-sufficient modules (objects), each containing all the information needed to manipulate its own data structure.

As it happens with cellular networks, the topology of software maps is both scale-free and modular (Fig. 6a) despite the fact that it emerges as the outcome of designed evolution (Valverde et al. 2002; Myers 2003; Valverde and Solé 2006). Obviously, there must be a basic building plan toward a final function or set of functions. The software engineer foresees the outcome of his work, although big software systems are in fact created by teams of developers. Each software engineer has been trained with some specific design principles (Pressman 2005) that should allow these systems to be of high quality, easily maintainable, faster, and evolvable. But there are a number of constraints affecting the software building process. Modularization is present and is the fingerprint of task allocation within the system: different blocks will be involved in more specific subfunctions. On the other hand, increased complexity leads to conflicts between different subparts. This is reflected for example at the topological level: small software maps tend to display a clean tree-like structure, whereas larger systems typically display much more complex patterns (Valverde and Solé 2006). But textbooks on software design principles make no mention of small worlds or scale-freeness. Actually, some universal patterns shown by these systems are in conflict with the basic rules of thumb used in software engineering. The reason for these patterns and why they seem inevitable stems from an unexpected actor: tinkering.

Duplications of software parts is much more common than one might think. They occur naturally at the level of code writing, when a given item to be created shares a number of common features with other previously created items. They are generally used at different scales, including large blocks involving millions of code lines. These blocks typically deal with some given tasks and define themselves as special subsets within software graphs. Since they are highly elaborated structures, once a given block has been generated, a large part of its structure might be reused, and it will. As the system becomes more and more complex, programmers are more prone to this reuse (known as cloning) since they know that a given structure already behaves well. New structures are thus developed within an increasingly complex and integrated structure where interdependencies force the reuse of previous parts. Moreover, empirical analysis of software evolution reveals that duplicated code is more stable than non-duplicated code (Krinke 2008).

An example of small-scale tinkering is illustrated in Fig. 6b. The nodes and their links correspond to part of a chessplayer program. Each element on the chessboard is characterized by its location in space, its pattern of movement, and its identity both as a chessman and a special class of it. All these generic features are required to define each piece and its behavior. If we have already written the code describing a Pawn, generating the new one for King is relatively easy. First, we clone the Pawn module and afterward changes are made so that the new chessman is created. As we would expect, the previous links to the general definition of chessmen and the movement object are also cloned. In other words, in order to create a new class of piece, we can take advantage of the already defined elements (Valverde and Solé 2005a). But in many other situations, reuse has to do with the intrinsic complexity of the system. It has been shown that the abundance of subgraphs in software networks can be explained as a result of the rules of network growth underlying their evolution (Valverde and Solé 2005a, b). These models are very similar to those described above in the context of proteome evolution.

If we take two different software projects and follow in time their evolution, some inevitable regularities will be found. They always evolve toward small-world, scale-free structures. On the other hand, the pattern of change displays universal trends, including an increase with time in the number of avalanches of modifications that need to be introduced, which follow a punctuated equilibrium pattern (Gorshenev and Pismak 2004). More importantly, the final outcomes of these projects are large-scale networks displaying very similar quantitative traits, including motif abundances (Valverde and Solé 2005a, b) and degree distributions. No matter the different natures of the function performed by each system, their structural pattern of organization converges toward the same, life-like architecture.