Self-organization is the emergence of a globally coherent pattern of organization out of the local interactions between initially independent components.1 Self-organization is a dynamic process through which patterns of organization emerge out of local level nonlinear interactions. This process can be understood in terms of a number of stages and prerequisites to the process, including an initial unregulated state, dense nonlinear interactions between agents through which positive feedback can amplify some small event into an attractor where agents synchronize their state to reduce transaction costs and leverage economies of scale, creating a stronger feedback mechanism. Out of this emerge distinct local patterns of organization that then have to either cooperate or compete on the global level, with the net result of this emergent process being a multi-level system with distinct irreducible levels of organization; what we might call institutions. Some of the key characteristics of these self-organizing systems include robustness, adaptability, and non-optimality.2
The ideas surrounding self-organization are in many ways counter-intuitive to our traditional beliefs about order having to be imposed from some external top-down design. This top-down approach is the default model that we inherit from the Industrial Age. It is the model to corporations, national economies, and most social organizations. But today, through the rise of information technology we see the emergence of new forms of distributed self-organizing networks that are capable of delivering value at a scale that we previously thought was not possible from this type of alternative organization, from car sharing to filesharing, to open source software. Now that information technology, the internet, and new software platforms have provided the enabling context, we are seeing that self-organization can in fact be and is increasingly becoming a mainstream economic model for production and consumption.
Our global economy is one of the most extraordinary examples of a complex system built largely through self-organization. Through the local incentives of millions of people and organizations around the world, we can produce something like a car that no individual in isolation could come close to producing and no one really controls or coordinates all of this. This order is an emergent property of millions of interactions around the globe, the same distributed dynamic that governs ant colonies, flocks of birds and the emergence of new cultures.3
But this process of self-organization and emergence is not a mystery. Through researchers having encountered it within many different areas, from chemistry and physics to engineering and sociology, we have pieced together a basic understanding of how it works and what conditions are required for it to take place. Self-organization is not a structure or static model like a hierarchy. It is a dynamic process that plays out over time. It is a process that is driven by an interplay between order and entropy. Unlike our traditional model that is all about maintaining order and stability, in the way that a government tries to regulate an economy so as to achieve some optimal state, self-organization is not like this at all. It inherently requires entropy to take place. By entropy, I mean some form of disorder or randomness.4
Entropy is the first condition in order to achieve self-organization. If we take a communist command and control economy where everything is regulated, with all the products having to be routed to a central authority and redistributed from there, this is a top-down model that is trying to hold the system in a particular configuration. As long as we have this structure, there cannot be self-organization.5 We have to have an initial state of randomness. When we have many people together in an unregulated environment, they will start to interact as they exchange goods and services between them. These are nonlinear interactions. They are happening in a distributed fashion. No one is coordinating them. They are created and driven by the local incentives of the agents.
The process of self-organization is facilitated by dense interactions. This is one reason why information technology is driving the emergence of self-organization systems because it enables a much greater density of distributed peer-to-peer interactions through which people can coordinate their activities. Cities are a good example. If you go to any city in a developing country, you will likely see self-organizing informal markets on many side streets because of the density of interactions that the urban environment enables.
Positive feedback plays a key role in the process of self-organization. Agents that interact more often and more intensely will come to synchronize their states, enabling more fluid, friction-free transactions. This could be people coordinating their activities to produce a product, or people trading on a regular basis. This coordination and reduction in transaction costs enable this subset of agents to be more productive, attracting others to this particular configuration in a positive feedback loop as we get the emergence of institutions, like a business, market or financial center.6
This positive feedback of economics of scale creates an attractor space, a particular set of states towards which any new component within the system will be drawn as it becomes a default. Cities are good examples of this. By having such a high density of people, they reduce transaction costs, increase economic coordination and leverage economies of scale as they become an attractor for anyone in the locality of the city looking for work, trade or business opportunities. We might cite Singapore as an example. Having offered itself as a center for free trade during the colonial era, it managed to reach a critical mass to become an attractor for trade and finance within Southeast Asia. But without global regulation and coordination, we will typically get a number of different local attractors forming. For example, Singapore is just one attractor within the global financial system. We also have New York, London, Tokyo, Shanghai, etc. Each of these is a different attractor that has emerged from their local context and now has to compete within this global environment.6
Order and Chaos
Self-organization is a dynamic process. Agents come together in a swarm-like fashion, like in a group of peers sharing files on the Internet, a flock of birds, or in bank syndication. The agents coordinate their activities but the organization that emerges is very much time dependent. There is no centralized force holding them in this configuration. They have to stay regenerating themselves in an evolutionary process that involves some interplay between order and entropy. We are far from properly understanding this dynamic. Thus, there are a number of hypotheses, such as the edge-of-chaos hypothesis, which is a metaphor for how some physical, biological, economic and social systems operate in a region between order and either complete randomness or chaos where the complexity is maximal. Researcher Stuart Kauffman has studied mathematical models of evolving computational systems in which the rate of evolution is maximized near this so-called edge of chaos.7
Put more simply, self-organization emerges out of entropy and randomness. In order for the system to stay regenerating itself over time, it may need a delicate balance between maintaining some coherent and structured patterns while also needing some randomness and entropy in order to say creating new patterns and adapting to changing circumstances. As an example, we might think about Schumpeter’s idea of creative destruction, in which new products, new forms of distribution and organization displace older forms. But in order to get this dynamic evolutionary process, the market needs to be able to both foster new structures emerging from entrepreneurs and disintegrate old structures. This is a very different vision to market than the traditional static equilibrium of supply and demand. It is a dynamic evolutionary one and we will be talking more about evolution in a later section.8
Self-organization is then both a creative process and a destructive one. Just as positive feedback loops can drive rapid self-organizing growth, they are also a part of disintegration. The sand pile model to self-organizing criticality best describes this phenomenon. In the sand pile model, grains of sand are dropped onto a pile one by one. As the pile starts to build up, some grains will roll off the side one by one or in small groups but some don’t as the side angle to the pile begins to reach a critical angle of repose before a very large avalanche takes place. The avalanche is a product of positive feedback. The more grains that are falling, the more likely they will displace additional grains that will then augment the size of the cascade, and so on. Thus, we get a positive feedback loop leading to a large avalanche.9
The size of the avalanche and the number of times it occurs represents a power law distribution, meaning there will be very many, very small cascades and very few, very large cascades. This power law distribution that is a common feature within nonlinear systems allows for events that are statistically virtually impossible within a linear system. These very large events are called black swan events. Many destructive phenomena exhibit this power law distribution including earthquakes, neuronal avalanches in the cortex, forest fires, landslides, epidemics and stock market crashes. Thus, these self-organizing systems like the sand pile are nonlinear in that a small perturbation to the system can have a very negligible effect or it can have a radically disproportionate one where a single grain inputted can cause a massive avalanche, and we don’t know when this will occur.9
Self-organization gives rise to new levels of organization, what are called integrative levels. It is out of this self-organization that we get the emergence of institutions from the micro level of a small local market to large business organizations, industries, economies and ultimately our whole global economy, which is a complex adaptive self-organizing system that has evolved over thousands of years. By looking at the economy as a self-organizing system, we can begin to recognize these emergent patterns that are not identifiable when we use standard linear systems, models where we simply aggregate up from the micro level. But with self-organization, we can get non-equilibrium and the emergence of attractors on different levels, with these attractors having their own emergent internal dynamics, meaning they can’t just be abstracted away or derived from simple aggregations of lower level phenomena, and they are very important to the behavior of the system.10
Self-generating systems of organization have a number of common features to them. Firstly, self-organizing systems are of course user generated. Because of this, we typically get much greater user engagement. When someone is self-employed, they will typically be more actively engaged than when they are employed by some centralized organization. We might think of how capitalism is able to harness the entrepreneurial spirit of people in a bottom-up fashion as they feel like they are able to change their lives through private enterprise, which is in contrast to more command and control economic system. Secondly, self-organizing systems are typically more robust because they are distributed. The system is not dependent upon one agent or a few agents for its future functionality. Many self-organizing systems can maintain themselves without the dependency upon highly specialized components, meaning many of the components can be easily swapped for any other if they fail.11
Thirdly, whereas the command and control organizational structures of hierarchies may be well suited to static and stable environments, self-organizing systems are well suited to dynamic, complex and volatile environments where needs are changing. They are inherently dynamic and adaptive because the agents are not held within a specific configuration by top-down administration. They are able to receive and respond immediately to events within their local environment, making them very flexible and adaptive. Lastly, there is no concept of optimization within self-organizing systems. There is no centralized authority designing or controlling the system in order to achieve an optimal outcome in the way that government typically does with economies. Thus, we will typically get results that are far from optimal.