Self-organization is one of the major themes within complex systems and a pervasive phenomenon in our world. Complex organizations like schools of fish, ant colonies, and car traffic, manage to organize themselves into emergent patterns without any form of global coordination. This is somewhat counter intuitive to our traditional assumptions, where we tend to assume that organization and order need to be imposed by some external force. But self-organization fascinates many people precisely because it is generated internally. Hurricanes, consciousness, and swarms of bees are other examples of organization emerging out of the internal interaction between component parts.
Because of the adaptive capability of humans, whenever they interact locally, we invariably get some form of self-organization. Thus, self-organizing forms of social organization have always been there – from local markets to the formation of villages and towns. But one characteristic of the modern era has been the formation of large formal organizations, like bureaucratic governments and corporations. As we formed these larger organizations, the local interaction between members across the whole organization was no longer possible in the way it was in small communities. Given the scale of the organization and the available communication tools, coordinating such large organizations was only really possible through a centralized model. A limited amount of people in the center understood and formulated plans and directions to coordinate the entire organization, with information from the local level flowing into the center and directions flowing back out.
Because of economies of scale and other facts, the industrial age selectivity favored these large systems of organization as being more efficient. Small self-organizing systems lived on but largely as a fringe activity as they were never able to get the scale and efficiency to compete with the centralized model. Our industrialized economies became increasingly dominated by these large formal centralized organizations, which were, up until recently, seen as the only viable means for the effective production of value within society at scale.
Today, information technology, among other factors, is fundamentally changing this dynamic. We have the tools and platforms for connecting people directly one-to-one or one-to-many at virtually any scale. From a small group of friends all the way up to the global level, we can do this at very low cost. This is truly a game changer for organizational structure. The reality is that people now have very powerful tools in their hands, and this fundamentally shifts the power dynamic away from large organizations, which were very dominant until recently, towards the individual. This is what Gartner the consulting company calls “the age of the customer” and they talk about it as such: “To put it simply the age of the customer is all about the shift in power between you and your customers, now that shift is away from you and towards your customer. They are empowered by digital technologies… real-time access to information.”
With the access to education and information that many now have and the low barriers to connect and collaborate with others, it is now very much viable for ad hoc self-organizing systems of organization to form and compete effectively and directly with traditional centralized organizations. This is both a radical and massive shift in power. Classical examples being Wikipedia that has displaced the traditional encyclopedia, or the Linux foundation that has a larger market share, running web servers and smartphones, than any formal business organization.
The traditional organization had relatively high overhead costs, it required an almost constant input of finance or other resources to maintain it. With the lowering of collaboration cost, it is much easier to build organizations that are more agile and less dependent on significant investment of capital. This opens the door to harnessing social value and other motives to create organizations that are less solely focused on financial outcomes, but also now socially engaged. Thus, we have seen the rise of the social enterprise. In developed nations, Millennials are growing up in a world of relative abundance with an almost over saturation of things, but they also inherit a world of real problems that are not so easy to ignore. And out of this mix of relative abundance, ease of collaboration and pressing real world problems, we see this social enterprise arise, harnessing the engagement of people along a number of dimensions. In short, due to the context that they have grown up in, millennials have a new set of expectations surrounding what work is and they are less willing to simply fit in with the traditional model. Acquiring and retaining this talent is becoming a problem for traditional organizations, as they need to develop a new mentality and a more self-organizing bottom-up structure to accommodate the new workforce.
The Self-Organization Process
Whereas the traditional centralized organization is a relatively static model, self-organization is a dynamic process that requires certain conditions. Most importantly, it requires autonomous members densely interacting locally.
Autonomy is a very important ingredient here. Self-organization always comes from the bottom-up. People have to be operating and driven by their own motives and agenda and as long as they are constrained through some central organization this can’t happen. This is one reason why self-organizing systems can be more flexible and responsive – everyone is responding locally and can act immediately to what they experience, instead of having to send the information to some centralized authority and waiting for instructions to return to act. This is part of why centralized organizations are seen as more mechanical and ridge as opposed to self-organizing systems that are more organic and fluid in nature. The autonomy of the members to act locally is key. If we look at the process of self-organization within a flock of birds or school of fish, we get these emergent self-organizing patterns through each fish or bird operating under their own set of simple instructions to follow their local neighbors, while also maintaining a certain distance from them. The important thing to note here is that, if you want self-organization, you have to value and respect the autonomy of the members highly. One needs to be aware that every time you impose an instruction on a member, this has a very high cost in terms of reducing the self-organizing emergent capabilities of the system.
The second essential element is dense peer-to-peer interactions between the members of the organization. Interactions and information need to be flowing horizontally, not vertically. One needs to create the conditions for connecting people locally. Dense interactions are important because all the members may start out with divergent opinions, activities or agendas, but the more they interact, the greater the requirement to coordinate their states. As long as we build a big wall separating us, we can all go on doing our different things without any coordination or emergent organization. But when we take down that wall, we may well all come into initial conflict. However, it will be more difficult for us in the long run to maintain our divergent activities and there will be a much higher reward for coordinating our activities. We might say this is one of the dynamics at play in our world today. Globalization and hyperconnectivity have taken down the boundaries that previously divided societies and cultures, which enabled them to remain divergent for a prolonged period. And with this connectivity, they are brought into contact and often conflict. However, at the same time, it is making common consensus more valuable and remaining divergency more costly. It is this dense peer-to-peer interaction which fuels the process of self-organization.
What we find then is that those members that interact more often and come to some form of consensus or coordination then form an attractor. Because they are now working together, they are more effective than the other members in the organization, thus generating better results which make it more attractive for others to join that organization – they will get a greater reward from it. We now get a positive feedback loop as, the more people that join the organization, the more valuable it becomes for future prospective members to do likewise. This is the social network effect that is driven by positive feedback, and it is behind the rapid expansion of many forms of organization, such a financial markets, where the more people there are in the market, the more liquidity, which reduces transaction cost, which attracts more traders and so on. Another classical example would be the spread of English as a global language. Whereas previously we had many different societies speaking different languages, with the increase in global connectivity, it has become more valuable for us to have some common global language. Now that English has largely occupied that position, there is a positive feedback loop around it – the more people that learn it, the more valuable it becomes for others to learn. Even though only a small percentage of the global population speaks English natively, and even though there are other languages that are spoken natively by many more people, English will likely remain the global language because of its initial head start and this feedback loop that drives its development. The net result of this process is the emergence of some global pattern of coordination.
A large section of our traditional management paradigm is built up around the idea that the function of management is to get people to do what they would not spontaneously do. This makes sense because, if people spontaneously did what you wanted them to do, then you obviously would not need management in this sense. So, within this traditional paradigm, the idea of self-organization looks totally contradictory to the whole endeavor of management. It might be legitimate to ask “do we need management if we have self-organization?” The answer is, of course, yes, or else there would be little point in this whole course. But management in this new context has to be redefined. It is all about leading by creating or designing the future context, and this is where design thinking fits into management. We are designing the context, instead of directly controlling and coordinating.
David Snowden talks about managing self-organizing teams as being like managing your child’s birthday party. One would take a very different approach to organizing a child’s birthday party, as one would to managing a traditional organization. You would not specify roles for each child that comes to the party, get them to clock in, set them deadlines for completing certain projects and so on. The reason you don’t do this is because you are not looking for a direct outcome to this organization. What we are hoping for is a self-organizing emergent outcome, where people interact and, out of that interaction, we get the emergence of the children enjoying themselves, a child’s party.
But even though we are not trying to manage the system in a formal fashion, if we want the desired outcome we still need to organize the party. it is not going to happen at all otherwise. And what we are doing when we do that is creating the context that will induce the children to interact and self-organize. We put toys out, we make a little sports field out back where they can kick a ball around, we get a clown, some balloons, put some music on, all of these are creating the context. We create this context and put all these things out, but we don’t tell the children which ones to play with. What we will see is that attractors will spontaneously form around some of the things and the children will be drawn into that for some time. And during that time we will get the emergence of the fun party that we hoped for.
Another example of this approach to management would be Google giving their employees time off to work on projects of their own interest. Google isn’t, of course, doing this out of the pure goodness of their heart. They, of course, hope for some desired outcome, namely new innovations. But they know that the development of new ideas takes time and space. Like learning, it is a self-organizing process. New different ideas have to be tested, interact and combined before some coherent functional pattern may emerge. In order for Google’s management to enable this, they have to create a space that engages people’s self-motivation for the process to take place. They can then, later on, reap the benefits of what emerges out of this.