Complex Analytics Overview
The confluence of major trends in cloud computing, new data sources, advances in algorithms and the rise of the internet of things are assuring in profound changes that take us into the ear of big data. The first wave of the information revolution based around the personal computer and the world wide web has created a torrent of new data sources from web blogs, internet search histories, large-scale e-commerce practices, retail transactions, RFID tags, GPS, sensor networks, social networks and mobile computing have all worked to create what we now call big data. But this mass of data would be no use without computing capacities to process it. Throughout human history, computing power was a scarce resource. However, with the recent advent of global scale cloud computing, high-end computing is now available to organizations of almost all size at low cost and on-demand. The third major element that has fallen into place is a powerful new set of algorithmic approaches. Breakthroughs in machine learning and deep learning in particular now provide the software systems to process these ever more complex data sets; algorithms that learn from data, that can deal with millions of parameters, that can coordinate vast digital platforms, optimizing logistics networks, automating financial trades, predicting maintenance on electrical grids.
Many compare this to building a new kind of microscope or telescope. But whereas with the microscope we revealed the microscopic mysteries of life and with the telescope the stars and galaxies, this tool lets us see the complex systems all around us. With these new tools, we are now peering into massive unstructured data sets using ever more sophisticated algorithmic frameworks to see what we could never before see. The implications of this are huge; just as the telescope changed our understanding of our place in the universe, complex analytics is changing our understanding of the world around us, the systems we form part of and this opens the door to a shift in the nature of how we make decisions and management is conducted. Data is opening up our ability to perceive things around us that were previously invisible; our evolved social, economic and technological systems that have become so complex we can no longer see them are being revealed to us in new ways. More and more governments, business sectors, and institutions begin to realize data is becoming the most valuable asset and its analysis is becoming core to competitiveness. Today data is becoming a new universal language, mastering it can win sports matches, can make movies a success, can win elections, can build smart cities, can make the right trade at the right time, it may even win wars.
Emergence of large unstructured data sets, process of datafication and quantification, the expanding universe of dark data, and the paradigm of dataism
Algorithms are coming to effect ever more spheres of our world as platforms with machine learning systems at their core come to expand to more and more spheres of the economy
The convergence of machine learning with the internet of things is starting to populate our world with systems that exhibit adaptive, responsive and autonomous behavior
The relationship between people and technology is changing fast with the emergence of a new form of analytics and data-driven networked organization