The information revolution that began in the mid-twentieth century is entering a new stage of development; the confluence of major trends in cloud computing, new data sources, advances in algorithms and the rise of the internet of things are assuring in profound changes that take us into the ear of big data. The first wave of the information revolution based around the personal computer and the world wide web has created a torrent of new data sources from web blogs, internet search histories, large-scale e-commerce practices, retail transactions, RFID tags, GPS, sensor networks, social networks and mobile computing have all worked to create what we now call big data.
But this mass of data would be no use without computing capacities to process it. Throughout human history, computing power was a scarce resource. However, with the recent advent of global scale cloud computing, high-end computing is now available to organizations of almost all size at low cost and on-demand. The third major element that has fallen into place is a powerful new set of algorithmic approaches. Breakthroughs in machine learning and deep learning in particular now provide the software systems to process these ever more complex data sets; algorithms that learn from data, that can deal with millions of parameters, that can coordinate vast digital platforms, optimizing logistics networks, automating financial trades, predicting maintenance on electrical grids.
With these new tools, we are now peering into massive unstructured data sets using ever more sophisticated algorithmic frameworks to see what we could never before see. Many compare this to building a new kind of microscope or telescope. But whereas with the microscope we revealed the microscopic mysteries of life and with the telescope the stars and galaxies, this tool lets us see the complex systems all around us. Data is opening up our ability to perceive things around us that were previously invisible; our evolved social, economic and technological systems that have become so complex we can no longer see them are being revealed to us in new ways.
The implications of this are huge; just as the telescope changed our understanding of our place in the universe, complex analytics is changing our understanding of the world around us, the systems we form part of and this opens the door to a shift in the nature of how we make decisions and management is conducted. More and more governments, business sectors, and institutions begin to realize data is becoming the most valuable asset and its analysis is becoming core to competitiveness. Today data is becoming a new universal language, mastering it can win sports matches, can make movies a success, can win elections, can build smart cities, can make the right trade at the right time, it may even win wars.
This course explores the world of complex data analytics: information systems that are able to analyze big data and transform these restless streams of data into insight, decisions, and action. Complex analytics focuses on how we extract the data from a complex system – such as a financial market, a transport network or a social network – and process that into meaningful patterns and actionable insights.
After starting the course with an overview to the subject we will look at the emergence of big data and the expanding universe of dark data; we talk about the ongoing process of datafication, the quantification of more and more aspects of our lives and the many issues that it brings with respect to privacy.
In the second section, we will talk about the rise of algorithms as they are coming to effect ever more spheres of our world. We will introduce you to the workings of machine learning systems and the different approaches used, we go more in-depth on neural networks and deep learning before assessing the limitations of algorithms.
The third section is dedicated to smart systems, as the convergence of machine learning with the internet of things is beginning to populate our world with systems that exhibit adaptive and responsive behavior, which are autonomous and can interact with humans in a natural way. Here we look at cyber-physical systems, smart platforms, and autonomous systems before discussing security issues.
The final section deals with the relationship between people and technology and the emergence of a new form of analytics and data-driven networked organization. We talk about the fundamental distinction between synthetic and analytical reasoning as a way of understanding the distinction between digital computation and human reasoning and as a means for interpreting the rapidly evolving relationship between the two.
This is not a technical course where you will learn the details of data modeling or how to build machine learning systems. What it does provide is an overview of this very exciting and important new area that will be of relevance to almost all domains, researchers, engineers and designers, business and the general public alike. The course aims to be a comprehensive overview to complex analytics, it aims to be inclusive in scope. We try to provide an understanding of the context to these major technological developments; a conceptual understanding of the methods and approaches of big data modeling and analysis; an overview to the underlying technology and address the issues and consequences both positive and negative of such technological developments.