Articles and other content published on the web around Big Data, with a particular emphasis on rights, humanitarian aid and conflict prevention. Curated by Sanjana Hattotuwa, ICT4Peace Foundation.
The online chatter of individuals, networks of friends or professionals, and the data being created by networked devices are growing exponentially, although our time to consume it all remains depressingly finite. There is a plethora of solutions that approach the challenge of data inflation in different ways. But what methodologies have worked at scale and how are they being used in the field?
GOOD with numbers? Fascinated by data? The sound you hear is opportunity knocking.
The potential and challenges of open data for crisis information management and aid efficiency A preliminary assessment Daniel Stauffacher, Sanjana Hattotuwa and Barbara Weekes 
The new book 'Big Data: A Revolution That Will Transform How We Live, Work and Think' written by Oxford University professor Viktor Mayer-Schonberger and The Economist journalist Ken Cukier -- is the definitive guide to a new age which, both authors promise, is going to revolutionize the way we live, work and think.
The ICT4Peace Foundation recently published The potential and challenges of open data for crisis information management and aid efficiency: A preliminary assessment, a short monograph in large part based on a longer report I wrote on Big and Open data about a month ago. It’s a work in progress. Our understanding and support of Big and Open Data is evolving rapidly – with new concerns as well as new found optimism even in such bureaucratic catacombs like India. In that sense, whatever is written on the topic is very quickly outdated. A personal challenge also remains to keep abreast of these developments in the midst of my own work in Sri Lanka, which this year I’m focussing more energy on. So warts and all, here’s my larger essay on Big and Open Data – and the sections I’m most interested in fleshing out and discussing more are around the ethics of generation, use and dissemination, the potential for investigative journalism that harnesses the skills and expertise of consumers, and the potential for peacebuilding.
In this episode, legendary photographer Rick Smolan turns his lens toward big data—the idea that insightful, cohesive narratives exist within all the data and information generated by our everyday actions, purchases, and web searches. What does big data say about you? What is this information used for?
From phone records to credit card transactions and pacemakers to browser history, in a world where everything we do creates data, new questions and issues surrounding ownership and privacy are looming conversations. Rick Smolan discusses the power of big data, the dangers it creates, and the conversations we should be having.
Let’s be clear on what we’re talking about. The term refers to something more specific than the general fact that companies and government agencies are collecting lots of personal information about people. What it refers to is the fact that once you store up huge amounts of information, you can mine those databases to discover subtle patterns, correlations, or relationships that our brains can’t perceive on their own because the scales involved are beyond our ability to process (either the time scales at work, or the sheer number of data points).
Imagine a real estate market where it is profitable to build new skyscrapers, even if they sit at 40 percent occupancy. Eventually you run out of land—not to mention the staggering maintenance and energy costs you rack up.
We now create an enormous amount of digital data every day on smart phones, social networks and sensors. So how do you make sense of all of it?
Innovations in technology and greater affordability of digital devices worldwide have ushered in an Age of “Big Data,” an umbrella term for the explosion in the quantity and diversity of high frequency digital data. This digital data is being produced, in real-time, at an unprecedented rate, even across the developing world. In addition, advances in computing and data science make it possible to process and analyze Big Data in real-time. But can it be useful for development, and enhancing resilience to global shocks? Can the innovations from data science and tech be applied to the field of development? If so, how?
Kalev Leetaru, a researcher at the University of Illinois, has been looking at the capacious volunteer-written encyclopedia as a Big Data resource, concentrating on the connections between cities around the globe over time. To understand these connections, he focuses on the type of language used to talk about a particular place, to see whether the writers have a generally positive or negative sentiment toward the place at that time.
What’s really driving so-called Big Data isn’t the volume of information. It turns out Big Data doesn’t have to be all that big. Rather, it’s about a reconsideration of the fundamental economics of analyzing data.
With Big Data we can now begin to actually look at the details of social interaction and how those play out, and are no longer limited to averages like market indices or election results. This is an astounding change. The ability to see the details of the market, of political revolutions, and to be able to predict and control them is definitely a case of Promethean fire—it could be used for good or for ill, and so Big data brings us to interesting times. We're going to end up reinventing what it means to have a human society.
It has been called the “biggest company you’ve never heard of”. In the business of “data-brokering,” Axciom is one of the largest firms of its kind, and one of the leading companies profiting from “big data.” Last year the company recorded $1.1 billion in sales from the “analytical services” collected from 144 million households in the United States.
We live in an age of “big data.” Data have become the raw material of production, a new source for immense economic and social value. Advances in data mining and analytics and the massive increase in computing power and data storage capacity have expanded by orders of magnitude the scope of information available for businesses and government. Data are now available for analysis in raw form, escaping the confines of structured databases and enhancing researchers’ abilities to identify correlations and conceive of new, unanticipated uses for existing information. In addition, the increasing number of people, devices, and sensors that are now connected by digital networks has revolutionized the ability to generate, communicate, share, and access data. Data creates enormous value for the world economy, driving innovation, productivity, efficiency and growth. At the same time, the “data deluge” presents privacy concerns which could stir a regulatory backlash dampening the data economy and stifling innovation. In order to craft a balance between beneficial uses of data and in individual privacy, policymakers must address some of the most fundamental concepts of privacy law, including the definition of “personally identifiable information”, the role of individual control, and the principles of data minimization and purpose limitation. This article emphasizes the importance of providing individuals with access to their data in usable format. This will let individuals share the wealth created by their information and incentivize developers to offer user-side features and applications harnessing the value of big data. Where individual access to data is impracticable, data are likely to be de-identified to an extent sufficient to diminish privacy concerns. In addition, organizations should be required to disclose their decisional criteria, since in a big data world it is often not the data but rather the inferences drawn from them that give cause for concern.
Robert Kirkpatrick, Director of UN Global Pulse, is harnessing the power of big data for “agile global development”. Already working with Facebook, Twitter, and the five largest wireless carriers in Indonesia (among others), he is promulgating the concept of “data philanthropy” to bring the full power of the real-time feedback economy to humanity’s most pressing needs, on a global scale.
Inside the Secret World of the Data Crunchers Who Helped Obama Win Read more: http://swampland.time.com/2012/11/07/inside-the-secret-world-of-quants-and-data-crunchers-who-helped-obama-win/#ixzz2TvLNRP4I
A citizen capability approach to big open data needs a critical pedagogy that fits with technological forms of life; a combination of critical peer learning and rapid prototyping that can be called Critical Hacktivism[7]. The gains of open data are not to be found only in statistical correlations but in the critical engagement of participants in examining and questioning what represents their world inside the data machine, and having the ability to intervene on behalf of their preferred futures.
Privacy and Big Data
The images and stories captured in The Human Face of Big Data are the result of an extraordinary artistic, technical, and logistical juggling act aimed at capturing the human face of the Big Data Revolution. Big Data is defined as the real time collection, analyses, and visualization of vast amounts of the information. In the hands of Data Scientists this raw information is fueling a revolution which many people believe may have as big an impact on humanity going forward as the Internet has over the past two decades. Its enable us to sense, measure, and understand aspects of our existence in ways never before possible.
Director of UN Global Pulse (www.unglobalpulse.org) presents a vision for utilizing big data and analytics to better understand human well-being at the O'Reilly Strata Conference in New York (September 25, 2012). What can Big Data analysis tell us about human well-being? About how people cope with unemployment, rising food prices, or about people's perceptions of HIV and other deadly diseases? A lot. The United Nations' Global Pulse initiative (www.unglobalpulse.org) was established on the premise that Big Data didn't just hold the answers to what products and services people prefer to consume. Big Data, we believe, also held information about how people were coping with global stresses like unemployment and natural disasters; and whether development programs were having the desired effect.
The Promise and Peril of Big Data, ASPEN INSTITUTE
Content for this paper is excerpted directly from the IDC iView "Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East," December 2012, sponsored by EMC. The multimedia content can be viewed at www.emc.com/leadership/digital-universe/index.htm.
The amount of data in the world is exploding - large portion of this comes from the interactions over mobile devices being used by people in the developing world - people whose needs and habits have been poorly understood until now. Researchers and policymakers are beginning to realize the potential for channeling these torrents of data into actionable information that can be used to identify needs & provide services for the benefit of low-income populations. This discussion note is a Call-to-action for stakeholders for concerted action to ensure that this data helps the individuals and communities who create it.
Experts say study of new streams of information will make us more nimble and adaptive; however, they express concerns over 'humanity's dashboard' being in government and corporate hands; and they point out numbers can lie.
Data has the power to transform our lives – collectively and individually. What is needed to unlock the profound opportunity data affords to improve the human condition – and to defend against a multitude of threats – is not technical, but an ethical framework for its use by and beyond those who initially collect it, including providing access to individuals.
UC Berkeley Course Lectures: Analyzing Big Data With Twitter
Big Data for the Public Good is a four-part seminar series hosted by Code for America in San Francisco and sponsored by Greenplum, a division of EMC, that explores how data scientists and change agents are harnessing the big data explosion to address the great opportunities and pressing challenges of our time. Do good with data - apply for a Code for America Fellowship.
As software eats the world, it consumes value chains and makes markets more efficient. It also leaves behind data, an artifact that speeds up the consumption by more software. 2014 will be the year that forces of data reshape journalism.
New York Times Compendium on Big Data
Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage,[3] search, sharing, transfer, analysis,[4] and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to "spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions.
Without a doubt, Big Data can produce significant social good. From slavery to starvation, and health to homicide, empirical analysis can provide actionable insight to aid social organizations in tackling the world’s hardest problems; however, there are few issues that organizations should consider when embracing Big Data: data quality, sources, and access control.
BigDataLandscape.com is a leading source for Big Data market information and insights. BigDataLandscape.com provides the most up to date version of The Big Data Landscape.
Big data - From The Guardian
As Dyson says, “Soon we will salt the oceans, the land, and the sky with uncounted numbers of sensors invisible to the eyes but visible to one another and to a variety of data collection devices. The vast, increasingly accurate data streams will combine and interact to produce more and more meaningful caches of knowledge.”
Data-centric specialties are growing fast, giving rise to a new vocabulary. In political science, this quantitative analysis is called political methodology. In history, there is cliometrics, which applies econometrics to history. In literature, stylometry is the study of an author’s writing style, and these days it leans heavily on computing and statistical analysis. Culturomics is the umbrella term used to describe rigorous quantitative inquiries in the social sciences and humanities.
Words and phrases are fundamental building blocks of language and culture, much as genes and cells are to the biology of life. And words are how we express ideas, so tracing their origin, development and spread is not merely an academic pursuit but a window into a society’s intellectual evolution.
Editor’s Note: Patrick Meier is an internationally recognised thought leader on the application of new technologies for crisis early warning, humanitarian response, human rights and civil resistance. He is currently Director of Social Innovation at the Qatar Computing Research Institute.
Big Data and other analytical tools have great potential to make governments more efficient and improve citizens’ lives, particularly in health and public safety sectors, according to a new study released by the TechAmerica Foundation.
“Data inherently has all of the foibles of being human,” said Mark Hansen, director of the David and Helen Gurley Brown Institute for Media Innovation at Columbia University. “Data is not a magic force in society; it’s an extension of us.”
Up to now, the focus on the power and implications of Big Data technology has been involved social media, business decision-making and online privacy. Those are big subjects in their own right. So it’s not surprising that the notion of a data-driven society has not been much considered.
We now live in a world in which vast amounts of data are created and stored on a constant and ongoing basis. The effects of globalization along with the advent of internet and mobile communications has created an opportunity for more effective analysis and response to crises through the use of big data analysis and network science theory. The world economies once separated by geography, culture, and language have become increasingly interconnected as the ability to transport products and communicate ideas has become easier. As the communication and physical divides continue to shrink, the need to address problems in the world more effectively and efficiently increases. The onset, pace, and scope of global disasters is growing faster than our collective ability to respond to such shocks. A byproduct of our world becoming interconnected is that disasters which directly affect a specific region also may have a global effect which transcends national borders. What started as a natural disaster in a given geographic area can evolve into an economic shock that hits regions not directly affected by the initial disaster. These secondary shocks invariably damage the regions least able to effectively manage them. The analysis of large data sets of complex systems, or big data analysis, allows for the study of trends and prediction of outcomes for future action. Applying network science theory to a large data set adds depth to this analysis by examining clusters of related entities and the value of their linkages. This dual approach allows for the identification of macrolevel trends through big data analysis and integral structural linkages within the data set via network science analysis. Using this technique, development and humanitarian aid can then be distributed in a more effective manner, getting aid to the areas that can benefit from it the most.
Jaron Lanier is a digital visionary with a difference. As the New Yorker once put it, he is a technology expert who dislikes what technology has become. He is a go-to person for those worried about the internet. His latest book, Who Owns the Future?, describes the assault on the middle classes by the Siren Servers. These are giant computer networks, devised by the smartest technical folk around, systems that gather data often without having to pay for it. Lanier has in his sights Facebook, Google and the other mega-tech firms (interest declared: I advise Google on free-expression issues).
A revelatory exploration of the hottest trend in technology and the dramatic impact it will have on the economy, science, and society at large. Which paint color is most likely to tell you that a used car is in good shape? How can officials identify the most dangerous New York City manholes before they explode? And how did Google searches predict the spread of the H1N1 flu outbreak? The key to answering these questions, and many more, is big data. “Big data” refers to our burgeoning ability to crunch vast collections of information, analyze it instantly, and draw sometimes profoundly surprising conclusions from it. This emerging science can translate myriad phenomena—from the price of airline tickets to the text of millions of books—into searchable form, and uses our increasing computing power to unearth epiphanies that we never could have seen before. A revolution on par with the Internet or perhaps even the printing press, big data will change the way we think about business, health, politics, education, and innovation in the years to come. It also poses fresh threats, from the inevitable end of privacy as we know it to the prospect of being penalized for things we haven’t even done yet, based on big data’s ability to predict our future behavior.
The power of big data -- analyzing huge swaths of information to uncover insights and make predictions that were largely impossible in the past -- is poised to transform business and society. Fueling it is the realization that data has a value beyond the primary purpose for which it was collected. Yet there is a dark side. Privacy is eroded like never before. And a new harm emerges: predictions about human behavior that may result in penalties prior to actual the infraction being committed. In this talk Mayer-Schönberger and Cukier take a look at big data's power, the dangers it poses and how to address them.
There is a mind-boggling amount of data floating around our society. Physicists at CERN have been pondering how to store and share their ever more massive data for decades - stimulating globalization of the internet along the way, whilst 'solving' their big data problem. Tim Smith plots CERN's involvement with big data from fifty years ago to today.
Is big data all it's cracked up to be? Kate Crawford of the MIT Centre for Civic Media goes behind the numbers to debunk five myths about big data. Read more: http://www.smh.com.au/it-pro/business-it/is-big-data-all-its-cracked-up-to-be-20130513-2jh55.html#ixzz2TvN0et5M
Big Data is all the rage in Silicon Valley. From Facebook to Netflix, companies are tracking and analyzing our searches, our purchases, and just about every other online activity that will give them more insight into who we are and what we want. And though they use the massive sets of data they collect to help create a better experience for their consumers (such as customized ads or tailored movie recommendations), their primary goal is to use what they learn to maximize profits. But can Big Data also create positive social change?
On Wednesday, the CIA's chief technology officer detailed the Agency's vision for collecting and analyzing all of the information people put on the Internet. Read more: http://www.businessinsider.com/cia-presentation-on-big-data-2013-3?op=1#ixzz2TwIJSzZ1
We'd love to hear from you. Please send questions or feedback to the below email addresses.
Before contacting us, you may wish to visit our FAQs page which has lots of useful info on Tiki-Toki.
We can be contacted by email at: hello@tiki-toki.com.
You can also follow us on twitter at twitter.com/tiki_toki.
If you are having any problems with Tiki-Toki, please contact us as at: help@tiki-toki.com
Close