Thursday, August 17, 2017

"Everybody Lies" the book about data

"Everybody Lies" is the title of a recent book by Seth Stephens-Davidowitz. The subtitle is "Big Data, New Data and What the Internet Can Tell Us About Who We Really Are". It's a quick and easy read and I found the subject fascinating both for the voyeuristic content as well as the implications and uses for the future of data analysis.

Stephens-Davidowitz analyzes massive amounts of data that he pulls from internet usage, primarily what people search for, and compares it with other databases. For example, he looked at how many people report they use condoms with the number of condoms sold. (There are far fewer condoms sold than people reporting they use them) He has a lot of examples like that, hence the title of the book.

What was particularly interesting to me was the rise and fall of specific searches after events. After the shooting in San Bernardino in December, 2015 the number of searches related to Muslims and negative words (words indicating harm, hate or revenge, terrorists, extremists)spiked. And then President Obama spoke about the attack in what was a well regarded message reminding Americans that this event did not mean Muslims were bad and exhorting us to treat each other fairly. The negative searches about Muslims spiked again. A couple of months later Obama spoke again but this time talked about Muslim Americans who were firefighters, doctors, soldiers and teachers. This time there was no increase in searches for negative terms regarding Muslims and in fact there was a spike in positive or neutral searches regarding Muslims.

The author explains this, "When we lecture angry people, the search data implies that their fury can grow. But subtly provoking people's curiosity, giving new information, and offering new images of the group that is stoking their rage may turn their thoughts in different, more positive directions."

This is important, especially as we confront people with radically different ideas than ours. There is too much violence, in word and deed, already. This is a better way to bridge the divide.