Shining light on the dark side of big data

Does the shift toward more data and algorithmic direction for our business decisions assure us that organizations and businesses are operating to everyone's advantage? There are a number of issues involved that some people feel need to be addressed going forward.

Numbers don't lie, or do they? Perhaps the fact that they are perceived to be absolutely objective is what makes us accept the determinations of algorithms without questioning what factors could have shaped the outcome.

That's the argument Cathy O'Neil makes in Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. While we tend to think of big data as a counterforce to biased, just decisions, O'Neil finds that in practice, they can reinforce biases even while claiming unassailable objectivity.

 “The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong.”   The math destruction posed by algorithms is the result of models that reinforces barriers, keeping particular demographic populations disadvantaged by identifying them as less worthy of credit, education, job opportunities, parole, etc. 

Now the organizations and businesses that make those decisions can point to the authority of the algorithm and so shut down any possible discussion that question the decision. In that way, big data can be misused to increase inequality. As algorithms are not created in a vacuum but are born of minds operating in a human context that already has some set assumptions, they actually can extend the reach of human biases rather than counteract them.  

“Even algorithms have parents, and those parents are computer programmers, with their values and assumptions, “Alberto Ibargüenhttps://www.knightfoundation.org/articles/ethics-and-governance-of-artificial-intelligence-fund,  president and  CEO and of the John S. and James L. Knight Foundation wrote.  “As computers learn and adapt from new data, those initial algorithms can shape what information we see, how much money we can borrow, what health care we receive, and more.”

I spoke with the foundation’s VP of Technology Innovation, John Bracken about its partnership with the MIT Media Lab and the Berkman Klein Center for Internet & Society as well as other individuals and organizations to create a $27 million fund for research in this area. 

The idea is to open the way to “bridging” together “people across fields and nations” to pull together a range of experiences and perspectives on the “social impact” of the development of artificial intelligence. As AI is on the road “to impact every aspect of human life,” it is important to think about sharping policies  for the “tools to be built” and how they are to be implemented.

Read more in 

Algorithms' Dark Side: Embedding Bias into Code

BI when and where it's needed

That was a critical factor in adopting WebFOCUS, Thiery says, because so many people rely on their phones more than on their desktop units. Consequently, reports that are not designed to be mobile-friendly are not as useful.

Generally, the visualizations are reviewed on a weekly basis at leadership meetings. Thiery explains that these meetings are where management "wants to see where we're at and where we're going." The meetings are also where managers make decisions about how many people they would hire.

As a result of the growth AudioNet has been experiencing, it's been adding on a large number of support people to keep up with the workload. "As our volume increases, so does our revenue," Thiery says.

The firm also uses WebFOCUS to analyze financial data. That includes revenue dollars, accounts and claims counts that factor into identifying an upward trend.

- See more at: http://www.baselinemag.com/business-intelligence/getting-business-intelligence-when-where-needed.html#sthash.BMoWyczs.dpuf

The perks of personalized marketing: Starbucks' use of AI

photo from https://c6.staticflickr.com/4/3133/2873861973_995e8a7902_b.jpg

“Would you like fries with that?” That one-size-fits-all recommendation was about as far as upselling at the point of sale used to go.  Starbucks has come a long way from that with marketing powered by data — and AI.  It's marketing that is uniquely tailored to the behaviors, preferences, and context of each customer.

Matt Ryan, Starbucks EVP and global CSO, and Gerri Martin-Flickinger, EVP and global CTO discussed the strategy at the company's Investor Day.  He explained the impact of digital on Starbucks sales, while she explained what their algorithms do, in The Digital Flywheel: Strategy and impact.

Ryan began by saying that for Starbucks digital represents the “core customer experience.”  He sees it as what distinguishes their business approach from “the rest of the retail world,” and what “gives us the advantage we have.” As proof he showed how the "digital flywheel" contributed to Starbucks' growth in the past three years, and the increase in its rewards program from five million to 12 million customers.

The incentives in the rewards program is one of the ways Starbucks personalizes its marketing.   Martin-Flickinger demonstrated the evolution of the personalization by showing how it has progressed in just a single year. Its baseline is the form of marketing that many businesses still use today, rooted in historical data that they view in spreadsheets, which in turn powers algorithms to fit their general customer base.

In January 2016, Starbucks' marketing messages were limited to 30 variants of weekly email messages that were based on data that was already two weeks old. In June last year, Starbucks moved to a to a real-time personalization engine capable of producing of 400,000 variants of hyper-personalized emails each week. The company continued to tweak its AI engine so that in October, it was able to launch real-time 1:1  personalized offers that are uniquely generated based on each individual customer's behaviors and marked preferences.

Read more in A.I. Informs Personalization for Starbucks

Data science training to fill the skills gap

 Datanami article looked at various estimates about how many more data scientists from firms like Gartner and McKinsey to arrive at a range of 100,000-190,000 shortfall by 2020 and 2017, respectively. That's about 60% more demand than available supply in the US. The upside is that salaries for those jobs go up, but the downside, of course, is that the businesses cannot progress on the data front as much as they would like to without the skilled people in-house.

The obvious solution to that problem is to get more people qualified for these jobs. If they didn't learn the skills in college programs, they can now get trained in specialty programs like the NYC Data Science Academy (NYCDSA). Vivian Zhang, the CTO of the school, set it up specifically to meet industry needs and get people the data science training needed to fill those open job slots.

Read more in 

Mind The Gap: Minting New Data Science and Analytics Professionals

Data visualization: you have to C it to believe it

 credit https://c1.staticflickr.com/9/8075/8448339735_e6626c28ff_b.jpg

I wrote this blog a couple of months before everyone started decrying the proliferation of fake news. Notice just about every fake news piece is accompanied by some sort of visualization, whether it is a graph or photo or video. They all capitalize on the "seeing is believing" concept, and one has to be extra vigilant about the lure of visual evidence.

As a regular big data blogger for several years now, I’ve noticed that in the last couple of years, data visualization has become a major focal point.  The old maxim of “Seeing is believing” is the real driving force behind visualizations of data.  While not all of us relate to spreadsheets, we tend to respond well to graphs, charts, and other visually appealing renderings of those numbers.  

As Brian Gentile, Senior VP and General Manager, TIBCO Analytics Product Group, TIBCO Software, wrote here there are business benefits to data visualizations.  They include making it easier to take in information, manipulating, data in various ways, and showing relationships.  On the latter, Gentile observes, that “finding these correlations among the data has never been more important.”

Indeed, the demand for that kind of instant insight that data visualizations can deliver is what drove Google to build its own data visualization product (currently in beta) called Data Studio. I saw a presentation of the features, including a report on the effectiveness of Olympics ads. It was that particular visualization that made me think of the danger inherent in relying completely on the story presented graphically.

In that analysis of the effects of ads on consumers, the report stresses that it asked people who saw the ads of particular brands what effect it had on their perception of them. Of course, the graphs are what grab your attention and that show that that 34.9% of viewers recall seeing the Coke ad. The graph does not show what the text admits that overall “only about 8% of viewers can recall both the brand and product in a specific advertisement.” So the graph here implies a much more positive effect for ad recall than the overall data actually shows.

 The next bar graph shows you that “Consumers who saw the ads were 18% more positive about the brand and were 16% more likely to find out more or purchase the product in the ad.” These are fairly modest numbers that don’t necessarily promise much bang for sponsor bucks. So this is followed by a third graph with the title “Which ads showed the greatest response?” That shows really impressive numbers ranging from 112%- 142% for the top 3 brands.

A mere glance would make you think that these show amazing results for the marketing efforts. Then when you read a bit, you realize that they merely reflect the increase in search.  In other words, the graph does not show that the McDonald’s commercial resulted in an increase of 42% in sales, merely an increase of that amount in online search that includes the brand. Still, you may say that is a positive metric that could possibly translate into improved sales down the road. But the chain of causation here is missing a few links. 

I got to speak to the Google people about Data Studio and asked if they had even determined if the people who were doing the search were the ones who had seen the ads as was the case for the first two graphical presentations. They had not.  True, it doesn’t say that the graph refers to the people who had seen the ads, but the context would make the viewer think that it does, and not everyone would even think to ask annoying questions like I do.

Ultimately, what makes data visualization so effective at conveying a point is that they don’t require much analysis on the viewer’s end because they’ve already done that kind of thinking for you. That’s both seductive and potentially misleading.

That’s exactly why we have to be careful about not merely accepting the visually expressed story at face value. Any data visualization should be subjected to a triple C test. 

Read about it here.

Also check out http://www.clickhole.com/article/greatest-all-time-statistical-portrait-babe-ruth-3983 

The one on the Babe versus the #12 may be my favorite example of the abuse of data visualization, and I'm not even a sports fan

Related post: 

EVERYBODY LIES WITH VISUALIZATIONS

Amazing visualization for sports

You don’t have to be a sports fan to be impressed by what real time analytics can do for sports. With

new offerings from Intel, we can now know exactly what speed and what heights a snowboarder achieved, and we can get a 3D view of basketball that goes far beyond anything we’ve ever seen before.....

 for data capture in basketball, Intel applied something completely different, 360-Degree freeD Replays. In February Intel partnered with the National Basketball Association to deliver 3-D experience to the fans watching live on TNT or on NBA.com, or through the NBA App.

Intel’s CEO Brian Krzanich said the “sweeping views of top plays from virtually any angle” make it possible for “people to experience NBA All-Star like never before.” The way it works is by setting out 28 ultrahigh-definition cameras to capture video images that can then be combined for 3D views from any position.

An article inThe Cauldron, "Digitization Is Upon Us — The Biggest Change In Sports In Over 100 Years," explained that the court is rendered into “a series of "voxels," small cubes to make up the volume of the area. Then, a computer can virtually place a viewer anywhere, providing a complete 360-degree view. See the videos here 

Read more in 

Sports Data Like You’ve Never Seen Before

Using data for good

We hear a lot about businesses looking for ways to tap into data to make better decisions. For Bloomberg, financial data is not just about the numbers that make up corporate profits. It's also about the metrics of a company's sustainability in terms of its social and environmental impact. - See more at: http://www.baselinemag.com/news/using-data-for-good-is-good-for-business.html#sthash.dDbMetvo.dpuf

Data Mining for Legislative Influence

If you want to learn about the process of getting a proposed bill passed, you can read the official explanation on a state senate site. It’s remarkably similar to the steps involved for federal legislation, according to the explanation offered to the protagonist of Mr. Smith Goes to Washington.

 What the explanations don’t reveal, however, are the entities behind the proposed legislation.

The actual authors of proposed legislation don’t sign their names, but they do leave signatures of a sort, the signals of individual style that can be found throughout their written work. All it takes is reading through thousands of proposed bills to find the textual clues that link bills to the same source. The only drawback is coming up with the time it takes for humans to read through it all. But this is one problem that technology can solve.

Read more in

Data for Good: Tracking Legislative Influence

Clouds and the Connected Car

Here we are, nearly at the end of 2015, and we still don't have flying cars. But what we do have are connected cars, and that technological advance has the potential to yield all new insights and solutions. Thanks to the combination of connected cars, analytic engines and the cloud, real-time data and real-time solutions are now possible.
Read more in 

Connected Cars, Data & the Cloud

Relatable robots

Advancements in robotics are not just about developing better robots to do work on their own. It’s about developing robots whose work is interacting with humans. Those robots have to be programmed with personality, which is not one-size-fits-all project.

The ideal robot companion for humans is not quite perfect. That’s the conclusion of PhD researcher Mriganka Biswas, supervised by Dr John Murray, from the University of Lincoln’s School of Computer Science who presented their findings the International Conference on Intelligent Robots and Systems (IROS) conference in Hamburg this October, as reported in WT VOX.: "A companion robot needs to be friendly and have the ability to recognise users’ emotions and needs, and act accordingly. Despite this, robots used in previous research have lacked human characteristics so that users cannot relate -- how can we interact with something that is more perfect than we are?”

The good news is that it is possible to program robots to come across as more human. As the French company Aldebaran has discovered in developing culture-specific programming for humanoid robots in conjunction with its parent company, Softbank, what people would want for their humanoid robot varies, according to cultural expectations. That’s what it’s working on in developing the robot Pepper for international markets.

The controlling idea for developing a companion robot like Pepper is interaction. On one level, the robot acts like Siri on wheels. It has the ability to tap into the Internet to answer questions about the weather or local entertainment offerings. Beyond recognizing your voice and responding to what you ask, Pepper is equipped with built-in cameras and sensors that enable it to analyze facial expressions and body language to identify what a person is feeling.

Pepper communicates to humans through eye movements, what appears on the tablet it wears, and speech. What Pepper says can be neutral, playful, or didactic. Which of those three should be the default depends on the expectations of the humans around, and that’s where specialized programming comes in.

Read more in 

Our Robots, Our Selves

Ho hum, another hack

Hacking is so common today that even major data breaches may be filed under the "hack du jour," but what's the answer -- laws and stronger penalties, threat-intelligence sharing or sharing responsibility across the business all the way to the top?
Read more in 

Data Breaches: The New Normal?

Smartphones are key for hotel rollout

In the future, hotel keys may be regarded as quaint, as guests will open doors with their own smartphones.

On August 11, Hilton Worldwide added a new feature to its Hilton HHonors app, the Digital Key. The hotel anticipates rolling it out to 250 Hilton properties in 2016, a brand that includes Waldorf Astoria, Contrad, and Canopy. The hotel chain had already offered digital check-in, but this allows guests to use their phones not only to reserve and register their rooms but to open them, eliminating the need to stop at the front desk altogether.

Credit: Flickr

Hilton is not the first chain to enable guests to use their smartphones as their room keys. The option has been offered at select locations of hotels chains, like Starwood. It’s a growing trend, according to Robert Cole, founder and CEO of RockCheetah, a hotel and destination marketing strategy and travel technology consulting practice. Cole anticipates a lot more hotels will adopt keyless entry options in the next few years. I spoke to him about the benefits that keyless systems offer the hotel industry.

Read more in 

Smartphones Open Doors, Literally

The trust factor for IoT

pic from https://farm8.staticflickr.com/7296/15768208714_9f9376cc7d_o_d.jpg

We talk a lot about IoT, referring of course, to the Internet of Things. But perhaps we should be thinking of making the T stand for trust. That’s what some are working on now in establishing industry standards to be worth of trust by committing to adopt best practices.

Essentially the OTA's best practices and standards boil down to two overarching considerations. One: Device manufacturers have to consider how they will secure the data collected on their devices. Two: The consumer has to be clearly informed about the nature and extent of the data collected. Having that information allows the potential purchaser to know exactly what they would be getting into with the Internet of Things (IoT) device, and whether or not they consider the gains are worth the risk. Having a universal standard also makes it clear how one company compares to another with respect to data privacy and security.

Read more in

Setting the IoT Standards

Retailers get into predictive analytics

Here on All Analytics, we’re generally sold on the value of predictive analytics. The question is: Are retailers, particularly those managed by people who believe in their gut intuition, sold on it? Even they are starting to appreciate what analytics can do for their business.

Dean Abbott

According to Dean Abbott, co-founder and chief data scientist at SmarterHQ and author ofApplied Predictive Analytics: Principles and Techniques for the Professional Data Analyst, it is ushering a cultural change for retail.

I recently chatted with Abbott about what the application of predictive analytics means to the retail space. Read more in 

Predictive Analytics: Data and Retail Expertise

Good data should not come at the cost of Goodwill

 https://upload.wikimedia.org/wikipedia/commons/a/a9/2008-08-04_CVS_Pharmacy_in_Durham.jpg

CVS recently ran an experiment that's generating a lot of buzz online. Unfortunately, it's not the kind of customer reaction retailer likes to get.

The drugstore giant chose the week of July 12 to deviate from its usual offer of special coupons or promotions in printed circulars. They were betting that what they gained in data insight for the week would be worth the price of alienating some customers.

- See more at: http://www.baselinemag.com/blogs/dont-sacrifice-goodwill-for-good-data.html#sthash.V3PGO2I9.dpuf

Mind the gap: information governance in the age of big data

"Many companies don't benefit from big data because of the gap between those who manage the data and those who apply it. The solution is information governance.


Sue Trombley, managing director of thought leadership at Iron Moutain, offered some perspective on the causes of this clarity gap.....

Going forward, Trombley asserts, businesses will have to adopt a new paradigm that enables each department to have direct access to the information it requires in order to extract value to meet its goals.  

- Read more  in Information Governance Is Key to Big Data's Value 

The science of empathy

"You never really understand a person until you consider things from his point of view, until you climb in his skin and walk around in it," declares Atticus Finch in Harper Lee's To Kill a Mockingbird. Gaining that kind of empathic insight is helpful not only for fostering humanity, but for improving business results....

Ultimately, there is no magical question that will elicit what the C-suite and IT leaders need to know about their customers. In fact, some customers may not even be able to articulate exactly what caused them to feel less than satisfied. The solution to that problem is not magic, but science.


Even when people can't find words for what they are feeling, sensors can pick up on the signs of stress that, when combined with contextual data, can reveal the emotional triggers that define a customer's experience. That's where design consultant Elliot Hedman comes in. Experience designer at mPath, Hedman has developed a methodology that combines stress-testing sensors with traditional observational techniques.

Graph from mPath

Read more in 

Can Data Teach Us Empathy?

When efficiency, algorithms, and labor laws collide

Timeclock Wikipedia Commons

Flexibility is considered a virtue and an essential component an agile organization which can respond to changing needs in real-time. However, when that type of flexibility comes at the expense of employees, the company may not only be crossing the line of ethics but of law.

On April 10, New York Attorney General Eric Schneiderman directed his office to send a letter (posted by the Wall Street Journal) to 13 major retailers.  What Gap Inc., Abercrombie & Fitch, J. Crew Group Inc., L. Brands, Burlington Coat Factory, TJX Companies, Urban Outfitters, Target Corp., Sears Holding Corp., Williams Sonoma Inc., Crocs, Ann Inc. and J.C. Penney Co. Inc were all asked were to account for questionable scheduling practices known as “on-call” shifts.

Read more in 

The Legal Limits for On-Call Shifts

Is a Smart Barbie a Smart Move?

Today's fashion doll is not your grandmother's Barbie. Nowadays, she comes equipped with WiFi, voice recognition, and access to data stored in the cloud.

Since 1959, the iconic Barbie doll has gone through multiple permutations as it evolved to reflect the fashions and expectations of each generation. Lately, that includes technology to enable conversations with the doll. And Mattel has discovered that a talking Barbie causes trouble. In the last generation, people objected to what Barbie said; now they object to what she hears.

Read more in 

Smart Barbie Puts Child's Play In The Cloud

Data for doctors: should there be limits on it?

This summer, Carolinas HealthCare System made the news rounds as a warning of the new levels of data mining available to healthcare companies. In Hospitals Are Mining Patients' Credit Card Data to Predict Who Will Get Sick, we get a very Big Brother type of picture of the invasiveness of such data mining with an illustrative picture showing a doctor saying, “Don’t lie to me, Susan, I know about the 2 a.m. Papa John’s deliveries.”

 It makes for dramatic copy, but it’s still in the realm of fiction rather than fact, as I found our when  I contacted Carolinas HealthCare and got a response from Jason Schneider, Director, Clinical PR. He explained that the article “focused on how providers could use data for in the future and didn't include details what data we are currently using and how we are using it.”

The data they are currently using does not follow an individual’s consumer trail but looks at things like socio-economic circles, neighborhood limitations, and cultural affiliation that could shape one’s access to healthcare. One example of that was identifying why patients in one particular area were not coming in for regular doctor’s visits. It turned out that it didn’t have reliable public transportation to a doctor's office. After identifying the geographic problem, Carolinas HealthCare set up a doctor in the neighborhood itself.

As the person quoted in each of the articles on Carolinas use of data is Dr.  Michael Dulin, chief clinical officer for analytics and outcomes research at Carolinas, I contacted him and spoke with him on the phone. He explained that Carolinas has a decade of experience in using data to improve healthcare by identifying individuals within contexts that could pose obstacles to care.

Read more in 

More Info in the Name of Better Healthcare