Unstructured data: Its all that stuff hiding in your emails, reports, customer records, spreadsheets, contracts, warranties, telephone/member listing books, advertisements, marketing materials, annual reports, customer calls, employee evaluations, ordering information, surveys, social media, blog posts, and Twitter feeds.
It’s all that stuff that’s probably not in your database … yet.
Does it matter? It does. Lots.
• In their book, Unstructured Data in a Big Data Environment, the authors say unstructured data is the largest piece of the data equation and the opportunities for using unstructured data are rapidly expanding.
• Wilson Raj, global customer intelligence director at SAS, writes in a whitepaper that the missing element in sales transactional data is what “goes on between buyer and seller in terms of negotiation, tone, emotion, motivation, and sentiment.” That information, says Raj, is hidden in unstructured data, which needs to be gathered, decoded, and then analyzed.
• Bill Inmon, the “Father of Data Warehousing” and author of Tapping Into Unstructured Data, says, “Data has to be put in a structure and a format that is particular and disciplined … Text needs to be lifted from paper media and converted into electronic format .. Like data found on paper, voice data likewise needs to be lifted from the media in which it was stored and reset into an electronic format … by means of voice character recognition.”
How the heck would we do that?
Marketers who’ve tried are enthusiastic.
Mary Grace Bateman, market manager for IBM, refers to practices that nonprofits can use to enhance unstructured data. “ … true gems can come from meaty blog posts or twitter feeds. Before, these types of unstructured data were only used as “listening” tools, with organizations unable to take the time and manpower to incorporate them into deep analysis. With predictive analytics, however, true sentiment can be extracted from text, putting structure around unstructured data that can be fed into analyses for actionable insight. Through sentiment analysis, for example, nonprofits can monitor public perception of their organization, understanding if a particular service is being talked about in a positive, negative, neutral, or ambivalent tone.”
In writing for TechRepublic, Mary Shacklett describes a scenario that encapsulates the value of unstructured data. “For years, non-profit aid organizations have been sending in field workers to advise local farmers on best agricultural practices. These workers file progress reports and keep tabs on agricultural projects to see if crop yields improve … The difficulty has been in collecting all of these reports, which come in many different forms–and then trying to glean insights into them after they become a monolithic body of unstructured and semi-structured data. By using Big Data collection, grooming and analytics techniques, humanitarian aid organizations are now able to compile all of these unstructured reports of field farming activity into databases-and then to mine these databases for information about which farming projects are succeeding, which are not, and why.”
Sunand Menon, president of New Media Insight, shares this: “… IF a registered user ‘likes’ a specific charity on Facebook, AND then ‘comments’ positively on a similar charity on LinkedIn, AND has identified themselves as previous donors to yet another similar charity, THEN one can assume there is a higher likelihood that donation requests from charitable organizations serving that cause will have a higher chance of success. Imagine the efficiency leap in such situations; they can actually be used to predict future behaviours and patterns.”
How much might this cost?
Menon suggests it’s not cheap. “The implementation cost can range from six figures to tens of millions of dollars, depending on the size and complexity of the data and analytics involved.” See more here.
Where to start?
Together, all of this looms large and overwhelms. No wonder companies are springing up everywhere to help marketers train for the unstructured data marathon. Spotfire suggests a few best practices for dealing with your data here.
Many hardware and software developers have products to help marketers tackle unstructured data from different angles. The following list is neither complete nor all inclusive, but I hope a look at some products being offered will help TDN readers understand what is happening in the unstructured data market. Feedback from readers, users, and vendors is most welcome and sincerely appreciated. Thank you!
1. Aspire offers a content processing system specifically designed for unstructured data
2. NVivo captures, manages, and analyzes unstructured data like videos, interviews, images, surveys, and social media.
3. NewBrandAnalytics deciphers and analyzes social media feedback and translates into operational insights.
4. Netessa seeks to give meaning to massive amounts of data.
8. IBM Smart Analytics organizes and mines information and performs customer behavioral analysis.
9. MapReduce processes large data sets, performing filtering and sorting procedures.
10. OpinionLab develops assessments of customer experience across company channels.