How Data Analytics Is Changing Up the Insurance Industry

November 23, 2017

4 minute read

The steepening change curve

Over the last ten years the explosion of available data from both internal and external sources, paired with advances in analytics technology and AI, have made it possible to build predictive models that have unparalleled accuracy and utility. These models, in an industry where success is predicated on the ability to analyse data to assess and manage risk, are changing the game for insurance companies. And not just in the back office, but in the board rooms and out in the field.  

For those companies that get ahead of the change curve and effectively implement data analytics the benefits will be immense:

  • better decision making at all levels
  • more precise predictive ability
  • better product design
  • deeper understanding of clients
  • ·near optimal risk assessment and management

But, as with any sea change, those who lag behind will suffer. Profit margins will be squeezed as competitors become more efficient and prices drop. Clients will seek out insurance companies whose products better meet their needs. And those companies who don’t get their analytics right will miss the benefit of improved risk assessment and management—the beating heart of insurance.

Migration to the digital realm

For large, traditional insurance companies the digital transformation powered by data analytics is ongoing—it will take time and a lot of resources to reach the finish line. But, while the older firms may not be as nimble as disruptive new entrants, they have a large potential advantage when it comes to one aspect of data analytics. The size and history of these traditional insurance companies give them a potential source of data even larger than the big data universe—a largely untapped resource that could dramatically improve the output of their data analytics.

Dark data

Take the case of an actuary using analytics to determine pricing for a product. They know that every extra percent of data they can access will improve the accuracy of their analysis by some significant percentage. Problem is, when making their decision, they typically are only working with 35-40% of the available data. That’s because the vast majority of data is unstructured—so-called dark data that data analytics can’t handle. Think; paper that can’t be accessed, emails, image files, that kind of thing. This type of content may be inaccessible, too expensive or time-consuming to access, or it may be the type of content that isn’t considered useful. In some insurance companies as much as 70% of all data may have this unstructured form.

A latent opportunity

Companies have entire histories—years or even decades of content that’s not accessible by data analytics. They may not even know what is in their data stores. They have paper and TIFFs and emails—contracts, policies, claims.

It’s a dark data treasure trove waiting to be exploited.

Because of the successes of data analytics, insurance companies are now driven to find ways to turn their dark data into actionable content. Some companies have already begun to tap the potential of the unstructured data lying unexamined within their legacy systems, and are seeing profound benefits. If they can convert their unstructured data into content that is usable for data analytics, insurance companies can gain an advantage over their competitors and, in some ways, inoculate themselves against disruption. Finding ways to make use of all that dark data could deliver greater operational efficiencies, better customized products and greater customer insights.

Cleaning unstructured content

To turn that data hoard into valuable content the data needs to be cleaned. Insurance companies at the leading edge of the digital transformation have found that the best solution involves the automated application of several steps.

First, the unstructured content must be standardized. That means content from any source needs to be converted into a common format. Unreadable formats need to be eliminated so that all data is usable by search and analytics algorithms. Conversion to a common format reduces the background noise (caused when analytics run into elements they can’t handle, like email headers or footers). Less noise means greater accuracy of the analytics.

Once the data has been standardized the next step is to understand how it is structured and to apply rules that will remove artifacts like footnotes that can interfere with machine interpretation. Duplicates need to be eliminated. And then the content can be categorized.

The final optimization of the content includes performing a variety of operations on the data to make it most suitable for achieving specific business outcomes—such as migration projects, creating clean content sets to massively elevate search efficiency, aggregating all content by case number for claims processing, and creating bullet-proof systems of record, among many others.  

The ongoing ingestion challenge

Once a company has dealt with its historical wealth of unstructured content its issues have still not been completely resolved. Insurance companies will, over the near term at least, continue to take in unstructured content. And it may take as long as ten years for that reality to change because it represents such a massive consumer shift.

The good news is, once an insurance company starts transforming its unstructured content, it necessarily creates and implements a process for content cleaning—one that is paperless, automated and takes much less human intervention. And that process can now be used to continually cleanse content as it is ingested. In a sense, creating a process for cleaning unstructured data future-proofs an organization—setting it up for greater improvements in the results from its data analytics projects as time goes on.

Wrap Up

Data analytics is changing the face of the insurance industry. In the early days big data was a driver of the digital transformation but companies now see need to come to terms with their vast amounts of dark data. Given that data analytics are improved by each increment of data available to be analyzed, gaining access to the 70% of all data that is unstructured represents a massive potential windfall. More data to analyze means improved outputs—which means better pricing, reduced risk exposure and better customer insights. Those companies that are best able to transform their unstructured content will be the ones to harness the true power of insurance data analytics and thrive in the new landscape.

Don’t forget to share this post