Organisations thought they had reached Nirvana, because disk space was so cheap, they never had to worry about organising files/data again. And why not, time was much more valuable than a few GB of hard disk. As time went by data volumes kept on growing and soon this impacted the performance of many applications, requiring infrastructure improvements to compensate. Still servers were much cheaper than they used to be so a few more wouldn’t hurt! Once more everyone was happy.

That was until the world got turned upside down by the financial crisis.  Governance and compliance became the watch words as regulations such as the Sarbanes-Oxley Act 2002 (SOX) and Dodd-Frank Act 2010 were brought in. Organisations now must define what is and is not a record, keeping items for defined periods of time and implementing defensible disposal. This became a hot issue for Chief Information Officers (CIOs) for the failure to comply carries significant financial penalties. New documents are not too difficult to handle but cleansing of all their repositories now becomes a significant task. While this problem was getting addressed, another big one started to emerge.

Security weaknesses started to cause large public cases of data theft. Credit card numbers, health records and even political affiliations are sold on the dark web, governments are forced to react. First, it was Europe with its General Data Protection Regulation 2016 (GDPR), which was followed by California Data Privacy Regulations 2018 (CCPA), and then gradually by the ROW, which is in a movement to implement laws referring to Personally Identifiable Information (PII), its storage and disposition.  Financial penalties are so high and, more importantly, damage to brand reputation is so severe that failure to take preventative action could threaten the stability of organisations.  Chief Security Officers (CSOs) are appointed to protect against the cyber threats whilst the CIOs need to analyse the organisations complete data to implement a new set of policies for mitigating the risk of PII data.

The one constant in all these changes is the role of Artificial Intelligence (AI) with its ability to understand unstructured data and help organisations both extract valuable information from their data whilst staying within the law!

In a series of blogs, I will show how IDOL-powered AI for unstructured data is applied within organisations to manage and discover data, as well as cover the overall functionality for connectivity, enrichment and analysis used in Enterprise search. I will also expand on the numerous use cases including ROT analysis, Records management and PII discovery used for data governance. So, please stay tuned.