The success of your Intelligent Automation strategy depends on your ability to extract necessary data from paper-based documents such as contracts, manuscripts, books, invoices, receipts, etc., and convert it into machine-readable text. Unless you want a manual data entry bottleneck in front of your whole automation process, step one of your automation strategy should include the automation of that data extraction process.
For years this has been done by a technology called Optical Character Recognition (OCR). OCR has been used to help automate business processes such as Procure-to-Pay in enterprises throughout the world. While OCR has improved over the years, it’s still not perfect. And guess what? That means even with OCR you probably still have that manual data entry bottleneck in front of your automation process as humans make manual corrections where OCR fails.
OCR has some inherent weaknesses, and one of the most significant is its dependence on templates. Fortunately, Artificial Intelligence (AI) technology can solve this problem by using a completely different approach that’s a powerful alternative to conventional template-based OCR conversion.
Let’s take a look at why template-based conversion creates some operational challenges and how a native AI-based technology can solve them.
What are Templates in OCR Conversion?
Have you ever played those hidden picture games where you have to find an object hidden somewhere in a giant image filled with tons of random things? It’s easier when you get a hint, like “The hidden turtle is somewhere in the lower-left quadrant just below the bookshelf near the chair.”
OCR needs hints like those to do its job better. If you tell it where to look, OCR can more accurately extract the data you want, and that’s what templates do for OCR.
Templates specify the location of information that needs to be extracted from the document that is being converted. The user marks coordinates and the software tool extracts text located at that specific location in the scanned image. This method of using coordinates is called zone OCR, and this helps OCR extract the right data so it can be stored in a structured database.
Zone OCR is effective when documents *reliably* have a structure similar to the template. As long as there is little or no variation among documents, your good to go.
Trending AI Articles:
2. How AI Will Power the Next Wave of Healthcare Innovation?
But if something changes or there’s too much variation, watch out.
For example, even if we set up a template for a standard invoice format, the total amount payable may appear in different positions because the number of items in the invoice varies and the total at the bottom gets pushed up or down. This variation causes all kinds of problems for your template. It’s as though that hidden turtle is crawling around and its location no longer matches the hint.
The Problem with OCR Templates
In fact, templates have a lot of limitations that impact accuracy and eat away at both your ROI and your ability to increase process automation. Just for fun, we made a list. See how many of these OCR template issues wreak havoc on your business processes:
Artificial Intelligence (AI) Finds Meaning in Text and Extracts it
As AI capabilities evolved and developed, a new approach to OCR emerged. Using AI takes an extraction solution well beyond what OCR is capable of.
Machine learning has enabled us to create algorithms trained on large volumes of data, that can extract data more efficiently. Now you can document variations with ease.
Natural language processing methods can be applied to understand the text and its context. Text analytics allow you to turn raw data into information.
Computer Vision (Deep Learning) systems are able to extract data from non-textual information such as stamps, diagrams, images, or tables.
A Bank Moves Beyond OCR
A large global bank offers a business loan service to companies in 15 countries. As part of the debt servicing process, the bank analyzes financial data and statements provided by the borrower. The analysis ensures that the borrower can repay the loan and is not at risk of default.
The bank used manual processing of the documents by skilled staff because of the documents:
- Varied from the borrower to borrower.
- Varied over time. One year’s annual report may be laid out differently than the next year’s report.
- Had important context within the documents that needed to be maintained. Footnotes, for example, make materially important changes to the figures in a table.
- Had data that was presented in a table and, sometimes, in nested tables.
This manual process worked, but it was slow, expensive, and often had accuracy problems.
An OCR, no matter how smart, could not handle the extraction job. The bank needed to move beyond OCR.
We worked with the bank to develop an AI-based solution that could extract all the data and context, and account for variations and changes.
(We also built an intelligent application for this bank that used natural language generation to generate a summary report from the extracted data.)
The outcome reduced operating expenses, decreased processing time, and improved accuracy. And the staff used for document extraction could be assigned to higher-value tasks such as analysis of the data.
Next-Gen OCR
Data extraction solutions now exist that are beyond OCR and beyond OCR templates.
These Intelligent Data Processing solutions now process what only humans could process a short time ago. And these solutions can do what OCR could never do: process unstructured, complex documents.
Don’t forget to give us your ? !
How AI Is Disruptive Innovation For OCR was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.
365 Data Science 