Sample Application to read receipts using Google Cloud Text Recognition API with Java

Sample Application to read receipts using Google Cloud Text Recognition API with Java

There has been numerous technologies, free and paid for recognition and detection for quite some time now. But the necessity for detection and recognition of objects and text has become a quintessential feature for AI and its dependent technology. I have tried quite a bit of OCR & detectors; mostly rendered, below average to average detection results with lots of ghost characters.

Here comes Google........

This is what google says about the API

"AutoML Vision makes it possible for developers with limited machine learning expertise to train high-quality custom models. After uploading and labeling images, AutoML Vision will train a model that can scale as needed to adapt to demands. AutoML Vision offers higher model accuracy and faster time to create a production-ready model.AutoML Vision makes it possible for developers with limited machine learning expertise to train high-quality custom models. After uploading and labeling images, AutoML Vision will train a model that can scale as needed to adapt to demands. AutoML Vision offers higher model accuracy and faster time to create a production-ready model."

The beauty of this framework is the ability to recognize with and/or without any external network connectivity. So we can now detect faces, barcodes, and text in image or video by having API installed on a device.

In this tutorial we will building a simple java program by using google Text Recognition API and we will scan a receipt using the API and see how far google can read the receipt.

Lets get started.

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicHeader;
import org.apache.http.protocol.HTTP;
import org.apache.http.util.EntityUtils;


public class GoogleReceiptScanner {

    public static void main(String[] args) {

        try {
            String key = args[0];
            String imageurl = args[1];
            new GoogleReceiptScanner().sendJSONData(key, imageurl);
        } catch (Exception E) {
            System.out.println("Exception Occured. " + E.getMessage());
        }
    }

    public String sendJSONData(String key, String imageurl) throws Exception {

        //creating map object to create JSON object from it
        String s = 
        "{\n" + 
        "'requests': [\n" + 
        "              {\n" + 
        "                   'image': {\n" + 
        "                        'source': {\n" + 
        "                        'imageUri': "+imageurl+"\n" + 
        "                    }\n" + 
        "              },\n" + 
        " 'features': [\n" + 
        "         {\n" + 
        "                'type': 'TEXT_DETECTION'\n" + 
        "         }\n" + 
        "       ]\n" + 
        "    }\n" + 
        " ]\n" + 
        "}" ;

        String url = "https://vision.googleapis.com/v1/images:annotate";
        DefaultHttpClient client = new DefaultHttpClient();
        HttpPost post = new HttpPost(url);
        post.setHeader("Accept", "application/json");
        post.setHeader("Authorization", "Bearer  " + key);
        StringEntity entity = new StringEntity(s, "UTF8");
        entity.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, 
                                              "application/json"));
        post.setEntity(entity);

        HttpResponse response = client.execute(post);
        HttpEntity resentity = response.getEntity();
        String det = null;
        if (resentity != null) {
            String content = EntityUtils.toString(resentity);
            int startindex = 
                content.indexOf("description") + "description".length() + 4;
            int endindex = content.indexOf("boundingPoly") - 7;
            det = content.substring(startindex, endindex).trim();
            det = det.substring(0, det.length() - 2);

        }
        System.out.println("==========================");
        String s1 = det;
        String replaceString = s1.replace("\\n", ":::");
        String[] words = replaceString.split(":::");
        for (String w: words) {
            System.out.println("=> " + w);
        }
        System.out.println("==========================");
        System.out.println("Response: " + response.getStatusLine());
        return response.getStatusLine().toString();
    }
}

Now Lets try reading the below Receipt image using the API Generated( Picked a random receipt from google)

javac GoogleReceiptScanner.java
java GoogleReceiptScanner ya29.GltOBmMjzyGXo2tcXSizAPBlD4NGI8a9jZ7sEk2JKsgR2ABx7nbf86RdKccoXt54rE5QKf32a_HI7unvm8SGmNtB-aozFZlAjzTPfae4olkjbiE6z2FuO-P3G6T2 'https://www.sunshinesupport.com/Images/receipt_example.jpg'