Flutter Plugins: Reading text using mobile vision

0 3

By Abhilasha Sinha Flutter August 19, 2020

In this week’s blog on plugins, we will look at a very special requirement – OCR(Optical Character Recognition) using the mobile vision of the device camera.

As per wikipedia, “Optical character recognition or optical character reader (OCR) is the electronic or mechanical conversion of images of typed, handwritten or printed text into machine-encoded text, whether from a scanned document, a photo of a document, a scene-photo (for example the text on signs and billboards in a landscape photo) or from subtitle text superimposed on an image”.

In today’s world of automation, where ease and performance of an application are of utmost priority, Optical character recognition is one such capability which facilitates automatic data entry and improves user experience in a variety of situations. There may be some need for user revalidation after the data is picked up, to ensure no mismatch and no wrong data entry.

In this blog, we will make use of the flutter_mobile_vision plugin to fulfill this requirement for OCR. This plugin is the Flutter implementation of Google mobile vision and currently supports Android only. It provides the three functionalities – recognize text, detect faces and scan barcodes.

In this blog, we will use this api to recognize and capture text through the vision of the camera and take it forward to be used in our application as needed.

Let’s look at how we can make use of this plugin and recognize any text anywhere.

We will be working on the same example from the last blog and enhance it for this feature. Alternatively, You can start by creating a new app as below:

flutter create new-app

1	flutter create new-app

Dependencies

Add the prerequisites in the pubspec.yaml which includes the plugins identified in the process.\

flutter_mobile_vision: ^0.1.3

1	flutter_mobile_vision: ^0.1.3

This plugin only supports Android. The below needs to be added to the AndroidManifest.xml

<activity android:name="io.github.edufolly.fluttermobilevision.ocr.OcrCaptureActivity" />

1	<activity android:name="io.github.edufolly.fluttermobilevision.ocr.OcrCaptureActivity" />

Creating the widget using the plugin

The widget ReadOCRLive will include the reader icon which when clicked will start the camera vision view with text reading capability.

The camera will be initialized as follows:

int _cameraOcr = FlutterMobileVision.CAMERA_BACK;

1	int _cameraOcr = FlutterMobileVision.CAMERA_BACK;

The build will include logic to render the button.

 @override
  Widget build(BuildContext context) {
    return     GestureDetector(
      child: Padding(
          padding: const EdgeInsets.only(top: 40, left: 130),
          child: Card(
            elevation: 15,
            child: Container(
                height: 80,
                width: 100,
                child: Column(
                  mainAxisAlignment: MainAxisAlignment.center,
                  children: [
                    Icon(Icons.camera),
                    Text("Scan",
                        style: TextStyle(fontWeight: FontWeight.bold))
                  ],
                )),
          )
      ),
      onTap: () => _read(),
    );
  }

@override

Widget build(BuildContext context) {

return GestureDetector(

child: Padding(

padding: const EdgeInsets.only(top: 40, left: 130),

child: Card(

elevation: 15,

child: Container(

height: 80,

width: 100,

child: Column(

mainAxisAlignment: MainAxisAlignment.center,

children: [

Icon(Icons.camera),

Text("Scan",

style: TextStyle(fontWeight: FontWeight.bold))

)),

)

onTap: () => _read(),

);

}

The onTap action handler is the _read method which will have the logic to use the flutter_mobile_vision plugin.

Future<Null> _read() async {
    List<OcrText> texts = [];
    List<String> values = [];
    try {
      texts = await FlutterMobileVision.read(
        multiple: true,
        camera: _cameraOcr,
        waitTap: true,
      );
      print(texts);
      print('bottom ${texts[3].bottom}');
      print('top ${texts[2].top}');
      print('left ${texts[4].left}');
      print('top ${texts[3].right}');
      print('top ${texts[1].language}');
 
      texts.forEach((val) => {
            values.add(val.value.toString()),
          });
      widget.getValue(values);
      if (!mounted) return;
    } on Exception {
      texts.add(new OcrText('Failed to recognize text.'));
    }
  }

Future<Null> _read() async {

List<OcrText> texts = [];

List<String> values = [];

try {

texts = await FlutterMobileVision.read(

multiple: true,

camera: _cameraOcr,

waitTap: true,

);

print(texts);

print('bottom ${texts[3].bottom}');

print('top ${texts[2].top}');

print('left ${texts[4].left}');

print('top ${texts[3].right}');

print('top ${texts[1].language}');

texts.forEach((val) => {

values.add(val.value.toString()),

});

widget.getValue(values);

if (!mounted) return;

} on Exception {

texts.add(new OcrText('Failed to recognize text.'));

}

The below code will read the multiple texts captured using the camera provided and give the result as you tap on the camera view, when the results will be returned in the texts variable in the form of an List<OcrText>.

texts = FlutterMobileVision.read(
        multiple: true,
        camera: _cameraOcr,
        waitTap: true,
      );

texts = FlutterMobileVision.read(

multiple: true,

camera: _cameraOcr,

waitTap: true,

);

This will include the text value along with the position dimensions(top, bottom, left, right) as well as the language.

This widget can be included in a parent widget by importing and calling as

 ReadOCRLive(getValue),

1	ReadOCRLive(getValue),

Where getValue is a function which will consume the data collected by the mobile vision and can be implemented as per the requirements.

Function getValue(List<String> values) {
    print(values);
    //You can include logic to handle the values
  }

Function getValue(List<String> values) {

print(values);

//You can include logic to handle the values

}

You will get the view like below showing OCR text in rectangles when you press the reader icon and hold the camera on top of any book/screen with text.

You can find the complete git repository for the implementation here.

Conclusion

In this blog, we saw the steps to implement text recognition capability of the flutter_mobile_vision plugin for android devices. The same plugin can also be used for barcode scanning and face detection as it is the implementation of Google Mobile Vision.

We will explore this further and other plugins in the coming weeks.