Summary

This code pattern discusses a way to create a progressive web application for mobile platforms that lets users classify images selected locally or taken with their mobile device’s camera. The app allows users to run a model offline in areas where internet connectivity might be sparse or nonexistent.

Description

After creating deep learning models, users want to be able to deploy their trained models in their own applications. There are several ways to do this, and how you do it depends largely on your use cases and requirements. One requirement is the ability to run a model offline in areas where internet connectivity might be sparse or nonexistent. The solution for this is to create native apps for mobile platforms that package and load a compressed version of your models. However, this has the overhead of needing developers with expertise in Android and iOS development.

This code pattern looks at an alternative, easier way to satisfy this offline mobile requirement by creating a progressive web application with the model using React and TensorFlow.js. Progressive web applications give a native app-like feel and can run on most modern web browsers. This makes cross-platform development much easier because the application only has to be developed once in HTML/JavaScript. Furthermore, through the use of service workers, progressive web applications can provide fully offline functionality.

With TensorFlow.js, we convert our pre-trained TensorFlow or Keras models into JavaScript to be run in the browser through the app. Then, we have a cross-platform application where users can classify images selected locally or taken with their mobile device’s camera. The app uses TensorFlow.js and a pre-trained model converted to the TensorFlow.js format to provide the inference capabilities. This model is saved locally in the browser using IndexedDB, and a service worker is used to provide offline capabilities.

With this solution:

• Data remains on-device and classification is performed locally
• Assets are stored in a browser cache and storage
• App can run on desktops and be ‘installed’ on mobile devices
• Content can still be updated by prompting the user

Flow

1. A pre-trained Keras/TensorFlow model is converted to the TensorFlow.js web-friendly format and integrated with the app.
2. The user launches the progressive web application.
3. The app assets and TensorFlow.js model files are downloaded from the web.
4. The assets and model are stored locally using the browser’s cache and IndexedDB storage.
5. The user takes a photo with a device camera or selects a local image.
6. The image is sent through the model for inference and top predictions are given.

Instructions

Find the detailed steps for this pattern in the README. Those steps show you how to:

1. Clone the repository.
2. Install the app dependencies.
3. Download and convert the pre-trained model.
4. Set up the configuration files.
5. Deploy the app and classify the images.

Paul Van Eck

Source: https://developer.ibm.com/patterns/create-a-progressive-web-application-for-offline-image-classification/