Comment on page
Coral Sign Language Tutorial
It also demonstrates how quantization can be used to optimize models for edge devices. The model from the Convolution Tutorial classifies pictures of sign language hand gestures representing the digits 0 through 9.
TensorFlow provides post-training conversion of models to reduce their size and increase inference speed at the expense of losing a bit of accuracy. Using Full integer quantization, all the 32-bit floating-point values in the model are converted to the nearest 8-bit fixed-point numbers. These values generally include weights and activation outputs. Specific types of hardware accelerators used for faster machine learning computations such as Coral's EdgeTPU only support fully-quantized models.
- Ensure that you have PerceptiLabs 0.11.4 or higher installed as this is the version where the functionality to export quantized models was added to PerceptiLabs.
Follow the steps below to export the model in PerceptiLabs to a trained TensorFlow Lite model:
- 1.Navigate to File > Export.
- 2.Enter a path.
- 3.Set Export as to TensorFlow Model.
- 4.Enable Quantized to set full integer quantization.
- 5.Click Export.
- 6.Locate the exported .tflite file in the path you specified in Step 2. This will be used in subsequent steps below.
To run the inference on the Dev Board, we need the following:
- Quantized model (.tflite file) exported from PerceptiLabs
- Input data to run the inference
- Python code to run the inference
- (Optional) Label data
Below is a sample Python wrapper class (
ClassificationEngine) that loads the image data and quantized (trained) model, and runs inference using the EdgeTPU API. Also included is a simple
main()function that instantiates that class and invokes its methods.
Copy and paste all of the code into a Python script (.py file) named hand_recognizer.py.
import numpy as np
from edgetpu.basic.basic_engine import BasicEngine
def __init__(self, model_path, device_path=None):
def load_data(self, dataset_path):
self.dataset = np.load(dataset_path)
def load_labels(self, labels_path):
self.labels = np.load(labels_path)
for sample in self.dataset:
input_tensor_shape = self.required_input_array_size()
sample = np.reshape(sample, input_tensor_shape).astype(np.uint8)
def get_loss(self, label, output, loss_fn):
return loss_fn(label, output)
parser = argparse.ArgumentParser(
'-m', '--model', required=True, help='File path of .tflite file.')
'-l', '--labels', help='File path of labels file.')
'-i', '--input', required=True, help='File path of input data file')
args = parser.parse_args()
engine = classificationEngine(args.model)
inference_step = engine.inference_step()
output = next(inference_step)
if __name__ == '__main__':
ClassificationEnginederives from Coral's
BasicEnginethat provides many useful methods which include calculating inference time, finding output shapes are included in the EdgeTPU API’s, etc.
ClassificationEngineadds the following key methods:
load_data(): loads the image data.
required_input_array_size(): returns the required input shape for the model. Using this, the input can then be adjusted to match the required input shape and subsequently fed to the model.
inference_step(): returns a generator that can be used to run inference on each input sample each step.
ClassificationEngineis used to run inference on one sample at a time and the output of the network is printed to the console.
Once you have all the files (Python script, quantized model, and data files) they can then be sent from your local computer to the Dev Board.
Follow the steps below to run the Python script at the edge (i.e., on the Dev Board):
3. Run the script on the Dev Board using the following command: python3 hand_recognizer.py --model tflite_model.tflite --input data/X.npy
python3 hand_recognizer.py --model tflite_model.tflite --input data/X.npy
This should output a 10-dimensional array with class probabilities for the current step, predicting what digit the current hand sign image represents.
4. Enter a non-empty value or a string to have the script continue to the next step.
You have now run image classification at the edge on a Coral Dev Board.