How to use Tensorflow straight from Delphi (and Freepascal)
How much time have you listen about artificial neural net, and the famous Tensorflow library for Python? But the good news is: you can use Tensorflow (TF in short) straight from Delphi! Thats is, without using Python as a proxy language. Of course you can also do in that mode, by mean of the great Python4delphi package.But there is a much more lightweight and fast mode, bypassing Python, and exploit the power of Tensorflow from Delphi. Read how.
About Tensorflow Lite
Tensorflow creators have crafted a special lightweight library, Tensorflow Lite. The reason is to exploit TF networks also in constrained software environments, as on cellular phones, microcomputer as Raspberry, and some type of microcontrollers, too. These platforms, compared to desktop pc and notebooks, have much less memory, less CPU power, and often they have no GPU chip . And sometimes there is no Python distribution at all for them. So the only choice is an autonomous library, having no need of Python, and optimized for the specific platform.
The nice thing is that the library is open source, and can be compiled also for regular pc operating system, i.e. in particular Windows and all sort of Linux. Of course there are some limitations: TFLite can only use an existing neural net to compute the output from assigned input (that is, can do only predictions or inference, in AI jargon); it cannot do the training of the net. Further, not all nets can be translated to the special format required by TF Lite.
Anyway, an entire world of opportunities is opened, having in mind the big diffusion of neural nets in TF format, just ready to use!
Now a brief description of input and ouput data format in TF Lite. I refer specially to convolutional neural network used on images: nets used to divide entire images in classes (cats, dogs, apples, pears, men, ...); and nets used to detect some object/class inside an image (example: all faces, all dogs, ...).
But note that TF Lite supports many other types of neural nets, from numeric regression simple nets to recurrent neural networks (RNN) for natural language processing.
A net can manipulate float data, or integer data. Every net require a specified type, that is you cannot feed float data to an integer net, and viceversa. Anyway, the library has some functions to query many net info, included type of input and output data.
I do some experiments, and found that the float format is much more fast on Windows (32 bit library version). But if you use an existing net, you are forced to use the format of the net, of course. Generally speaking, the input to a net is a 2 dimensional matrix of numbers, either float or single. And the returned output is another matrix with from 1 to 4 rows.
The number of rows depend from the type of net: a classification net return 1 vector, only scores; a object detection net return 4 vectors, classes, scores, 4 corners of detection rectangle (vector of vectors), and a vector with a single number that is number of detections (this indeed is always 10).
Scores are the probability that network assign to every classes detected; 1.0 is absolute certainty. TFLite return a variable number of results, depending on the net. Typical for object detection nets is a return vector of the 10 classes with highest scores.
The input matrix must have the dimensions required by the net. If the input is an image, so you must resize it to that dimensions. Some nets required also that the input data are normalized in a specific way. For example the image bytes have values between 0 and 255, but some nets want values between 0 and 1 normalized dividing by 127,5 (of course these nets want also float inputs!).
The various output rows/vectors are not in the same order for different nets. That is, a net can have the scores as the first output row , another net can have the classes as the first row. But by some common sense rules it's possible to find in code the correct order of output vectors.
Last, every net has two files: one is the same net data (neurons, layers, weights, ecc.); the other is the list of names associated with numerical classes. TF Lite doesn't use the names list, but this is useful to present data to users, of course.
I want to thanks the authors of original experiments on Delphi with TF Lite.
You can find them at: https://github.com/Embarcadero/TensorFlow-Lite-Delphi
Predicting by an artificial neural net in TF Lite
In Windows platform, TF Lite presents itself as a normal dynamic library (DLL; in Linux is a SO), with exported functions in plain C format. So, it's simple to interface with Delphi!The unit unTFLite.pas contains some classes for use TF Lite. Methods can: load a net and define it to TFLite engine; access some descriptive info of the net; prepare the input data (input image) in the required format; call the prediction function; read the predicted data.
There is a base class TTFLiteInterpreterBase, it has methods to process input for a standard base network: max input image dimensions 1024x1024, input data type byte or single, optional input normalization only for single type input.
You don't must use directly the standard class, but use the derived classes.
In the unit there are 3 derived classes:
- TTFLiteMobilenetV2 to use the image classification network Mobilenet version 2
- TTFLiteSSDMobilenet to use the object detection network SSD/Mobilenet that requires byte data input
- TTFLiteSSDMobilenetFloat to use a type of SSD/Mobilenet that requires float data input
- TTFLiteCenternetMobilenet to use the object detection network Centernet/Mobilenet
The class produces an output array containing the predictions, one record for every prediction. Every record has class name and score. In case of object detection network, in record there is also the detection box (rectangles corners).
One of the class properties is minScore. Only predictions with score greater of a minimum are added to output array. The result record, array and box types are defined in unTFLiteResults, with some useful methods, as one to draw boxes in image.
Program txConvnet is a demo for SSD/Mobilenet network, file detect.tflite, pretrained on COCO dataset of images. This set contains many dozens of objects, animals and human classes. So the network can recognize all that classes in a single image.
There is also the program TFnetinfo, that read a TFLite network and print useful informations on input and output vectors. For images it's particularly useful to view the input image dimension. For example, for detect.tflite it's 300x300.
Using TF Lite in Freepascal and Lazarus
The units unTFLite.pas and unTFLiteResults.pas can be used equally well in Freepascal and Lazarus projects. Demo program txConverter for Lazarus is provided.