An innovative solution for autosamplers
Modern laboratories are all about precision, (sample) safety and efficiency. Within this framework, Spark Holland developed an innovative solution for smart identification of sample plates with AI. The method combines image processing with a two-dimensional Fast Fourier Transform (2D-FFT) and a compact neural network that can run on a microcontroller or small processor. This approach also allows the instrument to reliably recognize the supported sample-plates, contributing to fewer errors and faster analyses.
The challenge
Thousands of samples are analyzed daily in laboratories using autosamplers, see figure. Setting the plate type correctly in the acquisition software and physically placing the correct sample plates in the instrument is a responsibility of the end user. An error in this can lead to malfunctions in the autosampler, causing a lab downtime and, in the worst case, loss of valuable samples.
Methods exist to identify sample plates with additional hardware, such as markers with optical sensors or RFID tags, but these require the use of specific plates. Instead, Spark Holland's goal is to be able to identify common, generic plates without modifications or additional hardware.
Recent developments in artificial intelligence (AI) have shown that image recognition is quite possible. However, a fully autonomous working instrument requires that the AI runs locally, without dependence on external (Cloud) computers. Therefore, Spark Holland is focused on implementing AI on a microcontroller so that recognition can be done directly by the instrument.
The solution
To achieve fast and reliable recognition of sample plates, the company combines algorithmic image processing with artificial intelligence. The implementation consists of a number of stages:
A camera takes an image of the sample plate in the holder. Because space inside an autosampler is limited and lighting conditions are not optimal, areas of the image can become distorted and unevenly exposed. The image below shows how this distortion occurs and how some areas are underexposed and others are overexposed. Therefore, editing of the image is necessary.
The first step
The first step is to focus on the central part of the image and apply a smooth transition to the edges. This avoids abrupt contrast differences that can cause noise in pattern recognition (see image below). After this operation is performed, we apply the two-dimensional Fast Fourier Transform (2D-FFT). This algorithm, which is also used in image processing and transfer, makes repeating patterns - such as the wells in a sample plate - more visible. In a 2D-FFT image, these repeating structures appear as bright points at fixed distances. This is also the case after the 2D-FFT of our image (see below).
The next step
The next step is actual image recognition by a neural network. Neural networks are widely used for image recognition. Their operation is based on recognizing patterns in large amounts of training data. In a sense, they simulate how the human brain learns by making connections between input images and corresponding categories.
For a neural network, for example, TensorFlow can be used, but that requires considerable computing power and hardware, which are not available in an autosampler. Therefore, Spark Holland itself has implemented a compact neural network in the C programming language. The neural network has a memory usage of less than 2 MB, allowing it to easily run on a microcontroller or other embedded platform. Training takes place with the 2D-FFT images of ‘known’ sample plates, where the network learns to distinguish between different plate types. After training, the neural network can accurately recognize and classify sample-plates without dependence on external hardware.
In the testing phase, we trained the neural network with 10,000 images of different sample plates in multiple orientations. Thanks to the compact design of the network, training time was limited to less than 15 minutes.
After training, the company had the neural network analyze 3,000 images of sample plates. Of these, 2,999 were recognized correctly - within one second. This resulted in a recognition accuracy of 99.97%, a promising result for a first concept with a compact neural network.
What was particularly striking was that the algorithm not only distinguished between different sample plates, but was also able to correctly recognize both high and low sample plates with the same well classification. This is remarkable, since these plates appear almost identical when viewed from the top - and thus from the camera's perspective. Moreover, the fast recognition time of less than a second hardly adds to the cycle time of an analysis.
Future developments
In the near future, Spark Holland plans to further expand the number of supported sample plates. An important advantage of a neural network is that it is “trainable.” Therefore, expanding with new sample plates is relatively easy by including them in the training set.
In addition, sample plates are often covered with a film, making the wells less visible. Possibly this problem could be solved by using better lighting (at a sharp angle, for example) to bring out the contours of the wells better. This will be an interesting topic of further research.
Spark Holland's current neural network is compact, enabling rapid training and recognition. In the future, the company plans to investigate whether a slightly more complex network can further improve recognition without sacrificing speed or feasibility on embedded systems.
Conclusion
Combining AI-based image processing, SPARK Holland has taken an important first step in autonomous recognition of sample plates by the new autosampler. The result - an impressive recognition accuracy of over 99.9% - is promising.
Spark has shown that technological challenges can be practically solved by using smart image processing algorithms and AI, and that this is also applicable to compact systems with limited computing power.
The company hopes that this project will inspire other companies within the innovation cluster to explore this technology further. Indeed, the possibilities extend beyond laboratories: fast and reliable object recognition can also be of great value in other fields and production processes.