{"id":6500,"date":"2023-06-30T12:46:39","date_gmt":"2023-06-30T10:46:39","guid":{"rendered":"https:\/\/eurocc.nscc.sk\/?p=6500"},"modified":"2023-06-30T12:46:39","modified_gmt":"2023-06-30T10:46:39","slug":"meranie-strukturalnych-parametrov-kapsul-pouzitim-technik-umelej-inteligencie-ai-a-strojoveho-ucenia-ml","status":"publish","type":"post","link":"https:\/\/eurocc.nscc.sk\/en\/meranie-strukturalnych-parametrov-kapsul-pouzitim-technik-umelej-inteligencie-ai-a-strojoveho-ucenia-ml\/","title":{"rendered":"Measurement of microcapsule structural parameters using artificial intelligence (AI) and machine learning (ML)"},"content":{"rendered":"
\n
\n
\n
\n

Measurement of microcapsule structural parameters using artificial intelligence (AI) and machine learning (ML)<\/strong><\/p>\n\n\n\n

<\/p>\n<\/div><\/div>\n\n\n\n

The main aim of collaboration between the National Competence Centre for HPC (NCC HPC) and the Institute of Polymers of SAV (IP SAV) was design and implementation of a pilot software solution for automatic processing of polymer microcapsules images using artificial intelligence (AI) and machine learning (ML) approach. The microcapsules consist of semi-permeable polymeric membrane which was developed at the IP SAV.<\/p>\n\n\n\n

<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/a>
illustrative image<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

<\/p>\n\n\n\n

Automatic image processing has several benefits for IP SAV. It will save time since manual measurement of microcapsule structural parameters is time-consuming due to a huge number of images produced during the process. In addition, the automatic image processing will minimize the errors which are inevitably connected with manual measurements. The images from optical microscope obtained with 4.0 zoom usually contain one or more microcapsules, and they represent an input for AI\/ML process. On the other hand, the images from optical microscope obtained with 2.5 zoom usually contain (three to seven) microcapsules. Herein, a detection of the particular microcapsule is essential.<\/p>\n\n\n\n

The images from optical microscope are processed in two steps. The first one is a localization and detection of the microcapsule, the second one consists of a series of operations leading to obtaining structural parameters of the microcapsules.<\/p>\n\n\n\n

Microcapsule detection<\/p>\n\n\n\n

YOLOv5 model with pre-trained weights from COCO128 dataset was employed for microcapsule detection. Training set consisted of 96 images, which were manually annotated using graphical image annotation tool LabelImg [3]. Training unit consisted of 300 epochs, images were subdivided into 6 batches per 16 images and the image size was set to 640 pixels. Computational time of one training unit on the NVIDIA GeForce GTX 1650 GPU was approximately 3.5 hours.<\/p>\n\n\n\n

The detection using the trained YOLOv5 model is presented in Figure 1. The reliability of the trained model, verified on 12 images, was 96%, with the throughput on the same graphics card being approximately 40 frames per second.<\/p>\n\n\n

\n
\"\"<\/a>
Figure 1: (a) microcapsule image from optical microscope (b) detected microcapsule (c) cropped detected microcapsule for 4.0 zoom, (d) microcapsule image from optical microscope (e) detected microcapsule (f) cropped detected microcapsule for 2.5 zoom.<\/figcaption><\/figure><\/div>\n\n\n

Measurement of microcapsule structural parameters using AI\/ML <\/p>\n\n\n\n

The binary masks of inner and outer membrane of the microcapsules are created individually, as an output from the deep-learning neural network of the U-Net architecture [4]. This neural network was developed for image processing in biomedicine applications. The first training set for the U-Net neural network consisted of 140 images obtained from 4.0 zoom with the corresponding masks and the second set consisted of 140 images obtained from 2.5 zoom with the corresponding masks. The training unit consisted of 200 epochs, images were subdivided into 7 batches per 20 images and the image size was set to 1280 pixels (4.0 zoom) or 640 pixels (2.5 zoom). The 10% of the images were used for validation. Reliability of the trained model, verified on 20 images, exceeded 96%. Training process lasted less than 2 hours on the HPC system with IBM Power 7 type nodes, and it had to be repeated several times. Obtained binary masks were subsequently post-processed using fill-holes [5] and watershed [6] operations, to get rid of the unwanted residues. Subsequently, the binary masks were fitted with an ellipse using scikit-image measure library [7]. First and second principal axis of the fitted ellipse are used for the calculation of the microcapsule structural parameters. An example of inner and outer binary masks, and the fitted ellipses is shown in Figure 2.<\/p>\n\n\n

\n
\"\"<\/a>
Figure 2: (a) input image from optical microscope (b) inner binary mask (c) outer binary mask (d) output image with fitted ellipses.<\/figcaption><\/figure><\/div>\n\n\n

Structural parameters obtained by our AI\/ML approach (denoted as \u201cU-Net\u201c) were compared to the ones obtained by manual measurements performed at the IP SAV. A different model (denoted as \u201cRetinex\u201d) was used as another independent source of reference data. The Retinex approach was implemented by RNDR. Andrej L\u00fa\u010dny, PhD. from the Department of Applied Informatics of the Faculty of Mathematics, Physics and Informatics in Bratislava. This approach is not based on the AI\/ML, the ellipse fitting is performed by the aggregation of line elements with low curvature using so-called retinex filler [8]. The Retinex approach is a good reference due to its relatively high precision, but it is not fully automatic, especially for the inner membrane of the microcapsule.<\/p>\n\n\n\n

Figure 3 summarizes a comparison between the three approaches (U-Net, Retinex, UP SAV) to obtain the 4.0 zoom microcapsule structural parameters.<\/p>\n\n\n

\n
\"\"<\/a>
(a)<\/figcaption><\/figure><\/div>\n\n
\n
\"\"<\/a>
(b)<\/figcaption><\/figure><\/div>\n\n
\n
\"\"<\/a>
(c)<\/figcaption><\/figure><\/div>\n\n\n

Figure 3: <\/strong>(a) microcapsule diameter for different batches (b) difference between the diameters of the fitted ellipse (first principal axis) and microcapsule (c) difference between the diameters of the fitted ellipse (second principal axis) and microcapsule. Red lines in (b) and (c) represents the threshold given by IP SAV. The images were obtained using 4.0 zoom.<\/p>\n\n\n\n

All obtained results, except 4 images of batch 194 (ca 1.5%), are within the threshold defined by the IP SAV. As can be seen from Figure 3(a), the microcapsule diameters calculated using U-net and Retinex are in a good agreement to each other. The U-Net model performance can be significantly improved in future, either by the training set expansion or by additional post-processing. The agreement between the manual measurement and the U-Net\/Retinex may be further improved by unifying the method of obtaining microcapsule structural parameters from binary masks.<\/p>\n\n\n\n

The AI\/ML model will be available as a cloud solution on the HPC systems of CS\u010c SAV. Additional investment into the HPC infrastructure of IP SAV will not be necessary. Production phase, which goes beyond the scope of the pilot solution, accounts for an integration of this approach into the desktop application.<\/p>\n\n\n\n

References:<\/strong>:<\/p>\n\n\n\n

[1] https:\/\/github.com\/ultralytics\/yolov5<\/a><\/p>\n\n\n\n

[2] https:\/\/www.kaggle.com\/ultralytics\/coco128<\/p>\n\n\n\n

[3] https:\/\/github.com\/heartexlabs\/labelImg<\/a><\/p>\n\n\n\n

[4] https:\/\/lmb.informatik.uni-freiburg.de\/people\/ronneber\/u-net\/<\/a><\/p>\n\n\n\n

[5] https:\/\/docs.scipy.org\/doc\/scipy\/reference\/generated\/scipy.ndimage.binary_fill_holes.html<\/a><\/p>\n\n\n\n

[6] https:\/\/scikit-image.org\/docs\/stable\/auto_examples\/segmentation\/plot_watershed.html<\/a><\/p>\n\n\n\n

[7] https:\/\/scikit-image.org\/docs\/stable\/api\/skimage.measure.html<\/a><\/p>\n\n\n\n

[8] D.J. Jobson, Z. Rahman, G.A. Woodell, IEEE Transactions on Image Processing 6 (7) 965-976, 1997.<\/p>\n\n\n\n


<\/p>\n\n\n\n

\n
Success-Stories<\/a><\/div>\n<\/div>\n\n\n
\"\"<\/a> Artificial Intelligence and a Supercomputer as a New Weapon Against Environmental Disasters<\/strong><\/a> 26 Mar<\/span> -<\/span> Scientists from Nitra, Slovakia are teaching machines to predict industrial failures before they can cause damage. Thanks to collaboration with the European supercomputer LUMI, they have developed a digital \u201cguardian\u201d capable of detecting pipeline leaks or manufacturing faults with high accuracy\u2014helping protect both the environment and companies\u2019 budgets.<\/span><\/div>
\"\"<\/a> The Slovak Recipe for Fair Play and Happier Players<\/strong><\/a> 25 Mar<\/span> -<\/span> Do you play games on your phone and sometimes feel like the game just doesn\u2019t understand you? Experts from Nitra, Slovakia, have used one of Europe\u2019s most powerful supercomputers to change that. Thanks to the Italian giant named Leonardo, they discovered how to read between the lines of player behavior and make the gaming experience more personal and fair.<\/span><\/div>
\"\"<\/a> Urban buildings awaken: Slovak AI gives a second chance to underused spaces<\/strong><\/a> 4 Mar<\/span> -<\/span> Mest\u00e1 s\u00fa \u017eiv\u00e9 organizmy, ktor\u00e9 sa neust\u00e1le menia. Mnoh\u00ed z n\u00e1s v\u0161ak v susedstve denne m\u00ed\u0148aj\u00fa tich\u00e9 svedectv\u00e1 minulosti \u2013 pr\u00e1zdne \u0161koly, nevyu\u017e\u00edvan\u00e9 \u00farady \u010di ch\u00e1traj\u00face verejn\u00e9 budovy. \u010casto si kladieme ot\u00e1zky: \u201ePre\u010do je to zatvoren\u00e9?\u201c \u201eNemohol by tu by\u0165 rad\u0161ej denn\u00fd stacion\u00e1r, \u0161k\u00f4lka alebo kult\u00farne centrum?\u201c<\/span><\/div><\/div>\n<\/div><\/div>","protected":false},"excerpt":{"rendered":"

The main aim of collaboration between the National Competence Centre for HPC (NCC HPC) and the Institute of Polymers of SAV (IP SAV) was design and implementation of a pilot software solution for automatic processing of polymer microcapsules images using artificial intelligence (AI) and machine learning (ML) approach. The microcapsules consist of semi-permeable polymeric membrane which was developed at the IP SAV.<\/p>","protected":false},"author":2,"featured_media":6513,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"templates\/template-full-width.php","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/posts\/6500"}],"collection":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/comments?post=6500"}],"version-history":[{"count":7,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/posts\/6500\/revisions"}],"predecessor-version":[{"id":6515,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/posts\/6500\/revisions\/6515"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/media\/6513"}],"wp:attachment":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/media?parent=6500"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/categories?post=6500"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/tags?post=6500"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}