Development of a cloud-based repository based on Azure

• SECTOR: CHEMISTRY
• Task: This customer project involved the development of a cloud-based infrastructure for the automated upload and download of laboratory measurement data and the subsequent analysis using Azure Machine learning.
• Team:
• Total of about 12 people consisting of POs, Backend- Frontend, FullStack developers, DevOps
• Project duration: +12 months

• Extending the Legacy Laboratory Data Management System with Azure
• Scaling: Steadily growing number of connected lab instruments and clients spread all over the world
• Unintuitive and slow UI of the legacy management system
• “Data – silo”, difficult access to decentralized data storage locations

• Implementation of a microservice architecture with NodeJs and Python
• Modular, simple and scalable front-end
• Fully automated provisioning with Terraform
• Fully automated pipelines

• Higher flexibility due to the modular structure of the software architecture
• Robustness through the use of Infrastructure as Code (Terraform Azure)
• Faster release cycles through CICD Azure DevOps
• Modern Secret Management with Keyvault

Development of Data Foundation Analytics
The customer’s objective was to extend its legacy data management tool. The main users are in-house employees. Due to the growing number of users based on different locations and the problem of data silos, the need for a central cloud-based data repository grew. Further challenges were an outdated, slow and unintuitive user interface design. This was to be remedied with a specially developed front-end application.

Cloud Infrastructure, Microservices and Infrastructure As Code
The challenges were solved with a modern microservice architecture using NodeJs, Angular and Python, IaC and Azure DevOps. Independent services allow teams to develop and deploy their components separately by loosely coupling them. Thanks to the newly introduced development and deployment workflows using Azure DevOps, consistent code quality is guaranteed and release cycles are accelerated. By using Infrastructure as Code with Terraform, it is possible to recover the entire infrastructure in a short time, making it more robust and greatly reducing the management overhead on the cloud infrastructure. In addition, the introduction of Azure Machine Learning is currently in progress, which will enable real-time analysis of the sample data from the ADLS. For this purpose, the data from the ADLS is fed into the respective machine learning models using Azure Datafactory in order to process it there. The use of Azure Machine Learning is designed in such a way that the deployment and training of different models can be easily scaled.

Project status and results
The system has been running in the productive environment for several months and the number of users as well as connected laboratory instruments is steadily increasing. Therefore, the development of the project is ongoing. Due to the applied best practices in software development and cloud DevOps, the system runs robustly and is flexible for new adaptations.

Cloud infrastructure:

• Azure

Data Services:

• Azure Data Factory
• Azure Machine Learning

CICD & Iac:

• Terraform
• Azure DevOps

Software development:

• TypeScript – NodeJs
• Angular
• Python