Migrating Nonacus data pipelines and workflows from Azure to AWS

  • Customer
  • Industry
  • Service
  • Segment
  • Author
  • Nonacus
  • Healthcare & Life Sciences
  • Migration
  • SMB
  • Jodie Rhodes
Explore more
At a glance

Nonacus was founded in 2015 with a singular purpose: to offer high quality, non-invasive, genetic testing, with the end-user at the forefront.

Challenge

The Nonacus team faced cloud scaling challenges with Microsoft Azure’s batch technology for genome sequencing.

Solution

Firemind migrated Nonacus’ data pipelines to AWS, creating a secure, scalable solution to address their cloud scaling challenges.

Services used
  • Amazon S3
  • AWS IAM
  • Amazon ECR
  • AWS ECS
Outcomes
  • 25-day migration
  • Optimised compute resources
  • Maximised cost efficiency
Business challenges

Overcoming cloud scalability hurdles

The Nonacus team had encountered many challenges with cloud scaling while utilizing Microsoft Azure’s batch technology for their genome sequencing solution. Genomic diagnostics demands the handling of vast volumes of intricate data, requiring precise and reliable results. The existing setup faced difficulties in meeting the scalability needs, leading to inefficiencies in processing and resource management.

To support their growing data and computation needs, a new approach was needed – one that could provide a seamless transition from existing workflows while ensuring a cost-efficient, fully scalable infrastructure. An effective and transformative architecture became critical, not just for the migration but also for sustaining long-term growth and performance in data-intensive tasks.

 
 

What our customers say

Hear directly from those who’ve experienced our services. Discover how we’ve made a difference for our clients.

Solution

Firemind's AWS-powered migration

For a company like Nonacus that handles the intricate data for oncology and pre-natal healthcare, ensuring a smooth migration with no loss of function was high priority. This project involved the ideation, design and deployment of substantial infrastructure on the AWS cloud, providing a full migration of the customer’s scoped workloads from Microsoft Azure to AWS services.

The project began by creating the main architecture for Amazon S3. This would allow Nonacus to store data securely as part of their Nextflow DSL 2 processes. Nextflow provides a convenient syntax extension that allows the definition of module libraries, as well as simplifying the writing of complex data analysis pipelines.

Our solution used intelligent tiering storage classes within S3, as a way to provide Nonacus the maximum cost efficiency, whilst not impacting their data access timelines.

Following this setup, we created a secure IAM role that allows Nextflow Tower SaaS to access the data in S3 for reading and writing. As Nextflow Tower SaaS didn’t appear to explicitly use AWS Key Management Service (KMS), we provided the IAM the right to decrypt using KMS, creating an S3 bucket policy that requires Nextflow Tower SaaS to use “aws:kms” encryption.

Now the roles, accesses and encryptions were all taken care of, Firemind built out an architecture that allowed Nonacus to migrate their pipelines and workflows to AWS from Microsoft Azure. We setup a Github Actions pipeline to build and deploy open source custom Docker images to Amazon Elastic Container Registry (ECR). ECR allowed us to push container images without installing or scaling infrastructure, as well as pull images using any management tool.

Setting up an ECS cluster to run AWS Fargate tasks, based on the images in ECR, was one of the final steps. We provisioned the Fargate task role to allow the tasks to access both S3 and KMS, to read and write any data by Nonacus.

We also used AWS Graviton Processors to ensure the best price performance within their computational workloads. AWS Graviton3 processors are the latest in the AWS Graviton processor family. They provide up to 25% better compute performance, up to 2x higher floating-point performance, and up to 2x faster cryptographic workload performance compared to AWS Graviton2 processors.

Optimised compute resources

With the introduction and use of AWS Batch, we could ensure the Nonacus Scientists, and Engineers, could efficiently run hundreds of thousands of batch and ML computing jobs, while optimising compute resources, so they can focus on analysing results and solving complex problems.

Effective migration

Confidently and securely migrating an existing workflow, especially from another cloud provider, can be a difficult task to achieve. Our thorough planning and extensive troubleshooting, step by step, ensured a smooth transition, with no loss of function to the complex and priority focused workloads of Nonacus.

Guidance and funding

As an Advanced Partner of Amazon Web Services (AWS), we were able to enable funding opportunities and avenues of cost savings, subject to certain criteria being met. This funding and project guidance becomes an invaluable asset, when planning, experimenting and building new workflows in the cloud.

Get in touch

Want to learn more?

Seen a specific case study or insight and want to learn more? Or thinking about your next project? Drop us a message!