Reduce risk by migrating to Google Cloud

One of Australia's leading owners and operators of integrated resorts in Australia was facing a number of challenges and risks related to their 20 year old on premise data platform.

Data ingestion and ETL management was cumbersome, prone to failure, and relied on significant manual intervention and schedule management. The proliferation of custom stored procedures and user specific functions generated significant change management overhead for support teams. Frequent outages driven by increased demand was driving down patron, guest and end user satisfaction as critical applications would go off-line due to failed data processing. 

Current data management and security capabilities were no longer aligned with government compliance and reporting requirements. On top of all of this, the technology was running out of vendor support and needed to be decommissioned as soon as possible. 

‍

Having made the strategic decision to migrate to Google Cloud, our senior consultant worked in partnership with the client’s SMEs, Google Professional Services and a team of data engineers, to design and deliver an Agile programme of works to reengineer their existing on-prem data platform and migrate the their data warehouse and data management services to Google Cloud Platform.

The migration involved establishing a new Google Cloud instance, Big Query Data Platform, data ingestion pipelines, and rebuilding over 600 custom stored procedures and user specific functions. Due to the high volume and limited time available, the project focused on utilising automation to accelerate delivery. It invested in building RegEx pattern matching and AI supported regeneration of stored procedures into functions in Big Query.

A complete data profile and data type correlation was conducted on all data sets to ensure the BigQuery data model was designed appropriately and migration into BigQuery would be error free. Data migration was prioritised according to data criticality, volumes and co-shared data domains.

‍

The solution included the following;

• New instances of Google Cloud Platform
• Big Query schemas including landing, staging and data marts
• Over 600 stored procedures migrated, tested and deployed in GCP within 6 weeks
• 1,200+ data objects, tables and view reengineered in Big Query
• Source to GCP automated event driven data ingestion pipelines deployed
• Migration of over 20 years of historical data into BQ
• Monitoring and alerting integration with client’s IT Service Desk

‍