SageBeans RPO
Job Title: BI Data Engineer – Hadoop to GCP Migration
Location: Toronto, ON (hybrid 3x/week in the office)
Job Type: Contract
37.5 hours/week
Contract Duration: 31 Dec 2024 (9 months)
• 1 consultant- to decommission Hadoop to GCP migration.
Background: Support our Network Cloud migration initiatives.
5+ years of experience
Looking for a contractor to assist with data transfers from on-prem Hadoop to GCS (Google Cloud Storage bucket) using either :
1 – gsutil: Google provided python based utility gold
2 – distcp: Use distributed copy provided by Hadoop distribution for copying data from on-premises Hadoop to Google Cloud Platform on GCS.
DETAIL:
Responsible for designing and implementing the infrastructure necessary to store, process, and analyze large sets of data. They work with stakeholders to understand their business needs and develop scalable and reliable data architectures. They also ensure that the infrastructure is optimized for performance and efficiency.