Experience: 5+ Years
Job Description:
- Participate in Requirements Gathering- work with key business partner groups and other Data Engineering personnel to understand Business unit wise data requirements for the analytics platform.
- Design Data Pipelines: work with other Data Engineering personnel on an overall design for flowing data from various internal and external sources into the Analytics platform.
- Build Data Pipelines from scratch with Python and SQL: leverage standard toolset and develop ETL/ELT code to move data from various internal and external sources into the analytics platform.
- Develop data migration, conversion, cleansing, retrieval tools and processes (ETL)
- Metadata driven data pipelines development experience
- Minimum Expertise of 3 years on Google Cloud Storage, data fusion, dataflow, pub/sub, Big query, bigtable, cloud functions, cloud composer, airflow, app engine, cloud data prep, cloud spanner, cloud dataproc,Document AI, Vertex AI
- Building batch & streaming large scale data pipelines in the enterprise data warehousing and data lake environments
- Overall, 5 to 8 years of rich data engineering experience
- Well versed with data file formats avro/parquet/json/orc processing
- Very good at SQL and advanced SQL
- Exposure to Looker & Data Studio
- Willing to work in the US overlap time zone.
- Good Attitude and Stake holder Communication is the key.
- Should have appetite and zeal to learn new technologies quickly and apply in the projects/POCs
