The Data Engineer will focus on modernizing, building out and maintaining our data technology infrastructure and processes. Processes include ETL tasks, data modeling and manipulation, and building out data pipelines to power our applications and products. The role is a unique opportunity to create and shape the technology, methods and data related processes.
What You’ll Do
- Develop, test and maintain data architecture.
- Design and implement secure data pipelines to prepare, process, ingest and organize data into data data lake / data warehouse from disparate on-premise and cloud data sources.
- QA and troubleshoot performance of data pipelines and queries accessing data warehouse
- Clean, transform and model data to power our analytics and user facing products
- Ensure proper data governance and privacy practices
- Partner with Analytics team on buildout of advanced data products
- Assist with automation and orchestration
Who We’re Looking For
- Capable coder with Python, Scala, and R
- Familiar with modern, cloud-native, scalable ETL solutions/tools, ex: Informatica, Stitch/Talend, Mulesoft/Salesforce, etc…
- Experience with workflow orchestration principles & platforms (DAGs, Airflow, DBT, Luigi, Dagster, Prefect, etc…)
- Prefer experience with Google Cloud Platform(Bigquery, Dataproc, Dataflow, Pub/Sub, etc…). Experience with AWS (DynamoDb, Kinesis Stream, etc…) is a plus
- Building an analytic engine, segmentation and grouping data
- Experience writing scripts to automate the provisioning and maintenance of systems in a distributed, virtualized infrastructure
- Familiarity with managed cloud-based options for building machine learning models
- RDBMS database development using SQL queries and stored procedures
- Nice to have experience with ElasticSearch, DataDog, Serverless microservices