Senior Data Engineer (Hybrid)

Job Summary:

Builds maintains big data pipelines to support advanced analytics and data science solutions. Identifies valuable internal and external data. Collaborates closely with data scientists to define data for the design, development, and deployment of new solutions that support strategic business priorities.

Ensure seamless integration of data across the enterprise and drive automation of the most common and repeatable processes. Performance Tuning and Optimization of all Data Ingestion and Data Integration processes, including Data Platform and databases

Job Responsibilities:

• Develops software that processes, stores, and serves data for use by others.
• Develops large scale data structures and pipelines to organize, collect and standardize data that helps generate insights and addresses reporting needs.
• Writes ETL (Extract / Transform / Load) processes, designs database systems and develops tools for real-time and offline analytic processing.
• Develops and maintains optimal data pipelines into the advanced analytics platform, including design of data flows, procedures, and schedules. Ensures that optimal data pipelines are scalable, repeatable, and secure.
• Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of sources, assuring that they adhere to data quality and accessibility standards.
• Anticipates and prevents problems and roadblocks before they occur.
• Interacts with internal and external peers and managers to exchange complex information related to areas of specialization.
• Collaborate with data scientists to prepare data for model development

• Bachelor's degree and at least 4 years of experience in data engineering OR a Graduate Degree in a technical discipline and at least 2 years of experience in data engineering
• Deep knowledge of SQL
• At least 2 years of experience with REST API development
• Experience establishing and maintaining key relationships with internal (peers, business partners and leadership) and external (business community, clients, and vendors) within a matrix organization to ensure quality standards for service.
• Experience diagnosing, isolating, and resolving complex business issues and recommending and implementing strategies to resolve problems.
• Experience presenting to all levels of an organization
• At least 2 years of experience contributing to financial decisions in the workplace.
• At least 2 years of direct leadership, indirect leadership and/or cross functional team leadership.
• Willing to travel up to 10% of the time for business purposes (within state and out of state).

• Experience deploying Python applications
• Experience with messaging systems such as Kafka or Azure Event Hubs
• Experience with Azure Databricks and Delta tables
• Experience with data engineering practices using Snowflake
• Experience with Git, merging, managing branches, and resolving conflicts.
• Experience with manual/automated functional testing and unit testing.
• Experience with project management practices, frameworks, and methodologies (Agile/Scrum/Waterfall)
• Experience with Data Platform concepts (Data Lake, Data Warehouse, and Delta Lake/Lakehouse). General understanding of landing, staging area and data cleansing