IT Senior Data Engineer

WHAT YOU'LL DO Understand the business needs through partnership with Marketing Data Hub Product Owner and Chapter Lead to obtain deep knowledge of data requirements for stakeholders. Independently design, build and launch new data extraction, transformation and loading processes in production, mentoring others around efficient queries. Support existing processes running in production and implement optimized solutions with limited guidance. Integrate data from a wide variety of sources, including on premise databases and external data sources with rest APIs and other replication tools. Participate in architecture design and implementation of high-performance, scalable and optimized data pipelines. Monitor daily execution, diagnose and log issues, and fix business critical pipelines to ensure SLAs are met with internal stakeholders Bridge gap between business requirements and ETL logic by troubleshooting data discrepancies and implementing scalable solutions Propose and implement data model and ETL code improvements to improve pipeline efficiency and data quality Recognize and adopt best practices in Data engineering: data integrity, test design, analysis, validation, and documentation Build data expertise and leverage data controls to ensure privacy, security, compliance, data quality, and operations YOU'RE GOOD AT Designing, developing & tuning medium-high complexity data applications/pipelines on large scale data platform Strong problem-solving skills and able to analyze data to identify deliverables, gaps, and inconsistencies. Able to manage conflicting priorities Excellent written and verbal communication skills and ability to succinctly summarize key findings Communicating effectively with teams in a matrixed organization with varying technical backgrounds & data expertise Intellectually curious and continuously striving to learn. Being a team player, open, pleasure to work with and positive in a group dynamic, ability to work collaboratively in virtual teams and someone who is self-starter and highly proactive. YOU BRING (EXPERIENCE & QUALIFICATIONS) Bachelor’s Degree in Computer Science or equivalent 4+ years of relevant experience in Data Engineering stream Expert in creating API integrated data pipelines using Python for extracting data from different sources. Advanced knowledge of different AWS technologies like AWS Lambda, AWS Glue, CloudFormation etc. Working knowledge of CI/CD based deployments using Github and Jenkins Expertise in SQL, especially within cloud-based data warehouses like Snowflake Exposure to unstructured datasets and ability to handle XML, JSON file formats Professional experience in building robust data pipelines using ETL tools (e.g. Alteryx, Talend, SSIS etc.) Understanding of version controlling tools like Github, SVN etc. Knowledge of Dbt cloud for orchestrating data models is preferred Experience of working on complex projects, including strategic and tactical planning and implementation, analysis, design, technology selection and deployment Experience with Agile and Scrum is preferred YOU'LL WORK WITH You will be a member of the Data Hub Squad, a team of passionate individuals whose mission is to ingest, transform, streamline and expose quality data to internal teams and to enable the Marketing function to make data driven decisions. This position will involve daily collaboration with the Chapter Lead, Product Owner, and the rest of the development teams within the Marketing Product Portfolio, as well as stakeholders and vendors throughout Agile design, development, implementation and operations of both infrastructure and business mappings to offer greater visibility and streamlined integrations. Candidates should be aware that BCG currently maintains a policy requiring all US & Canada based employees to be fully vaccinated against COVID-19. Newly hired employees must be fully vaccinated prior to their employment start date. BCG is an equal opportunity employer and will provide a reasonable accommodation to those unable to be vaccinated for medical or religious reasons where it is not an undue hardship to the company to do so as provided under applicable federal, state, provincial and local law.