Manager – Data Engineering

Manager – Data Engineering

Location: United States Remote

Experience: 10-12 yrs

Tech Stack: AWS (S3, Redshift, Glue, Athena, Lambda, Step Functions, DMS), SQL Server (T-SQL), SSIS, Python, Data Warehousing/ETL, AI-Assisted Development

Role Overview

We are looking for a Lead Data Engineer to own the technical direction and delivery of data engineering initiatives across our health plan technology platform, spanning AWS-native data services and existing SQL Server/SSIS ETL processes. Working alongside software engineering, Data/BI & Integration, DevOps, and Edifecs EDI teams, you will set engineering standards, lead the migration and modernization of legacy workloads onto AWS-native pipelines, and mentor a team of data engineers. This role combines deep hands-on technical ownership with emerging team leadership, acting as the primary technical escalation point for data pipeline design, performance, and reliability.

Key Responsibilities

● Lead the design, build, and delivery of ETL/ELT pipelines using AWS Glue, Lambda, and Step Functions to ingest, transform, and load data across AWS-based data platforms (S3, Redshift, Athena)

● Own the technical roadmap for migrating and modernizing legacy SQL Server/SSIS ETL workloads to AWS-native data pipelines

● Maintain, enhance, and troubleshoot existing SQL Server-based ETL processes built in SSIS, including packages, control flows, data flows, and error handling

● Define coding standards, design patterns, and best practices for data engineering across the team

● Write and optimize complex T-SQL queries, stored procedures, and views supporting reporting and downstream applications

● Architect and govern data models (dimensional/star schema) supporting analytics and reporting use cases

● Lead use of AWS Database Migration Service (DMS) and related tooling to drive data migration from on-premises SQL Server to AWS

● Define and enforce data quality, validation, monitoring, and lineage standards across ETL pipelines

● Drive pipeline performance optimization, reliability improvements, and cost efficiency across both AWS-native and SQL Server/SSIS workloads

● Serve as the primary technical partner for BI, integration, and application teams (including Power BI and Edifecs/EDI teams) to ensure data availability and consistency

● Lead technical design reviews and contribute to architecture decisions at the program level

● Own production incident response and root-cause analysis for data pipeline issues; implement long-term reliability improvements

● Mentor and develop data engineers; guide technical growth and conduct meaningful code reviews

● Maintain thorough documentation of data flows, pipeline architecture, and data models to support team knowledge sharing

Required Qualifications

● Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field

● 10 - 12 years of experience in data engineering or ETL development, with demonstrated progression toward technical leadership

● Expert, hands-on experience with AWS data services: S3, Redshift, Glue, Athena, Lambda, and Step Functions

● Expert, hands-on experience with SQL Server, including T-SQL development, query optimization, and performance tuning

● Expert, hands-on experience building and maintaining ETL packages in SSIS (control flow, data flow, error handling, and deployment)

● Strong proficiency in Python for scripting, automation, and Glue/PySpark-based ETL development

● Deep expertise in dimensional data modeling (star schema) for analytics and reporting use cases

● Proven experience leading data migration programs using AWS DMS or equivalent tooling for on-premises to AWS transitions

● Strong command of data quality, governance, and lineage practices -- able to define and enforce standards across a team

● Strong debugging, performance-tuning, and production-support skills across complex ETL pipelines

● Experience leading technical projects or guiding a team of data engineers

● Excellent written and verbal communication skills for cross-functional and stakeholder engagement

AI Knowledge & AI Assisted Development (Required)

● Daily, practical use of AI coding/assistant tools (e.g., Claude Code, GitHub Copilot, Cursor, or similar) to accelerate development of Glue/PySpark scripts, SSIS package logic, and T-SQL queries

● Able to critically review and validate AI-generated code, queries, and transformations for correctness, performance, and data integrity before deployment

● Defines and enforces team-level standards for responsible AI-assisted development, including validation rigor, security review of AI-generated code, and prompt hygiene

● Practical use of AI tools to assist with data profiling, anomaly detection, and technical documentation

● Deep understanding of secure and compliant AI tool usage, including never entering PHI, member data, or other sensitive information into prompts or external AI tools

● Identifies and leads adoption of AI-driven automation to improve pipeline development, testing, and migration efficiency across the team

Preferred Qualifications

● Experience in the US health insurance or payer domain: claims, eligibility, enrollment, provider, or member data, with HIPAA-aware data handling practices

● Exposure to healthcare data standards (X12 EDI, HL7, FHIR)

● Experience with Power BI or other BI/reporting tools consuming the data pipelines you build

● Experience with additional AWS data services: EMR, Kinesis, or Redshift Spectrum

● Experience with modern orchestration tools (e.g., Apache Airflow) as an alternative or complement to SSIS

● Relevant certifications: AWS Certified Data Engineer/Analytics Specialty, Microsoft SQL Server certifications