Data Engineer Manager

About the Role

The Data Engineering Manager will lead a high‑performing team responsible for designing, building, and operating Associa’s enterprise data platform. This role combines hands‑on technical leadership with people management, Agile delivery, and architectural oversight. The manager will ensure reliable data products that empower analytics and data science throughout the organization. Technical duties include creating scalable ETL/ELT pipelines using Python and SQL, and architecting batch, streaming, and hybrid ingestion patterns. The manager will develop solutions on Databricks—leveraging Spark, Delta Lake, Unity Catalog, DLT, and Workflows—while applying medallion (Bronze/Silver/Gold) architecture for data transformation and governance. They will also define data‑quality standards, monitoring, alerting, and maintain platform diagrams and extensibility guidance. Leadership responsibilities involve partnering with a Product Owner to shape the transformation backlog, co‑hosting sprint ceremonies, and guiding the team through solutioning and estimation. The manager will coordinate code merges, testing, and releases with DevOps, oversee deployments from development to production, and track scrum metrics and risk escalations to the VP of Engineering. Additional duties include hiring, onboarding, performance reviews, mentoring, and establishing sandbox and data‑staging frameworks for real‑world scenarios. • Design, build, and maintain scalable ETL/ELT pipelines using Python, SQL, and Databricks (Spark, Delta Lake, Unity Catalog, DLT, Workflows). • Architect batch, streaming, and hybrid ingestion patterns and implement medallion (Bronze/Silver/Gold) data models. • Establish data‑quality standards, monitoring, alerting, and governance across the platform. • Collaborate with data scientists, analysts, and business stakeholders to deliver reliable data products. • Lead Agile sprint ceremonies, manage the transformation backlog, and provide technical estimates and demos. • Coordinate code merges, testing, release management, and production deployments with DevOps. • Oversee hiring, onboarding, performance reviews, mentorship, and resolution of personnel issues. • Define and educate teams on client sandbox architecture and data‑staging frameworks for real‑world use cases.

Key Responsibilities

• ▸etl pipelines
• ▸data modeling
• ▸data quality
• ▸agile delivery
• ▸devops release
• ▸team management

What You Bring

Candidates must have at least five years of data‑engineering experience and two years in a formal management role, with proven success delivering production‑grade pipelines at scale. Required technical skills include deep expertise in Databricks, Python (including PySpark and pandas), advanced SQL, data ingestion technologies (Auto Loader, CDC, streaming), lakehouse architecture, and CI/CD practices. Strong communication, risk‑management, and empathetic people‑leadership skills are also essential. Preferred qualifications include Databricks certifications, experience with AWS or Azure services, familiarity with orchestration tools such as Airflow or Prefect, and knowledge of real‑time streaming platforms like Kafka or Kinesis. Candidates with two or more years of engineering‑team leadership and Agile/Scrum certifications (CSM, SAFe) are highly valued. • Required: 5+ years data‑engineering experience, 2+ years management; expertise in Databricks, Python (PySpark, pandas), SQL, data ingestion (Auto Loader, CDC), lakehouse architecture, CI/CD. • Preferred: Databricks certification, AWS/Azure experience, Airflow/Prefect, Kafka/Kinesis, Agile/Scrum certification.

Requirements

• ▸databricks
• ▸python
• ▸sql
• ▸ci/cd
• ▸aws/azure
• ▸agile

Work Environment

Hybrid