Senior Data Engineer

About the job

JOB LEVEL P40 EMPLOYEE ROLE Individual Contributor About the Role Adobe is seeking a highly skilled Senior Data Engineer to join our Doc Cloud Product & Business Analytics organization. This role focuses on building and operating large-scale analytical data pipelines and analytics-ready data tables that power product adoption, usage metrics, experimentation (A/B testing), and executive dashboards across a massive monthly active user base. You will work primarily on Azure Databricks, playing a key role in setting up and evolving our Databricks-based analytics data warehouse, including curated tables, metric foundations, and scalable data models. This role is ideal for a hands-on data engineer with deep expertise in SQL, Python, Spark, and Delta Lake, and a strong understanding of analytics-driven data warehousing and product analytics. Key Responsibilities • Design, build, and maintain large-scale, production-grade data pipelines on Azure Databricks using Apache Spark and Python • Write and optimize complex, high-performance SQL for data transformation, aggregation, and analytics workloads at scale • Develop and maintain analytics-ready data models (fact tables, dimensions, rollups, metric layers, and gold layer tables) used for dashboards and reporting • Optimize Databricks workloads for performance, reliability, and cost efficiency, including tuning Spark jobs and Delta Lake tables • Establish and apply Delta Lake best practices, including incremental and idempotent processing, MERGE patterns, partitioning, and table optimization. • Partner closely with product analysts, business analysts, data scientists, and product managers to enable reliable, self-serve analytics, supporting product-led growth use cases. • Implement data quality checks, validation frameworks, and monitoring to ensure accurate and trusted analytics metrics • Apply strong Data engineering best practices, including version control and documentation • Contribute to solutions that integrate structured and unstructured data, including selective use of GenAI / LLM-based capabilities where relevant Required Qualifications Education • Bachelor’s degree or higher in Computer Science, Engineering, or a related field Experience • 6–12 years of professional experience in Data Engineering • Proven experience building and supporting production-grade analytical data pipelines at scale Technical Expertise (Must-Have) • Expert-level SQL skills, including complex joins, window functions, performance tuning, and large-scale aggregations • Advanced Python proficiency for data processing, pipeline development, and automation • Deep hands-on experience with Azure Databricks and Apache Spark • Strong understanding of Delta Lake and optimization techniques (partitioning, Z-ordering, compaction) • Experience designing data models optimized for analytics and BI consumption Cloud & Platform • Strong experience with Microsoft Azure, including data lake storage and access controls • Familiarity with lakehouse architectures and enterprise data governance concepts Preferred Qualifications • Experience with streaming or near–real-time data pipelines on Databricks • Prior experience supporting product analytics, feature adoption, or MAU-based metrics • Exposure to MLOps, LLM deployment, or GenAI-enabled data applications • Familiarity with BI tools such as Tableau or Power BI and their performance considerations • Experience mentoring analysts or junior data engineers