Sr. Specialist - Data AI Engineering

About the job

Job Description: Key Responsibilities: Data Engineering and Integration: • Design, develop, and optimize scalable data pipelines and ETL processes using Palantir Foundry for data integration, transformation, and creating data models to support analytics and business use cases. • Build and manage APIs and microservices using Python and Java to integrate data across systems and manage data processing and application logic. Data Modeling and Architecture: • Design and implement robust, scalable, and efficient data models in Palantir Foundry. • Collaborate with data architects to define data governance, data lineage, and data quality standards. • Develop and maintain reusable pipelines and templates for data transformation and enrichment. Azure Cloud Expertise: • Utilize Azure Synapse, Azure Data Lake, and Azure Storage for data storage and processing. • Implement secure and efficient data workflows on Azure, ensuring compliance with organizational and regulatory policies. • Monitor and troubleshoot Azure data pipelines for performance optimization. Programming and Automation: • Write clean, maintainable, and efficient code in Python and Java for data processing, automation, and integration tasks. • Develop scripts to automate repetitive tasks and improve overall system efficiency. • Hands on Experience on Snowflake in writing procedures/functions, Snowpipe, Data Pipelines, Data Transformation. Collaboration and Stakeholder Engagement: • Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights. • Provide technical support and training on Palantir Foundry to internal teams. • Participate in Agile development processes and collaborate with DevOps teams to ensure seamless deployment. Performance Monitoring and Optimization: • Monitor data pipelines and applications for reliability, scalability, and performance. • Implement best practices for error handling, logging, and alerting to ensure system stability. Required Skills and Qualifications: • Technical Expertise: • Proficiency in Palantir Foundry for data integration, modeling, and pipeline development. • Strong programming skills in Python and Java. • Hands-on experience with Azure Data Engineering tools (e.g., Azure Data Factory, Databricks, Synapse, Data Lake, etc.). • Solid understanding of data structures, algorithms, and software engineering principles. • Data Engineering Skills: • Experience in building and optimizing ETL/ELT pipelines for large-scale data processing. • Proficiency in SQL for data querying and transformation. • Familiarity with data governance, data lineage, and data security practices. • Cloud Expertise: • Strong knowledge of Azure cloud services and infrastructure for data engineering. • Experience with CI/CD pipelines, containerization (e.g., Docker), and orchestration tools (e.g., Kubernetes). • Problem Solving and Collaboration: • Excellent problem-solving skills and ability to troubleshoot complex issues in data systems. • Strong communication skills to collaborate effectively with technical and non-technical stakeholders. Preferred Qualifications: • Certification in Palantir or Microsoft Azure Data Engineering or related cloud certifications. • Experience in working in Agile/Scrum environments.

Requirements

• Palantir Foundry
• Python
• Java
• Azure

Preferred Technologies

• Palantir Foundry
• Python
• Java
• Azure

About the company

Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.