Data Engineer

About the job

Exp : 6 to 8 years Location : Trivandrum, Kochi, chennai, pune Notice Period : 0 to 15 days Skill : Data Engineering, python, GCP, SQL JD- Key Responsibilities We are seeking a skilled Data Engineer to design, build, and maintain data pipelines and data models that support analytical and business intelligence needs. The ideal candidate will have hands-on experience with Python or SQL, Google Cloud Platform (GCP), and a strong understanding of data management, quality, and security best practices. Key Responsibilities • Build and maintain moderately complex data pipelines, ensuring data flow, transformation, and usability for analytical projects. • Design and implement data models, optimizing for performance and scalability. • Apply knowledge of data characteristics and supply patterns to develop rules and tracking processes that support data quality models. • Prepare data for analytical use by gathering, integrating, cleansing, and structuring data from multiple sources and systems. • Perform design, creation, and interpretation of large and highly complex datasets. • Troubleshoot pipeline and data issues to ensure accuracy and reliability. • Stay up-to-date with GCP advancements and recommend innovative solutions. • Implement security best practices within data pipelines and cloud infrastructure. • Collaborate with global teams to share and adopt best practices in data management, maintenance, reporting, and security. • Develop and execute data quality checks to ensure consistency and integrity. • Work with credit data products and perform analysis using tools like Google BigQuery, BigTable, DataFlow, and Spark/PySpark. Mandatory Skills • Python or SQL Proficiency: Experience with Python or SQL and intermediate scripting for data manipulation and processing. • GCP & Cloud Fundamentals: Intermediate understanding and experience with Google Cloud Platform (GCP) and overall cloud computing concepts. • Data Pipeline Construction: Proven ability to build, maintain, and troubleshoot moderately complex pipelines. • Data Modeling & Optimization: Experience designing and optimizing data models for performance. • Data Quality Governance: Ability to develop rules, tracking processes, and checks to support a data quality model. • Data Preparation & Structuring: Skilled in integrating, consolidating, cleansing, and structuring data for analytical use. • Security Implementation: Knowledge of security best practices in pipelines and cloud infrastructure. • Big Data Analysis Tools: Hands-on experience with Google BigQuery, BigTable, DataFlow, Scala + Spark or PySpark. • Advanced Data Formats: Experience working with JSON, AVRO, and PARQUET formats. • Communication & Best Practices: Strong communication skills to promote global best practices and guide adoption. Preferred Qualifications • Cloud certification (e.g., GCP Data Engineer, AWS, Azure). • Experience with credit data products. • Familiarity with data governance frameworks and metadata management tools. Technical Skills Python | SQL | GCP | BigQuery | BigTable | DataFlow | Spark / PySpark | JSON | AVRO | PARQUET

Requirements

• Data Engineering
• Python
• GCP
• SQL

Qualifications

• Bachelor's degree

Preferred Technologies

• Data Engineering
• Python
• GCP
• SQL