Senior AI QA Test Engineer

About the job

We are seeking a Senior AI - QA Test Engineer with extensive experience in AI / ML Quality Engineering, specializing in Agentic GenAI systems, evaluation frameworks, observability, and reliability testing. You will ensure the quality and consistent performance of our enterprise-scale AI ecosystem across Snowflake, AWS, and Azure. Responsibilities • Design and implement QA strategies for LLM, RAG, and Agentic AI pipelines. • Develop automated test suites for agentic workflows, LLM applications, APIs, and vector-search systems. • Validate retrieval workflows, reasoning behavior, grounding quality, and reduce hallucination tendencies. • Build evaluation pipelines with robust metrics and regression frameworks (covering grounding, hallucination risks, and retrieval evaluation). • Implement AI observability: telemetry, drift monitoring, quality dashboards. • Integrate automated testing into CI/CD workflows. • Establish enterprise QA best practices for AI governance and reliability.

Requirements

• QA/Quality Engineering
• Python
• CI/CD
• AI/ML systems

Qualifications

• 5+ years of experience in QA/Quality Engineering
• 2+ years testing AI systems

Preferred Technologies

• QA/Quality Engineering
• Python
• CI/CD
• AI/ML systems