Software Engineer QA - AI

About the job

Software Engineer QA - AI Are you passionate about ensuring the quality and reliability of AI-driven software that transforms how professionals work? Join a dynamic and highly skilled team at Thomson Reuters, where we invest deeply in AI technologies and explore emerging fields with the backing of a global leader. As a Senior QA Software Engineer - AI, you will play a key role in testing and validating intelligent systems that power the next generation of expert solutions across legal, tax, risk, and compliance domains. About The Role: As a Software Engineer QA - AI, you will collaborate with a cross-functional team of product managers, UX designers, AI engineers, and software developers to design, implement, and execute comprehensive testing strategies for AI-powered solutions. Test AI-Driven Systems: • Design and implement comprehensive test strategies for advanced AI systems, including multi-component pipelines, retrieval-augmented generation (RAG), and custom AI agents with multi-step reasoning. • Develop automated testing frameworks for AI model performance, accuracy, and reliability in production environments. • Create test cases specific to AI/ML systems, including model validation, output quality assessment, and edge case testing. • Validate AI model integrations with production software through robust APIs and scalable data pipelines. • Test AI systems adapted to specialized domains, ensuring quality for expert systems in areas such as legal, tax, and compliance. Innovate in AI Testing: • Evaluate and prototype cutting-edge testing techniques for AI systems to address unique quality challenges. • Develop proof-of-concept automated testing frameworks for new AI-driven features. • Stay current with AI testing methodologies, quality metrics, and emerging technologies. • Implement testing approaches for AI model drift, bias detection, and fairness validation. Provide Technical Leadership: • Break down functional requirements into comprehensive test specifications and automation strategies. • Mentor junior QA engineers and facilitate technical discussions on AI testing best practices. • Contribute to MLOps and LLMOps practices with a focus on quality assurance, continuous testing, and monitoring. • Act as a thought leader in AI quality assurance, sharing expertise in company-wide forums and representing the organization in emerging testing technologies. Ensure Quality & Operations: • Implement comprehensive automated testing frameworks and monitoring systems for AI model performance, accuracy, and reliability. • Create and maintain test automation suites using industry-standard tools and frameworks (Selenium, pytest, etc.). • Ensure compliance with ethical AI principles, security standards, and regulatory requirements through thorough testing. • Conduct systems analysis and recommend operational improvements to testing processes. • Report deviations of quality to development teams and create informative bug reports with effective follow-up. • Create detailed test reports documenting all issues found during testing iterations. Collaborate Across Functions: • Work closely with AI researchers, engineers, designers, and product teams to translate AI quality requirements into effective test strategies. • Participate in daily stand-ups and iteration meetings in an Agile/Scrum environment. • Optimize testing processes considering factors like test coverage, execution time, and resource usage. • Support application feature enhancements by ensuring robust quality assurance for AI capabilities. About You: You are a fit for the position of Software Engineer QA - AI if your background includes: Required Skills and Experience: • Bachelor's degree in Computer Science or equivalent experience. • 3+ years of experience in software quality assurance and test automation; at least 2 years focused on testing AI/ML systems. • 70% automation/development experience, 30% manual testing experience. • Proficiency in Python and Java with strong coding skills for test automation. • Experience with test automation tools and frameworks (e.g., Selenium, pytest, JUnit). • Strong understanding of machine learning principles, evaluation metrics, and AI system testing methodologies. • Knowledge of MLOps/LLMOps and the end-to-end lifecycle of AI-powered software applications. • Experience testing AI model integrations in production systems using APIs and data pipelines. • Good knowledge and experience working with relational databases (SQL, Oracle) and ability to write complex SQL scripts. • Experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization tools (e.g., Docker, Kubernetes). • Hands-on experience with source control and CI/CD systems such as Git and Jenkins. • Good knowledge and experience working with Web Services and RESTful APIs. • Excellent problem-solving skills and ability to work independently in a fast-paced environment. • Strong communication skills and experience working in cross-functional teams. • Ability to work on multiple projects while setting priorities to complete tasks within set timelines. Preferred Qualifications: • Experience testing AI-driven systems, agent-based architectures, or AI APIs from providers like OpenAI and Anthropic. • Knowledge of testing vector databases, embeddings, or search-based AI systems. • Experience with AI/ML frameworks (e.g., PyTorch, TensorFlow) for understanding model behavior. • Familiarity with AI model evaluation techniques, including accuracy metrics, performance benchmarks, and bias testing. • Experience creating effective end-to-end test plans for complex AI systems. • Knowledge of NoSQL databases. • Domain knowledge in legal, tax, or accounting. • A portfolio of projects demonstrating expertise in testing AI/ML solutions and building quality frameworks for LLMs.

Requirements

• software quality assurance
• test automation
• AI/ML systems
• Python
• Java

Qualifications

• Bachelor's degree in Computer Science
• 3+ years of experience

Preferred Technologies

• software quality assurance
• test automation
• AI/ML systems
• Python
• Java

Benefits

• flexible hybrid working environment
• supportive workplace policies

About the company

Thomson Reuters informs the way forward by bringing together the trusted content and technology that people and organizations need to make the right decisions. We serve professionals across legal, tax, accounting, compliance, government, and media.