AI Test Architect

About the job

Driven by the passion to improve quality of people’s lives, WS Audiology continues to grow as market leader in the hearing aid industry. With our commitment to increase penetration in an underserved hearing care market, we want to accelerate our business transformation in order to reach more people, more effectively. We’re looking for an AI Test Architect to define, implement, and scale the next generation of quality engineering powered by AI. You will own the test architecture strategy end-to-end—combining LLM-driven automation, computer vision, and hardware-in-the-loop (HIL) systems—to deliver robust, scalable, and privacy-first testing for mobile applications and hardware-integrated products. This is a senior, hands-on architecture role: you’ll set the strategy, establish standards, lead technical decision-making, and build reusable platforms and frameworks while mentoring teams across QA, Dev, and DevOps. What You’ll Do • AI Test Strategy & Architecture • Define and maintain the enterprise test architecture, roadmap, and standards spanning functional, non-functional, integration, mobile, and HIL layers. • Drive a shift-left and automation-first culture; architect frameworks that are modular, resilient, and easy to evolve. • Establish test design principles, risk-based testing approaches, and coverage models aligned with business goals and compliance requirements. • Lead architecture reviews and decision forums; evaluate build vs. buy for AI tooling and frameworks. • AI-Driven Test Innovation (LLMs, RAG, CV/OCR) • Architect and implement RAG-based test generation using local LLMs (e.g., Ollama, llama.cpp) and frameworks like LangChain/LlamaIndex to reason over requirements, app states, and logs. • Build AI agents that can: interpret acceptance criteria, propose and prioritize test scenarios, and auto-generate test cases/scripts. • Develop computer-vision and OCR pipelines (OpenCV, Tesseract, or equivalent) for precise UI validation, visual diffing, and visual regression analysis. • Design models for UI anomaly detection, flakiness prediction, and self-healing locators beyond traditional Appium-style selectors. • Automate localization testing for text, layout, and formatting across languages and screen sizes using AI. • Hardware-in-the-Loop (HIL) & Mobile Systems • Define and evolve a Python-based HIL framework for end-to-end validation of Mobile/Desktop/WebApp interacting with hardware (e.g., medical devices, sensors, wearables). • Architect communication interfaces (USB, Bluetooth, Serial) and test harnesses to control/observe device interactions reliably at scale. • Incorporate AI-driven adapters that learn device behaviors, detect drift, and improve robustness of HIL scenarios over time. • Partner with mobile teams (Android/iOS) to integrate Appium/Espresso/XCUITest/FlaUI where appropriate and augment with AI components. • On-Prem Infrastructure & MLOps for Test at Scale • Design on-prem/private-cloud inference infrastructure for low-latency, high-throughput model execution with strong data privacy guarantees. • Containerize models and agents (Docker) and integrate into CI/CD (Jenkins, GitLab CI) for parallel execution and test-on-commit workflows. • Implement continuous learning loops to leverage test failures, telemetry, and labels to retrain and improve models. • Establish model lifecycle practices (versioning, evaluation, rollback, governance) using tools like MLflow/Langchain, self-hosted vector stores, and caching. • Automation Platforms & Tooling • Extend or replace traditional frameworks (e.g., Appium) with AI-assisted components that enhance stability and coverage. • Build Python-based toolchains for model training/inference and integrations into test workflows; standardize reusable libraries and templates. • Define reference architectures for UI, API, performance, security, and resilience testing; ensure seamless observability, logging, and triage workflows. • Quality Governance, Metrics & Risk • Define quality gates, entry/exit criteria, and release readiness bars; ensure compliance and privacy-by-design. • Track and publish leading indicators and outcome metrics—defect escape rate, test yield, visual regression detection rate, flakiness, mean-time-to-detect/triage, inference latency, infra utilization. • Conduct root cause analyses and drive systemic fixes across tooling, test design, and pipelines. • Audit processes for adherence to standards; champion continuous improvement. • Technical Leadership & Enablement • Mentor QA engineers and SDETs on AI/ML testing, HIL, and architectural best practices. • Facilitate collaboration across QA, Development, Data/ML, Security, and DevOps. • Curate internal playbooks, patterns, sample repos, and training content to scale adoption. What You Bring Must-Have • 10+ years in Quality Engineering / Software Development, with 5+ years in test architecture / QA leadership. • Strong Python expertise and hands-on experience with AI/ML in testing contexts. • Proven experience training or fine-tuning models using PyTorch, TensorFlow, or Hugging Face. • Solid background in Computer Vision (OpenCV) and OCR for UI/visual validation. • Track record building automation frameworks for mobile and/or device-integrated systems, and clear understanding of limitations of traditional tools (e.g., Appium). • Experience with CI/CD, Docker, and deploying AI models into test or production environments. • Demonstrated ability to define strategy, set standards, and lead cross-functional technical initiatives. Highly Desirable • Hands-on with local LLMs (Ollama, llama.cpp) and RAG using LangChain/LlamaIndex. • Prior experience with Hardware-in-the-Loop (HIL) or Software-in-the-Loop (SIL) testing. • Familiarity with Android/iOS ecosystems and tooling (ADB, Xcode, Fastlane, Espresso, XCUITest, Detox). • Observability and analytics (e.g., Prometheus/Grafana, OpenTelemetry) for test health and triage. • Experience in regulated environments (e.g., medical devices) and familiarity with risk management and compliance (e.g., ISO 13485, IEC 62304, IEC 62366, ISO 14971, HIPAA/GDPR) is a plus. Soft Skills • Strategic thinker with a builder’s mindset and strong systems design instincts. • Excellent communicator who simplifies complex concepts and aligns stakeholders. • Strong ownership, bias for action, and comfort navigating ambiguity. • Passion for mentoring and uplifting engineering excellence across teams. Success Metrics (Examples) • >40% reduction in flaky test failures and >30% faster triage through AI triage/visual diffing. • >25% increase in critical defect detection pre-release (vs. prior baseline). • P95 AI inference latency within target SLA for CI pipelines. • Adoption of standardized AI-assisted frameworks across X product teams within Y quarters. • Documented compliance with privacy and model governance standards. Tech Stack (Indicative) • Languages/Frameworks: Python, PyTorch/TensorFlow, Hugging Face, OpenCV, Tesseract OCR, LangChain/LlamaIndex • LLMs/Inference: Ollama, llama.cpp, local vector DBs (FAISS/Chroma), Triton Inference Server (optional) • Mobile & Automation: Appium, Espresso, XCUITest, FlaUI, and Azure DevOps • HIL/Protocols: USB, Bluetooth, Serial; vendor SDKs; Python-based device controllers • CI/CD & Infra: Azure DevOps, self hosted Ubuntu / MacOS instances • Observability: Prometheus, Grafana, OpenTelemetry, structured logging Personal competencies • Strong leadership and coordination skills. • Excellent communication skills in professional written and spoken English. • Detail-oriented with a structured and analytical approach. • Ability to work collaboratively across global teams. • Proactive mindset with a focus on quality and compliance. Who we are At WS Audiology, we provide innovative hearing aids and hearing health services. Together with our 12,000 colleagues in 130 countries, we invite you to help unlock human potential by bringing back hearing for millions of people around the world. With us, you will become part of a truly global company where we care for one another, welcome diversity and celebrate our successes. Sounds wonderful? We can't wait to hear from you. WS Audiology is an equal-opportunity employer and committed to creating an inclusive employee experience for all. Regardless of race, color, religion, national origin, age, sex, gender, gender identity, gender expression, sexual orientation, marital status, medical condition, ancestry, disability, military or veteran status we firmly believe that our work is at its best when everyone feels free to be their most authentic self.

Requirements

• Quality Engineering
• AI Test Architecture
• Test automation
• Python
• Computer Vision

Preferred Technologies

• Quality Engineering
• AI Test Architecture
• Test automation
• Python
• Computer Vision

About the company

Driven by the passion to improve quality of people’s lives, WS Audiology continues to grow as market leader in the hearing aid industry. With our commitment to increase penetration in an underserved hearing care market, we want to accelerate our business transformation in order to reach more people, more effectively.