A recruitment partner working with an innovative AI company is seeking a Senior QA Automation Engineer (AI & LLM Systems) for a fully remote, long-term contract role accessible from Abu Dhabi, UAE — or anywhere in the world. This is not traditional software testing. You will validate LLMs, RAG pipelines, AI agents, and predictive ML systems, building automated frameworks that ensure AI products are reliable, scalable, and production-ready across enterprise-scale platforms.
About This AI & LLM QA Opportunity
Product Environment: Production AI systems — LLMs, RAG pipelines, AI agents, and predictive ML models
Remote Flexibility: Fully remote — work from anywhere in the world, including Abu Dhabi, UAE
Contract Type: Long-term contract with exciting greenfield AI projects
Mission: Ensuring AI products are reliable, scalable, and production-ready at enterprise scale
Why This AI QA Role Stands Out
Cutting-Edge AI Stack: Build and test LLM, RAG, and AI agent systems using the latest generative AI technologies
World-Class Team: Work alongside world-class AI engineers, data scientists, and product teams
High-Impact Scope: Enterprise-scale AI platform integrations and greenfield AI product quality engineering
Future of QA: Shape the quality of production AI rather than traditional software — a rare, specialised opportunity
Position Overview
This Senior QA Automation Engineer (AI & LLM Systems) builds automation frameworks for AI, ML, and LLM applications, validates RAG pipelines and AI-generated outputs, develops UI, API, and backend automation using Python, integrates automated testing into CI/CD pipelines, designs regression testing for AI models and prompt changes, tests enterprise integrations with large-scale platforms, and works closely with AI Engineers, Data Scientists, and Product teams to improve product quality throughout the SDLC.
Why This Role Matters: As Senior QA Automation Engineer (AI & LLM Systems), you validate the quality of production AI systems powering enterprise applications, build automation frameworks for LLMs, RAG pipelines, and AI agents from the ground up, work alongside world-class AI researchers and engineers, enjoy full remote flexibility from Abu Dhabi or anywhere globally, and take a high-impact role on greenfield AI projects where quality engineering directly shapes product reliability at scale.
Key Responsibilities
AI & LLM Automation Framework Development
- Build automation frameworks for AI, ML, and LLM applications from the ground up
- Validate RAG pipelines and AI-generated outputs for accuracy, reliability, and production readiness
- Design regression testing frameworks for AI models and prompt changes
Python Development, API & CI/CD Integration
- Develop UI, API, and backend automation using Python (PyTest, Playwright, Selenium)
- Integrate automated testing into CI/CD pipelines including GitLab
- Perform API testing using Requests or similar frameworks
- Test enterprise integrations with large-scale AI platforms
Cross-Functional AI Product Collaboration
- Work closely with AI Engineers, Data Scientists, and Product teams throughout the SDLC
- Improve product quality by embedding QA best practices into AI product development cycles
Qualifications & Requirements
Experience Requirements
- 7+ years of work experience with QA Automation
- 5+ years of work experience with Python (Programming Language)
- 4+ years of work experience with Artificial Intelligence (AI)
- Experience with PyTest, Playwright, and/or Selenium
- API testing using Requests or similar frameworks
- Experience with SQL, CI/CD, and GitLab pipelines
Highly Desirable Skills
- Knowledge of LLM evaluation, RAG, or Generative AI is highly desirable
- Experience testing AI, Machine Learning, or data-driven applications
- Comfortable working in fast-paced product engineering environments
About This Opportunity — The Future of QA is AI
This role is a rare opportunity to step away from traditional software testing and work at the cutting edge of AI quality engineering — validating LLMs, RAG pipelines, AI agents, and predictive ML systems for an innovative AI company. Fully remote and long-term, this contract role offers Abu Dhabi-based or globally distributed engineers the chance to work on enterprise-scale AI platforms alongside world-class AI researchers and engineers, building a specialist career in one of the fastest-growing disciplines in software engineering today.
Career Excellence: Shape the quality of production AI and LLM systems at an innovative AI company — fully remote, long-term, high-impact.
Who Should Apply?
- Senior QA Automation Engineers: With 7+ years of automation experience and strong Python skills
- AI & ML QA Specialists: With 4+ years testing AI, machine learning, or LLM-powered applications
- RAG & LLM Testers: Familiar with LLM evaluation, RAG pipeline validation, and generative AI quality frameworks
- Python & PyTest Practitioners: Comfortable building test automation from scratch in Python-first environments
- Remote-First Engineers: Looking for long-term contract flexibility from Abu Dhabi, UAE, or anywhere globally
Recently Opening Job👇
