At Mastech Digital, we solve meaningful business problems using data, AI, and modern digital technologies. Our teams work closely with global enterprises to build solutions that create real, measurable impact. We bring deep industry expertise across Utilities & Energy, Financial Services, Healthcare, Retail, and Technology. With an AI-first mindset and a collaborative culture, we empower our people to innovate, grow, and help clients move forward with confidence.

Role: QA Engineer – AI Applications

Location: Bangalore/Chennai, India

Employment Type: Full-Time

Years of Experience: 6-10 years

Education/Qualification: Bachelor’s degree in a related field

Submit Resume To: recruitment.helpdesk@mastechdigital.com

Role Description: The QA Engineer – AI Applications must have 6-10 years of experience. As an AI Engineer, you will design, build, and scale intelligent AI-driven systems as part of an enterprise-grade agentic platform. This role focuses on developing LLM-powered applications, agentic workflows, and evaluation frameworks, ensuring production-ready, scalable AI solutions.

You will be part of a POD-based delivery model working across AI workflows, search systems, embeddings, evaluation stacks, and backend services, contributing to both platform capabilities and business-facing AI solutions.

This role combines hands-on AI engineering, system design, and delivery execution, with a strong emphasis on building reliable, observable, and high-performance AI systems.

In this role, you’re expected to -

- Demonstrate strong ownership of AI solution delivery within POD-based teams

- Design and build LLM-powered applications and agentic workflows

- Work on search, retrieval, embeddings, and evaluation pipelines

- Contribute to end-to-end delivery: design → build → deploy → operate → optimize

- Collaborate across platform, product, data, and DevOps teams

- Ensure quality, scalability, and performance of AI systems in production

- Continuously improve prompt design, evaluation coverage, and model performance

Responsibilities:

- Develop and deploy AI agents, RAG pipelines, and orchestration workflows

- Build and optimize retrieval systems (search relevance, embeddings, ontology expansion)

- Implement evaluation frameworks, test harnesses, and LLM assurance pipelines

- Design and fine-tune prompts, policies, and agent behaviors

- Develop backend services supporting AI workflows using Python-based frameworks

- Integrate AI systems with enterprise APIs, data pipelines, and business applications

- Work on multi-model orchestration and routing strategies

- Improve latency, reliability, and scalability of AI systems

- Implement observability, logging, and monitoring for AI workflows

- Collaborate with QA/Eval engineers to ensure robust validation of AI outputs

- Participate in architecture discussions and solution design reviews

- Support A/B testing, experimentation, and continuous improvement of AI features

- Ensure adherence to security, governance, and compliance practices in AI systems

Qualifications:

Experience

- 6–10 years in software/AI engineering

- Hands-on experience building AI/ML or LLM-based applications in production

AI Engineering

- Experience with LLM frameworks (LangChain, LangGraph, AutoGen, DSPy, etc.)

- Strong knowledge of RAG pipelines, embeddings, and vector search

- Experience in prompt engineering and model tuning

- Understanding of evaluation strategies, golden datasets, and benchmarking

Backend & Platform

- Strong Python experience (FastAPI, Flask)

- Experience building microservices and distributed systems

- Familiarity with API design and integration patterns

Data & Systems

- Experience with vector databases, relational DBs, caching layers (Redis)

- Understanding of data pipelines and streaming (Kafka or similar)

Cloud & DevOps

- Experience with AWS / Azure / GCP

- Familiarity with Docker, Kubernetes, CI/CD pipelines

- Exposure to observability tools and production monitoring

Additional Capabilities:

- Strong problem-solving and debugging skills

- Ability to work in agile POD-based delivery models

- Good collaboration with cross-functional teams

Mastech Digital is an Equal Opportunity Employer - All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or disability.

Telecom Communications Privacy Notice: By submitting for this position you will receive a text message from us and you agree and authorize us to send you text messages; you can reply STOP at any time to opt-out of text messaging from us.