AI Test Architect (Antioquia)

Why This Role Matters

Caseware is a leading Fintech company with an integral audit and accounting software platform. We are accelerating an AI‑first future by embedding generative AI and autonomous agents to deliver smarter, faster user experiences.

What You’ll Be Doing

1. AI‑Driven Quality Strategy & Architecture
   Architect a comprehensive “Quality Intelligence” platform to predict defect hotspots, optimize regression suites, auto‑generate tests, and enable self‑healing automation.
   Define an enterprise‑wide AI‑first testing strategy, including continuous monitoring for drift, hallucination, and bias.
   Establish ethical governance for AI testing, aligning with emerging standards.
2. LLM & Agent Evaluation Frameworks
   Design advanced benchmarks, red‑team protocols, and adversarial testing for AI agents.
   Build evaluation pipelines using tools such as LangFuse, LangSmith, DeepEval, RAGAS, or Arize Phoenix for faithfulness, context precisi...