W
Sr Machine Learning Engineering Manager - AI Quality and Governance
Workiva
80d ago
0$193k - $308kDataRemote, USjobspy_indeed
remoteindeed
Job Description
Join Workiva as a **Sr Machine Learning Engineering Manager \- AI Quality and Governance** and help establish how we build, evaluate, release, and operate trustworthy AI products at scale. You will lead a multidisciplinary team of software, machine learning, and quality engineers responsible for two connected missions: advancing end\-to\-end quality across Workiva's AI platform and products, and building shared evaluation and governance capabilities that make our AI systems measurable, observable, reliable, and ready for enterprise use.
Your team's scope spans generative AI and agentic products, including AI platform services, agent frameworks and runtimes, conversational experiences, and RAG/knowledge systems. You will partner across Product, Engineering, Data Science, Security, Risk, and Legal to establish practical quality standards and embed evaluation and governance throughout the AI development lifecycle.
**What You'll Do**
------------------
**Leadership \& Team Development**
----------------------------------
* Lead, mentor, and develop a multidisciplinary team of software, ML, and quality engineers
* Build a culture of technical excellence, quality ownership, experimentation, and continuous improvement
* Establish clear team priorities while balancing platform investments, product needs, and enterprise risk
* Recruit engineers with complementary expertise across software quality, ML evaluation, platform engineering, and governance automation
**AI Product Quality**
----------------------
* Define and drive a comprehensive quality strategy for Workiva's AI platform and products, spanning unit, integration, end\-to\-end, performance, resilience, security, and production testing
* Establish measurable quality bars, release\-readiness criteria, and automated quality gates for AI and agentic capabilities
* Advance testing approaches for nondeterministic systems, including RAG pipelines, agents, prompts, models, tools, and multi\-step workflows
* Detect regressions, model or data drift, unsafe behavior, and degraded customer experiences before and after release
**AI Evaluation Platform**
--------------------------
* Lead architecture and delivery of a scalable, self\-service evaluation platform for generative AI, RAG, and agentic systems
* Enable teams to create, manage, version, and reuse evaluation datasets, golden test sets, task\-specific metrics, graders, and benchmarks
* Support deterministic checks, statistical metrics, model\-based graders, human evaluation, adversarial testing, and domain\-expert review
* Build capabilities for offline evaluation, pre\-release regression testing, online experimentation, production sampling, and continuous evaluation
* Ensure evaluation results are reproducible, explainable, actionable, and integrated into developer workflows, CI/CD pipelines, and operational dashboards
**AI Governance \& Assurance**
------------------------------
* Translate Workiva's Responsible AI principles into practica
