Job Description

A bit about this role:

The Quality Program Manager for UM Operations is a high-impact, hands-on leader responsible for driving clinical quality, regulatory compliance, and operational efficiency across the Utilization Management (UM) department.

This role combines advanced UM expertise, AI/LLM integration, and data-driven decision-making to transform clinical review, audit, and workflow operations. The Program Manager will oversee quality programs, lead AI-powered audit initiatives, and ensure all UM processes meet Regulatory and CMS Regulations.

This is a fast-paced, high-change environment requiring someone who thrives on operational challenges, can manage multiple priorities simultaneously, and is comfortable leading innovation at the intersection of clinical operations and technology

Your Responsibilities and Impact will include:

AI & LLM Program Integration

• Co-lead design, testing, and implementation of LLM-based tools supporting clinical summarization, level-of-care determinations, and documentation accuracy.

• Translate clinical workflows into AI-friendly use cases and prompts, ensuring human-in-the-loop review for all high-risk outputs.

• Track AI performance metrics (accuracy, turnaround time, variance reduction) and escalate potential compliance or bias risks.

• Partner with Data Science and Engineering to refine prompt logic, accuracy thresholds, and feedback loops for continuous improvement.

• Develop training resources and calibration exercises for clinical staff to effectively use AI/LLM tools in daily operations.

Quality Oversight & Audit Governance:

• Identify opportunities for automation within audit workflows and lead initiatives to scale AI- and LLM-assisted reviews, ensuring consistency, accuracy, and regulatory adherence.

• Lead the UM quality and audit program to ensure decision accuracy, timeliness, and full compliance with Regulatory Guidelines and CMS Regulations.

• Oversee both internal and delegated entity audits, including pre-authorization, concurrent, and retrospective chart reviews.

• Analyze audit trends and develop proactive, data-driven action plans to address operational inefficiencies, compliance gaps, and clinical inconsistencies.

• Drive operational transparency by developing dashboards, Balanced Scorecards, and executive-level reports that quantify quality, compliance, and efficiency outcomes.

• Monitor and respond to high-risk issues, demonstrating strong judgment and agility in a fast-moving clinical and regulatory environment.

Operational Excellence & Data Analytics:

• Monitor and analyze operational performance metrics to identify inefficiencies and opportunities for improvement across UM programs.

• Implement dashboards and reporting mechanisms to track audit accuracy, decision quality, workflow efficiency, and AI impact.

• Lead cross-functional initiatives to streamline UM workflows, reduce administrative burden, and increase productivity across internal and delegated teams.

• Proactively identify and resolve barriers to adoption of AI-assisted workflows, ensuring scalability and sustainability.

Compliance & Risk Management:

• Ensure all AI-assisted and manual UM activities comply with CMS Regulations, Regulatory Guidelines, and internal policies.

• Investigate audit discrepancies and AI-identified anomalies, taking corrective action and mitigating compliance or clinical risk.

• Document governance and escalation processes for AI outputs and UM operational decisions.

How You’ll Get It Done:

• Lead with influence and innovation: Champion AI adoption while ensuring clinical accuracy and regulatory compliance.

• Operate in a high-change, fast-paced environment: Quickly adapt to new regulations, operational priorities, and technology tools.

• Use data to drive decisions: Leverage dashboards, reports, and audit data to inform operational improvements and automation strategies.

• Collaborate cross-functionally: Build strong partnerships with Clinical, Compliance, Data Science, Engineering, and vendor teams to achieve operational goals.

• Implement AI thoughtfully: Translate complex clinical and operational workflows into structured AI tasks with clear validation and governance.

• Maintain operational discipline: Ensure all audit and review processes are standardized, documented, and continuously improved.

• Educate and influence teams: Train, guide, and support clinicians and auditors to adopt AI-enabled workflows effectively and responsibly.

Required skills and experience:

• Active RN or LPN license with 5+ years of UM experience (inpatient, outpatient, post-acute, or appeals).

• 3+ years of experience in UM quality, audit, or program management.

• In-depth knowledge of UM processes, medical necessity criteria, Regulatory Guidelines, and CMS Regulations.

• Demonstrated ability to implement AI or automation tools within healthcare operations.

• Strong analytical, communication, and project management skills.

Desired skills and experience:

• Experience designing or managing LLM/AI-powered workflows in a healthcare environment.

• Familiarity with prompt engineering, AI validation, and accuracy monitoring.

• Proficiency with data visualization and operational tracking tools ( Looker, Google Workspace, Monday.com, Jira).

#LI-DS1
#LI-Remote

Salary Range: $80,000-$120,000 / year

Our ranges are purposefully broad to allow for growth within the role over time. Once the interview process begins, your talent partner will provide additional information on the compensation for the role, along with additional information on our total rewards package. The actual base salary offered may depend on a variety of factors, including the qualifications of the individual applicant for the position, years of relevant experience, specific and unique skills, level of education attained, certifications or other professional licenses held, and the location in which the applicant lives and/or from which they will be performing the job.

Our Total Rewards package includes:

• Employer sponsored health, dental and vision plan with low or no premium

• Generous paid time off

• $100 monthly mobile or internet stipend

• Stock options for all employees

• Bonus eligibility for all roles excluding Director and above; Commission eligibility for Sales roles

• Parental leave program

• 401K program

• And more....

*Our total rewards package is for full time employees only. Intern and Contract positions are not eligible.

Healthcare equality is at the center of Devoted’s mission to treat our members like family. We are committed to a diverse and vibrant workforce.

At Devoted Health, we’re on a mission to dramatically improve the health and well-being of older Americans by caring for every person like family. That’s why we’re gathering smart, diverse, and big-hearted people to create a new kind of all-in-one healthcare company — one that combines compassion, health insurance, clinical care, service, and technology - to deliver a complete and integrated healthcare solution that delivers high quality care that everyone would want for someone they love. Founded in 2017, we've grown fast and now serve members across the United States. And we've just started. So join us on this mission!

Devoted is an equal opportunity employer. We are committed to a safe and supportive work environment in which all employees have the opportunity to participate and contribute to the success of the business. We value diversity and collaboration. Individuals are respected for their skills, experience, and unique perspectives. This commitment is embodied in Devoted’s Code of Conduct, our company values and the way we do business.

As an Equal Opportunity Employer, the Company does not discriminate on the basis of race, color, religion, sex, pregnancy status, marital status, national origin, disability, age, sexual orientation, veteran status, genetic information, gender identity, gender expression, or any other factor prohibited by law. Our management team is dedicated to this policy with respect to recruitment, hiring, placement, promotion, transfer, training, compensation, benefits, employee activities and general treatment during employment.