OneDigital Health Logo

OneDigital Health

AI Quality Assurance Analyst - Remote

Posted 2 Days Ago
Remote
Hiring Remotely in USA
Mid level
Remote
Hiring Remotely in USA
Mid level
Test and validate AI-powered applications for output quality and functional behavior. Design and run test plans, evaluate LLM responses for accuracy and hallucination, document defects in issue trackers, maintain repeatable test cases, perform regression testing, and collaborate with engineers and product managers to define AI quality standards across internal and client-facing tools.
The summary above was generated by AI

No other company in our industry is supercharging the way they work and serve their clients like OneDigital. Fresh thinking has always been the core of OneDigital’s vision and growth strategy. It’s how we stand out in our industry, it’s how we stay competitive and resilient in a changing world. Most importantly, our innovative approach is helping more people do their best work and live their best lives. Innovation fuels our employee experience by making it easier to do your best work anytime, anywhere and from any device. And our tech-based products for clients are a game changer in our industry. If you thrive on change and innovation, OneDigital is the career choice for you.

Our Newest Opportunity: 

OneDigital | Product Team
No other company in our industry is supercharging the way they work and serve their clients like OneDigital. Fresh thinking has always been the core of OneDigital’s vision and growth strategy. It’s how we stand out in our industry, it’s how we stay competitive and resilient in a changing world. Most importantly, our innovative approach is helping more people do their best work and live their best lives. Innovation fuels our employee experience by making it easier to do your best work anytime, anywhere and from any device. And our tech-based products for clients are a game changer in our industry. If you thrive on change and innovation, OneDigital is the career choice for you.

Our Newest Opportunity
OneDigital is seeking a detail-oriented and curious AI Quality Assurance Analyst to join our embedded product team. This role is responsible for ensuring that the AI-powered applications built by our product team are reliable, accurate, and ready for real-world use across OneDigital’s business lines.

As AI becomes core to how we work and serve clients, quality assurance takes on a new dimension. This is not just about catching bugs. It is about evaluating how AI behaves: whether it responds accurately, handles edge cases gracefully, and delivers a consistent experience for users. You will partner closely with our Principal Engineer, AI Builders, and Product Managers to test both the AI output layer and the application functionality built around it. This is a hands-on individual contributor role for someone who is methodical, curious, and excited about the frontier of AI product development.

Essential Duties and Responsibilities

  • Design and execute test plans for AI-powered applications across both output quality and functional behavior

  • Evaluate AI responses for accuracy, consistency, hallucination, and edge case handling

  • Test application workflows, UI behavior, and integrations across internal and client-facing tools

  • Identify, document, and track defects using standard issue tracking tools (e.g., Jira or equivalent)

  • Partner with AI Builders and Product Managers to define acceptance criteria and quality standards for AI features

  • Develop and maintain repeatable test cases and QA documentation as AI applications evolve

  • Conduct regression testing as applications are updated or new features are released

  • Collaborate with the Principal Engineer and product team to surface quality risks early in the build process

  • Contribute to defining what “good” looks like for AI output quality at OneDigital, helping shape internal QA standards for LLM-based applications

  • Support testing across multiple business lines including Employee Benefits, Retirement, Wealth, and P&C

What You Will Work On

  • AI-powered tools built for internal productivity and workflow automation

  • Client-facing AI applications across OneDigital’s business lines

  • Applications built on modern LLM APIs, including prompt-driven and agent-based workflows

  • A growing library of AI products created by an embedded team of Product Managers and AI Builders

Qualifications, Skills and Requirement

  • Experience in a QA, testing, or quality-focused role, with some exposure to AI or LLM based products preferred

  • Strong ability to evaluate AI-generated outputs for quality, accuracy, and appropriateness

  • Familiarity with prompt-driven systems and an intuition for how language models can fail or produce unexpected results

  • Experience writing and executing test cases and documenting defects clearly

  • Comfort working in no-code and low-code environments

  • Strong written communication skills, with the ability to describe issues precisely and concisely

  • Curiosity and adaptability in a fast-moving, emerging technology space

  • Ability to work independently and manage multiple testing workstreams at once

Education, Training and Experience

  • Bachelor’s degree or equivalent experience

  • Prior experience in a QA role within a software, SaaS, or technology-adjacent environment

  • Exposure to AI or LLM-based tools, either professionally or through independent exploration

  • Familiarity with issue tracking tools (e.g., Jira, Linear, or similar)

  • Understanding of AI governance, data privacy, and responsible AI considerations

The typical base pay range for this role nationwide is $100,000 to $110,000 per year 

Your base pay is dependent upon your skills, education, qualifications, professional experience, and location. In addition to base pay, some roles are eligible for variable compensation, commission, and/or annual bonus based on your individual performance and/or the company’s performance.  We also offer eligible employees health, wellbeing, retirement, and other financial benefits, paid time off, overtime pay for non-exempt employees, and robust learning and development programs. You will receive reimbursement of job-related expenses per the company policy and may receive employee perks and discounts. 

To learn more, visit: www.onedigital.com/careers  

OneDigital is an equal opportunity employer. Not only as a matter of standard, but to honor and celebrate our differences. We believe that the power of ONE starts with you. We are committed to cultivating and preserving a culture that celebrates diversity, insists on equity and inclusion, and connects us. Ensuring our people feel seen, valued, respected, and supported is fundamental to our core values and business goals.

OneDigital provides equal employment opportunities to all employees and applicants for employment regardless of their: veteran status, uniformed servicemember status, race, color, religion, sex, sexual orientation, gender identity, age (40 and over), pregnancy (including childbirth, lactation and related medical conditions), national origin or ancestry, citizenship or immigration status, physical or mental disability, genetic information (including testing and characteristics) or any other category protected by federal, state or local law (collectively, “protected characteristics”). A copy of the Federal EEO poster is linked here.

Pursuant to local Fair Chance Ordinances, we will consider qualified applications with arrest or conviction records for employment. For applicable candidates, the following ordinances are linked here to inform you of your rights as an applicant:

City and County of San Francsico

City of Los Angeles

County of Los Angeles

Employment decisions shall comply with all other applicable federal, state and city/county laws prohibiting discrimination in employment. OneDigital complies with all criminal history inquiry [or ‘ban the box’] laws in California, Connecticut, Colorado, Hawaii, Illinois, Maine, Maryland, Massachusetts, Minnesota, New Jersey, New Mexico, Oregon, Rhode Island, Vermont and Washington.

In short, we believe in hiring the most qualified applicant for the position, regardless of background.

If you have questions about our hiring policies and practices, we would be happy to discuss upon receiving your application. We hope to welcome you to OneDigital and look forward to hearing from you.

 

OneDigital understands the immense responsibility and opportunities provided by Artificial Intelligence. We utilize advanced Artificial Intelligence [AI] technologies to enhance our recruitment process. This includes using AI to filter candidates based on their qualifications and to rediscover potential candidates from our existing applicant pool. Our AI systems help us efficiently identify the best fit for our open positions, ensuring a streamlined and effective hiring experience. However, AI does not replace the humans in our process. If you have concerns about our use of AI, you may opt out where laws allow.

Thank you for your interest in joining the OneDigital team!

Similar Jobs

2 Hours Ago
Remote or Hybrid
Mid level
Mid level
Machine Learning • Payments • Security • Software • Financial Services
The Incident & Problem Consultant manages high-impact IT incidents, coordinates response teams, monitors events, and drives issue resolution, ensuring effective communication throughout.
Top Skills: ItilMicrosoft Teams
2 Hours Ago
Remote or Hybrid
Mid level
Mid level
Machine Learning • Payments • Security • Software • Financial Services
Provide high-volume, phone-based technical support and real-time troubleshooting for hardware, software, network, and access issues. Create and manage incidents in ServiceNow or similar ITSM tools, escalate complex problems, collaborate with internal teams, and recommend process improvements while maintaining security, data privacy, and excellent customer service in a remote workspace.
Top Skills: Call Center TechnologiesHardwareHelp DeskItsmNetworkingRemote Support ToolsServicenowSoftwareTicketing Systems
4 Hours Ago
Remote or Hybrid
Senior level
Senior level
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Lead product marketing for ServiceNow's telecommunications portfolio: craft messaging, drive releases and sales enablement, support deals with positioning and competitive insight, and embed AI-first workflows for scalable content and research.
Top Skills: Ai ToolsBssOssServicenow

What you need to know about the Calgary Tech Scene

Employees can spend up to one-third of their life at work, so choosing the right company is crucial, not just for the job itself but for the company culture as well. While startups often offer dynamic culture and growth opportunities, large corporations provide benefits like career development and networking, especially appealing to recent graduates. Fortunately, Calgary stands out as a hub for both, recognized as one of Startup Genome's Top 100 Emerging Ecosystems, while also playing host to a number of multinational enterprises. In Calgary, job seekers can find a wide range of opportunities.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account