Generative AI Scientist - (Model Risk Validation)

Role overview

Responsibilities

• check_circle As a Generative AI Scientist within Cotiviti you will be responsible for delivering solutions that help our clients identify payment integrity issues, reduce the cost of healthcare processes, or improve the quality of healthcare outcomes. You will work as part of a team and will be individually responsible for the delivery of value associated with your projects.
• check_circle Conduct independent model validation of existing models for benchmarking, assessment, and gauging effectiveness
• check_circle Determine aspects of model drift and related data drift for the purpose of model risk management (MRM) to both reduce risk and also find opportunities to drive new revenue growth and innovation
• check_circle Apply deep expertise with AI/ML/GenAI model development, including hands-on experience with model building and model evaluation
• check_circle Benchmark and potentially rebuild existing models as needed using updated data, and potentially newer, more modern and effective algorithms
• check_circle Actively drive improvements in model monitoring activities, including methods for model registration, model metadata management, and conceptualizing approaches for related tools and techniques
• check_circle Complete all responsibilities as outlined in the annual performance review and/or goal setting.
• check_circle Complete all special projects and other duties as assigned.
• check_circle Must be able to perform duties with or without reasonable accommodation.
• check_circle Graduate Degree in a quantitative discipline such as Computer Science/Engineering, Statistics, Operations Research covering Advanced Statistics, Machine learning and AI.
• check_circle Experience with the latest techniques in natural language processing including transformers, fine-tuning LLMs, measuring/benchmarking and deploying LLMs with tools such as HuggingFace, Langchain, LLAMA/Mistral and OpenAI, vector databases.
• check_circle 1+ years of hands-on data science/AI experience, using typical machine learning and data science tools including pandas, scikit-learn, keras, nltk, and TensorFlow/PyTorch, GPU.\
• check_circle Experience building production-grade machine learning deployments on AWS, Azure, or GCP.
• check_circle Experience working with Apache Spark™ and large-scale distributed datasets.
• check_circle Experience communicating technical concepts to non-technical and technical audiences is a plus.
• check_circle Passion for collaboration, learn it all mindset and driving value with AI.

Preferred qualifications

• Familiarity with healthcare payor ecosystem and related data.
• General understanding of Responsible AI (RAI), including explainability (XAI), AI NIST RMF, and related AI risk management frameworks.
• Experience and understanding evaluating models for bias and fairness, with aptitude for detecting bias in the model design and data, as well as using metrics such as SHAP and LIME.
• Understanding appropriate model metrics and techniques for managing, evaluating and monitoring GenAI models and LLMs
• Understanding and familiarity with model governance and data governance best practices.
• Strong understanding of technology systems for model development (e.g., Python, DataRobot, AWS Sagemaker), model deployments (AWS, Azure, DataRobot, DataBricks), model monitoring (AWS Model Monitor, MLFlow, NannyML, FiddlerAI, Arize) and related tools for model management and metadata management.
• Ability to work independently as well as collaborate as a team with a sense of urgency.
• Professional with ability to properly handle confidential information.
• Be value-driven, understand that success is based on the impact of your work rather than its complexity or the level of effort.
• Ability to handle multiple tasks, prioritize and meet deadlines.
• Ability to work within a matrixed organization.
• Proficiency in all required skills and competencies above.
• Communicating with others and teamwork.
• Assessing the accuracy, neatness, and thoroughness of the work assigned.
• Flexibility to work with global teams as well geographically dispersed US based teams.
• Remaining in a stationary position, often standing or sitting for prolonged periods.
• Repeating motions that may include the wrists, hands and/or fingers.
• Must be able to provide high-speed internet access/connectivity and office setup and maintenance.
• Must be able to provide a dedicated, secure work area.