Standardizing COA Administration with AI Technology: Ensuring Consistency and Accuracy in Clinical End Points

In the life sciences, the outcome of a clinical trial often hinges on the performance of the clinical rater (i.e., independent assessor), who is responsible for evaluating patient symptoms through the administration of a clinical outcome assessment (COA), such as the Montgomery–Åsberg Depression Rating Scale. Raters play a central role in determining the efficacy and safety of a clinical trial by accurately capturing and interpreting patient-reported experiences and behavioral observations, ultimately contributing critical data that influences trial success and regulatory decision-making. A significant challenge with commonly used COAs in mental health is that many are semi-structured, resulting in significant variability in how they are administered and scored. To date there are no standardized requirements for training or clinical experience among assessment raters which further complicates standard practice in administration.

While regulators and sponsors work to understand why nearly half of trials involving approved antidepressants fail to show drug efficacy over placebo (see Kahn et al., 2002; Katz et al., 2002), limited attention has been given to COA raters whose clinical assessments ultimately drive trial outcomes. In response to this challenge, mpathic, a science-backed conversational intelligence company specializing in the life sciences and healthcare, has developed an AI tool that provides standardized just-in-time 100% quality oversight of clinical trials, including standardization and scoring accuracy of COAs.

In this Phase I SBIR project, mpathic will leverage pre-SBIR research efforts by (Aim 1) expanding the gathering of a robust, clinically validated training dataset (n = 500) for COA model development. The primary dataset will specifically focus on real-world assessments of the MADRS, HAM-D, SCID and C-SSRS. Further, this real-world data will contain critical metadata and varied symptom presentation across depression and suicidality. Next (Aim 2), mpathic will develop AI models that target COA administration, inclusive of scoring accuracy on the MADRS, HAM-D, SCID and C-SSRS. Discrepancy analysis will also be performed at this stage to ensure consistency in facilitator behavior and scoring across heterogeneous populations (e.g., age, geographic region). Upon development of acceptable AI models (f = .70; IRR > .67); finally, (Aim 3) mpathic will benchmark COA models on Transcend Therapeutic's archived COA data from two PTSD trials, specifically IMPACT 1 (NCT05741710; 64 patients) and IMPACT 2 (NCT06215261; 79 patients). At this stage, we will further refine our COA models to Transcend Therapeutic’s specific use case, determining end user functionalities for trial sponsors and clinical research organizations ahead of widescale commercialization.
mpathic seeks to become the leading COA quality oversight company in the nation, ensuring 100% oversight of clinical assessments, thereby revolutionizing clinical endpoints by leveraging conversation intelligence to expand human understanding and precision medicine, thereby helping to bring new, effective drugs to market.

Grant Number: 1R43MH142353-01
NIH Institute/Center: NIH
Principal Investigator: Alison Cerezo