NEW YORK – Precision oncology trials — most of which struggle to enroll sufficient populations of patients with unique biomarkers — may broaden their pool of eligible participants by relaxing overly restrictive eligibility criteria without altering trial results or endangering patients, according to research out of Stanford University and Roche subsidiary Genentech.
The analysis, based on real-world data and using artificial intelligence, was published earlier this month in Nature and suggests that eliminating or relaxing certain eligibility criteria such as laboratory values with defined cutoffs would minimally alter the trials' survival outcomes. The framework behind the analysis, dubbed Trial Pathfinder, is part of a collaboration between Stanford and biopharmaceutical industry partner Genentech aimed at improving clinical trial design.
Using their data-driven approach, the researchers specifically found that when overly restrictive criteria were relaxed, the population of patients eligible for oncology trials more than doubled, and the efficacy of the treatments evaluated in the trials — as assessed by the survival benefit in the treatment arm — remained essentially unchanged, if slightly improved. The findings, according to study authors including Stanford's James Zou, could encourage trial designers to re-evaluate and relax their eligibility criteria, in turn broadening the pool of eligible patients including those from underrepresented patient populations.
Roche is already applying the approach to designing prospective clinical trials, and according to Zou, Stanford hopes to expand the approach to partnerships with other pharmaceutical companies, too.
"We are currently developing a software tool to incorporate elements of Trial Pathfinder to assist in the design of future oncology trials," Shemra Rizzo, Genentech's senior data scientist for personalized healthcare and one of the authors on the Nature study, said. "Our methodology is open-source, and we are hoping further research takes place on the topic across the healthcare industry."
Indeed, direct access to all of the open-source Python code for Trial Pathfinder is included in the Nature paper, as the researchers hope trial sponsors beyond Genentech will implement the framework for their trials, too.
Assessing Trial Pathfinder in NSCLC
The Trial Pathfinder framework allows researchers to simulate how different combinations of eligibility criteria might alter trial outcomes. For their initial analysis, the researchers retrospectively considered eligibility criteria from 10 completed clinical trials as well as real-world data from patients' electronic health records within the Flatiron Health database. To begin with, they zeroed in exclusively on advanced non-small cell lung cancer (NSCLC) patients and trials because the NSCLC patient population — totaling 61,094 patients — comprised the largest proportion of the Flatiron database. On average, 5,167 patients in the database corresponded to each of the 10 selected NSCLC trials, meaning they received the respective treatment in the real-world setting.
The 10 trials evaluated either single-agent treatment or chemotherapy combinations involving one of the checkpoint inhibitors pembrolizumab (Merck's Keytruda), nivolumab (Bristol Myers Squibb's Opdivo) or atezolizumab (Roche's Tecentriq) or one of the targeted therapies osimertinib (AstraZeneca's Tagrisso), afatinib (Boehringer Ingelheim's Gilotrif), and bevacizumab (Roche's Avastin). Eligibility criteria varied significantly across the trials, several of which required patients to harbor specific mutations or biomarkers, and many of which excluded patients on the basis of their ages, performance scores, prior treatment regimens, or laboratory values.
Stanford's Zou described the overarching framework of the Trial Pathfinder approach. "We looked at patients who have taken particular drugs [in the real-world setting], and their longitudinal outcomes for different lines of treatment, then the algorithms took these data and basically emulated thousands of synthetic clinical trials" he said, explaining that each "synthetic" clinical trial corresponded to a variation on the original trial but with a different combination of eligibility criteria.
Integrating the treatment outcomes from the real-world data patient cohort, the algorithms then evaluated, based on the patients who fit the criteria for each synthetic trial, what the overall survival benefit would be with each respective criteria tweak.
"The Trial Pathfinder emulation framework makes it possible to systematically vary the eligibility criteria in silico and quantify how the hazard ratio of overall survival changes with different combinations of criteria," wrote Zou and co-authors in the Nature paper.
The researchers removed an average of nine pieces of inclusion or exclusion criteria for each trial without detrimentally affecting the trial's outcomes. They referred to the list of criteria after these exclusions as the "data-driven criteria."
Initially, of patients who received the same drugs in the real-world setting that had been evaluated in the trials, only about 30 percent fit all of the original trials' eligibility criteria. When the data-driven criteria were applied to the real-world dataset, however, the number of eligible patients for the trials jumped, on average, from 1,553 patients to 3,209 patients, representing a 107 percent increase in the pool of eligible patients for each trial. The overall survival hazard ratio, on the other hand — that is, the risk of dying with the treatment versus the control arm — decreased an average of 0.05 with the data-driven criteria versus the full eligibility criteria.
"[This] demonstrated that broadening existing standards for several eligibility criteria could potentially open opportunities for more people who could have benefited from the treatment," Genentech's senior data scientist for personalized medicine said, explaining that the decrease in the hazard ratio meant that people who, based on eligibility criteria, would have been excluded from the original trial actually ended up benefiting from the treatment in the real-world setting.
Validation, safety assessment
To further validate their results, the researchers ran the data through the algorithm again, but with the effect of eligibility criteria on progression-free survival as their focus as opposed to overall survival. Again, the pool of eligible patients expanded while the trial results were minimally changed. They also performed the analysis using data from the Flatiron Health-Foundation Medicine clinicogenomic database, which included NSCLC patients who had undergone genomic profiling via Foundation Medicine's FoundationOneCDx next-generation sequencing panel.
"We used the [NGS test results] as additional information to create even better controlled synthetic clinical trials," Zou explained. "Because the goal of the Trial Pathfinder is to emulate synthetic clinical trials, and the more biomarker information we have about the patients, the better the algorithms are able to create better-matched treatment and control arms to emulate these trials."
Including the comprehensive biomarker information, in other words, improved the algorithm while validating the findings from the initial Flatiron database. They also repeated the analysis in three non-NSCLC trials — including melanoma, colorectal cancer, and breast cancer trials — and found similar results.
Finally, Zou and colleagues assessed the effect of their data-driven criteria on safety, using the percentage of NSCLC patients in the real-world dataset who had to stop treatment due to toxicity as a measurement. Here, they found that patients who fit the more relaxed eligibility criteria did not have higher safety risks. Broadening their analysis beyond NSCLC, moreover, they applied their data-driven eligibility criteria to 22 completed Roche oncology trials in different cancer types including melanoma, breast cancer, and lymphoma and found that those trials with more relaxed eligibility criteria did not report significantly more safety-associated withdrawals than their stricter eligibility-criteria counterparts.
'Doubling the pie' to diversify trials
According to Rizzo and Zou, a key advantage of the Trial Pathfinder approach to designing precision oncology trials is its potential to make these trials more diverse and inclusive. When the pool of eligible patients expands, the thinking goes, so do participation opportunities for patients from underrepresented groups such as racial and ethnic minorities and older patients.
"Data-driven approaches like Trial Pathfinder can provide supporting evidence that clinical trials should have more inclusive approaches to capture more diverse populations," Rizzo said. "Broadening eligibility criteria is one key approach to expand inclusivity to historically underrepresented populations in oncology trials."
In the NSCLC analysis specifically, the researchers found increases in the percentage of female patients, Black patients, and older patients, on average, who were eligible for the trials when they applied the data-driven criteria. In all trials evaluated, Rizzo noted, the median age of participants was five to seven years higher in the real-world data cohorts than in the trial populations. The proportion of female participants was also much higher. For example, in the Keynote407 trial, which evaluated pembrolizumab plus chemotherapy versus chemotherapy alone as frontline treatment for metastatic NSCLC, the initial proportion of female patients enrolled in the treatment arm was just 21 percent, but this share rose to 50 percent of the real-world data population when then data-driven eligibility criteria were applied.
While race information was not available for all the original trial publications assessed, Rizzo pointed out that across all trials that did report on the proportion of Black patients enrolled the researchers observed an increase in their real-world emulated trials. Reported proportions of Black patients in the original trials ranged from just one to four percent, whereas the proportion of Black patients in the real-world data-derived cohorts was about 10 percent.
"Some of these patients from these demographic groups may have been excluded because of these more restrictive, exclusion-based criteria, and once we standardize these criteria, we have the benefit of bringing in more diverse populations," Zou said. "If we double the pie, then everybody gets more opportunities."
In both Zou and Rizzo's view, the main takeaway from the Trial Pathfinder project is the potential this framework has to make precision oncology trials more inclusive. Looking at NSCLC in particular, Rizzo pointed out, 86 percent of clinical trials fail to complete recruitment within their targeted time.
"The current enrollment of patients in these trials can be very challenging and also very expensive," Zou added. "And what we know from the insights from our study is that we could actually take this approach using real-world data [and] provide evidence that we could make these trials more inclusive, which benefits both patients and the biopharma companies."