Stanford Team Proposes Automated Clinical Trial Accrual Strategy, Increased Trial Annotation

NEW YORK – Nam Bui has a piece of advice he wished he could share with more cancer patients: try to find a clinical trial.

Still, the Stanford University hematologist, academic oncologist, and medical oncology researcher recognizes the many obstacles to enrollment — from geography to time-consuming trial criteria assessments. Not only is it difficult for clinicians to keep tabs on all of the trials open at a center at any given time, he argued, but the process of matching a patient to a trial can be time consuming.

"The accrual from clinical trials in America is actually very low," Bui said. "It's much lower than where we want it to be."

That assertion is backed by research. According to an Institute of Medicine report in 2010, only around 3 percent of adult cancer patients participate in clinical trials. Moreover, an analysis of more than 500 trials within the National Cancer Institute's Cancer Therapy Evaluation Program revealed that 40 percent did not meet minimum patient enrollment goals. More than three out of five of these trials were Phase III studies.

Although these numbers are from a time when trials of molecularly targeted treatments were less common, the problem persists today, according to Bui and his colleagues, and is exacerbated by the additional layer of information needed to match patients to trials based on the presence of specific mutations, fusions, or other genetic alterations.

Matching cancer patients to appropriate trials "becomes more laborious as there are more trials out there, especially the biomarker-driven trials where the mutation of the tumor informs whether someone can go on the trial or not," explained Stanford pathologist Henning Stehr. "I think that is kind of an open problem in the community as a whole: how to improve this process, the trial matching of patients."

In an effort to demonstrate the value of more streamlined precision medicine trial matching strategies, Bui, Stehr and their colleagues set out to develop an automated clinical trial accrual approach — work they described in a recent proof-of-principle paper in the Pacific Symposium on Biocomputing 2020. There, they described an algorithm designed to match Stanford patients to trials underway at that center, which was spearheaded by the paper's first author Jessica Chen, who received a clinical data science fellowship to pursue the problem.

"We present the development of a feature matching algorithmic pipeline that identifies patients who meet eligibility criteria of precision medicine clinical trials via genetic biomarkers and apply it to patients undergoing treatment at the Stanford Cancer Center," the authors explained, noting that conventional clinical trial eligibility screening "is a labor- and time-intensive manual process that is susceptible to errors and missed enrollment as the volume of patients and clinical trials increases."

The method has not been rolled out widely at Stanford. In the long term, the investigators are hoping to get a better sense of how the algorithm impacts clinical trial accrual and patient outcomes, for example.

Even so, the current findings suggest that there could be a benefit to standardizing and structuring clinical trial descriptions more broadly. In contrast, the current analysis included biomarker-informed clinical trials that the Stanford team systematically and laboriously annotated, Stehr explained.

Other centers and commercial services have come up with their own solutions to matching cancer patients to trials based on tumor mutational data. At Dana-Farber, for example, investigators developed open-source MatchMiner software, while Chicago-based precision medicine firm Tempus launched an automated trial matching service based on a proprietary platform last spring.

For the proof-of-principle application of their own bioinformatics pipeline, the Stanford researchers attempted to match cancer patients to a dozen biomarker-informed trials listed on the institute's OnCore Enterprise clinical research management system, or to 18 multicenter precision medicine basket trials in which Stanford investigators were participating. 

"Even though clinical trials are internationally run operations, they are — at least for our practice in the Phase I clinic — very local," Bui noted. "Most patients really only consider trials that are in their area."

In particular, the team scanned data for more than 2,000 cancer patients whose formalin-fixed, paraffin-embedded (FFPE) tumor biopsy samples had been profiled using version 2 of the center's Solid Tumor Actionable Mutation Panel (STAMP) assay — a targeted capture- and next-generation sequencing-based test used to search for clinically actionable somatic mutations in some 130 cancer-related genes.

The algorithm at the heart of this pipeline relied on a "feature mapping" strategy: investigators pulled out specific pieces of pertinent information, including altered biomarker genes and pathogenicity classifications, and other features by delving into a centralized Stanford database that contains patient diagnostic data from the center's pathology lab, along with potential biomarkers that had been tracked down with the STAMP assay and annotated by molecular genetic pathology clinical fellows.

The algorithm used the standardized features extracted from raw STAMP data files and the precision medicine clinical trial selection criteria to match patients to appropriate and accessible trials, the team explained.

"The pipeline we developed uses a hierarchical decision tree-based algorithm to determine whether any of the STAMP-identified mutations per patient satisfy the criteria of the clinical trials of interest and thereby render the patient potentially eligible for at least one clinical trial," the authors wrote, noting that "each step examines whether a feature of the clinical trials matches the corresponding feature of the STAMP entries being queried, where each branch represents a decision and the leaves are the potential outcomes in the diagnostic report generated per patient."

For example, patients could be matched based on the presence of clinically actionable, pathogenic mutations, gene fusions, or copy number variants, the researchers noted. They did not include files from patients whose tumors contained synonymous, benign, unclassified, or unknown variant profiles.

In a subset of 366 STAMP-profiled cases used to validate the algorithm, the pipeline matched 93 patients to one or more precision medicine basket trials, with 22 cases matching criteria for two trials. Compared with manual trial matches, the researchers put the precision of the pipeline at just shy of 37 percent, with 91 percent specificity and a recall of nearly 94 percent. They noted that the specificity and recall increased further when they took into account specific disease exclusions that had been used for the manual assessments.

The team also analyzed more than 1,500 STAMP assay-profiled tumors retrospectively, looking at the type and frequency of mutations in them. Using the automated method, the researchers reported an overall match rate of more than 44 percent. Many of those matches stemmed from specific mutations in lung tumors, the authors noted. Excluding lung cases brought the overall match rate down to under 18 percent.

"The integration of such bioinformatic tools into the existing clinical workflow is advantageous for translational research," they reported, "as it redistributes the limited resources currently allocated to tasks that may be automated to tasks that [require] active physician engagement."

The approach is not intended to replace clinical trial matching by clinicians or molecular tumor boards, but to complement them, Stehr noted. And, he said, the algorithm, in its current form, is not intended for widespread use.

Instead, the work is "almost meant as a suggestion for the community, to see if we can find a standard — a common way of annotating trials and matching genomic profiles to it," he said, cautioning that the algorithm is "very specific to our internal data format."

"There are a lot of drugs that can change patients' lives, transform their lives, but being able to be on clinical trials or being able to get their molecular profiles back can be a problem," Bui added. "That's why I think trying to automate it in any sort of way is helpful." 

At centers where panel sequencing and other biomarker detection tests are routinely implemented, the team believes automated matching methods could complement manual clinical trial matching efforts that are currently done by molecular tumor boards or by the individual oncologists, pathologists, geneticists, and other providers responsible for a patient's care.

"This is never meant to replace the standard process of identifying clinical trials based on all other clinical data," Stehr said. "It's really more of a supplement, if you do panel testing anyways, to just include this automated screening process to identify cases that might otherwise be missed."

Despite the encouraging results achieved in matching molecularly profiled patients to well-characterized clinical trials at Stanford, Stehr cautioned that there is a need to standardize and annotate clinical trial requirements within widely used resources in the field if automated trial matching is to be attempted more broadly. For example, consistent, extractable information is difficult to glean from existing large clinical trial databases, such as ClinicalTrials.gov. 

"We put a lot of manual work into annotating all the trial arms [at Stanford]," he said. "That's a big limitation right now, and I think what is really needed is some public repository of structured clinical trial data."