Are clinical trial professionals tracking the right metrics? Do any metrics matter, given that every company has its own quirky list—and that no list corresponds to other firms in the industry? Are there any metrics that should be granted early retirement? Or should the industry be tracking more?
Ken Getz, senior research fellow at the Tufts Center for the Study of Drug Development, says that the industry's efforts to monitor its own performance are limited and must become more comprehensive. "The only way that you can improve efficiency and lower variance is to become better at tracking your activities and performance" says Getz. "This is part of a much larger trend in companies looking to improve their ability to manage an increasingly complex array of activities and providers that support those activities."
As with everything the industry does, the best practices of one therapeutic area or corporate department are seldom adopted companywide, much less shared by the project's contract research organizations or software vendors. For those working at the lower level of the organization, the sheer profusion of metrics can create the impression of an exercise in futility, as orders are given to robotically track numbers that may have little bearing on real efficiency or closing the study in a prompt manner.
"Historically, companies have been so fragmented in their application of metrics," Getz says. "Whatever one functional area or a single therapeutic area was using was a different set of metrics than was used elsewhere. Sometimes it was driven by political winds—whoever was heading up a group in a power center, they set up the metrics they wanted to use."
Fortunately, Getz says, this is beginning to change. "More and more companies are trying to coordinate and standardize the metrics within their organization and centralize them so that they can access the data and achieve a level of internal consistency." So-called "dashboards" of information found in both proprietary software and garden variety commercial systems, he says, are helping sponsors settle on a small number of metrics.
The Metrics Champion Consortium, a nonprofit based in Indiana, is trying to drive the process. Getz thinks there are parallels between the approach of MCC and that of the well-regarded Clinical Data Interchange Standards Consortium (CDISC), an early proponent of standards for clinical databases.
In time, Getz says, the industry is aiming for something that many other more technologically sophisticated industries take for granted. Namely, simple, quick metrics to gauge the operational efficiency of an organization or activity. That could allow senior managers to see the return on a scientific investment on a broad level. Today, the metrics and data are too fragmented, too varied from one firm to the next.
On the marketing or commercial side of the industry, Getz says, there is an ability to measure performance. But the industry has not been able to bring the same tools, the same consistency, to research. "We do not have a core competency in management and operating metrics," Getz says. "There is an acknowledgment in industry that we have to strike a balance in achieving scientific goals but also in achieving operating needs. That's where metrics are going to play a part."
This does not mean the industry needs umpteen new ways to assess clinical trials. Rather, the challenge is to find a few metrics that all stakeholders can agree on. "Having better metrics does not mean more metrics," he says. "It's not quantity. It's the quality, accuracy and relevance that matter."
Getz says that select sponsors, including Pfizer, are starting to pick a few key metrics, and develop ways to incentivize personnel and partners when the milestones are achieved. And that could present some new ways of thinking.
Last Patient In
An example. Enrolling the first patient has long been an industry talisman, ostensibly signaling that a trial has begun. But does it have true significance? What if a burst of activity to find the first patient is followed by months of inactivity and waiting around?
As additional paperwork, protocol amendments, and technology systems come online, the real start of the trial may occur well after the first patient is logged. Pfizer is not walking away from first patient methodology. But according to Getz, the firm is examining the significance of the last patient. That would shift the emphasis from the sites, which are not under Pfizer's control, back to the sponsor itself.
As the sponsor community begins to appreciate new metrics, Getz says that in some cases, it's not clear where the core competency around a particular metric will reside. Some metrics may be under the purview of the contract research organization (CRO). Others may rest with the sponsor. The situation is fluid. Says Getz: "As pharma and biotech transfer day to day managing responsibilities to CROs, perhaps the CRO becomes the domain where the competency resides, and pharma needs to rely on a smaller set of management metrics that are harvested and collected by the CRO."
Editor's note: Today marks the debut of a new editorial department on ClinPage: Clin Ops. We encourage readers to submit relevant topics for coverage—or to write articles themselves.d9A2t49mkex