The drug development industry has spun its wheels for years, trying to come up with universal metrics to measure speed and efficiency. But metrics that accurately measure quality in a trial? No one has attempted that yet.
Until now. The Metrics Champion Consortium (MCC), a non-profit collaborative effort of sponsors, contract research organizations (CROs) and vendors, was launched five years ago. Its stated goal: to improve the way sponsors and suppliers of services and technology work together.
Born out of an effort at Covance's central labs, the MCC sought standardized performance metrics to which all parties should strive. The group chugged along, getting heavyweights in the industry to brainstorm on and amass useful metrics that applied to central labs, imaging, and electrocardiography. (Here's a story we wrote about the consortium's efforts in 2007.)
Last year, though, the organization decided to go further, launching an effort to focus on performance metrics for clinical trials in a broader sense. A working group of about 100 people was divided into subgroups (biometrics; business operations; clinical operations; drug supply; regulatory). Before long, they came up with a beta set of 30 metrics which the consortium released in April of last year.
Soon after that, it occurred to the MCC's VP of operations, Linda Sullivan, that something was missing. As she participated in all web conferences and calls between parties trying to develop and revise various metrics, she began to notice that the people using process-improvement strategies—think lean sigma, six sigma and the like—seemed better able to take new metrics into their companies, apply them and get exciting results. She also noticed that part of that process-improvement culture is a commitment to matters beyond speed and efficiency.
Says Sullivan: “Process-improvement people look at things two ways: trying to drive cycle time but they also want to make sure the quality, the percentage of defects, is better. They're not just trying to do things faster, but better.”
“It Doesn't Matter”
Sullivan was drawn to this, and began to talk more with black belts and other people deep into process improvement. Their hunch? The consortium was missing quality measurements. “And that's because they don't exist,” she says. “But quality is an issue that plagues everybody. Because you can drive, say, your query rate down. But if you don't measure the quality of the data, it doesn't matter.”
And so, to remedy that as best it can, the consortium has launched a new process-improvement group, comprised of about 20 people who are focusing on developing the first-ever metrics for quality in clinical trials. First stop: study start-up. The early efforts will zero in on the hallmarks of a quality protocol. And can quality be measured within site selection? “I think we're right on the edge of some very exciting stuff,” says Sullivan.
The results will be released in April, 2010, along with the 1.0 version of the consortium's clinical trials performance metrics. Initially, because they will be so new, so virginal, the quality metrics will likely be published with an exploratory or provisional designation. Currently, the volunteer members of the process-improvement group are beta-testing them within their own companies.
The consortium now has 40 members, including Abbott, Covance, PPD, ICON, AstraZeneca, Medidata, Roche, and “collaborators” like the Association of Clinical Research Organizations (ACRO), the Clinical Data Interchange Standards Consortium (CDSIC) and the Pharmaceutical Research and Manufacturers of America (PhRMA). Those on the working groups coming up with and revising the metrics are employees of these organizations; they volunteer their time.
In addition to developing relevant metrics for quality, the consortium is asking its new process-improvement group to apply lean or six sigma-like principles to each of the five working groups within the clinical trials group. The first task is to scrutinize the 30 performance metrics and determine whether anything process-oriented is being missed. The second is to look at the definitions of terms in the metrics and make sure they are useful for process improvement within companies that adopt them. Finally, they'll create a useful guidance document that will be released with the metrics. “What we really want is to drive process improvement using our metrics,” says Sullivan.
After that, more six sigma. The process-improvement thinking will then be applied to the upcoming revisions of metrics in the central labs, electrocardiography and imaging arenas. Version 2.0 of those metrics are due out later in 2010. But for now, the consortium is still knee-deep in its far-ranging clinical trial metrics. We asked Sullivan if there were any well-worn metrics that seem to be falling out of favor these days. She immediately mentioned first patient, first visit, adding that it seems to be losing its relevance.
“We left in first patient, first visit, but also put in another metric: first patient, first visit for all sites, which says a lot more,” she explains. “Wall Street is convinced that [first patient, first visit] is an important indicator, but really it's just a milestone—but companies can't not collect it now because there are people who expect to see that number. However, using it is not the best way to manage your trial.”
What are the chances that the consortium's clinical trials metrics will be adopted by industry? Decent. Says Guy Mascaro, the consortium's president, “We went from being an entity where people sat back and listened and said, 'Maybe this could work,' to an organization where people really now want to become involved. Service providers now understand how the metrics and best practices can help them.” Nowadays, even the FDA sits in on some of the consortium's communications with its steering committees. As an added indication the metrics are not just rhetorical, some are being written into contracts.
The time has come for standardized metrics, says Mascaro. After all, outsourcing is becoming the default approach to trials; it's only logical, then, that both sides would streamline their relationships in a quantifiable manner.
“Outsourcing is becoming more and more the norm,” he says. “Improving the way you're able to work with an external partner and form long-term relationships just makes sense. There's a learning curve that always occurs when using an outside organization; we learn how you do things and you learn how we do things. But at the end of the day, there's got to be a way to make sure that relationship is as productive as possible.”
And, adds Mascaro, “It's not just metrics themselves, but the dialog between service providers and sponsors about those metrics.”
—by Suz Redfearn
Editor's note: This week marks the debut of a new editorial department on ClinPage: Clin Ops. We encourage readers to submit relevant topics for coverage—or to write articles themselves.d9A2t49mkex