The most notable nugget from the Center for Business Intelligence’s (CBI) Arlington, Virginia meeting on patient-reported outcome (PRO) research late last month was not so much from the podium, though there was plenty of excellent material there.

Some of the most interesting news was in the questions from attendees. Your correspondent, for example, had assumed that utterances by regulators about PRO data have been lucid enough to assuage the doubts of the sponsor community. Wrong.

From audience comments at the conference, there remains considerable ambiguity, even frustration, about the FDA stance on PRO data. Some of it seems linked to the industry’s usual torpid approach to using any new-ish technique or tool. But some of it is linked to the agency itself flashing a warning light on PRO studies. Perhaps the rumored publication (later this year or early next, or by 2019 at the latest) of a new final PRO guidance document will resolve the qualms and questions in the community.

In fairness, one speaker at the CBI conference did insist the FDA’s communications with sponsors have been clear and predictable. “We are seeing actual citations to actual pieces of the [draft] guidance,” noted Jean Paty co-founder and senior VP of regulatory affairs at invivodata. “It is completely unambiguous that they are saying, ‘if you are going to do something, give us the rationale, and document the justification for what you just told us.’ ”

“One of the most fundamental messages is, and we have seen this from Laurie Burke, is begin with the end in mind,” said Paty. “What are you trying to show here? Tell us what your claims are. Tell us where you are trying to go.”

Merck’s Method

On an operational level, the details of best practices in trials using patient-reported outcomes are far from clear. Simply planning translations of questionnaires in new tongues can be a challenge, as Josephine Norquist, an epidemiologist at Merck, pointed out. Her slide for just the translation component of PRO trials contains eleven different boxes arranged in several loops.

Beyond that, Merck takes the validation of the questions in a PRO seriously, testing them in focus groups of 20-50 people. “We ask them what they think of the question,” Norquist said.

Like Paty and other speakers at the meeting, she stressed the importance of having a plan and sharing it with regulators. “We have to treat a PRO like any other input in a trial,” said Norquist. “We need to prespecify the endpoint.”

Terminology Issues

Norquist stressed anticipating complications like missing data, statistical issues, and the conceptual connection between the outcome and claims on the label. “You may have patients that have answered your questionnaires but their data are not in the analysis,” Norquist said. “You would have to prespecify how you are going to handle that.”

A premeeting seminar at the conference covered, among other topics, the nuances of terminology used by the FDA. Diane Wild, director of Oxford Outomes, endured some sharp questioning from the audience abut stuck to her guns, citing the FDA itself.

“Some people call conceptual frameworks ‘measurement models,’ ” Wild noted. “What we called ‘end point models’ would now be considered to be overly simplistic by the FDA. We’re moving on to conceptual framework. This is the terminology that comes directly from the FDA.”

A conceptual framework can be rendered in a diagram or (to our uneducated brain) imagined as a way to connect a disease, the patient’s experience of the disease, and an intervention to change that experience. In addition to biological and physiological variables, a conceptual framework includes symptoms, functional status, perceptions, psychology and quality of life.

Not rigorous enough? Fine. Here are two other definitions of “conceptual framework” from Wild’s presentation: “A diagram of proposed causal linkages among a set of concepts believed to be related to a particular public health problem/disease” (from Earp, Ennet). Or: “a taxonomy of patient outcomes according to the underlying health concepts [sic] they represent and proposes specific causal relationships between different health concepts” (Wilson, Cleary).

Start At The End

End point models won’t disappear, Wild predicted. They will get more complex. But diffuse and generalized claims may be harder to take to the agency. Says Wild: “We tend to advise clients to stay away from health related quality of life (QOL). You might be better off focusing on psychological well being, where the function is easier to define than quality of life.”

Another essential regulatory concept is the target product profile, or TPP, which you can read about on the FDA website. To a degree, using PRO instruments can help sponsors generate a TPP. These two tools can evolve together.

In another talk at the CBI meeting, ClinPhone’s Keith Wenzel explained that PRO measures are being used in scientific quests that don’t necessarily involve the registration of a drug for approval.

For him, the PRO technology itself is not confined to a piece of paper or a personal digital assistant from Palm, but encompasses digital pens, interactive voice response (IVR) systems, tablets and patients visiting websites. Indeed, our subjective impression from the CBI conference is that the jury remains out on just which form of electronic PRO (via IVR, web, tablet or handheld smart phone) is best.

To Do List: 186,000 Calls

But Wenzel’s presentation on the capability of IVR in the PRO arena was impressive. For starters, PRO can be used as an expeditious way to facilitate an adaptive clinical trial (which we wrote about here and here, with the experiences of earlier groups of patients funneled directly into the algorithms determining the randomization for later patients. In one project, the savings to the sponsor were $2.1 million. The links between technology, cost savings and scientific rigor have rarely been more effectively presented.

In another project, Wenzel noted that his firm had helped study side effects to a vaccine. The investigators found 6,016 pediatric patients in 11 days, and ClinPhone’s system automatically generated 186,000 reminder calls, far beyond what the staff at the sites could have handled. Human-initiated calls to parents numbered a mere 28,000. There was no falloff in compliance during the trial. “These kinds of studies are almost impossible to conduct on paper,” said Wenzel.

The technology allowed the sponsor to cast a wider net for sites that participated, Wenzel said. “These are non-traditional study sites. It’s almost like a Phase IV site, where the study staff are not necessarily geared to traditional study design.”

Wenzel also presented a few slides on a Phase III study for eye pain. An IVR system collected primary end point data; subjects had no problems with using the telephone to record their symptoms and experiences. The average length of each call hovered at five minutes or so.

Lost In Translation

We were unable to hear GSK’s Ingela Wiklund, director of patient reported outcomes. Still, her slides are fascinating and clearly suggest a key link between advertising based on label claims and the techniques used to gather the underlying PRO data to support those claims.

Wiklund’s slides show a paper diary in which an “enthusiastic” patient filled out a page with all manner of barely legible commentary, all of it needing to be deciphered and scrutinized for years to come by countless employees of her own and other organizations. (The handwriting sample alone, we must confess, was enough to make us cringe, and wonder why any paper diaries are still accepted by regulators.)

The Swedish Wiklund explored a few of the linguistic issues with PRO trials. Words like “heartburn” should not be blithely translated from one language to another, lest they spawn confusion across oceans or cultures. Even a term like “gardening” may not make sense to all socioeconomic groups.

Wiklund’s list of must-have items in any registration dossier using a PRO instrument was surprisingly detailed and (we presume) born of hard experience. We reproduce it here for those unable to attend the conference:
• All targeted claims (including ones supported by PRO)
• Endpoint model (with justification of instruments, relationship to all endpoints)
• PRO conceptual framework, diagram
• Cognitive validity documentation
• Focus group composition, number of patients
• Translatability and lexibility assessment
• Translation/cultural adaptation (translation process, qualifications of translators)
• Measurement properties (including psychometric performance)
• Interpretation of score (with definition of responder MCID)
• Protocol and analysis plan
• User manual, training material
• Mode of administration (paper, electronic, IVR) with evidence of concordance
• References
• Additional information (examples: interview guides, focus group transcripts)
• Rationale for modification of instruments (if applicable)
• Protocol documentation
• Copy of PRO instrument, supporting references