How to generate better patient-reported outcomes data

By Rachel Smith, Executive Director, Rare Disease, Center of Excellence

8 min

Lessons learned from a Parexel analysis of FDA-approved orphan drugs

Sponsors, regulators, health technology assessment (HTA) agencies, and patient advocacy groups (PAGs) require direct input from patients about a new drug’s impact on quality of life, symptoms, and side effects. Patient-reported outcomes (PROs) are a critical tool to achieve that: they measure how a patient feels and functions without amendment or interpretation by a clinician or anyone else. Many companies know this: they spend time and effort collecting PROs in pivotal efficacy trials and submit the data to the FDA, but the information often does not appear on product labels. As a result, it’s not readily available to U.S. providers and patients. Why? It turns out that selecting relevant PRO tools—and collecting and analyzing the data with sufficient rigor for regulators—is challenging.

Parexel analyzed the FDA clinical review documents for 164 novel orphan drugs and biologics approved from 2017-2022 to understand the issue.¹ We found that just 13% (21) of the products got PRO data on the label, even though 63% (103) of the sponsors collected it during pivotal efficacy trials.

FDA-approved orphan drugs, 2017-2022 (n=164)

PROS Data Chart

See footnote 1 for a description of the dataset.

Of course, sponsors collect PRO data for various reasons. Since the FDA often redacts information about labeling negotiations, it’s hard to know how many companies seek a formal U.S. label claim. Even if FDA reviewers disallow PRO data for the label, they may refer to it as “supportive” evidence in clinical review documents.

Frequently, PRO data is not intended for the FDA at all. Sponsors may be aiming at the EMA, which, according to a recent study, allows more PRO data on product labels because it is “more likely to accept data from open-label studies and broad concepts such as health-related quality of life.”² Other sponsors may collect the data primarily to support European HTA agencies’ reimbursement decisions.

Nevertheless, our analysis of FDA review documents for orphan drugs suggests that PRO data is not utilized efficiently, suffers from persistent methodological problems, and needs to be more widely disseminated to patients, at least in the United States.

If patients are to reap the benefits of patient-focused drug development, the art and science of developing, collecting, and analyzing PRO data must advance.

Persistent problems with PROs

Problems with PROs frequently cited by FDA staff in clinical and statistical reviews and clinical outcome assessment (COA) consultations for the 164 orphan drug approvals we analyzed included:

  • Missing data due to disease progression, death, or high dropout rates, especially in the setting of advanced, refractory cancers
  • The lack of a prespecified analysis plan with hierarchical testing that corrects for multiple endpoints
  • Lack of prior communication with FDA about PRO selection and rationale
  • Potential for bias in single-arm, open-label trials
  • Inadequate, unvalidated PRO instruments and failure to define what constitutes a “clinically meaningful” effect on the scale, score, or index

How to generate higher-quality PRO data

PROs burden patients who must fill out sometimes lengthy questionnaires or record events in a paper or electronic diary (e-diary) or during already lengthy clinic visits. As a result, PRO data collected but later deemed uninterpretable represents an unethical use of patients’ time and effort. For sponsors, the ideal strategy is to identify high-quality measures of patient health that can be used to construct a meaningful, evidence-based PRO endpoint.

Parexel recommends three best practices to ensure the scientific rigor of PROs.


PRO instruments must have “content validity,” meaning they must be relevant and specific to the disease being studied. That requires a clear understanding of the natural history of the disease, the key symptoms that may improve from treatment, and an appropriate length of time to see a clinical benefit. Specify and define concepts—such as signs, symptoms, and impacts—important to the target patient population and likely to demonstrate meaningful and interpretable changes in clinical trials. Seek input from patients and expert clinicians.

It takes time—sometimes years—and effort to do this right. At Parexel, we are currently working with a sponsor to design a PRO for a rare pediatric central nervous system disorder. Clinical trials of this disease traditionally use the primary endpoint of reducing seizures. But seizure control is not the only symptom critical to quality of life in this setting: these children often cannot feed themselves or verbalize, functions that families, caregivers, and patients prize. So, we are developing a new PRO to quantify changes in self-feeding and verbalization. We have worked closely with patients and caregivers and scoured the medical literature to determine what would represent a meaningful change in vocalization and how to measure it precisely. We are combining modules from multiple already-validated PROs into a novel instrument we can validate in this disease and use in clinical trials.

Initially, we will use the new PRO as a secondary or exploratory endpoint in a Phase 1/2 trial. If successful, we will use it for the pivotal trial. We hope to gain FDA guidance and acceptance of this approach during Type B and C meetings. Developing a new PRO requires careful planning, strong rationale, and high-quality evidence. 

Sponsors must also ensure that the patient population can validly and reliably self-report (some patients may be too young or sick) and select a PRO that accommodates a spectrum of cognition and mobility levels. Also, make sure the recall period for a PRO is at most two weeks. Otherwise, the FDA may question the validity of the results. 

For statistical validity and comparability, the mechanism used to report PRO data should be consistent throughout a trial. A drug recently approved by the FDA for an orphan metabolic disorder used a daily questionnaire to measure patients’ hunger levels. However, partway through the pivotal study, patients switched from answering the questions on paper to using an e-diary. The sponsor did not subject the new e-diary device to usability testing or conduct patient cognitive interviews to assess its functionality, questionnaire comprehension, and ease of use. The FDA concluded there was insufficient data to demonstrate that patients understood one of the questions. Although the PRO data, in this case, made it onto the label (the PRO was the primary endpoint of the study), it’s a cautionary tale for other sponsors.

FDA reviewers who analyzed and rejected PRO data contained in orphan drug applications from 2017-2022 frequently noted there was no prespecified statistical analysis plan (SAP) or correction for Type 1 errors (results due to chance). One way to avoid this problem is to agree with the agency on the SAP before initiating a pivotal trial. When a PRO is designated as a secondary endpoint, a hierarchical testing framework with an alpha level (the level of significance required to show that the result is not due to chance) adjusted for multiple comparisons can help ensure valid results. 

Another considerable problem cited by the FDA was missing data due to death, disease progression, or a high dropout rate for the trial. Sicker patients and those who do not respond to treatment are less likely to complete patient-reported quality-of-life instruments as a trial proceeds, skewing results. In one orphan cancer trial that enrolled heavily pretreated patients with advanced disease, patient completion of a 36-item health-related quality of life (HRQoL) questionnaire dropped so precipitously from baseline (about 90%) to week 48 (about 30%) that the data were deemed invalid by FDA reviewers.

During a 2022 FDA conference on patient-focused drug development (PFDD), senior clinical analysts from the agency noted that sponsors should have a plan to understand why data is missing and how to handle it.³ To address potential bias and interpretability issues due to missing data, sponsors can plan a worst-case scenario analysis in which they manage patients with missing HRQoL data differently. More sophisticated statistical methods, such as pattern mixture models, may further elucidate the effect of missing data on study findings.

Although missing data is inevitable in trials, sponsors can take steps to prevent it. If the burden of filling out questionnaires and e-diaries for multiple PROs in a single study is too high, patients will not complete them. Sponsors thus need to ensure that study designs make sense, are practical, and will achieve the goals outlined in the protocol.

Of the 21 orphan drug labels from 2017-2022 that contained PRO data, just four (19%) reported data from open-label studies.

Most PRO data the FDA approved for inclusion on product labels were generated in blinded, controlled studies.

PRO data from open-label or single-arm studies has limited interpretability. That’s because treatment effects can be systematically over- or under-estimated by patients who know the treatment assignment. 

Randomized, controlled trials (RCTs) are the most efficient way to neutralize confounding factors and, therefore, the best way to study risks and benefits. Double-blinded assignment to treatment and control can provide unbiased comparisons for known and unknown factors. Sometimes, a comparator arm is neither feasible nor ethical for some modalities, such as cell and gene therapies with remarkable activity and indications, such as advanced cancers or ultra-rare diseases with no existing treatments. Use of active controls, external control arms, natural history studies, and historical controls as comparators can help.

However, even a single-arm open-label trial can collect useful PRO data. In our analysis of orphan drugs, we found a CAR-T cell therapy for cancer that the FDA approved based on a single-arm, open-label Phase 2 study. This pivotal study captured three different PROs as secondary endpoints. Although the PRO data did not meet the FDA’s standard for inclusion on the product label, the data did appear on the EMA label and was later published in a peer-reviewed journal.⁴ An editorial accompanying the journal report stated: “…the authors should be commended for their efforts to provide the scientific community with unprecedented PROs information on the burden of this CAR T-cell therapy. Despite several open questions, which should be elucidated in further studies, the results reported in their article are highly encouraging and hopefully will stimulate other high-quality research initiatives in this area.⁵

PROs can enrich the risk-benefit analysis for novel drugs

The most important lesson from our review is that sponsors should select or develop a PRO in collaboration with patients and disease specialists to agree with regulators ahead of pivotal trials. Wider capturing and publishing of the effects of drugs as experienced by patients will help drive more holistic, patient-focused therapies.

Contributing Expert

¹ Numbers include original new drug applications (NDAs) and biological license applications (BLAs) only, not supplemental applications (label extensions or new formulations). The dataset includes only orphan-designated products that achieved U.S. licensure for the first time between 2017 and 2022 and contained a new molecular entity or new active moiety. For CBER approvals, we excluded assays, fractionated plasma products, patch tests, reagents, vaccines, and tissue transplant products.

² A Review of Patient-Reported Outcomes Labeling for OncologyDrugs Approved by the FDA and the EMA, 2012-2016 (February 2022, Value in Health).

³ Missing Data Hinders Inclusion of Patient Experience in Drug Labeling, US FDA Says (July 2022, Pink Sheet).

Patient-reported quality of life after tisagenlecleucel infusion in children and
young adults with relapsed or refractory B-cell acute lymphoblastic leukemia: a global, single-arm, phase 2 trial (December 2019, Lancet Oncology).

Quality of life and CAR-T cell therapy in children, adolescents, and
young adults with haematological malignancies (December 2019, Lancet Oncology).