By Steve Jones
Vice President, Data Analytics, Medable
At the beginning of April 2023, the FDA published a draft of the fourth in a series of Guidance Documents entitled “Patient-Focused Drug Development: Incorporating Clinical Outcome Assessments (COA) Into Endpoints For Regulatory Decision-Making”. Once finalized these four documents will be combined and will replace the 2009 Guidance, “Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims”. Part 3, which was released in draft in June 2022, provides advice around the development and validation of new COA (including the modification of existing COA).
This final part considers the impact of this extra step on the study design, the statistical analyses, and the reporting required to demonstrate not only a clinically relevant treatment effect but also a treatment effect that is meaningful and of overall benefit to patients.
As with all types of endpoint, it should be stressed that statistical significance is not the same as clinical and patient relevance but is an important consideration at the design stage when planning the number of analyses and the impact of this on the study power once all adjustments for multiple comparisons have been made. Early on during the design and construction of the drug-development plan, Sponsors should initiate a dialogue with the FDA and provide a rationale for the proposed selection of endpoints. Readers with sufficient knowledge of statistical practices adopted for the design and analysis of non-COA endpoints will see a lot of similarities with what is being suggested for COA endpoints; this is not surprising as analysis specification is more dependent on the data type (continuous, ordinal, categorical) and their expected statistical distribution.
From my perspective, there are three main issues emanating from the inclusion of COA data as a non-exploratory endpoint.
The first issue is the availability of data in published literature for PRO endpoints pertaining to the intended patient population. For many therapeutic areas, where there exists objective measures of efficacy, the PRO endpoints are usually defined as exploratory (to avoid the need to adjust for multiplicity) and as such rarely are reported in scientific literature to the level of detail required to assess sample-size requirements. For Pharma Companies without the required data from previous trials from the same therapeutic area, one solution would be to include PRO evaluation in Phase I/II clinical trials as exploratory endpoints in order to collect the data required to formally include the planned PRO analyses in Phase III. With appropriate planning and discussions, it may be possible to reuse the data as part of a meta-analysis to strengthen the evidence of a correlation between clinical and patient benefit.
The second issue is the impact to the power and the need for the adjustment of significance level for multiple comparisons (resulting in the need for a larger trial in terms of patients). There is discussion on the use of multi-component and personalized endpoints, where an algorithm for combining the outcome from the individual endpoints thus removing the need for adjustment for multiple comparisons. The discussion also highlights the challenges around defining such endpoints. Interestingly, there is no mention of defining a hierarchical approach to testing, eg. testing components in a pre-specified order and with a predefined significance level (eg. Benjamini-Hochberga). Whilst this approach does not remove the need for multiplicity adjustment, it does alleviate the exacerbation of my first issue, caused by defining a novel approach/combination of endpoint components that definitely won’t be described to the required level of detail.
My third issue is the Impact on the duration of the trial, owing to a longer time required for patients to experience a “meaningful benefit” to treatment; for example, a surgical procedure may provide almost immediate clinical benefit, but the recovery time may lessen the immediate patient benefit and a longer follow-up to reassess patient benefit may be required. The use of a hierarchical approach may allow for an earlier assessment of clinical benefit, which if demonstrated would allow the later assessment of patient benefit (and possibly allow future trials to be initiated earlier), whereas, with a multi-component endpoint, all data are analyzed at the later time.
I am certain that readers of the Guidance Documents will have differing concerns. One thing for certain is that the publication of these Patient-Focussed Guidance Documents represents an increase in the importance of COA endpoints and the need to design trials to scientifically capture these data with sufficient credibility and rigor.
Benjamini, Y. and Hochberg, Y., 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57(1), pp.289-300.
