Are unanchored matching-adjusted indirect comparisons a non-starter in EU Joint Clinical Assessments?

The publication of the first Joint Clinical Assessment (JCA) report, which evaluated the comparative efficacy of tovorafenib in pediatric patients with low-grade gliomas¹, is an important milestone in the implementation of the EU Health Technology Assessment Regulation (HTAR). The assessment covered eight PICOs (Population, Intervention, Comparator, and Outcomes). Six PICOs lacked meaningful comparative evidence, and one PICO relied on data published in a conference abstract (PICO 7), which the assessors deemed inadequate, leaving only a single PICO for which the assessors critiqued the relative effect estimates (PICO 5). 

To address PICO 5, the health technology developer (HTD) submitted an unanchored matching-adjusted indirect comparison (MAIC) analysis as its primary source of indirect comparative evidence. Here, we reviewed the assessors’ technical criticism of this MAIC analysis, and discussed the implications of the first JCA report for methodological best practices and overall strategic value in future JCA submissions employing this approach. 

PICO 5 concerned comparison of data for tovorafenib in the phase 2 FIREFLY-1 trial² vs dabrafenib + trametinib in the phase 1/2 NCT02124772 trial³ for the subpopulation of patients with BRAF V600E mutation, and encompassed objective response rate, progression-free survival, and safety outcomes. The unanchored MAIC analysis drew heavy criticism from the assessors, who highlighted serious methodological concerns relating to the covariate selection process:  

1. Unsuitability of sensitivity analyses  
2. Limited reporting of relevant patient characteristics in the comparator data  
3. Small sample size  

Ultimately, the assessors concluded that the treatment effect estimates obtained from the MAIC were not reliable. 

The extent of the scrutiny to which the analyses were subjected may leave other HTDs with the impression that unanchored MAICs are unlikely to fulfil the evidence requirements expected by assessors.⁴ Nonetheless, the report provides detailed insight into how a defensible MAIC analysis should be designed, so that HTDs considering an unanchored MAIC for future JCA submissions can position the generated evidence for maximum impact. 

Systematically identify clinically relevant baseline characteristics  

An unanchored MAIC uses propensity score weighting of individual patient data from the trial to achieve balance in key baseline characteristics with a published comparator population, to estimate treatment effects in the aggregate population.^5,6 These models rely on strong assumptions^7-9, most critically, that all clinically relevant treatment effect modifiers and prognostic factors have been captured in the model.^10,11 This condition is extremely challenging to satisfy in practice^12-14, and for this reason the Member State Coordination Group on HTA (HTACG) regards an unanchored MAIC as one of the lowest forms of evidence for establishing relative efficacy.¹⁵ 

In the first JCA report, the assessors commented that the criteria used to judge importance of candidate covariates for inclusion in the MAIC model were opaque, and the mechanisms by which input from clinical experts was elicited and utilized were not explained. The assessors stressed that comprehensiveness and transparent reporting of the covariate selection process is an essential prerequisite for an acceptable MAIC analysis. Furthermore, the assessors had a serious concern that not all relevant baseline characteristics were accounted for, since some of the key characteristics were not mutually reported in the datasets, and one key identified factor was silently excluded from the analysis of response outcomes. This feedback makes it clear that the burden of proof sits firmly with the HTD to demonstrate that validity of the population-adjustment model is justified, by following best practice to ensure that no important variable has been overlooked: 

• The list of priority prognostic factors and treatment effect modifiers should be informed by both structured clinical expert opinion and quantitative evidence from a systematic literature review, using clearly defined criteria for importance and a documented consensus process. 
• Unless there is explicit justification otherwise, the same covariate set should be applied consistently across analyses of all outcomes. 
• Where a priority prognostic factor is excluded from the model, the rationale for exclusion must be clearly stated. 
• Results of prespecified diagnostic tests for adequate model convergence, such as weight distributions, should be reported in the dossier.¹⁶ 
• Simple univariate statistical analysis of the patient-level trial data should be used to identify additional covariates to include in sensitivity analyses. 

Employ measures to help achieve adequate effective sample size 

The small effective sample sizes after reweighting, and the poor overlap of the trial populations implied by the substantial reduction in sample size compared to the unadjusted trial data — approximately 35-50% depending on the analysis set were among the key issues identified by the JCA assessors. Loss of precision is a pervasive problem in MAICs^9,17, and the assessors did not offer specific solutions to counter this issue, but HTDs should nonetheless seek to employ various measures to avoid unnecessarily high uncertainty, including: 

• Excluding lower-priority covariates from the base case model, instead reserving them for sensitivity analyses. 
• Adopting alternative reweighting strategies that optimise the effective sample size.¹⁸ 
• Employing an appropriate variance estimator given the features of the specific problem.¹⁷ 
• Applying multiple imputation by an appropriate method (e.g., MICE) to handle missing covariate data prior to reweighting. 

Conduct appropriate quantitative bias and sensitivity analyses 

The JCA assessors decisively refuted the quantitative bias analyses presented in the dossier as “incorrect”, demonstrating that not only the base case analyses, but also the supporting and sensitivity analyses, are subject to intense scrutiny. Three specific criticisms were particularly notable: 

1. Shifted null hypothesis testing based on minimal clinically important differences was said to be of limited use, as the assessors’ primary concern was to establish whether the extent of unmeasured confounding required to explain the treatment effect was plausible. 
2. Related metrics (i.e., E-values¹⁹) were said to have little meaning when identified relevant baseline characteristics were not included in the model. 
3. Sensitivity analyses in which fewer covariates were included than in the base case model were said to not be useful for the assessment. 

Together, this feedback illustrates the crucial role of quantitative bias and sensitivity analyses in exploring potential limitations of, and contextualizing results from, the base case analyses. HTDs should perform sensitivity analyses to confirm the robustness of results under expanded covariate sets, and quantitative bias analyses should be predicated on pre-defined thresholds that characterize implausibly high residual confounding.  

Conclusion 

The first JCA report has set a high bar for the acceptability of indirect evidence for relative efficacy from unanchored MAIC analyses, but it has also provided a rich and actionable source of guidance for sponsors seeking to meet that bar in future submissions. Most of these learnings, including on aspects of covariate selection and quantitative bias analyses, are equally applicable to anchored MAICs and to external control arm analyses using patient-level comparator data.²⁰ The criteria for generating fit-for-purpose estimates from such approaches are now well-defined and, in therapeutic areas where the evidence base is less limited, can in principle be met. 

It is important to note that the challenges facing the MAIC analysis to support relative efficacy of tovorafenib were, in many respects, insurmountable. A small sample size in a rare disease setting, combined with incomplete reporting of key baseline characteristics in the published comparator data, were fundamental limitations that no amount of methodological rigour could fully overcome. In these situations, following a thorough feasibility assessment, HTDs should make a pragmatic judgement about whether executing an unanchored MAIC for a requested PICO is likely to generate credible evidence, or whether the exercise risks producing results that will be dismissed, undermining confidence in the broader dossier. 

If this piece has raised questions for your JCA planning considerations, we'd welcome the opportunity to continue the conversation. 
 

Start a conversation

References 

1. Member State Coordination Group on Health Technology Assessment. Joint Clinical Assessment report of tovorafenib, Version 1.0. 2026. https://health.ec.europa.eu/publications/joint-clinical-assessment-report-tovorafenib-ojemda_en 
2. Kilburn LB, Khuong-Quang D-A, Hansford JR, et al. The type II RAF inhibitor tovorafenib in relapsed/refractory pediatric low-grade glioma: the phase 2 FIREFLY-1 trial. Nature Medicine. 2024/01/01 2024;30(1):207-217. doi:https://doi.org/10.1038/s41591-023-02668-y 
3. Bouffet E, Geoerger B, Moertel C, et al. Efficacy and Safety of Trametinib Monotherapy or in Combination With Dabrafenib in Pediatric BRAF V600-Mutant Low-Grade Glioma. J Clin Oncol. Jan 20 2023;41(3):664-674. doi:https://doi.org/10.1200/jco.22.01000 
4. Aballéa S, Toumi M, Wojciechowski P, et al. Between Rigor and Relevance: Why the EU HTA Guidelines on Indirect Comparisons Miss the Mark. Journal of Market Access & Health Policy. 2026;14(2):30. doi:https://doi.org/10.3390/jmahp14020030 
5. Signorovitch JE, Sikirica V, Erder MH, et al. Matching-adjusted indirect comparisons: a new tool for timely comparative effectiveness research. Value Health. Sep-Oct 2012;15(6):940-7. doi:https://doi.org/10.1016/j.jval.2012.05.004 
6. Ishak KJ, Proskorovsky I, Benedict A. Simulation and matching-based approaches for indirect comparison of treatments. Pharmacoeconomics. Jun 2015;33(6):537-49. doi:https://doi.org/10.1007/s40273-015-0271-1 
7. Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. Methods for Population-Adjusted Indirect Comparisons in Health Technology Appraisal. Medical Decision Making. 2018;38(2):200-211. doi:https://doi.org/10.1177/0272989x17725740 
8. Phillippo DM, Dias S, Ades AE, Welton NJ. Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study. Statistics in Medicine. 2020;39(30):4885-4911. doi:https://doi.org/10.1002/sim.8759 
9. Remiro-Azócar A. Two-stage matching-adjusted indirect comparison. BMC Med Res Methodol. Aug 8 2022;22(1):217. doi:https://doi.org/10.1186/s12874-022-01692-9 
10. Hatswell AJ, Freemantle N, Baio G. The Effects of Model Misspecification in Unanchored Matching-Adjusted Indirect Comparison: Results of a Simulation Study. Value Health. Jun 2020;23(6):751-759. doi:https://doi.org/10.1016/j.jval.2020.02.008 
11. Remiro-Azócar A, Heath A, Baio G. Methods for population adjustment with limited access to individual patient data: A review and simulation study. Res Synth Methods. Nov 2021;12(6):750-775. doi:https://doi.org/10.1002/jrsm.1511 
12. Cassidy O, Harte M, Trela-Larsen L, et al. A Comparison of Relative-Efficacy Estimate(S) Derived From Both Matching-Adjusted Indirect Comparisons and Standard Anchored Indirect Treatment Comparisons: A Review of Matching-Adjusted Indirect Comparisons. Value in Health. 2023;26(11):1665-1674. doi:https://doi.org/10.1016/j.jval.2023.07.001 
13. Jiang Z, Liu J, Alemayehu D, et al. A critical assessment of matching-adjusted indirect comparisons in relation to target populations. Res Synth Methods. May 2025;16(3):569-574. doi:https://doi.org/10.1017/rsm.2025.10 
14. Ren S, Ren S, Welton NJ, Strong M. Quantitative bias analysis for unmeasured confounding in unanchored population-adjusted indirect comparisons. Research Synthesis Methods. 2025;16(3):509-527. doi:https://doi.org/10.1017/rsm.2025.13 
15. Member State Coordination Group on Health Technology Assessment. Practical guideline for quantitative evidence synthesis: direct and indirect comparisons. 2024. https://health.ec.europa.eu/latest-updates/practical-guideline-quantitative-evidence-synthesis-direct-and-indirect-comparisons-2024-03-25_en 
16. Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Statistics in Medicine. 2009/11/10 2009;28(25):3083-3107. doi:https://doi.org/10.1002/sim.3697 
17. Chandler CO, Proskorovsky I. Uncertain about uncertainty in matching-adjusted indirect comparisons? A simulation study to compare methods for variance estimation. Res Synth Methods. Nov 2024;15(6):1094-1110. doi:https://doi.org/10.1002/jrsm.1759 
18. Jackson D, Rhodes K, Ouwens M. Alternative weighting schemes when performing matching-adjusted indirect comparisons. Res Synth Methods. May 2021;12(3):333-346. doi:https://doi.org/10.1002/jrsm.1466 
19. VanderWeele TJ, Ding P. Sensitivity Analysis in Observational Research: Introducing the E-Value. Annals of Internal Medicine. 2017/08/15 2017;167(4):268-274. doi:https://doi.org/10.7326/M16-2607 
20. Jaksa A, Louder A, Maksymiuk C, et al. A Comparison of Seven Oncology External Control Arm Case Studies: Critiques From Regulatory and Health Technology Assessment Agencies. Value Health. Dec 2022;25(12):1967-1976. doi:https://doi.org/10.1016/j.jval.2022.05.016 

Return to Insights Center