As previously announced, we have started a project that will look at issues relating to “Methodological aspects of evaluating transferability of Real World Evidence results between countries / settings”.
As part of this project, we will have a blog-series about the “illusion of simplicity in the transferability of outcomes”. We believe that the issue of transferability is fundamental to the entire discussion about the use of real-world evidence by decision makers.
Our hypothesis is that the topic is more complex and important than what has been previously understood.
The discussion about the difference in results between RCT’s and real world, would tend to mainly be attributed to a difference in population, which is an important question. For some treatments this is the case and then it is rather simple to evaluate the issues of transferability. In general, this would be for interventions that have a stand-alone effect with limited impact from the healthcare delivery or the patient’s response.
On the contrary, we would suggest that for some interventions where the intervention is one of many, there is a large impact of healthcare delivery and patient response. Here, we are facing a much more complex question to understand the relevance of the data from a clinical trial in the real world.
In the coming weeks we are going to review a few disease areas to see if our hypothesis holds true. Based on a couple of examples, we have found an analysis of outcomes across regions in national cohort which has been controlled for differences in the population/case-mix.
If any of you have examples to contribute which illustrate the differences in outcomes, we would very much appreciate if you could share them:
- RCT vs. RWE results
- Differences in outcomes between countries.
The important questions are:
- How relevant are the results from RCT’s to understand the effectiveness in the local context?
- Is it possible to use the results from a study in Sweden, in the US or in Germany? Or rather how do we know and is it possible to control for differences that impact the outcomes.
Below is a hypothetical example to illustrate the key issue
An international RCT has been carried out with ‘standard of care’ as a comparator against a new drug. There is meaningful improvement in the outcomes.
An analysis has been done on the outcomes of the current standard of care in two countries (A + B) and the results indicate that the outcomes are better in country B than country A.
The question would then be:
- What relevance does the comparative efficacy in the RCT have in country B?
- Is it relevant to use the RWE study from country A in country B, and vice versa?
The example may seem irrelevant due to the obvious problem. However, due to differences in the health delivery, the outcomes may vary quite significantly. When we demonstrate the variability in outcomes in the upcoming blogs, it will become clear why this is indeed a very relevant question.
Is this a relevant question? Please engage in the discussion by joining our linkedin group about the topic.