Conquer Magazine: How Real-World Evidence Has the Potential to Shape Real-World Outcomes in Multiple Myeloma
As seen on Conquer Magazine (special issue sponsored by Takeda Oncology)
For patients with cancer, participating in a clinical trial is one way in which they can contribute to the development of new medications for their disease. For decades, randomized controlled trials, often known as RCTs, have been the gold standard for confirming that a new medicine is both safe and effective. But what about those patients who are unable to participate in a clinical trial? Is there any way in which their experience can be captured and used to inform treatment decisions? In recent years, advances in technology, such as the switch to electronic patient records and the use of smartphones, have allowed everyday patients to generate large amounts of portable healthcare data. Today, healthcare professionals (HCPs), scientists, and regulatory agencies, among others, are looking at how to use these data to help us understand how well certain medications work in routine clinical practice. Collected from large numbers of patients, this realworld evidence (RWE) is helping to address clinical questions, providing additional understanding of treatment risks and benefits in routine clinical practice, and allowing current patients to potentially help improve disease management for future patients.
RWE is having an impact across all fields of medicine. Here, through interviews with patient advocates and industry leaders, we show how ongoing programs are generating RWE to support treatment decisions for patients with multiple myeloma.
What Is Multiple Myeloma?
Multiple myeloma is a kind of hematologic (blood) cancer that affects a type of blood cell called a plasma cell, which is located in the bone marrow.1 The bone marrow—the soft tissue inside the bones— is where blood cells, including plasma cells, redblood cells, white blood cells, and platelets, grow and mature.1-3
Survival rates among patients with multiple myeloma have improved dramatically over the past 20 years.3 These improved outcomes have largely been the result of the approval of new therapies and treatment regimens. Since 2015, a total of 7 new agents have been approved for the treatment of multiple myeloma in either the United States or Europe.3-5 With more new agents becoming available, the number of possible therapeutic combinations from which to select has also increased. With more choices available for patients with newly diagnosed and with relapsed/refractory multiple myeloma, making a treatment decision can be challenging. This is particularly true outside of major academic research centers and in community oncology practices, which is where most patients are treated. In these community practices, physicians may not be specialists in any one type of cancer, instead treating many different forms of cancer.
What Is Real-World Evidence?
Real-world evidence is information on the safety, effectiveness, and actual use of a medication in routine clinical practice, outside of clinical trials.
Why Are Clinical Trials Important?
RCTs allow researchers to confirm the efficacy and safety of new treatments. With this type of study, the participants are randomized, or divided by chance, into separate treatment groups: (1) an experimental treatment group, in which patients are treated with the treatment regimen under investigation, and (2) a control group, in which patients receive the current standard treatment (Figure 1). Randomization helps to ensure that the treatment groups will be similar in terms of patient characteristics and that the effects of the treatments can be compared more fairly. All new prescription medications must be shown to be safe and effective before they are approved for use in patients, and this is most often demonstrated with the use of RCTs.
Although clinical trials have been fundamental to the advancement of modern medicine, there are some limitations when the study results are later applied to routine clinical practice. For example, one key feature of an RCT is that only patients who fit a certain strict profile are eligible to participate, and these patients are then consistently treated according to a set procedure throughout the duration of the trial. This level of control is necessary, so that unintended factors do not affect the conclusions about the efficacy and safety of a particular treatment. “It is important to isolate the effects of the treatment, so the design characteristics of trials ensure that patients are treated in a particular way,” noted Dr. Dasha Cherepanov, Associate Director in the Global Outcomes Research and Epidemiology Group at Takeda.
A disadvantage of this approach is that many patients have characteristics or other medical conditions that disqualify them from participating in clinical trials. Thus, new drugs may not be directly evaluated in some types of patients. For example, multiple myeloma trials often exclude patients who are unable to care for themselves, have impaired kidney function, or have weakened immune systems. 6 Therefore, patients enrolled in clinical trials tend to be healthier than the average patient with multiple myeloma. In fact, studies have shown that approximately 40% to 70% of patients with multiple myeloma would not be eligible to participate in most RCTs.3,7 “Most treatment clinical trials have chosen groups of patients—they’re highly selected,” said Dr. James Omel, a multiple myeloma survivor and patient advocate. “Real-world patients would often not match study inclusion criteria for randomized cancer treatment trials.” Partly because of this selection process, the efficacy and tolerability of antimyeloma agents reported in clinical trials do not always translate directly to what is observed in the real world.7,8
Why Can’t All Patients Participate in Every Clinical Trial?
To be eligible to participate in a clinical trial, patients need to have certain characteristics. Also known as inclusion criteria, these are the features that a patient must have, such as a specific type of cancer, in order to participate in a particular study. There may also be exclusion criteria, or certain features that would prevent a patient from participating in a particular study.
For example, patients with a history of kidney failure or reduced kidney function may be excluded from a clinical trial, as proper kidney function is important for the normal elimination of many medications from the body. In this case, by only including those patients with normal kidney function, researchers can be more confident that patients in the trial will have about the same amount of the drug in their bloodstream.
This level of control allows researchers to limit the number of factors that might affect their study, thus enabling them to better understand the effects of the medication itself.
How Can We Learn From Real-World Evidence?
Real-world data (RWD) can help us learn how medicines work in patients outside of clinical trials. What is RWD, and where does it come from? RWD are data generated during daily life, outside the scope of RCTs. This means that the data recorded during a typical visit to an HCP, such as blood pressure readings, clinical laboratory values, and medication dosing, contribute to RWD. In addition to patient medical records, RWD can also be collected from many other sources, including medical records, insurance claims databases, patient registries, mobile and wearable devices, observational studies, and patient and caregiver surveys, all of which are defined in Figure 2.9,10
By analyzing RWD, researchers can then generate real-world evidence (often called RWE). One can say that RWE comprises the lessons learned from the analysis of RWD. According to Dr. Dawn Marie Stull, Scientific Director and lead of the INSIGHT MM observational study at Takeda Oncology, “RWE is important in terms of understanding the effectiveness of treatments and whether they are tolerable in a broader population of patients than can be enrolled in an RCT.” For example, researchers can ask such questions as, “How many patients with multiple myeloma need to change their medication dose because of side effects?” or “How well do elderly patients with multiple myeloma tolerate this medication?”
Compared with an RCT, RWE provides insights into what occurs in actual clinical practice (see Table).11 “RWE gives a more complete picture and understanding of practical treatment considerations,” Dr. Stull explained. With the use of RWE, effectiveness, patient adherence (that is, how well patients take their medication as prescribed by their physician), long-term patient outcomes, and new safety issues can be revealed for larger numbers of patients. This can be particularly helpful when trying to find differences among drugs that have not been directly compared in a clinical trial.
An Example of the Power of Real-World Evidence
Real-world evidence is not a new concept; rather, it has been contributing to improved cancer care for decades. One commonly cited example relates to the use of bisphosphonates—a class of medications for the treatment of multiple myeloma and certain forms of cancer that cause bone damage.
Many randomized controlled trials have demonstrated that bisphosphonates are important medications for the prevention of bone fractures and can help to eliminate pain in patients with multiple myeloma.12 It was only when 36 dental patients in Florida who had been taking these medications developed a rare, serious jaw complication, however, that the association with the longer-term use of bisphosphonates was recognized.13 With this new information available, we now know how to reduce the risk for this jaw complication. Patients should undergo a thorough dental examination before beginning treatment with bisphosphonates and should be encouraged to maintain good oral hygiene during their treatment.12,14
The value of RWE extends beyond assessing the effectiveness of a particular medication: RWE can help HCPs understand the practical and logistical issues that patients and their caregivers face during treatment, and how their quality of life can be affected (see An Example of the Power of Real-World Evidence on page 5).12-14 According to Dr. Stull, by understanding how medicines work in the real world, researchers and HCPs can identify areas that need improvement. She added that the discoveries made with RWE can become the basis for new RCTs.
Regulatory agencies, such as the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA), as well as health authorities in Japan, China, and Canada, have begun to recognize the value of RWE. In fact, both the FDA and EMA have published guidance on the use of RWE to support clinical trial data.15-17 Insurance companies and other healthcare payers have also begun recognizing the value of RWE.18,19 “Using RWE is a field that is gaining more and more traction,” said Dr. Cherepanov. “Clinical trials are still the gold standard for assessing drug efficacy and safety, but RWE is an important component and complement to that gold standard.” Although the use of RWE is growing in all aspects of healthcare, room still exists for greater incorporation of RWE into treatment decision-making.
As with clinical trials, there are some limitations with RWE studies as well. One such limitation is that real-world studies are not randomized, meaning that known and unknown human characteristics, and even choices, can affect the study results. For example, HCPs may, purposefully or without intending to, prescribe one treatment for healthier patients and another treatment for sicker patients. Also, unlike in those phase 3 clinical trials that are double-blind, both HCPs and patients in RWE studies know which treatment is being used. This may influence how outcomes are reported. Finally, the ever- present challenge exists of missing information in RWD. For example, HCPs may not record a patient’s height, weight, or blood pressure at all office visits.10 Knowing the limitations of both clinical trials and RWE is important for their appropriate use. “The first step in addressing these challenges is to ensure that the readers and users of these data…understand that these results [from real-world and clinical trials] complement each other and enhance our understanding of how therapies perform in a broader population of patients,” Dr. Stull indicated.
Collecting and maintaining reliable RWD is important for generating high-quality RWE. Often, large amounts of data on a medical condition can be obtained from patient registries, which collect standardized information on outcomes from a group of patients who share a common medical condition or experience. The quality of these data can be highly variable, depending on where they are collected. “Europe consists of 30-plus countries. Each of them has different data sets and registries,” noted Emilie Prazakova, Patient Advocacy Lead at Takeda in the EU region. “Not all countries have good registries for real-world evidence.” The creation of larger, better-maintained data sets is one way in which to overcome these problems. Within the multiple myeloma community, a number of initiatives, including INSIGHT MM, HealthTree, and Noona, are doing just that.
Insight MM Observational Study
The INSIGHT MM observational study is evaluating the real-world effectiveness of treatments for patients with multiple myeloma from around the world (Figure 3).3 Designed to study more than 4,000 patients with newly diagnosed or relapsed/refractory multiple myeloma in 15 countries for at least 5 years, INSIGHT MM is the largest observational study of its kind to date. This real-world study was initiated in response to the rapidly changing and growing complexity of the multiple myeloma treatment landscape. As Dr. Stull explains, “The data collected in INSIGHT MM will allow us to gain a better understanding of how multiple myeloma is managed globally, assess the impact of different therapies on patient outcomes, and identify gaps in treatment, both at the country level and on a global basis.” In addition, the study collects data submitted directly by patients. These data, referred to as patient-reported outcomes, include such issues as quality of life (that is, how patients perceive their physical and mental health over time) and the use of healthcare resources. INSIGHT MM is also exploring relationships among disease characteristics, symptoms, treatment patterns, and clinical outcomes.
Since the onset of the INSIGHT MM study in 2016, researchers have already observed differing trends in multiple myeloma treatment. “We have learned that there really is no standard of care in how patients with multiple myeloma are treated globally, which is more predominant in the relapsed/refractory setting than in the newly diagnosed setting,” said Dr. Stull. “This might be attributed to differences in drug availability or to the treatment guidelines that are available.”
Many patients who would be ineligible to participate in an RCT are contributing to the findings of the INSIGHT MM study. Indeed, a 2019 analysis has shown that nearly 40% of the patients included in INSIGHT MM would not have been eligible to enroll in RCTs.20 In particular, patients with a history of other cancers, decreased kidney function, or an irregular heartbeat would likely be excluded from other studies, but are eligible for INSIGHT MM.20
With its relatively few eligibility criteria, the INSIGHT MM study is allowing a broader population of adult patients with documented information about their disease and treatment to participate. This is particularly important, as multiple myeloma is a disease that varies significantly from patient to patient, requiring that treatments be tailored to each individual. Because of this variability, it is difficult to define one global standard of care. With its large collection of data, INSIGHT MM has highlighted other challenges faced by patients in different regions of the world. One particular challenge is access to effective treatment. As Dr. Omel, who is also a member of the INSIGHT MM steering committee, explained, “Part of the problem for patients in Europe, Asia, and absolutely in Latin America, is that they don’t have access to many of the drugs that we have in the United States…in general, we don’t have the accessibility issues that they have in these other parts of the world.” These accessibility issues are a significant problem for patients with multiple myeloma, among whom the best outcomes depend on receiving combinations of treatments, one after the other. In those situations in which patients have fewer available options, treatment strategies may be more limited and result in poorer outcomes. “They use the drug that they’re taking to try to get as much benefit out of it as they can, and they don’t have other agents to back it up,” Dr. Omel pointed out.
INSIGHT MM at a Glance
The goal of INSIGHT MM is to identify treatment patterns based on patient and disease characteristics to achieve improved outcomes in individuals with multiple myeloma.
|Start date:||July 2016|
|Number of patients:||Approximately 4,200 patients with newly diagnosed or relapsed/refractory multiple myeloma|
|Countries:||Belgium, Brazil, China, Colombia, France, Germany, Greece, Israel, Italy, Mexico, Spain, Taiwan, Turkey, United Kingdom, United States|
|How to get involved:||INSIGHT MM is no longer enrolling new patients, but new data derived from the study are published regularly. Study details and results can be followed using the ClinicalTrials.gov identification code NCT02761187.|
Results collected from INSIGHT MM are shared regularly with the multiple myeloma community, to advance our understanding of how patients with multiple myeloma are treated, and the outcomes when certain therapies are used as part of routine clinical practice. Clinicians can also request data directly by contacting the INSIGHT MM study steering committee. By sharing this important information on how to best use treatments to improve patient outcomes, INSIGHT MM is not only helping today’s patients, but is also contributing to the design of future studies. Assessing the results from both RCTs and from real-world studies such as INSIGHT MM will allow us to better understand the effectiveness of treatments and to help reveal patients’ unmet medical needs.
HealthTree is a software tool that allows patients with multiple myeloma to enter their own information and merge it into a large patient network. It also provides patients with access to a number of interactive educational and healthcare tools (Figure 4).21 “You can track your disease. You can see your personally relevant treatment options at your stage…you can see clinical trials that you can join,” noted Jenny Ahlstrom, a survivor of multiple myeloma who is also a patient advocate, founder of Myeloma Crowd, and creator of HealthTree.
By giving patients the opportunity to input their own data, HealthTree is designed to capture information that other sources of real-world data may miss. According to Ms. Ahlstrom, “We realized that electronic health records contained only about 8% of a patient’s complete data.” Questions such as, “What can you do on your own?” “Do you have an autoimmune disorder?” and “Do you have quality- of-life issues?” are not commonly asked or recorded by HCPs. Making matters even worse, the data that are collected can often be spread across numerous different databases and may not include the information that patients have provided. HealthTree provides a single database that is populated by patients and can be linked to their online electronic healthcare records on their mobile devices, which can then help them and their healthcare team work together during treatment. Patient profiles are further validated against both the paper and the online medical records.
HealthTree at a Glance
By providing patients with the opportunity to input their own data, HealthTree is designed to capture information that may be overlooked by other sources of real-world data.
|Start date:||October 2018|
|Number of patients:||More than 5,000 patients with multiple myeloma|
|How to get involved:||Patients, caregivers, or physicians interested in participating in HealthTree can do so directly by accessing the website www.healthtree.org.|
Researchers can also access the HealthTree database to create RWE. By analyzing the survey data from the more than 5,000 US-based patients registered in HealthTree, investigators can pose new scientific questions. “For example, one of the doctors said, ‘I want to know if patients have been vaccinated after myeloma therapy,’” Ms. Ahlstrom indicated. “He asked about the flu, he asked about shingles, he asked about pneumonia vaccines… about everything. We were able to get 550 patient responses in a few weeks.” Other ongoing studies are exploring possible links between multiple myeloma and the microbiome (that is, the native bacteria that live on and inside our bodies) or immune- mediated disorders such as the chronic skin condition psoriasis. Because RWD provide information on large numbers of patients, even potentially rare combinations of conditions (for example, having both multiple myeloma and psoriasis) can be studied in a way that is impossible to do in a clinical trial. This is one way in which RWD are giving a voice to the voiceless.
What Can Users Do on HealthTree?
HealthTree offers patients, caregivers, and physicians a number of ways in which to participate in the multiple myeloma community. Some features of the platform include the following21:
- Track My Myeloma
- See My Treatment Options
- Learn at HealthTree University
- Find and Connect with Other Participants
- Find Relevant Clinical Trials
- Find Financial Assistance
- View Reports and Data
- Accelerate Myeloma Research
- Share Your Myeloma Story
- Donate Records
Challenges with HealthTree are similar to those with other programs that rely on patient-reported data. One obstacle has been how to make contact with “difficult-to-reach” patients. Because patients with multiple myeloma tend to be older, they may not be as active or as comfortable with using these types of online platforms or with sharing their medical history. Another challenge is ensuring that patient- reported data are sound enough to be able to draw strong conclusions. “What we’re working on now is more data reporting, more data validation… going back and making sure we’ve validated the data against their medical records and their chart,” said Ms. Ahlstrom.
With enhanced reliability of the database, Ms. Ahlstrom is looking forward to expanding the use of HealthTree in multiple myeloma research. “We want to thoroughly validate our data. We want the researchers to trust the insights and have confidence that the data have undergone a vetting process. We are building a long-term relationship with patients over the lifetime of their disease,” she said.
Noona is a third-party Internet-based service for patients with cancer that is designed to capture patient-reported outcomes. To date, the service has been used by more than 10,000 patients with cancer in 10 large cancer centers across the United States and the European Union. Noona was created with the goal of taking advantage of the high-quality healthcare data collected in Finland and develop a system to drive earlier interventions and better outcomes for patients with cancer. Initially tested in patients with breast cancer, the service has since been rolled out to patients with multiple myeloma, in a partnership between Varian (the owner of Noona) and Takeda. “After 1 or 2 years of piloting, the Finnish pilot clinics have enrolled more than 200 myeloma patients onto the Noona platform,” said Carmen Cucul, Customer Experience and Learning & Development Lead at Takeda. “Based on that [success], we are now scaling up in several European countries.”
Noona at a Glance
Noona allows an oncology patient’s healthcare team access to information on real-time symptoms, thus enabling them to recognize early warning signs, provide patients with self-care instructions, schedule a clinic visit, or change a prescription.
|Number of patients:||Approximately 200 patients with multiple myeloma|
|Countries:||Finland, Germany, Spain (active as of time of publication) Switzerland, The Netherlands (wave 2)|
|How to get involved:||Patients from select clinics from the above countries can participate in the Noona project by speaking with their care team for details.|
Noona offers patients the ability to monitor their symptoms in real time and to communicate directly with their healthcare team (Figure 5). These features are designed to improve clinical care while generating RWD at the same time. Like the HealthTree program, Noona is based on outcomes reported by the patient rather than on outcomes reported by the HCP. This allows the patient to be an active participant in his or her own care.
Asking patients about their symptoms at office visits has its limitations. For example, patients often remember recent symptoms better and experience a difficult time describing symptoms to their HCP. “Even a patient diary or patient monitoring checklist… it’s not the most up-to-speed monitoring,” Ms. Prazakova explained. “[Noona] is almost 24-hoursa- day, real-time monitoring,” she added, “and we’ve seen that continuous symptom monitoring in other cancers can improve outcomes for patients.” In addition to forgetting information during the time between clinic visits, patients can be unsure of the severity of their symptoms or experience difficulty providing answers to medical questions. Furthermore, care teams are unable to adjust therapy when a patient is at home. They may not be able to assess the severity of a patient’s complaint by phone and may be unable to deal with unscheduled phone calls or clinic visits.
Noona addresses many of these challenges faced by patients and their care teams. Patients can report their symptoms from home at any time and have access to their healthcare team via the platform. Most importantly, they only need to click on their symptom and the severity without having to describe anything or to think of the right word. For care teams, Noona allows access to real-time symptom information, thus enabling them to recognize early warning signs, provide patients with selfcare instructions, schedule a clinic visit, or eventually change a prescription.
Studies have shown that programs such as Noona, which provides real-time tracking of patient- reported symptoms, have many benefits in routine cancer care. Although not yet demonstrated in patients with multiple myeloma, researchers in both the United States and Europe have reported that such programs can improve patient survival, enhance quality of life, and lead to fewer emergency department visits and hospitalizations compared with standard clinical follow-up.22-24 Patients using Noona have also reported emotional benefits, including greater peace of mind, as well as a feeling of greater control over their own health and treatment management. “If you are using Noona, you have, in a way, an open channel of communication. It’s not like you would be able to chat with your nurse immediately, but what we have heard from the patients is that the response is something between 2 to 4 hours, on average,” Ms. Prazakova explained. “That gives you a pretty fast response. It also gives you a different feeling of security…of being taken care of.” Patients have also highlighted the self-explanatory, user-friendly nature of the Noona application.
HCPs have also observed benefits with use of the Noona platform. Care teams can track a patient’s symptoms over a long period of time, which allows them to make improved treatment decisions and permits interventions outside of a scheduled visit. Communication between patients and their healthcare team is therefore made more efficient, with better identification of those patients in need of closer follow-up, along with overall time-savings because of fewer phone calls, emergency department visits, and hospitalizations.
The Noona project in multiple myeloma is currently ongoing, with plans to continue European expansion outside of Finland. “We plan to have, by end of this year , up to 10 clinics with approximately 50 patients each,” said Ms. Cucul. “Why do we hope to get such large number of patients in this program? Because then we start kicking in with the evidence-generation part of the project.” Moving forward, Takeda Oncology has plans to leverage the electronic patient-reported outcomes (ePROs) collected in the Noona platform: symptoms and quality of life, to be more precise. At a later stage, the company might decide to sponsor studies linking these ePROs with clinical and treatment-related data from patients’ health records. If pursued, such studies could investigate the effectiveness of treatments for multiple myeloma and whether symptom monitoring through the Noona platform improves such outcomes, in addition to increasing patient and healthcare provider satisfaction with care, and reducing the use of healthcare resources.
In patients with multiple myeloma, approved treatments have the potential to help them lead longer, healthier lives. Although RCTs have been and will continue to be critical for testing the efficacy and safety of these agents, RWE is necessary to confirm their effectiveness in the broad patient population observed in routine clinical practice. In other words, RCTs and RWE can be used together to better understand how treatments work across various patient types. Similar to building a structure, RCTs provide the foundation, whereas RWE provides more details. Only by evaluating all of the available data will HCPs become fully informed on the best combinations and order of these treatments. The number of questions that can be addressed with robust RWE is numerous. Patients can contribute to these efforts either by collaborating directly with programs such as HealthTree or by speaking with their care team about opportunities in their clinical area. Patient organizations and online communities may provide additional opportunities for patients to share their experience.
- American Cancer Society. What is multiple myeloma? Updated February 28, 2018. www.cancer.org/cancer/multiple-myeloma/about/what-is-multiplemyeloma. html. Accessed January 20, 2020.
- National Cancer Institute. Plasma cell neoplasms (including multiple myeloma) treatment (PDQ)-patient version. Updated November 8, 2019. www.cancer.gov/types/myeloma/patient/myeloma-treatment-pdq. Accessed March 24, 2020.
- Costello C, Davies FE, Cook G, et al. INSIGHT MM: a large, global, prospective, non-interventional, real-world study of patients with multiple myeloma. Future Oncol. 2019;15:1411-1428.
- US Food and Drug Administration. FDA grants accelerated approval to selinexor for multiple myeloma. July 3, 2019. www.fda.gov/drugs/resourcesinformation-approved-drugs/fda-grants-accelerated-approval-selinexormultiple-myeloma. Accessed January 20, 2020.
- Sarclisa (isatuximab-irfc) [prescribing information]. Bridgewater, NJ: sanofi- aventis U.S. LLC; March 2020.
- Huang H. Multiple myeloma: out of the clinic and into the (real) world. Pharmafile. 2018;65:13-14.
- Chari A, Romanus D, Palumbo A, et al. Randomized clinical trial representativeness and outcomes in real-world patients: comparison of 6 hallmark randomized clinical trials of relapsed/refractory multiple myeloma. Clin Lymphoma Myeloma Leuk. 2020;20:8-17.
- Richardson PG, San Miguel JF, Moreau P, et al. Interpreting clinical trial data in multiple myeloma: translating findings to the real-world setting. Blood Cancer J. 2018;8:109.
- Gottlieb S. FDA budget matters: a cross-cutting data enterprise for real world evidence. June 10, 2018. www.fda.gov/news-events/fda-voicesperspectives-fda-leadership-and-experts/fda-budget-matters-crosscutting-data-enterprise-real-world-evidence. Accessed January 20, 2020.
- McDonald L, Malcolm B, Ramagopalan S, Syrad H. Real-world data and the patient perspective: the PROmise of social media? BMC Med. 2019;17:11.
- Kim HS, Lee S, Kim JH. Real-world evidence versus randomized controlled trial: clinical research based on electronic medical records. J Korean Med Sci. 2018;33:e213.
- Terpos E, Roodman GD, Dimopoulos MA. Optimal use of bisphosphonates in patients with multiple myeloma. Blood. 2013;121:3325-3328.
- Marx RE. Pamidronate (Aredia) and zoledronate (Zometa) induced avascular necrosis of the jaws: a growing epidemic. J Oral Maxillofac Surg. 2003;61:1115-1117.
- Dimopoulos MA, Kastritis E, Bamia C, et al. Reduction of osteonecrosis of the jaw (ONJ) after implementation of preventive measures in patients with multiple myeloma treated with zoledronic acid. Ann Oncol. 2009;20:117-120.
- US Food and Drug Administration. Submitting documents using realworld data and real-world evidence to FDA for drugs and biologics: guidance for industry. May 2019. www.fda.gov/media/124795/download. Accessed January 20, 2020.
- US Food and Drug Administration. Framework for FDA’s Real-World Evidence Program. December 2018. www.fda.gov/media/120060/download. Accessed January 15, 2020.
- European Medicines Agency. Real world evidence (RWE) - an introduction; how is it relevant for the medicines regulatory system? April 2018. www.ema.europa.eu/en/documents/presentation/presentation-real-worldevidence-rwe-introduction-how-it-relevant-medicines-regulatory-systememas_en.pdf. Accessed January 15, 2020.
- Malone DC, Brown M, Hurwitz JT, et al. Real-world evidence: useful in the real world of US payer decision making? How? When? And what studies? Value Health. 2018;21:326-333.
- Bruce F. Real-world evidence and the quest for European market access. November 22, 2016. https://invivo.pharmaintelligence.informa.com/IV004954/RealWorld-Evidence-And-The-Quest-For-European-Market-Access. Accessed January 15, 2020.
- Hungria VTM, Lee HC, Abonour R, et al. Real-world multiple myeloma patients remain under-represented in clinical trials based on standard laboratory parameters and baseline characteristics: analysis of over 3,000 patients from the INSIGHT MM global, prospective, observational study. Presented at 61st annual meeting of the American Society of Hematology; December 7-10, 2019; Orlando, FL. Abstract 1887.
- Healthtree. Home page. www.healthtree.org/. Accessed January 15, 2020.
- Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. 2017;318:197-198.
- Basch E, Deal AM, Kris MG, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34:557-565. Errata in: J Clin Oncol. 2016;34:2198; J Clin Oncol. 2019;37:528.
- Denis F, Lethrosne C, Pourel N, et al. Randomized trial comparing a web-mediated follow-up with routine surveillance in lung cancer patients. J Natl Cancer Inst. 2017;109:djx029. Erratum in: J Natl Cancer Inst. 2018; 110:djy020.