Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that enables research into pragmatic trials. It shares clean trial data and ratings using PRECIS-2, allowing for multiple and diverse meta-epidemiological studies to compare treatment effects estimates across trials with different levels of pragmatism as well as other design features.
Background
Pragmatic trials are increasingly acknowledged as providing evidence from the real world for clinical decision-making. The term "pragmatic", however, is a word that is often used in contradiction and its definition and measurement need further clarification. Pragmatic trials should be designed to inform clinical practice and policy decisions, rather than to prove an hypothesis that is based on a clinical or physiological basis. A pragmatic trial should also aim to be as similar to the real-world clinical environment as possible, including in the selection of participants, setting and design of the intervention, its delivery and implementation of the intervention, determination and analysis of outcomes as well as primary analyses. This is a key distinction from explanatory trials (as described by Schwartz and Lellouch1) which are intended to provide a more thorough proof of the hypothesis.
Trials that are truly pragmatic must not attempt to blind participants or clinicians, as this may lead to bias in estimates of treatment effects. Practical trials should also aim to recruit patients from a wide range of health care settings, so that their results can be applied to the real world.
Finally, pragmatic trials must concentrate on outcomes that are important to patients, such as the quality of life and functional recovery. This is particularly relevant when trials involve the use of invasive procedures or could have dangerous adverse consequences. The CRASH trial29 compared a 2-page report with an electronic monitoring system for patients in hospitals with chronic cardiac failure. The catheter trial28 on the other hand was based on symptomatic catheter-related urinary tract infection as the primary outcome.
In addition to these characteristics pragmatic trials should also reduce the requirements for data collection and trial procedures to cut costs and time commitments. In the end these trials should strive to make their results as relevant to real-world clinical practices as possible. This can be achieved by ensuring that their analysis is based on the intention to treat approach (as described within CONSORT extensions).
Despite these requirements, many RCTs with features that defy pragmatism have been incorrectly self-labeled pragmatic and published in journals of all kinds. This could lead to false claims of pragmatism and the term's use should be standardized. The creation of the PRECIS-2 tool, which offers a standard objective assessment of practical features is a great first step.
Methods
In a practical study, the goal is to inform policy or clinical decisions by showing how an intervention could be integrated into routine treatment in real-world settings. This is distinct from explanation trials, which test hypotheses about the causal-effect relationship in idealized conditions. Therefore, pragmatic trials might have less internal validity than explanatory trials, and could be more susceptible to bias in their design, conduct, and analysis. Despite these limitations, pragmatic trials can contribute valuable information to decision-making in the context of healthcare.
The PRECIS-2 tool evaluates the degree of pragmatism in an RCT by assessing it across 9 domains ranging from 1 (very explicit) to 5 (very pragmatic). In this study the domains of recruitment, organisation, flexibility in delivery, flexibility in adherence, and follow-up received high scores. However, the principal outcome and method of missing data were scored below the practical limit. This suggests that it is possible to design a trial with excellent pragmatic features without damaging the quality of its results.
It is hard to determine the amount of pragmatism that is present in a study because pragmatism is not a have a binary characteristic. Some aspects of a research study can be more pragmatic than others. Additionally, logistical or protocol modifications made during the trial may alter its score in pragmatism. Koppenaal and colleagues discovered that 36% of the 89 pragmatic studies were placebo-controlled or conducted prior to licensing. Most were also single-center. This means that they are not very close to usual practice and can only be called pragmatic in the event that their sponsors are supportive of the lack of blinding in these trials.
Another common aspect of pragmatic trials is that the researchers try to make their results more meaningful by analysing subgroups of the trial sample. This can result in imbalanced analyses and lower statistical power. This increases the chance of missing or misdetecting differences in the primary outcomes. In the case of the pragmatic trials included in this meta-analysis, this was a significant problem since the secondary outcomes were not adjusted for variations in the baseline covariates.
Furthermore, pragmatic studies may pose challenges to gathering and interpretation of safety data. It is because adverse events are usually self-reported, and therefore are prone to delays, inaccuracies or coding errors. It is crucial to improve the quality and accuracy of outcomes in these trials.
Results
While the definition of pragmatism may not require that all clinical trials are 100% pragmatist, there are benefits to including pragmatic components in trials. These include:
Increasing sensitivity to real-world issues, reducing cost and size of the study and allowing the study results to be more quickly implemented into clinical practice (by including patients who are routinely treated). However, pragmatic trials may also have disadvantages. The right amount of heterogeneity for instance could help a study expand its findings to different settings or patients. However the wrong kind of heterogeneity can reduce the sensitivity of an assay and thus reduce a trial's power to detect small treatment effects.
A variety of studies have attempted to classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 developed a framework for distinguishing between research studies that prove a clinical or physiological hypothesis, and pragmatic trials that help in the selection of appropriate therapies in the real-world clinical setting. The framework consisted of nine domains that were evaluated on a scale of 1-5 with 1 being more explanatory while 5 was more pragmatic. The domains were recruitment setting, setting, intervention delivery and follow-up, as well as flexible adherence and primary analysis.
The original PRECIS tool3 was built on the same scale and domains. Koppenaal et. al10 devised an adaptation of the assessment, called the Pragmascope that was simpler to use for systematic reviews. They discovered that pragmatic reviews scored higher on average in all domains, but scored lower in the primary analysis domain.
This distinction in the primary analysis domain can be due to the way in which most pragmatic trials analyze data. Certain explanatory trials however don't. The overall score was lower for systematic reviews that were pragmatic when the domains of the organization, flexibility of delivery and follow-up were combined.
It is crucial to keep in mind that a pragmatic study should not mean a low-quality trial. In fact, there is a growing number of clinical trials that use the word 'pragmatic,' either in their title or abstract (as defined by MEDLINE however it is neither sensitive nor precise). The use of these terms in titles and abstracts could indicate a greater understanding of the importance of pragmatism, but it is unclear whether this is manifested in the contents of the articles.
Conclusions
As the value of evidence from the real world becomes more popular and pragmatic trials have gained momentum in research. They are randomized trials that compare real world care alternatives to clinical trials in development. They involve patient populations more closely resembling those treated in regular medical care. This method is able to overcome the limitations of observational research for example, the biases that are associated with the use of volunteers and the limited availability and codes that vary in national registers.
Other advantages of pragmatic trials include the ability to use existing data sources, and a higher likelihood of detecting meaningful changes than traditional trials. However, these trials could still have limitations that undermine their validity and generalizability. For instance, participation rates in some trials could be lower than expected due to the healthy-volunteer effect and financial incentives or competition for participants from other research studies (e.g., industry trials). Many pragmatic trials are also limited by the need to recruit participants quickly. Some pragmatic trials also lack controls to ensure that observed differences aren't due to biases during the trial.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs self-labeled as pragmatist and published until 2022. 프라그마틱 무료스핀 -2 tool was employed to assess pragmatism. It covers domains such as eligibility criteria, recruitment flexibility as well as adherence to interventions and follow-up. They discovered that 14 of these trials scored as highly or pragmatic practical (i.e. scores of 5 or more) in one or more of these domains and that the majority of them were single-center.
Studies with high pragmatism scores tend to have more lenient criteria for eligibility than conventional RCTs. They also include patients from a variety of hospitals. The authors argue that these characteristics could make pragmatic trials more effective and useful for everyday clinical practice, however they do not necessarily guarantee that a pragmatic trial is free from bias. Furthermore, the pragmatism of trials is not a definite characteristic and a pragmatic trial that doesn't contain all the characteristics of an explanatory trial can produce valuable and reliable results.