Flawed meta analytic methodology is common in many fields such as oncology and. We have summarized the stateoftheart in the field, discussing the key assumptions of the statistical models and provided guidance for researchers interested in. The authors have published over twenty innovative meta analyses from the turn of the century till now. Outlines the role of meta analysis in the research process shows. There are a lot of issues one could discuss here but the factor i think is most. The historical roots of metaanalysis can be traced back to 17th century studies of astronomy, while a paper published in 1904 by the statistician karl pearson in the british medical journal which collated data from several studies of typhoid inoculation is seen as the first time a metaanalytic approach was used to aggregate the outcomes of multiple clinical studies. The validity of the random effects model used in the article is dependent on a large number of individual studies, each with sufficiently large samples. In fact, though, it is the meta analysis, which incorporates data from five randomized trials rather than one, that has the more powerful position. Special problems arise when metaanalysis is applied to quasiexperiments. Methodological considerations in network metaanalysis. He is the pi on several nih grants to develop software for meta analysis and is the developer, with larry hedges, julian higgins, hannah rothstein and others, of comprehensive meta analysis, a bestselling computer program for meta analysis. However, meta analysis of adverse events has associated methodological challenges.
This book provides a clear and thorough introduction to metaanalysis, the process of synthesizing data from a series of separate studies. Authors of this chapter have participated in authoring several of the software packages discussed in this chapter. It also utilizes robust metaanalysis methods so that many of the problems. Fourth, when a researcher includes lowquality studies in a metaanalysis, the limitations of these studies impact the mean effect size i. Having contributed chapters to two books on metaanalysis, she coedited publication bias in metaanalysis. Sampling is a foundational step in conducting any type of thorough research.
This letter, however, focuses on concerns about methodological issues in the meta. Figure 2 shows the results of a metaanalysis of metoclopramide compared with. Software for statistical metaanalysis 175 finally, there are standalone packages for metaanalysis that come in many different flavors. One may find it easy to read and comprehend the various conceptual and methodological issues related to metaanalysis and their applicability using r. Third, some scientists argue that the objective coding procedure used in metaanalysis ignores the context of each individual study, such as its methodological rigor. Publications on methodological issues cochrane comparing. Methodological issues in the metaanalysis of observational. Criticisms of metaanalysis introduction one number cannot summarize a research field the file drawer problem invalidates metaanalysis mixing apples and oranges garbage in, garbage out important studies are ignored metaanalysis can disagree with randomized trials metaanalyses are performed poorly. There are 3 types of heterogeneity commonly considered in metaanalysis. A meta analysis containing only old information may not be very useful. The second objective is to apply a meta analysis to determine whether the exhibited variation in the prevalence estimate is associated with methodological factors including the following. Oct 07, 2005 since the mid1990s, dr borenstein has lectured widely on meta analysis. There are good metaanalysis methods available, but even when they are carefully and optimally applied there remain some unresolved statistical issues. The first trial has a parallel design independent outcomes and the second trial has a crossover design.
Reviewers may choose to reject studies with such problems due to quality. Nov 26, 20 the statistical methods group has played a pivotal role in the cochrane collaboration over the past 20 years. Similar comments apply to other metaanalysis software. Methodological and clinical heterogeneity and extraction. Metaanalysis has become a critically important tool in fields as diverse as medicine, pharmacology, epidemiology, education, psychology, business, and ecology. Methodological issues in a metaanalysis original manuscript. The second objective is to apply a metaanalysis to determine whether the exhibited variation in the prevalence estimate is associated with methodological factors including the following. Methodologically, it involves the identification of where the study takes place e.
Careful consideration of threats to each studys validity may help the metaanalyst to avoid these pitfalls. To facilitate better understanding, each of the commonly used methods is covered with illustration using real data sets. The beckerbalagtas approach is an estimation method used in the metaanalysis of crossover trials with binary outcomes. Metaanalysis is a quantitative technique that uses specific measures e. Acknowledgment we cordially acknowledge public health evidence south asia phesa and dr. Unfortunately, problems among metaanalyses such as the. However, metaanalysis of adverse events has associated methodological challenges. Metaanalysis in biological sciences, especially in ecology and evolution which we refer to as biological metaanalysis faces somewhat different methodological problems from its counterparts in medical and. Background metaanalysis of randomized controlled trials rcts is usually based. Methodological issues in the meta analysis of observational studies. She has authored several metaanalyses as well as articles on methodological issues in the area, and made numerous presentations on the topic. Figure 2 shows the results of a meta analysis of metoclopramide compared with.
We investigated the methodological steps used by authors of srmas of clinical trials via an author survey. The nma can be as valid as a standard pairwise meta analysis if these methodological issues are taken care of. These random effects models and software packages mentioned above. The team which does a meta analysis needs to include persons. Besides the demographic characteristics, methodological questions regarding the source, extraction and synthesis of data were included. The 27item questionnaire was developed to study the methodological steps used by authors when conducting a sr ma and the demographic characteristics of the respondent. While the metaanalytic methodology is similar for systematic and rapid. In may 2010, we hosted a meeting on network metaanalysis methodology at the johns hopkins bloomberg school of public health. This qualitative interview study aimed to understand researchers understanding of complexity and heterogeneity and the factors which may influence the choices researchers.
Effect of metformin on clinical, metabolic and endocrine outcomes in women with polycystic ovary syndrome. All material and software is as is with no guarantees of functionality or correctness. Prevalence of psychotic disorders and its association with. Metaanalysis of erp investigations of pain empathy. The three meta analysis estimates look quite unimportant by comparison.
Our discussion document that describes all relevant methodologies for indirect comparisons suggested in the scientific literature to date. Effectiveness of probiotics in irritable bowel syndrome. The authors have published over twenty innovative metaanalyses from the turn of the century till now. If possible the results are statistically combined into a metaanalysis in which the data are weighted and pooled to produce an estimate of effect. Having contributed chapters to two books on meta analysis, she coedited publication bias in meta analysis.
Despite a recent flurry of publications related to network metaanalyses, only a handful of articles have focused on key methodological issues and most of these have covered statistical approaches 24, 816. Meta analysis, a set of statistical techniques for synthesizing the results of multiple studies, is used when the guiding research question focuses on a quantitative summary of study results. Metaanalysis in biological sciences, especially in ecology and evolution which we refer to as biological metaanalysis faces somewhat different methodological problems from its counterparts in medical and social sciences, where metaanalytic techniques were originally developed. Brunelle and colleagues 14 report a metaanalysis of large randomized clinical trials of insulin lispro versus regular insulin. The technique emphasizes results across multiple studies as opposed to results from a single investigation. We focus on those that are the most flexible and the most suited to the types of analyses carried out by ecologists and evolutionary biologists. Further, as new studies become available, metaanalyses need frequent updating. Jun 17, 2014 cytokine aberrations in autism spectrum disorder. Chs is a coauthor of metaanalyst, hr is a coauthor of com. Methodological issues and advances in biological metaanalysis.
Methodological issues in the metaanalysis of observational studies. A meta analysis of crossover trials with binary data. These are distinguished by colour, but otherwise look like the other studies. Both clinical heterogeneity and methodological heterogeneity are sources of.
Includes a full description of an improved windowsbased metaanalysis software package for applying the metaanalysis methods presented in the book an expanded discussion of the issues involved in path analyses based on metaanalytic correlation matrices has been added. The trials were identified from a pharmaceutical companys database. This article provides an introduction to the meta analysis literature and discusses the challenges of applying meta analysis to human dimensions research. Metaanalysis in biological sciences, especially in ecology and evolution which we refer to as biological metaanalysis faces somewhat different methodological problems from its counterparts in medical and social sciences, where metaanalytic. An overview and methodological assessment of systematic. Statistical considerations in indirect comparisons and network metaanalysis. The complete set of forest plots for each component at each location is shown in supplementary figures s1s8. Meta analysis has become a critically important tool in fields as diverse as medicine, pharmacology, epidemiology, education, psychology, business, and ecology. Includes a full description of an improved windowsbased meta analysis software package for applying the meta analysis methods presented in the book an expanded discussion of the issues involved in path analyses based on meta analytic correlation matrices has been added. We conducted an emailbased crosssectional study by contacting corresponding authors of srmas that were published in 2015 and 2016 and. Oct 28, 2010 if possible the results are statistically combined into a meta analysis in which the data are weighted and pooled to produce an estimate of effect. Hannah rothstein is cochair of the methods group of the campbell collaboration, and a member of the collaborations steering group.
Meta analysis is a quantitative technique that uses specific measures e. Software packages supporting clinical metaanalyses include the excel. A metaanalysis containing only old information may not be very useful. She has authored several meta analyses as well as articles on methodological issues in the area, and made numerous presentations on the topic. The historical roots of meta analysis can be traced back to 17th century studies of astronomy, while a paper published in 1904 by the statistician karl pearson in the british medical journal which collated data from several studies of typhoid inoculation is seen as the first time a meta analytic approach was used to aggregate the outcomes of multiple clinical studies. A metaanalysis is a statistical analysis that combines the results of multiple scientific studies. Although there was substantial agreement in some methodological areas there was also. The team which does a metaanalysis needs to include persons.
Binder 1statistics canada retired 49 bertona street, nepean, ontario, canada k2g 4g7 abstract papers by lauren griffith, george wells, and karla fox are discussed. This book provides a clear and thorough introduction to meta analysis, the process of synthesizing data from a series of separate studies. The results from the metaanalysis of the effect of the observation of pain stimulation on the erp components that were the most frequently analysed are shown in figures 6 and and7. A metaanalysis of crossover trials with binary data. The beckerbalagtas estimation method is applied when combining data from different designs in a metaanalysis. She is also a member of the cochrane collaborations reporting bias methods group. Work package 4 methodological guidance, recommendations. It was recognised, however, that the field was in its infancy and. Home publications methodological publications meta analysis. Sampling, methodological issues in sage research methods.
Brunelle and colleagues 14 report a meta analysis of large randomized clinical trials of insulin lispro versus regular insulin. Rothstein has been first author of four published metaanalyses of employment selection methods and has written many articles on. Statistical considerations in indirect comparisons and network meta analysis. The three metaanalysis estimates look quite unimportant by comparison. In recent years, a number of new methods have been developed to meet these challenges.
Metaanalysis, a set of statistical techniques for synthesizing the results of multiple studies, is used when the guiding research question focuses on a quantitative summary of study results. There are 3 types of heterogeneity commonly considered in meta analysis. The target audience includes postgraduate students conducting a metaanalysis or beginning researchers in metaanalysis. Methodological standards for metaanalyses and qualitative. The nma can be as valid as a standard pairwise metaanalysis if these methodological issues are taken care of. This qualitative interview study aimed to understand researchers understanding of complexity and heterogeneity and the factors which may influence the choices researchers make in synthesising complex data.
572 82 867 1463 584 763 449 1296 1474 262 1139 434 1290 1185 1605 22 831 446 1628 399 1490 869 929 1334 1139 1146 1182 329 1161 622 1200 722 1036