The bounds of meta-analytics and an alternative method

OBJECTIVES Meta-analysis is a statistical appraisal of the data analytic implications of published articles (Y), estimating parameters including the odds ratio and relative risk. This information is helpful for evaluating the significance of the findings. The Higgins I2 index is often used to measure heterogeneity among studies. The objectives of this article are to amend the Higgins I2 index score in a novel and innovative way and to make it more useful in practice. METHODS Heterogeneity among study populations can be affected by many sources, including the sample size and study design. They influence the Cochran Q score and, thus, the Higgins I2 score. In this regard, the I2 score is not an absolute indicator of heterogeneity. Q changes by bound as Y increases unboundedly. An innovative methodology is devised to show the conditional and unconditional probability structures. RESULTS Various properties are derived, including showing that a zero correlation between Q and Y does not necessarily mean that they are independent. A new alternative statistic, S2, is derived and applied to mild cognitive impairment and coronavirus disease 2019 vaccination for meta-analysis. CONCLUSIONS A hidden shortcoming of the Higgins I2 index is overcome in this article by amending the Higgins I2 score. The usefulness of the proposed methodology is illustrated using 2 examples. The findings have potential health policy implications.


INTRODUCTION
The genesis of meta-analysis can be traced to the work of an eminent statistician [1] who compared evidence from several studies on typhoid inoculation.Meta-analysis is intended to identify patterns of similarities and differences among studies with the same aim.Glass et al. [2] and others have discussed it in detail.Meta-analysis has been criticized for averaging the differences of studies with sample data from heterogeneous populations.A systematic review precedes the meta-analysis for the sake of appraising critical evidence in the publications.A meta-analysis is performed in sequential steps.As exemplified by research aiming to establish the impact of vitamin D on protecting patients from coronavirus disease 2019 (COVID-19) , the steps include focusing on establishing research questions, formulating the population, conducting a literature search for published results, selecting published studies of appropriate quality, and evaluating whether the summary measures in studies are comparable, whether the model to integrate the studies should involve fixed or random effects, and whether the heterogeneity among the study populations is acceptable in order for the findings of the metaanalysis to yield meaningful insights into the issue at hand.
Other noteworthy recent meta-analytic studies include Pearson [24] and more [25][26][27][28][29][30][31].Recently, Hong et al. [14] published an article on the importance of meta-analysis in the journal of the Korean Society of Epidemiology, Epidemiology and Health.Using the inverse of the estimated variance of the studies, the fixed type of meta-analysis provides a weighted average estimate.When popu-OBJECTIVES: Meta-analysis is a statistical appraisal of the data analytic implications of published articles (Y), estimating parameters including the odds ratio and relative risk.This information is helpful for evaluating the significance of the findings.The Higgins I2 index is often used to measure heterogeneity among studies.The objectives of this article are to amend the Higgins I 2 index score in a novel and innovative way and to make it more useful in practice.lations are heterogeneous, the random type is appropriate, and it is applied with inverse variance as weights or no weights at all.A disadvantage of meta-analysis is that the sources of bias are not accounted for in the calculations of heterogeneity.When the findings of studies lack significance, the results are often not reported in any publication; this phenomenon is known as publication bias (or the "file drawer" problem).The role of publication bias is beyond the scope of this paper.The reader is referred to Borenstein et al. [8] for the role of the Higgins score in relation to the heterogeneity of the sampled populations in meta-analyses and to Chernikova et al. [32] and Blumenfeld [33] for a discussion of simulation-based learning meta-analysis.The probability distribution of the Higgins statistic, I 2 ≥ 0, is assumed to have a chisquared distribution.However, in some studies, when is not true, I 2 does not have a chi-squared distribution.Note that and df refer to the Cochran Q score and degrees of freedom (df), respectively.In this manuscript, a modified approach is given to rectify this shortcoming in the Higgins statistic-based approach.The approach is illustrated by applying it to 2 examples-cognitive impairment and COVID-19 vaccination-for meta-analysis.

An alternative meta analytic approach
Epidemiologists, biostatisticians, and investigators in other disciplines utilize the Higgins statistic, , in meta-analysis, where Q and df are Cochran's score and df, respectively.It is straightforward and obvious that I 2 ≥ 0 and it is necessary that while conducting a meta-analysis.In some studies, this requirement is not satisfied.Consequently, the Higgins statistic, I 2 , does not have a chi-squared distribution.We offer a modified approach to rectify this shortcoming in the Higgins statistic-based approach as follows: The probability pattern of the Higgins statistic is explored, and the expression Corr( ,y) = 0 is utilized (Supplementary Material 1), where Q and df are the Cochran Q score and df, respectively.The random number, Y = y, corresponding to the number of studies on a topic available for the meta-analyst to consider at a point in time, must be y = 2,3,...,θ, in a meta-analysis, where θ is an unknown upper-bound uniform parameter.Its transformation follows a probability density function (pdf) .An additional transformation has the sample space , and its pdf is .We note that and the variance is (Supplementary Material 2).The survival function of w is .The incremental rate of researchers performing additional studies is , which stabilizes at the asymptote .The conditional pdf of the statistic is, for a given w (Supplementary Material 2). (1) Consequently, and .We have shown that and (Supplementary Material 3).We have obtained a statistical procedure to find the critical value of the new statistic, (4) as the expression (4) follows a chi-squared distribution with 1 df.In other words, the p-value of a data base S 2 is .These results would help the practitioner to have more confidence in conducting meta-analysis.

Ethics statement
In the article, 2 publicly available data sets were used for illustrating the usefulness of the proposed methodology.Informed consent was not required.

Example 1
As an illustration, we consider the recent data collected by Chen et al. [34] and Chen et al. [35] on the global prevalence of mild cognitive impairment (MCI) among elder adults living in nursing homes.The occurrence of MCI is caused by aging and/or dementia.The data they analyzed in various studies using the statistical software Stata, compiling the Q-values and the df from 53 published articles in 17 countries, are reproduced in Table 1.They concluded that there is significant heterogeneity in the studies.The Higgins statistic, I 2 , has been described to follow the chisquared distribution, whose sample space should be non-negative (that is, I 2 ≥ 0).The values of are negative (Table 1, last column) in the data for Europe and Central Asia and for the upper middleincome category.The negative values of I 2 clearly attest that the Higgins statistic does not always follow the chi-squared distribution.Hence, a refined version of the Higgins statistic is a necessity, and such a revised version is our statistic, S 2 , whose values are displayed in Table 1.

Example 2
Parents were concerned about vaccinating their children with the then-untested COVID-19 vaccine.A combined worldwide study using a meta-analysis was used to probe patterns in these concerns.A total of 98 papers across 69 different countries with 413,590 participants were examined by Alimoradi et al. [4].The authors found that countries' income level, location, and data collection methods were significant moderators of parents' willingness to vaccinate their children against COVID-19.The data collection method was another significant factor influencing parental willingness.Studies collected using phone interviews had the lowest prevalence of willingness.None of the studies were thought to have exhibited heterogeneity.
Once again, the Higgins statistic, I 2 , exhibited negative values which violate the required non-negative sample space of the chisquared distribution (see the last column in Table 2) in the data for all groupings.A refined version of the Higgins statistic is, once again, a necessity.For comparison, our revised statistic, I 2 , is displayed in Table 2.

DISCUSSION
A word of caution is necessary when interpreting the Higgins I 2 value and its impact.There are 3 challenges in using the Higgins score, I 2 : (1) It is mentioned by Higgins et al. [36] that I 2 is the percentage of variation across the studies that is due to heterogeneity rather than sheer chance.Khan [16] commented that "….The I 2 values of 25%, 50%, and 75% indicate low, moderate, and high heterogeneity, respectively, among the population effect sizes.I 2 ≤ 25% of studies are considered to be homogeneous." (2) Corr ( ,Y) = 0 does not imply that and Y are independent.(3) The I 2 statistic can be negative when Q is less than df.For this situation, it is commented by Higgins et al. [36]: "Negative values of the I 2 are put equal to zero so that I 2 is between 0% and 100%." This causes users to doubt the validity of the score and have less confidence in using it.These shortcomings are overcome by our refinement of the Higgins score, which we explain below: The exact probability structure of the popularly utilized Higgins score in meta-analytic studies to assess the consistency of the findings in various studies about a healthcare topic is derived.With this probability structure, a method of finding the p-value for the Higgins score and its interpretation is devised and demonstrated.The exact new expression (4) for the score S 2 is a refined version of the Higgins standardized score, which follows the chisquared distribution with 1 df.With these new innovative results, meta-analytic researchers do not have to follow the subjective interpretations of the estimated Higgins score.Instead, the researchers could obtain the p-value for the calculated standardized S 2 score based on the chi-squared distribution and conduct an objective, exact interpretation.The values of the new score S 2 are objective.The authors show both the conditional and unconditional probability structures of the Higgins statistic, including how the correlation between and Y is derived and utilized for and Y to be uncorrelated and independent.
Had Higgins followed the line of the traditional thinking of statistical discipline, he could have defined the I 2 score as the ratio