epiH Search


Epidemiol Health > Volume 38; 2016 > Article
Shin and Shin: Validity analysis on merged and averaged data using within and between analysis: focus on effect of qualitative social capital on self-rated health



With an increasing number of studies highlighting regional social capital (SC) as a determinant of health, many studies are using multi-level analysis with merged and averaged scores of community residents’ survey responses calculated from community SC data. Sufficient examination is required to validate if the merged and averaged data can represent the community. Therefore, this study analyzes the validity of the selected indicators and their applicability in multi-level analysis.


Within and between analysis (WABA) was performed after creating community variables using merged and averaged data of community residents’ responses from the 2013 Community Health Survey in Korea, using subjective self-rated health assessment as a dependent variable. Further analysis was performed following the model suggested by WABA result.


Both E-test results (1) and WABA results (2) revealed that single-level analysis needs to be performed using qualitative SC variable with cluster mean centering. Through single-level multivariate regression analysis, qualitative SC with cluster mean centering showed positive effect on self-rated health (0.054, p<0.001), although there was no substantial difference in comparison to analysis using SC variables without cluster mean centering or multi-level analysis.


As modification in qualitative SC was larger within the community than between communities, we validate that relational analysis of individual self-rated health can be performed within the group, using cluster mean centering. Other tests besides the WABA can be performed in the future to confirm the validity of using community variables and their applicability in multi-level analysis.


With increasing attention on the effect of environment around individuals as a social decisive factor on health, many multi-level analytic studies are now assessing the simultaneous reflection of health-related occurrence factors at the individual and local community level. Recently, social capital (SC) has gained prominence as a factor affecting individual health. SC, a form of capital developed from social relationship rather than individual relationship, is a concept different from economic, human, or cultural capital [1]. Although there are different viewpoints on defining SC, some quantitative and qualitative factors built from interpersonal relationship affecting SC are reflected in prior studies, including social participation, quality of family relationship, contact frequency, reliability, and reciprocity.
Many studies have confirmed the relationship between SC and health, and recent studies are shifting towards multi-level analysis utilizing the characteristics of SC at the regional level. Two primary data selection methods exist for multi-level analysis of SC reflecting community characteristics. The first involves consideration of administrative data such as turnout of voters, number of volunteer camps, and community-based organizations [2] as proxy variables of SC. The second method utilizes the merged average of individual responses or the response rate. Average participation level in community organizations [2], confidence levels of seven community groups, and individual responses on network level of 11 different types of community organizations were created as community variables [3]. Happiness index calculated by averaging the satisfaction scores of five different sections including individual health and attitude scores as a property of SC, calculated by averaging attitude scores of four different sections including attitude score against disabled people were reflected in the analysis [4].
Although one can ensure objectivity of the data using administrative data as a proxy variable, it is difficult to distinguish whether the data is a result of SC or is a representation of SC. On the contrary, newly created data using individual data allows selection of factors that are closely related to SC compared to administrative data, although potential bias between individuals can result in reduced reliability of the accumulated data. Therefore, one can overcome limitations of the data obtained from administrative data, using survey data through a statistically validated method to represent the community, avoiding unnecessary logical leaps when explaining the relationship between the SC and health. However, a statistical test is needed to validate merging to convert individual responses into community variables. In other words, the procedure to validate whether or not the measurements from combining individual scores are adequately representing the characteristics of the community is necessary [5]. Within and between analysis (WABA) is a method suggested by previous studies to assess construct validity in multi-level analysis. As a tool to assess validity in data merging and adequacy of multi-level analysis, the WABA can provide statistical test results on whether or not the newly calculated community variables can be used as a valid, shared construct, and whether it can be used as an individual variable or must be manipulated into a comparative variable. Domestic studies involving validation of upper-level variables derived from individual variables were often at the organizational level [5-7], and in the domestic public health field, only one study by Lee et al. [8] assessed the validity of utilizing merged and averaged variables from individual self-rated health, stress, and oral health variables, as community variables to discover regional variables affecting the health-related quality of life. There has been no previous study assessing the validity of merged variables from SC and their applicability in multi-level analysis.
Therefore, this study aims to assess the validity of regional level SC variables calculated through merging and averaging the data from community residents’ responses in reliability and reciprocity section of 2013 Community Health Survey, and applicability in multi-level analysis using subjective self-rated health assessment as a dependent variable through WABA.


Subjects and data

The analysis uses data from the 2013 Community Health Survey. This nation-wide survey was conducted from August to November of 2013, for statistical data to understand the health conditions of community residents and to establish and evaluate health policy based on the evidences from the survey. Around 900 people over 19 years of age selected through stratified sampling were included in health-related surveys at the city, district, and borough-levels of the public health centers. Out of 221,816 surveys, 215,727 surveys were used in the analysis excluding 6,089, which did not have responses for key questions.

Analyzed variables

Self-rated health question of “What do you think of your usual health conditions?” with very bad, bad, average, good, and very good as potential answers was used as dependent variable. This is a continuous variable where a higher score is assigned to the more positive answer for the question.
The independent variables were reliability and reciprocity. The question to assess reliability was “Can you and your neighbors within the community trust each other?”, and the one to assess reciprocity was “Do people within the community have the tradition to help each other for congratulations and condolences?” with an option to choose “yes” or “no” for both questions. Results were converted into continuous variables by assigning one point for every “yes” answer and zero point for every “no” answer, with the total score ranging from zero to two, and this was termed qualitative SC. Qualitative SC was manipulated into community variables by merging and averaging community residents’ individual responses, and finally the cluster group mean centering variable was calculated by subtracting merged and averaged community scores from individual scores, which was then termed “mean-centered qualitative SC”.
Control variables included sex, age, education level, income, marital status, and contact frequency with family, friends, and neighbors. Sex (male, female), age (19-45, 46-65, 65+), educational level (none, elementary school graduate, middle school graduate, high school graduate, above college attending or graduate), monthly income (Korean won [KRW]; <1, 1-2 million KRW, 2-3 million KRW, ≥4 million KRW, no answer), and marital status (single, married, married but without partner) were all designed as nominal variables. Contact frequency with family, friends, or neighbors changed from frequency variable (>4 times/wk, <3 times/wk to >1 times/mo, <1 times/mo) to nominal variable (frequent, sometimes, rarely).

Statistical analysis

First, basic statistics were calculated for variables in the analysis to understand the basic characteristics of the data. Next, reliality and reciprocity variables were merged, and the community variable was produced using community residents’ average responses, followed by assessment of Spearman’s rank correlation between control and independent variables. Validity and adequacy analysis was performed through the WABA. The WABA provides not only the validity of merging but also analysis level based on relationship between the variables. The analysis is divided into WABA(1) and WABA(2). WABA(1) provides validity of community variables using results from analysis of variance for individual independent variables, while WABA(2) allows determination of analysis level by looking at differences between and within the group(s) in the relationship between independent and dependent variables [5,8,9].
WABA(1) considers the sizes of between group variation and within group variation, testing using the value of ηBW. ηW is calculated as the square root of SSW/SST, and ηB is calculated as the square root of SSB/SST, and E-test is performed using these values. E-test involves the two vectors (B and W) forming the two axes forming 90° and between and within group deviations changing between the two axes. E values of 1.3 (at 52.5°, +7.5° away from the middle 45°) and 0.77 (at 37.5°, -7.5° away from the middle 45°) are used as test standards. E≥1.3 represents differences between the groups and within group homogeneity, while E≤0.77 represents large differences within the group and relatively smaller differences between the groups suggesting analysis through group mean centering, which is also known as the Frog-Pond model. In cases of 0.77<E<1.3, where between and within group variations are similar, individual-level analysis is recommended [5,8-10].
E = ηBηw   
WABA(2) differs from WABA(1) in that it not only uses the η values, but also considers the correlation between independent and dependent variables. The formula for WABA(2) is as follows:
Total correlation=between component + within component                                  
  • ηBX =between-unit η value for independent variable

  • ηBY =between-unit η value for dependent variable

  • rBXY =between-unit correlation coefficient between independent and dependent variables

  • ηWX =within-unit η value for independent variable

  • ηWY =within-unit η value for dependent variable

  • rWXY =within-unit correlation coefficient between independent and dependent variables

If between component (BC) value is larger than within component (WC) value, between-unit analysis is suitable, while within-unit analysis is appropriate if the BC and WC values are the same. A within-unit comparative analysis model needs to be established if the BC value is smaller than the WC value. Comparative analysis model involves manipulated variables from individual variables through cluster mean centering.
Lastly, statistical significance level was set to 0.05, and multivariate regression analysis was performed as per the WABA results; subsequently, the multi-level analysis was performed for purposes of comparison.


Basic characteristics of the subjects and community variables

Table 1 outlines basic characteristics of 215,737 subjects involved in the analysis. There were more females than males, and self-rated health scores were somewhat higher in males (3.31) than in females (3.06). The respondents aged 19-45 were 34.8% of the total, the other 37.5% were 46 to 65, and the remaining 27.7% were above 65. Self-rated health scores gradually decreased with aging. In addition, lower education level and income showed correspondingly lower self-rated health score. Majority of the subjects were married (68.5%), but single subjects had highest self-rated health score (3.65). Increasing contact frequency to neighbors was correlated with increased self-rated health score, but there was no correlation observed between contact frequency with family or friends and self-rated health score.
Table 2 outlines the results of assessing correlation between the individual variables. Regional level qualitative SC and gap SC – gap between regional level SC and individual level SC – had a very high correlation at 0.88. Contact frequency with family, neighbors, or friends and individual level qualitative SC had low correlation, indicating the possibility of concurrent introduction with other sociodemographical or social-economical control variables.

Within and between analysis

To assess the validity of selected variables from community residents’ average responses of qualitative SC, a WABA was performed. From the WABA(1) analysis of regional level qualitative SC variables, ηB value was 0.219 and ηW value was 0.781. E value calculated according to the equation (1) is 0.280, and fits into E≤0.77 group with E (15°) as a standard, indicating large modification within group and the consequent adequacy of analysis using single-level comparative model.
Next, according to the analysis using the WABA(2) equation, BC was 0.005, and WC was 0.072. Since WC score was higher than BC score, similar to the suggestion from WABA(1), the analysis using the single-level comparative model is the appropriate method.
Based on the WABA results, a single-level analysis was performed and an additional multivariate regression analysis using cluster mean centered individual variables was performed as a comparative model (Table 3).

Single- and Multi-level regression analysis

WABA results indicated that analysis of qualitative SC between groups was not appropriate, and analysis using comparative model was considered. A new variable was generated by cluster mean centering of qualitative SC for further analysis. Increased cluster mean centered qualitative SC was correlated with increased self-rated health score (0.054, p<0.001). It is notable that this result was not very different from the result of model 1 which involved qualitative SC, which was not cluster mean centered.
The comparisons between model 1, model 3, and model 4 provide results for unconditional multi-level analysis and multi-level analysis reflecting every variable. Unconditional model 3 without variable introduction had intra coefficient correlation (ICC) value of 0.031, and model 4 which had all variables introduced showed statistical significance in upper-level variable of regional average value for individual qualitative SC (0.068, p<0.001) with ICC value of 0.01 (Table 4).


Since the 2000s, health-related studies using multi-level analysis are on the rise; however, there is still no clear standard for statistical test procedures prior to the analysis, unlike single-level multivariate regression analysis. Only a few studies provide preliminary validation procedures to show regional differences causing different characteristics of dependent variables or to assess validity in the manipulation of individual responses into community variables. In other words, there is possibility of artificial analysis results due to unfulfilled statistical requirements, even if the multi-level analysis model is appropriate in theory. The WABA can partially solve these level-issue problems. Therefore, this study performed statistical validation using WABA to assess the validity of merged and averaged community variables calculated from the community residents’ responses of reliability and reciprocity variables – measuring qualitative aspect of SC – in 2013 Community Health Survey, and the possibility of utilizing the result in multi-level analysis.
From the two analysis methods suggested by WABA, both WABA(1) – utilizing individual variables to assess within group variance and between group variance – and WABA(2) – considering correlation between independent and dependent variables and within group variance and between group variance of these variables – results demonstrated appropriateness of using the single-level comparative model.
Based on the above result, single-level multivariate regression analysis was performed using cluster mean centered qualitative SC, calculated by subtracting merged and averaged community scores from individual scores. There was a statistically significant association between the cluster mean centered qualitative SC and self-rated health. However, these results—compared to the values from merging individual-level reliability and reciprocity scores—are neither very high nor significantly affect the coefficient value. Since the correlation between cluster mean centered qualitative SC and simple merged qualitative SC is very high (0.88), introduction of any variable would yield similar results. In this analysis, manipulation of two “yes or no question” about regional awareness in to continuous variable resulted in a very limited score range. Therefore, it appears that statistical significance was achieved even though the meanings between the two variables are practically different. Other factors such as love for the community, sense of belonging, and trust level in policies—aside from reliability and reciprocity, which were assessed in this study —are important [11], but these psychological characteristics were not reflected properly. Therefore, using a validating tool to assess regional qualitative SC, or re-designing the survey to reflect Korean culture and better examine and compare regional qualitative SC and SC differences between communities, can also be considered.
Moreover, to compare multi-level analysis results, we looked at the variance size in unconditional model, and it was statistically significant at 0.03, exhibiting community-level variance although at low level. However, ICC analysis involving measurement of community-level variance from the whole variance indicated variance due to regional characteristics is only at 0.031 (3.1%). Typically, a multi-level analysis is not performed if the ICC analysis result is less than 5.0% as between group homogeneity is not secured [12], and community-level variance can change by introduction of individual-level control variables 3.1% from the null model, have no independent variables and only set outcome variable, is a comparatively small value. On the other hand, multi-level analysis with all variables introduced showed correlation between self-rated health score and regional qualitative SC, but there was not much difference in the averaged qualitative SC coefficient value from the single-level model. There is a possibility that significance observed in regional variables from multi-level analysis is a statistical artifact, looking at within group variance and the size of between group variance. In conclusion, the single-level comparative model suggested by both the WABA(1) and WABA(2) test result is suitable for the analysis.
Meanwhile, external factors for inadequacy of multi-level analysis for WABA using self-rated health and qualitative SC include potential lack of difference in self-rated health among cities, districts, or boroughs in Korean society. This is due to frequent moving and shortened time periods for stable housing; furthermore, the majority of interactions performed in communities or different regions is outside the house [14]. Even if a more accurate method is used to assess self-rated health as a variable, since it is unlikely to be explained by community variables, a multi-level analysis is practically not feasible. However, some domestic multi-level analysis studies indicate an association between regional characteristics and health [14-17]. This might imply that people having high sensitivity to exposures at the community level is more reactive with qualitative SC of regional level for the analysis, rather than there are only small differences on health-impacting community-level qualitative SC among the communities; therefore, the study hereafter need to be performed on a selected cohort of samples based on high-sensitive to the environment. In other words, as the value from the WABA(2) analysis reflect the correlation between independent and dependent variables, a follow-up study introducing highly-correlated health variable is needed.
There are still a few limitations in the effective usage of the WABA as an analytical tool to test whether or not multi-level analysis can be performed. First, the WABA is an analysis-of-variance based method, and only works when the dependent variable is a continuous variable. Therefore, dichotomous variables such as disease morbidity or health behaviors limit the WABA. Moreover, according to previous studies, when Between variance and Within variance values are exactly same in WABA(2) analysis, individual single-level analysis is appropriate. However, the two values being identical is an extremely rare case; therefore, the chance of single-level analysis being concluded as an appropriate method is very low. These limitations of the WABA indicate that when performing multi-level analysis, validation using WABA is a sufficient but not a necessary condition. Nevertheless, WABA is a useful analytical tool, when manipulating continuous variable that can represent a community into community-level variables and assessing its utility in multi-level analysis.
This study is a first attempt—within the scope of our investigation—to check if the manipulation of SC variable from individual responses into community variable is appropriate, and also if the manipulated variable is adequate in multi-level analysis, using WABA. Although our research showed that qualitative SC was deemed inappropriate for multi-level analysis due to large within group variance, validity testing should be performed using the WABA as a statistical tool in future multi-level health studies involving merged and averaged data.


The authors have no conflicts of interest to declare for this study.


Supplementary material (Korean version) is available at http://www.e-epih.org/.

Table 1.
Basic characteristics of the subjects
Characteristic n % Average± standard deviation
 Male 96,721 44.8 3.31 ±0.93
 Female 119,006 55.2 3.06±0.95
Age (yr)
 19-45 75,121 34.8 3.56±0.76
 46-65 80,957 37.5 3.23±0.86
 ≥65 59,649 27.7 2.59±0.99
 None schooling 26,833 12.4 2.38±0.96
 Elementary 35,032 16.2 2.77±0.95
 Middle 24,145 11.2 3.07±0.89
 High 71,584 33.2 3.41 ±0.84
 Above college 58,133 26.9 3.54±0.76
Household monthly income (million Korean won)
 <1 52,884 24.5 2.65 ±1.01
 1-2 39,022 18.1 3.16±0.92
 2-3 39,477 18.3 3.35±0.84
 3-4 25,044 11.6 3.42±0.81
 >4 52,800 24.5 3.47±0.81
 No response 6,500 3.0 3.13±0.95
Marital status
 None married 36,916 17.1 3.65±0.82
 Divorced, bereaved 30,961 14.4 2.72±0.82
 Married 147,850 68.5 3.19±0.90
Neighborhood contact
 Rare 80,144 37.2 3.00±0.99
 Occasional 74,919 34.7 3.23±0.93
 Frequent 60,664 28.1 3.33±0.88
Friend contact
 Rare 51,534 23.9 3.28±0.95
 Occasional 118,495 54.9 3.28±0.87
 Frequent 45,698 21.2 2.79±1.02
Family contact
 Rare 46,076 21.4 3.15±0.96
 Occasional 131,510 61.0 3.19±0.93
 Frequent 38,141 17.7 3.15±0.99
Table 2.
Correlation between variables
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11)
(1) Self rated health 1 -0.13*** -0.39*** 0.39*** 0.29*** -0.07*** 0.00 0.14*** -0.16*** -0.10*** -0.05***
(2) Sex 1 0.04*** -0.18*** -0.05*** 0.14*** -0.08*** -0.10*** 0.06*** 0.01 0.01
(3) Age 1 -0.67*** -0.44*** 0.03*** -0.05*** -0.38*** 0.14*** 0.36*** 0.25***
(4) Education 1 0.49*** -0.15*** 0.03*** 0.39*** -0.14*** -0.34*** -0.20***
(5) Income 1 -0.19*** 0.01*** 0.25*** -0.11*** -0.20*** -0.08***
(6) Marriage 1 0.09*** 0.06*** -0.03** -0.06*** -0.07***
(7) Family contact 1 0.19*** 0.11*** -0.11*** -0.06***
(8) Neighborhood contact 1 0.05*** -0.51*** -0.35***
(9) Friend contact 1 0.00 -0.01
(10) Qualitative SC 1 0.88***
(11) Gap SC1 1

SC, social capital.

1 Gap between regional level SC and individual level SC.

** p<0.01,

*** p<0.001.

Table 3.
Within group and between group variance analysis
Variable Correlation Etas
Between Within Between Within Between Within
Self rated health 0.032 0.968
 Qualitative social capital 0.724 0.219 0.781 0.756 0.095 0.005 0.072 BC< WC parts

BC, between component; WC, within component.

Table 4.
Estimates of single-level and multi-level regression analysis
Model 1 Model 2 Model 3 Model 4
Intercept -3.044*** -2.917*** -2.825*** -3.020***
 Male Reference Reference Reference
 Female -0.115*** -0.120*** -0.118***
Age (yr)
 19-45 Reference Reference Reference
 45-64 -0.161*** -0.158*** -0.147***
 Above 65 -0.455*** -0.451*** -0.437***
 None schooling Reference Reference Reference
 Elementary 0.204*** 0.199*** 0.200***
 Middle 0.355*** 0.344*** 0.350***
 High 0.520*** 0.504*** 0.510***
 Above college 0.578*** 0.560*** 0.570***
Household monthly income (million KRW)
 <1 Reference Reference Reference
 1-2 0.171*** 0.168*** 0.174***
 2-3 0.235*** 0.231*** 0.238***
 3-4 0.254*** 0.248*** 0.260***
 >4 0.288*** 0.282*** 0.296***
 No response 0.165*** 0.162*** 0.167***
Marital status
 None married Reference Reference Reference
 Divorced, bereaved 0.180*** 0.180*** 0.177***
 Married -0.023*** -0.026*** -0.027***
Family contact frequency
 Frequent Reference Reference Reference
 Occasional -0.001 0 0
 Rare -0.036*** -0.040*** -0.050***
Neighborhood contact frequency
 Frequent Reference Reference Reference
 Occasional -0.036*** -0.050*** -0.046***
 Rare -0.072*** -0.106*** -0.104***
Family contact frequency
 Frequent Reference Reference Reference
 Occasional -0.049*** -0.049*** -0.053***
 Rare -0.202*** -0.201*** -0.215***
Qualitative SC 0.071*** - -
Centering SC - 0.054*** -
Level 2
Qualitative SC (regional) 0.068***
σ2 (level 1) 0.869*** 0.681***
τ00 (level 2) 0.028*** 0.007***
ICC 0.031 0.010

KRW, Korean won; SC, social capital; ICC, intra coefficient correlation.



1. Kim HY. A study of the effects of social capital on life satisfaction of the residents participating in mobilization program of community resources. J Korean Soc Welf Adm 2011; 13: 1-29 (Korean).

2. Jung M, Cho BH. A multi-level study of contextual effects of community capacity on health status among Seoul residents: focused on social quality. Korean J Health Educ Promot 2011; 28: 1-14 (Korean).

3. Kim HY. Community inequalities in health: the contextual effect of social capital. Korean J Sociol 2010; 44: 59-92 (Korean). PMID: 2010
4. Kim YH, Cho YT. Impact of area characteristics on the health of vulnerable populations in Seoul. Korea J Popul Stud 2008; 31: 5-30 (Korean). PMID: 2008
5. Park WW, Ko S. Procedures and methods of multilevel analysis: with a focus on WABA. Seoul J Bus 2005; 39: 59-90 (Korean). PMID: 2005
6. Han TY, Tak JK. Transformational/transactional leadership: revisit using a multilevel approach. Korean J Ind Organ Psychol 2005; 18: 337-360 (Korean).

7. Lee YH, Lee KB, Lee SW. Individual perception or team perception? Application of multilevel analysis for securing validation of team level psychological evaluation. Korean J Sport Psychol 2010; 21: 71-81 (Korean).

8. Lee JH, Lee DR, Kim S, Lim DS, Heo TY. A study on the utilization of multilevel model using community health survey data. J Korean Data Anal Soc 2013; 15: 1289-1302 (Korean). PMID: 2013
9. Klein KJ. From micro to meso: critical steps in conceptualizing and conducting multilevel research. Organ Res Methods 2000; 3: 211-236.
10. Dansereau F, Cho J, Yammarino F. Avoiding the “fallacy of the wrong level”: a within and between analysis (WABA) approach. Group Organ Manage 2006; 31: 536-577.
11. So JK. Toward developing social capital indicators. J Korean Reg Dev Assoc 2004; 16: 89-117 (Korean). PMID: 2004
12. Kim KK, JeKarl J, Kwon YM, Park MS. Effects of the community environmental characteristics on drinking problems of adults: a multilevel analysis. Korean J Health Educ Promot 2012; 29: 103-113 (Korean). PMID: 2012
13. Jung SW, Cho YT. Neighborhood characteristics and individual health under Korean context. J Prev Med Public Health 2005; 38: 259-266 (Korean). PMID: 16323625
14. Ko JE, Lee SH. A multilevel modeling of factors affecting depression among older Korean adults. Ment Health Soc Work 2012; 40: 322-351 (Korean).

15. Park YM, Kim Y. A spatially filtered multilevel model to account for spatial dependency: application to self-rated health status in South Korea. Int J Health Geogr 2014; 13: 6. PMID: 24571639
crossref pmid pmc
16. Han S. Compositional and contextual associations of social capital and self-rated health in Seoul, South Korea: a multilevel analysis of longitudinal evidence. Soc Sci Med 2013; 80: 113-120. PMID: 23261256
crossref pmid
17. Jung M, Bigman-Galimore CA, Viswanath K. Contextual effects of community mobilization and communication capacity as a positive factor for self-rated health status: a multi-level analysis. Int J Public Health 2014; 59: 289-299. PMID: 24362447
crossref pmid


Browse all articles >

Editorial Office
Graduate School of Cancer Science and Policy, National Cancer Center
323 Ilsan-ro, Ilsandong-gu, Goyang 10408, Korea
TEL: +82-2-745-0662   FAX: +82-2-764-8328    E-mail: enh0662@gmail.com

Copyright © 2019 by Korean Society of Epidemiology. All rights reserved.

Developed in M2community

Close layer
prev next