Measurement Equivalence Between Men and Women in the Abbreviated Social and Emotional Loneliness Scale for Adults (SELSA)

Thomas V. Pollet*a, Tamsin K. Saxtona, Melanie Mitchella


Health practitioners, policy-makers, and psychologists point to legitimate concerns about the negative impact of loneliness. To help resolve such negative impact, we need to better understand the psychometric structure of loneliness. Men’s and women’s differing social roles may mean that they experience different sources of loneliness. After matching via exact matching, we compared men and women’s scores (N = 273) on the abbreviated form of the Social and Emotional Loneliness Scale for Adults (SELSA) using confirmatory factor analysis and measurement invariance testing. We replicated the three-factor structure of the SELSA, thereby providing further evidence for differing etiologies of family, romantic, and social loneliness. We found no good evidence for gender differences in the structure of the questionnaire answers, indicating that the SELSA can be used to further illuminate the implications of loneliness for men and women.

Keywords: structural equation modelling, measurement equivalence, loneliness

Interpersona, 2018, Vol. 12(2), doi:10.5964/ijpr.v12i2.316

Received: 2018-06-28. Accepted: 2018-11-05. Published (VoR): 2018-12-21.

*Corresponding author at: NB 165, Northumberland Building, 2 Ellison Place, NE18ST, Newcastle upon Tyne, UK. E-mail:

This is an open access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Loneliness has been consistently associated with negative health outcomes (review in Rico-Uribe et al., 2018). It has also been placed on the national social and political agenda in countries such as the United Kingdom (Yeginsu, 2017). One of the earliest attempts to conceptualise loneliness by Weiss (1973) moved away from treating it as a unitary state which only varies in frequency or intensity, towards differentiating emotional from social loneliness. Emotional loneliness arises from a lack of close emotional ties (e.g., spouse, parent). In contrast, social loneliness is a consequence of not having affiliative relationships such as friendships. This differentiation between social and emotional loneliness has led to the development of multidimensional measurement instruments such as the Social and Emotional Loneliness Scale for Adults (SELSA, DiTommaso & Spinner, 1993, 1997); the authors further divided emotional loneliness into family and romantic emotional loneliness, following principal component analysis of this scale.

Research exploring gender differences in loneliness presents mixed findings, with some research suggesting that women report more loneliness than men do (e.g., Borys & Perlman, 1985; Pinquart & Sorensen, 2001), others that men report more (e.g., Lee & Goldstein, 2016; Russell, 1996), and still others that do not find a gender difference (e.g., Tornstam, 1992). Much of this research has relied on a unidimensional approach to loneliness rather than a multidimensional approach. There is reason to expect that men and women might differ from each other when specific aspects of loneliness are considered separately, as part of a multidimensional approach. For example, women have been argued to be kin keepers and to have more intimate ties with family members than men have (e.g., Hagestad, 1986; Moore, 1990), and so might have lower levels of family loneliness in general, but might suffer more from loneliness if they do not achieve the family relationships they desire (Maes, Klimstra, Van den Noortgate, & Goossens, 2015). Similarly, research has shown that men attach greater value to having a romantic partner than women do (e.g., Dykstra & Fokkema, 2007), again perhaps indicating greater variation in men’s than women’s romantic loneliness. Such reasoning has previously led researchers to examine the measurement invariance across genders of various loneliness scales (Buz & Pérez-Arechaederra, 2014; Maes et al., 2015), and to recommend that similar analyses should be carried out on other multidimensional loneliness scales (Maes et al., 2015). Indeed, the testing of measurement invariance across genders is a regular part of the development of psychometric questionnaires (Schmitt & Kuljanin, 2008; Vandenberg & Lance, 2000). If a questionnaire does not demonstrate measurement invariance across different groups (such as genders), this would indicate that the questionnaire could be measuring a different underlying construct in each group. Here, we examine whether the abbreviated version of the SELSA (Cramer, Ofosu, & Barry, 2000) measures the same construct in men and women by testing measurement invariance across genders. We thereby set out to uncover more about how the sources of loneliness may differ between men and women, alongside testing some of the standard psychometric properties of an established questionnaire.

Method [TOP]

Participants [TOP]

In order to obtain a large sample that extended beyond students, participants were recruited via the personal networks of students enrolled at a large Dutch university (see Pollet, Roberts, & Dunbar, 2013 for full description). There is a substantial number of German participants as this study proved popular with German-speaking students who had fewer options of studies to participate in for credit. 458 surveys were processed (301 women, M = 30.97 years, SD = 14.55 years, 3 did not report age or gender); due to some non-responses, 382 of those were able to be used for the current research project.

Given that we wanted to compare men and women, it was important to match these samples to ensure that any differences were not due to other sample characteristics. Using the ‘MatchIt’ algorithm (Ho, Imai, King, & Stuart, 2011), we exactly matched men and women on age, marital status (“Single (never married)”, “Living with partner”, “Married (first marriage)”, “Remarried”, “Divorced”, “Separated”, “Widowed”), educational attainment (“no diploma/degree”,“primary education”,“lower secondary”, “higher secondary”, “higher education/ non-university degree”, “university degree”,“other”), and nationality (“Dutch”, “German”, “Other”). This left us with 273 cases (age: M = 25.97 years, SD = 11.17 years; 181 women). The majority were German (78.75%) and never married (78%). In terms of educational attainment, 40.65% had previously obtained a higher secondary school degree (i.e. qualifications derived from education until around the age of 16), 35.9% a non-university higher, 11.36% had completed a university degree, and the remainder had completed a lower secondary school degree or a lower degree (i.e. qualifications derived from education up until around the age of 14). It is important to note that we asked about educational attainment, and therefore the category of secondary school degree also contains students currently enrolled upon a university degree.

Procedure [TOP]

After providing informed consent, participants completed a paper-based survey in German or Dutch depending on participant preference. Participants first provided basic sociodemographic data, including age, gender, nationality and educational attainment (similar to Roberts, Dunbar, Pollet, & Kuppens, 2009). They then completed a questionnaire on their living relatives (see Pollet et al., 2013; data not used here), and finally the SELSA.

Materials [TOP]

The abbreviated form of the SELSA consists of 15 items, of which 5 items make up the family loneliness sub-scale, 6 items make up the romantic sub-scale, and 4 refer to social loneliness (Cramer et al., 2000) (Table 1). All items were answered on a 7-point Likert scale (1=Strongly Disagree to 7=Strongly Agree) and then reverse scored with the exception of ‘Romantic3’.

Table 1

Descriptive Statistics (N = 273) of Questionnaire Items (1-7 Scale; All Items Except Romantic3 Were Reverse-Scored, Such That Higher Scores Here Indicate Greater Loneliness). Items Were Translated in Dutch or German.

Item M SD Min Max
I really belong in my family (Family1) 1.780 1.062 1 6
I feel part of my family (Family2) 1.722 1.079 1 6
My family really cares about me (Family3) 1.879 1.096 1 7
My family is important to me (Family4) 1.593 0.790 1 6
I feel close to my family (Family5) 1.886 1.042 1 7
I have a romantic partner with whom I share my most intimate thoughts and feelings (Romantic1) 3.593 2.310 1 7
I have a romantic or marital partner who gives me the support and encouragement I need (Romantic2) 3.582 2.328 1 7
I have an unmet need for a close romantic relationship (Romantic3) 3.352 2.044 1 7
I am in love with someone who is in love with me (Romantic4) 3.645 2.314 1 7
I have someone who fulfils my needs for intimacy (Romantic5) 3.344 2.112 1 7
I have a romantic partner to whose happiness I contribute (Romantic6) 3.659 2.308 1 7
I have friends that I can turn to for information (Social1) 1.484 0.718 1 5
I can depend upon my friends for help (Social2) 1.703 0.991 1 7
I have friends to whom I can talk about the pressures in my life (Social3) 1.670 0.912 1 6
I have a friend(s) with whom I can share my views (Social4) 1.648 1.068 1 7

Data Analysis [TOP]

All the analyses were conducted in R 3.5.1 (R Development Core Team, 2008). We used ‘lavaan’ (Rosseel, 2012) to perform confirmatory factor analysis and examine measurement invariance (Sass & Schmitt, 2013; Schoot, Lugtig, & Hox, 2012; Vandenberg & Lance, 2000). The data, code, and analysis document are available from the Open Science Framework (

Results [TOP]

Descriptive Statistics [TOP]

Table 1 shows the descriptive statistics for all items, following reverse scoring.

Measurement Invariance [TOP]

The model proved a good fit in both CFI (.922) and TLI (.906) but not in RMSEA (.105), something that could be a consequence of the relatively small sample size for our models (Kenny, Kaniskan, & McCoach, 2015). We also attempted a single factor model which proved to be a substantially worse fit to the data (CFI = .333, TLI = .222, RMSEA = .302). This corroborates that the SELSA is composed of the three factors identified previously (Cramer et al., 2000). Figure 1 summarizes the loadings for men and women.

Figure 1

SEM Plots for men and women. Fml = family loneliness, Rmn = romantic loneliness, Scl = social loneliness.

Table 2 shows the measurement invariance summary. It shows that there is no substantial loss in fit when moving from configural to metric, and from metric to scalar measurement invariance. However, there is a loss of fit when moving from scalar invariance to residual invariance. This implies that we cannot conclude that the latent structure is measured identically in men and women. In conclusion, we can establish scalar invariance, i.e. we can compare men and women on their latent variable scores. We were able to recover the proposed three factor structure in both groups of men and women. Given that we did not achieve residual invariance, we did not examine structural invariance (Milfont & Fischer, 2010).

Table 2

Measurement Invariance Summary

Invariance χ2 df Δχ2 df p CFI ΔCFI RMSEA ΔRMSEA BIC ΔBIC
Configural 435.1 174 NA NA NA 0.922 NA 0.105 NA 11413.9 NA
Metric 455.0 186 20.0 12 .068 0.920 0.002 0.103 0.002 11366.6 47.4
Scalar 471.6 198 16.6 12 .165 0.918 0.001 0.101 0.002 11315.9 50.7
Residual 529.0 213 57.3 15 < .001 0.906 0.013 0.104 -0.004 11289.1 26.8
Mean 535.9 216 7.0 3 .074 0.904 0.001 0.104 0.000 11279.2 9.9

Correlations Between the Three Latent Constructs (Family, Romantic and Social Loneliness) [TOP]

Figure 1 shows the correlations between the latent constructs. For both men and women, there was a significant association between Family and Social loneliness (respectively: 0.333+/-0.10 and 0.358+/-0.07, p = .002 and p < .001). However, there were no significant associations between Family and Romantic loneliness (men: p = .067 and women: p = .161) or between Romantic and Social loneliness (men: p = .412 and women: p = .161).

Gender Differences [TOP]

We summed all items to generate scores for family, romantic and social loneliness. With 80% power and a two-sided significance level of 5%, we can detect a relatively small effect size (Cohen’s d = .36, Cohen, 1988). There was no good evidence for a difference between men and women in terms of family loneliness (Mwomen = 8.58, SD = 4.4; Mmen = 9.41, SD = 4.29; t(187.27) = 1.5, p = .13), social loneliness (Mwomen = 6.29, SD = 2.9; Mmen = 6.93, SD = 3.28; t(164.61) = 1.60, p = .11), or romantic loneliness (Mwomen = 22.14, SD = 11.90; Mmen = 19.27, SD = 11.52; t(188.41) = 1.93, p = .056). Using the latent means, rather than the sum scores, leads to the same conclusions (see ESM).

Discussion [TOP]

In this short paper, we used a fairly large German and Dutch adult sample to replicate the three-factor structure that has previously been reported for the abbreviated form of the SELSA from a sample of Canadian students (Cramer et al., 2000) and, in a 5-point scale version of the questionnaire, from a sample of Brazilian students (Rabelo, 2017). This three-factor structure indicates that satisfying relationships with friends, family, and romantic partners are distinct; people may have fulfilling relationships with family but not partner, for example. Indeed, the correlations between scores on the friends, family, and romantic partners components (Figure 1) indicate a moderate correlation between family and social loneliness, but no statistically significant relationship between romantic loneliness and the other two types of loneliness. Perhaps as a corollary, romantic loneliness cannot be ameliorated by improving general social connections, and social loneliness cannot be solved through a strong relationship with a romantic partner (Weiss, 1973). The questionnaire, it seems, does not measure a unitary construct, which might be the case if participants’ answers were merely tapping into a single dimension such as self-esteem, negative affect, or people skills.

Our study did not detect a significant difference between men and women in their family, social, or romantic loneliness. Previous research using the same questionnaire to compare men and women found that male Chinese university students reported higher family loneliness (Deng, 2016), while male students at universities in Taiwan reported significantly greater social loneliness, and significantly lower romantic loneliness (Ong, Chang, & Wang, 2011). Gender similarities and differences in types of loneliness might arise from a variety of influences, including cultural norms around men’s and women’s social networks and romantic aspirations (e.g., Adamczyk, 2016; Perrin et al., 2011).

Consistent with Cramer et al. (2000), we found that respondents scored higher on the romantic loneliness than the family or social loneliness subscales. Three of the six romantic loneliness questionnaire items depend upon having a romantic partner, and participants should select the response coded as representing the greatest loneliness simply if they are single. Thus, the higher romantic loneliness scores might merely reflect participants’ single status. In this case, use of the phrase ‘romantic loneliness’ is perhaps misleading, because ‘loneliness’ has negative implications. How much does romantic loneliness matter, compared to social and family loneliness? Although causation is difficult to demonstrate, commitment to a romantic relationship corresponds positively with enhanced wellbeing (Dush & Amato, 2005), whereas social loneliness, but not romantic or family loneliness, corresponds to symptoms of depression and anxiety (DiTommaso & Spinner, 1997). Among adolescents, it is specifically family loneliness that has been associated with depression, anxiety, and some psychopathologies (Lasgaard, Goossens, Bramsen, Trillingsgaard, & Elklit, 2011). In future research, it would also be worthwhile to examine whether the relationships between the loneliness subscales, and outcome variables such as depression and anxiety, are also invariant between men and women.

Given the different consequences of the different types of loneliness, the three subscales of the SELSA are probably most usefully considered separately, rather than being combined into a single total. Further, the different subscales appear to correlate differently with other loneliness measures (Cramer et al., 2000), a pattern that is equally apparent when another variant on the SELSA is used (the SELSA-S; DiTommaso, Brannen, & Best, 2004; Goossens et al., 2009). Logically, a questionnaire such as the SELSA that asks about relationships with different people (i.e. family and partner), asking participants to evaluate statements such as, “I have a romantic partner with whom I share my most intimate thoughts and feelings”, will be predisposed to cleave along those joints; an individual might have family relationships but not a partner relationship, or vice versa. However, loneliness clearly also has a common core (DiTommaso et al., 2004). This is perhaps most apparent when questionnaires focus on the emotional experience of loneliness by asking about regular feelings of loneliness, as in the Revised UCLA Scale, which asks participants to evaluate such statements as, “I feel alone”, and which tends to be associated with unidimensional approaches to loneliness (Goossens et al., 2009). Whether loneliness is best considered as unidimensional or multidimensional depends on what aspects of loneliness (e.g. its experience or its sources) are the focus of the research.

Our findings corroborated the previously-established three-factor structure of the abbreviated form of the SELSA instrument, and also established its scalar invariance. Other versions of the SELSA exist, including the more widely-used SELSA-S (DiTommaso et al., 2004), which provides a balance between positively and negatively worded items (e.g. “I feel part of a group of friends” and “I don’t have any friends who share my views, but I wish I did”). In contrast, the abbreviated form of the SELSA (Cramer et al., 2000) that we used only contains one negatively worded item, in the romantic loneliness subscale. The inclusion of positively and negatively worded items is generally undertaken to try to control for participant bias to agree with any statement presented to them (acquiescence bias; Ray, 1983). However, some scholars argue that the inclusion of negatively worded items is not a sensible solution to acquiescence bias, and recommend against this practice (see e.g., Solís Salazar, 2015; Sonderen, Sanderman, & Coyne, 2013; Zhang & Savalei, 2016). We conclude that the abbreviated form of the SELSA is a useful tool within the kit of instruments available to loneliness researchers.

Funding [TOP]

The authors have no funding to report.

Competing Interests [TOP]

The authors have declared that no competing interests exist.

Acknowledgments [TOP]

The data were collected while the first author was at the University of Groningen. He wishes to thank Jana Niemann, Sam Roberts, and Robin Dunbar for assistance with the questionnaire development and his students for their assistance with data collection and processing.

Data Availability [TOP]

The data are freely available (see the Supplementary Materials section).

Supplementary Materials [TOP]

The data, code, and analysis document are available from the Open Science Framework (

Index of Supplementary Materials

  • Pollet, T. V., Saxton, T. K., & Mitchell, M. (2018). Measurement equivalence between men and women in the abbreviated Social and Emotional Loneliness Scale (SELSA) [Supplementary materials].

References [TOP]

  • Adamczyk, K. (2016). An investigation of loneliness and perceived social support among single and partnered young adults. Current Psychology, 35(4), 674-689. doi:10.1007/s12144-015-9337-7

  • Borys, S., & Perlman, D. (1985). Gender differences in loneliness. Personality and Social Psychology Bulletin, 11(1), 63-74. doi:10.1177/0146167285111006

  • Buz, J., & Pérez-Arechaederra, D. (2014). Psychometric properties and measurement invariance of the Spanish version of the 11-item De Jong Gierveld loneliness scale. International Psychogeriatrics, 26(9), 1553-1564. doi:10.1017/S1041610214000507

  • Cohen, J. (1988). Statistical power for the social sciences. Hillsdale, NJ, USA: LEA & associates.

  • Cramer, K. M., Ofosu, H. B., & Barry, J. E. (2000). An abbreviated form of the Social and Emotional Loneliness Scale for Adults (SELSA). Personality and Individual Differences, 28(6), 1125-1131. doi:10.1016/S0191-8869(99)00162-2

  • Deng, X. (2016). The study on the relationship among the temperament type, parental rearing pattern, and sense of loneliness of college students. Studies in Sociology of Science, 7(4), 66-74. doi:10.3968/8888

  • DiTommaso, E., & Spinner, B. (1993). The development and initial validation of the Social and Emotional Loneliness Scale for Adults (SELSA). Personality and Individual Differences, 14(1), 127-134. doi:10.1016/0191-8869(93)90182-3

  • DiTommaso, E., & Spinner, B. (1997). Social and emotional loneliness: A re-examination of Weiss’ typology of loneliness. Personality and Individual Differences, 22(3), 417-427. doi:10.1016/S0191-8869(96)00204-8

  • DiTommaso, E., Brannen, C., & Best, L. A. (2004). Measurement and validity characteristics of the short version of the Social and Emotional Loneliness Scale for Adults. Educational and Psychological Measurement, 64(1), 99-119. doi:10.1177/0013164403258450

  • Dush, C. M. K., & Amato, P. R. (2005). Consequences of relationship status and quality for subjective well-being. Journal of Social and Personal Relationships, 22(5), 607-627. doi:10.1177/0265407505056438

  • Dykstra, P. A., & Fokkema, T. (2007). Social and emotional loneliness among divorced and married men and women: Comparing the deficit and cognitive perspectives. Basic and Applied Social Psychology, 29(1), 1-12. doi:10.1080/01973530701330843

  • Goossens, L., Lasgaard, M., Luyckx, K., Vanhalst, J., Mathias, S., & Masy, E. (2009). Loneliness and solitude in adolescence: A confirmatory factor analysis of alternative models. Personality and Individual Differences, 47(8), 890-894. doi:10.1016/j.paid.2009.07.011

  • Hagestad, G. O. (1986). The family: Women and grandparents as kin-keepers. In A. Pifer & L. Bronte (Eds.), Our aging society: Paradox and promise (pp. 141–160). New York, NY, USA: WW Norton.

  • Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2011). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42(8), ). doi:10.18637/jss.v042.i08

  • Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods & Research, 44(3), 486-507. doi:10.1177/0049124114543236

  • Lasgaard, M., Goossens, L., Bramsen, R. H., Trillingsgaard, T., & Elklit, A. (2011). Different sources of loneliness are associated with different forms of psychopathology in adolescence. Journal of Research in Personality, 45(2), 233-237. doi:10.1016/j.jrp.2010.12.005

  • Lee, C.-Y. S., & Goldstein, S. E. (2016). Loneliness, stress, and social support in young adulthood: Does the source of support matter? Journal of Youth and Adolescence, 45(3), 568-580. doi:10.1007/s10964-015-0395-9

  • Maes, M., Klimstra, T., Van den Noortgate, W., & Goossens, L. (2015). Factor structure and measurement invariance of a multidimensional loneliness scale: Comparisons across gender and age. Journal of Child and Family Studies, 24(6), 1829-1837. doi:10.1007/s10826-014-9986-4

  • Milfont, T. L., & Fischer, R. (2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3(1), 111-130. doi:10.21500/20112084.857

  • Moore, G. (1990). Structural determinants of men’s and women’s personal networks. American Sociological Review, 55(5), 726-735. doi:10.2307/2095868

  • Ong, C.-S., Chang, S.-C., & Wang, C.-C. (2011). Comparative loneliness of users versus nonusers of online chatting. Cyberpsychology, Behavior, and Social Networking, 14(1-2), 35-40. doi:10.1089/cyber.2009.0321

  • Perrin, P. B., Heesacker, M., Tiegs, T. J., Swan, L. K., Lawrence, A. W., Smith, M. B., . . . Mejia-Millan, C. M., (2011). Aligning Mars and Venus: The social construction and instability of gender differences in romantic relationships. Sex Roles, 64(9-10), 613-628. doi:10.1007/s11199-010-9804-4

  • Pinquart, M., & Sorensen, S. (2001). Gender differences in self-concept and psychological well-being in old age: A meta-analysis. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 56(4), P195-P213. doi:10.1093/geronb/56.4.P195

  • Pollet, T. V., Roberts, S. G. B., & Dunbar, R. I. M. (2013). Going that extra mile: Individuals travel further to maintain face-to-face contact with highly related kin than with less related kin. PLoS One, 8(1), Article e53929. doi:.doi:10.1371/journal.pone.0053929

  • R Development Core Team. (2008). R: A language and environment for statistical computing. Vienna, Austria: author.

  • Rabelo, A. L. A. (2017). The proposal and initial exploration of the Connectedness and Loneliness Model (CLM) (Doctoral thesis, Universidade de Brasília). Retrieved from{\_}Andr{\'{e}}LuizAlvesRabelo.pdf

  • Ray, J. J. (1983). Reviving the problem of acquiescent response bias. The Journal of Social Psychology, 121(1), 81-96. doi:10.1080/00224545.1983.9924470

  • Rico-Uribe, L. A., Caballero, F. F., Martín-María, N., Cabello, M., Ayuso-Mateos, J. L., & Miret, M. (2018). Association of loneliness with all-cause mortality: A meta-analysis. PLoS One, 13(1), Article e0190033. doi:10.1371/journal.pone.0190033

  • Roberts, S. G. B., Dunbar, R. I. M., Pollet, T. V., & Kuppens, T. (2009). Exploring variation in active network size: Constraints and ego characteristics. Social Networks, 31(2), 138-146. doi:10.1016/j.socnet.2008.12.002

  • Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1-36. doi:10.18637/jss.v048.i02

  • Russell, D. W. (1996). UCLA Loneliness Scale (Version 3): Reliability, validity, and factor structure. Journal of Personality Assessment, 66(1), 20-40. doi:10.1207/s15327752jpa6601_2

  • Sass, D. A., & Schmitt, T. A. (2013). Testing measurement and structural invariance. In T. Teo (Ed.), Handbook of quantitative methods for educational research (pp. 315–345). Rotterdam, The Netherlands: SensePublishers. doi:10.1007/978-94-6209-404-8_15

  • Schmitt, N., & Kuljanin, G. (2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18(4), 210-222. doi:10.1016/j.hrmr.2008.03.003

  • Schoot, R. van de, Lugtig, P., & Hox, J. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486-492. doi:10.1080/17405629.2012.686740

  • Solís Salazar, M. (2015). The dilemma of combining positive and negative items in scales. Psicothema, 27(2), 192-200. doi:10.7334/psicothema2014.266

  • Sonderen, E. van, Sanderman, R., & Coyne, J. C. (2013). Ineffectiveness of reverse wording of questionnaire items: Let’s learn from cows in the rain. PLoS One, 8(7), Article e68967. doi:10.1371/journal.pone.0068967

  • Tornstam, L. (1992). Loneliness in marriage. Journal of Social and Personal Relationships, 9(2), 197-217. doi:10.1177/0265407592092003

  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. doi:10.1177/109442810031002

  • Weiss, R. S. (1973). Loneliness: The experience of emotional and social isolation. Cambridge, MA, USA: The MIT Press.

  • Yeginsu, C. (2017, January). U.K. appoints a minister for loneliness. Retrieved from

  • Zhang, X., & Savalei, V. (2016). Improving the factor structure of psychological scales: The expanded format as an alternative to the Likert scale format. Educational and Psychological Measurement, 76(3), 357-386. doi:10.1177/0013164415596421