This is an open access article distributed under the terms of the Creative Commons
Attribution License (Loneliness has been consistently associated with negative health outcomes (review in Rico-Uribe et al., 2018). It has also been placed on the national social and political agenda in countries such as the United Kingdom (Yeginsu, 2017). One of the earliest attempts to conceptualise loneliness by Weiss (1973) moved away from treating it as a unitary state which only varies in frequency or intensity, towards differentiating emotional from social loneliness. Emotional loneliness arises from a lack of close emotional ties (e.g., spouse, parent). In contrast, social loneliness is a consequence of not having affiliative relationships such as friendships. This differentiation between social and emotional loneliness has led to the development of multidimensional measurement instruments such as the Social and Emotional Loneliness Scale for Adults (SELSA, DiTommaso & Spinner, 1993, 1997); the authors further divided emotional loneliness into family and romantic emotional loneliness, following principal component analysis of this scale.
Research exploring gender differences in loneliness presents mixed findings, with some research suggesting that women report more loneliness than men do (e.g., Borys & Perlman, 1985; Pinquart & Sorensen, 2001), others that men report more (e.g., Lee & Goldstein, 2016; Russell, 1996), and still others that do not find a gender difference (e.g., Tornstam, 1992). Much of this research has relied on a unidimensional approach to loneliness rather than a multidimensional approach. There is reason to expect that men and women might differ from each other when specific aspects of loneliness are considered separately, as part of a multidimensional approach. For example, women have been argued to be kin keepers and to have more intimate ties with family members than men have (e.g., Hagestad, 1986; Moore, 1990), and so might have lower levels of family loneliness in general, but might suffer more from loneliness if they do not achieve the family relationships they desire (Maes, Klimstra, Van den Noortgate, & Goossens, 2015). Similarly, research has shown that men attach greater value to having a romantic partner than women do (e.g., Dykstra & Fokkema, 2007), again perhaps indicating greater variation in men’s than women’s romantic loneliness. Such reasoning has previously led researchers to examine the measurement invariance across genders of various loneliness scales (Buz & Pérez-Arechaederra, 2014; Maes et al., 2015), and to recommend that similar analyses should be carried out on other multidimensional loneliness scales (Maes et al., 2015). Indeed, the testing of measurement invariance across genders is a regular part of the development of psychometric questionnaires (Schmitt & Kuljanin, 2008; Vandenberg & Lance, 2000). If a questionnaire does not demonstrate measurement invariance across different groups (such as genders), this would indicate that the questionnaire could be measuring a different underlying construct in each group. Here, we examine whether the abbreviated version of the SELSA (Cramer, Ofosu, & Barry, 2000) measures the same construct in men and women by testing measurement invariance across genders. We thereby set out to uncover more about how the sources of loneliness may differ between men and women, alongside testing some of the standard psychometric properties of an established questionnaire.
Method
Participants
In order to obtain a large sample that extended beyond students, participants were recruited via the personal networks of students enrolled at a large Dutch university (see Pollet, Roberts, & Dunbar, 2013 for full description). There is a substantial number of German participants as this study proved popular with German-speaking students who had fewer options of studies to participate in for credit. 458 surveys were processed (301 women, M = 30.97 years, SD = 14.55 years, 3 did not report age or gender); due to some non-responses, 382 of those were able to be used for the current research project.
Given that we wanted to compare men and women, it was important to match these samples to ensure that any differences were not due to other sample characteristics. Using the ‘MatchIt’ algorithm (Ho, Imai, King, & Stuart, 2011), we exactly matched men and women on age, marital status (“Single (never married)”, “Living with partner”, “Married (first marriage)”, “Remarried”, “Divorced”, “Separated”, “Widowed”), educational attainment (“no diploma/degree”,“primary education”,“lower secondary”, “higher secondary”, “higher education/ non-university degree”, “university degree”,“other”), and nationality (“Dutch”, “German”, “Other”). This left us with 273 cases (age: M = 25.97 years, SD = 11.17 years; 181 women). The majority were German (78.75%) and never married (78%). In terms of educational attainment, 40.65% had previously obtained a higher secondary school degree (i.e. qualifications derived from education until around the age of 16), 35.9% a non-university higher, 11.36% had completed a university degree, and the remainder had completed a lower secondary school degree or a lower degree (i.e. qualifications derived from education up until around the age of 14). It is important to note that we asked about educational attainment, and therefore the category of secondary school degree also contains students currently enrolled upon a university degree.
Procedure
After providing informed consent, participants completed a paper-based survey in German or Dutch depending on participant preference. Participants first provided basic sociodemographic data, including age, gender, nationality and educational attainment (similar to Roberts, Dunbar, Pollet, & Kuppens, 2009). They then completed a questionnaire on their living relatives (see Pollet et al., 2013; data not used here), and finally the SELSA.
Materials
The abbreviated form of the SELSA consists of 15 items, of which 5 items make up the family loneliness sub-scale, 6 items make up the romantic sub-scale, and 4 refer to social loneliness (Cramer et al., 2000) (Table 1). All items were answered on a 7-point Likert scale (1=Strongly Disagree to 7=Strongly Agree) and then reverse scored with the exception of ‘Romantic3’.
Table 1
Descriptive Statistics (N = 273) of Questionnaire Items (1-7 Scale; All Items Except Romantic3 Were Reverse-Scored, Such That Higher Scores Here Indicate Greater Loneliness). Items Were Translated in Dutch or German.
| Item | M | SD | Min | Max |
|---|---|---|---|---|
| I really belong in my family (Family1) | 1.780 | 1.062 | 1 | 6 |
| I feel part of my family (Family2) | 1.722 | 1.079 | 1 | 6 |
| My family really cares about me (Family3) | 1.879 | 1.096 | 1 | 7 |
| My family is important to me (Family4) | 1.593 | 0.790 | 1 | 6 |
| I feel close to my family (Family5) | 1.886 | 1.042 | 1 | 7 |
| I have a romantic partner with whom I share my most intimate thoughts and feelings (Romantic1) | 3.593 | 2.310 | 1 | 7 |
| I have a romantic or marital partner who gives me the support and encouragement I need (Romantic2) | 3.582 | 2.328 | 1 | 7 |
| I have an unmet need for a close romantic relationship (Romantic3) | 3.352 | 2.044 | 1 | 7 |
| I am in love with someone who is in love with me (Romantic4) | 3.645 | 2.314 | 1 | 7 |
| I have someone who fulfils my needs for intimacy (Romantic5) | 3.344 | 2.112 | 1 | 7 |
| I have a romantic partner to whose happiness I contribute (Romantic6) | 3.659 | 2.308 | 1 | 7 |
| I have friends that I can turn to for information (Social1) | 1.484 | 0.718 | 1 | 5 |
| I can depend upon my friends for help (Social2) | 1.703 | 0.991 | 1 | 7 |
| I have friends to whom I can talk about the pressures in my life (Social3) | 1.670 | 0.912 | 1 | 6 |
| I have a friend(s) with whom I can share my views (Social4) | 1.648 | 1.068 | 1 | 7 |
Data Analysis
All the analyses were conducted in R 3.5.1 (R Development Core Team, 2008). We used ‘lavaan’ (Rosseel, 2012) to perform confirmatory factor analysis and examine measurement invariance (Sass & Schmitt, 2013; Schoot, Lugtig, & Hox, 2012; Vandenberg & Lance, 2000). The data, code, and analysis document are available from the Open Science Framework (https://osf.io/rq5sy/).
Results
Descriptive Statistics
Table 1 shows the descriptive statistics for all items, following reverse scoring.
Measurement Invariance
The model proved a good fit in both CFI (.922) and TLI (.906) but not in RMSEA (.105), something that could be a consequence of the relatively small sample size for our models (Kenny, Kaniskan, & McCoach, 2015). We also attempted a single factor model which proved to be a substantially worse fit to the data (CFI = .333, TLI = .222, RMSEA = .302). This corroborates that the SELSA is composed of the three factors identified previously (Cramer et al., 2000). Figure 1 summarizes the loadings for men and women.
Figure 1
SEM Plots for men and women. Fml = family loneliness, Rmn = romantic loneliness, Scl = social loneliness.
Table 2 shows the measurement invariance summary. It shows that there is no substantial loss in fit when moving from configural to metric, and from metric to scalar measurement invariance. However, there is a loss of fit when moving from scalar invariance to residual invariance. This implies that we cannot conclude that the latent structure is measured identically in men and women. In conclusion, we can establish scalar invariance, i.e. we can compare men and women on their latent variable scores. We were able to recover the proposed three factor structure in both groups of men and women. Given that we did not achieve residual invariance, we did not examine structural invariance (Milfont & Fischer, 2010).
Table 2
Measurement Invariance Summary
| Invariance | χ2 | df | Δχ2 | df | p | CFI | ΔCFI | RMSEA | ΔRMSEA | BIC | ΔBIC |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Configural | 435.1 | 174 | NA | NA | NA | 0.922 | NA | 0.105 | NA | 11413.9 | NA |
| Metric | 455.0 | 186 | 20.0 | 12 | .068 | 0.920 | 0.002 | 0.103 | 0.002 | 11366.6 | 47.4 |
| Scalar | 471.6 | 198 | 16.6 | 12 | .165 | 0.918 | 0.001 | 0.101 | 0.002 | 11315.9 | 50.7 |
| Residual | 529.0 | 213 | 57.3 | 15 | < .001 | 0.906 | 0.013 | 0.104 | -0.004 | 11289.1 | 26.8 |
| Mean | 535.9 | 216 | 7.0 | 3 | .074 | 0.904 | 0.001 | 0.104 | 0.000 | 11279.2 | 9.9 |
Correlations Between the Three Latent Constructs (Family, Romantic and Social Loneliness)
Figure 1 shows the correlations between the latent constructs. For both men and women, there was a significant association between Family and Social loneliness (respectively: 0.333+/-0.10 and 0.358+/-0.07, p = .002 and p < .001). However, there were no significant associations between Family and Romantic loneliness (men: p = .067 and women: p = .161) or between Romantic and Social loneliness (men: p = .412 and women: p = .161).
Gender Differences
We summed all items to generate scores for family, romantic and social loneliness. With 80% power and a two-sided significance level of 5%, we can detect a relatively small effect size (Cohen’s d = .36, Cohen, 1988). There was no good evidence for a difference between men and women in terms of family loneliness (Mwomen = 8.58, SD = 4.4; Mmen = 9.41, SD = 4.29; t(187.27) = 1.5, p = .13), social loneliness (Mwomen = 6.29, SD = 2.9; Mmen = 6.93, SD = 3.28; t(164.61) = 1.60, p = .11), or romantic loneliness (Mwomen = 22.14, SD = 11.90; Mmen = 19.27, SD = 11.52; t(188.41) = 1.93, p = .056). Using the latent means, rather than the sum scores, leads to the same conclusions (see ESM).
Discussion
In this short paper, we used a fairly large German and Dutch adult sample to replicate the three-factor structure that has previously been reported for the abbreviated form of the SELSA from a sample of Canadian students (Cramer et al., 2000) and, in a 5-point scale version of the questionnaire, from a sample of Brazilian students (Rabelo, 2017). This three-factor structure indicates that satisfying relationships with friends, family, and romantic partners are distinct; people may have fulfilling relationships with family but not partner, for example. Indeed, the correlations between scores on the friends, family, and romantic partners components (Figure 1) indicate a moderate correlation between family and social loneliness, but no statistically significant relationship between romantic loneliness and the other two types of loneliness. Perhaps as a corollary, romantic loneliness cannot be ameliorated by improving general social connections, and social loneliness cannot be solved through a strong relationship with a romantic partner (Weiss, 1973). The questionnaire, it seems, does not measure a unitary construct, which might be the case if participants’ answers were merely tapping into a single dimension such as self-esteem, negative affect, or people skills.
Our study did not detect a significant difference between men and women in their family, social, or romantic loneliness. Previous research using the same questionnaire to compare men and women found that male Chinese university students reported higher family loneliness (Deng, 2016), while male students at universities in Taiwan reported significantly greater social loneliness, and significantly lower romantic loneliness (Ong, Chang, & Wang, 2011). Gender similarities and differences in types of loneliness might arise from a variety of influences, including cultural norms around men’s and women’s social networks and romantic aspirations (e.g., Adamczyk, 2016; Perrin et al., 2011).
Consistent with Cramer et al. (2000), we found that respondents scored higher on the romantic loneliness than the family or social loneliness subscales. Three of the six romantic loneliness questionnaire items depend upon having a romantic partner, and participants should select the response coded as representing the greatest loneliness simply if they are single. Thus, the higher romantic loneliness scores might merely reflect participants’ single status. In this case, use of the phrase ‘romantic loneliness’ is perhaps misleading, because ‘loneliness’ has negative implications. How much does romantic loneliness matter, compared to social and family loneliness? Although causation is difficult to demonstrate, commitment to a romantic relationship corresponds positively with enhanced wellbeing (Dush & Amato, 2005), whereas social loneliness, but not romantic or family loneliness, corresponds to symptoms of depression and anxiety (DiTommaso & Spinner, 1997). Among adolescents, it is specifically family loneliness that has been associated with depression, anxiety, and some psychopathologies (Lasgaard, Goossens, Bramsen, Trillingsgaard, & Elklit, 2011). In future research, it would also be worthwhile to examine whether the relationships between the loneliness subscales, and outcome variables such as depression and anxiety, are also invariant between men and women.
Given the different consequences of the different types of loneliness, the three subscales of the SELSA are probably most usefully considered separately, rather than being combined into a single total. Further, the different subscales appear to correlate differently with other loneliness measures (Cramer et al., 2000), a pattern that is equally apparent when another variant on the SELSA is used (the SELSA-S; DiTommaso, Brannen, & Best, 2004; Goossens et al., 2009). Logically, a questionnaire such as the SELSA that asks about relationships with different people (i.e. family and partner), asking participants to evaluate statements such as, “I have a romantic partner with whom I share my most intimate thoughts and feelings”, will be predisposed to cleave along those joints; an individual might have family relationships but not a partner relationship, or vice versa. However, loneliness clearly also has a common core (DiTommaso et al., 2004). This is perhaps most apparent when questionnaires focus on the emotional experience of loneliness by asking about regular feelings of loneliness, as in the Revised UCLA Scale, which asks participants to evaluate such statements as, “I feel alone”, and which tends to be associated with unidimensional approaches to loneliness (Goossens et al., 2009). Whether loneliness is best considered as unidimensional or multidimensional depends on what aspects of loneliness (e.g. its experience or its sources) are the focus of the research.
Our findings corroborated the previously-established three-factor structure of the abbreviated form of the SELSA instrument, and also established its scalar invariance. Other versions of the SELSA exist, including the more widely-used SELSA-S (DiTommaso et al., 2004), which provides a balance between positively and negatively worded items (e.g. “I feel part of a group of friends” and “I don’t have any friends who share my views, but I wish I did”). In contrast, the abbreviated form of the SELSA (Cramer et al., 2000) that we used only contains one negatively worded item, in the romantic loneliness subscale. The inclusion of positively and negatively worded items is generally undertaken to try to control for participant bias to agree with any statement presented to them (acquiescence bias; Ray, 1983). However, some scholars argue that the inclusion of negatively worded items is not a sensible solution to acquiescence bias, and recommend against this practice (see e.g., Solís Salazar, 2015; Sonderen, Sanderman, & Coyne, 2013; Zhang & Savalei, 2016). We conclude that the abbreviated form of the SELSA is a useful tool within the kit of instruments available to loneliness researchers.