# QuantMethodsSession2.pdf

51Quant. Methods 1 (M. Messner)

Concepts and their measurement

Pixabay.com

Nasser Alajmi

52Quant. Methods 1 (M. Messner)

Concepts and their measurement

• Research is usually interested in relationships between different phenomena or concepts.

– E.g., Intelligence, organizational performance, customer satisfaction, strategic position, etc.

– Some phenomena are directly observable and can thus be (rather) easily measured (e.g., age, industry affiliation, revenues, etc.)

– Other phenomena are much harder to capture.

• Construct validity is an important concern in quantitative research

Nasser Alajmi

53Quant. Methods 1 (M. Messner)

Concepts and their measurement

Concept measurement depends on the method of data collection:

• Archival research: Use of secondary data obtained from firms, public sources, databases, etc.

• Survey studies: Collection of data through questionnaires

• Experiments: Collection of data through laboratory replication of a real-world situation

Nasser Alajmi

54Quant. Methods 1 (M. Messner)

Concepts and their measurement

Examples for archival proxies:

• “To proxy and control for the firm’s innovative capability or capacity, we include the variable cumulative number of patents, which is the total number of patents that the firm has generated since the year of its establishment until the year before the observation year.” (Jia et al. 2019, AMJ)

• “As a proxy for project complexity, I used the total number of electrical engineering hours on a project”. (Young-Hyman, 2017, ASQ)

Nasser Alajmi

55Quant. Methods 1 (M. Messner)

Concepts and their measurement

Using surveys to measure concepts

 Example

nasseralajmi
Sticky Note
Ordinar values are problomatic cause we can't put them on a regression
Nasser Alajmi

56Quant. Methods 1 (M. Messner)

Concepts and their measurement

• Surveys often rely on so-called Likert scales to measure the extent to which the respondent agrees or disagrees with a statement.

– Likert-scales can have different degrees of detail, but most common are 7-point scales.

– End-points represent the most extreme positions on a question, e.g.:

• 1 = definitely false; 7 = definitely true; 1 = strongly disagree, 7 = strongly agree, etc.

• The different questions represent “items“. The construct (representing the concept of interest) is then a combination of these items.

– Items either taken from prior research or newly developed scale

– # of items: balance length of questionnaire with measurement quality; at least 3 items for a (reflective) construct

nasseralajmi
Highlight
nasseralajmi
Sticky Note
We do this cause it is most likely already proven
nasseralajmi
Sticky Note
Why more questions are better?- Help to find if people who answer actually answer it well.- It gives a good confidence at the end
Nasser Alajmi

57Quant. Methods 1 (M. Messner)

Concepts and their measurement

Compare the two regressions:

Manifest variables (items) x1-x5:

y = ß0 + ß1* x1 + ß2 * x2 + ß3 * x3 + ß4 * x4 + ß5 * x5 + u

One latent construct z1:

y = ß0 + ß1* z1 + u

nasseralajmi
Sticky Note
The more valuable we have , the bigger the regression becomes and this is problematic
Nasser Alajmi

58Quant. Methods 1 (M. Messner)

Concepts and their measurement

There are two types of multi-item (latent) constructs:

• Reflective: Items represent manifestations of the construct

• Formative: Items represent elements (components) of the construct

This distinction is important, because construct validity and reliability of these constructs must be assessed differently.

nasseralajmi
Sticky Note
We can do statistical relation here
Nasser Alajmi

59Quant. Methods 1 (M. Messner)

Concepts and their measurement

Monitoring difficulty

It is not possible to supervise the salesperson’s activities closely.

It is difficult for us to evaluate how much effort this salesperson really puts into her/his job.

It is relatively easy for this salesperson to turn in falsified sales call reports.

Our evaluation of this salesperson cannot be based on his/her activity and sales call reports.

Salesperson’s ability

This salesperson has a high degree of competence in tailoring his/her sales approach to the specific situation on hand.

This salesperson has been very creative in designing relevant solutions to customers’ problems.

This salesperson is a skilled and persuasive negotiator.

This salesperson is capable of closing a deal in a tough selling situation.

This salesperson is able to learn from past experiences and adapt them to current circumstances.

This salesperson is skilled in extracting the unique problems faced by and the requirements of his/her customers.

Lo et al. (2011, JMR)

Nasser Alajmi

60Quant. Methods 1 (M. Messner)

Concepts and their measurement

Construct validity (Do we really measure what we seek to measure?) and reliability (Do we measure reliably?) are a frequent concern in survey research.

Construct validity and reliability should be approached from a judgmental and a technical perspective:

• Judgmental (for both reflective and formative constructs):

– Do the questions really reflect what I think the phenomenon is about? (often referred to as “face validity”)

Nasser Alajmi

61Quant. Methods 1 (M. Messner)

Concepts and their measurement

• Technical (for reflective constructs only):

– Internal consistency and convergent validity: do the items converge on the construct?

– Discriminant validity: are the constructs sufficiently different from each other?

Nasser Alajmi

62Quant. Methods 1 (M. Messner)

Concepts and their measurement

Reflective constructs Formative

constructs

(composite

constructs)

Face validity Items should all reflect the underlying construct (in a similar way)

Items should together add up to the construct (in a comprehensive way)

Internal consistency & convergent validity

Factor analysis:• High factor loadings of each item• High Average Variance Extracted

(AVE)• High composite reliability• Good goodness-of fit criteria

correlations between constructs (Fornell-Larcker Test)

Nasser Alajmi

63Quant. Methods 1 (M. Messner)

Concepts and their measurement

The basis for assessing construct validity and reliability of reflective

constructs is typically factor analysis.

• Factor analysis:

– Identifies (or confirms) the relationship between items and underlying constructs (factors)

– Assumes that the correlation between items can be explained by underlying factors (of which the items are manifestations)

– Exploratory factor analysis (EFA): no ex ante specification of factor structure: see which factors emerge from the data

– Confirmatory factor analysis (CFA): factors and their relationships to items are specified a priori and then tested. Confirm if items load as they should.

• Different uses in studies:

– Only EFA or only CFA

– First an EFA, then a CFA

nasseralajmi
Sticky Note
Hypothetical construct
nasseralajmi
Highlight
nasseralajmi
Highlight
nasseralajmi
Sticky Note
The automatic one!
Nasser Alajmi

64Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Example

We are interested in 4 (reflective) constructs and, to this end, collected 16 variables through a survey.

Our hypothesized factor structure is as follows:

Factor1: v2 + v9 + v13 + v16 + v17Factor2: v6 + v7 + v8 + v11Factor3: v1 + v3 + v5 + v20Factor4: v4 + v12 + v14

Nasser Alajmi

65Quant. Methods 1 (M. Messner)

Exploratory factor analysis

v1 v2 v3 v4 v5 v6 v7 v8 v9 v11 v12 v13 v14 v16 v17 v20v1 1,00v2 -0,05 1,00v3 0,44 0,08 1,00v4 0,00 -0,07 0,10 1,00v5 0,41 0,00 0,40 0,13 1,00v6 0,02 0,02 0,07 0,02 0,19 1,00v7 0,01 -0,09 0,00 0,13 0,08 0,34 1,00v8 0,05 -0,02 0,03 0,02 0,09 0,31 0,37 1,00v9 -0,05 0,25 -0,06 0,03 -0,03 -0,12 -0,06 -0,04 1,00v11 0,04 -0,10 0,08 0,04 0,16 0,47 0,48 0,37 -0,10 1,00v12 0,11 0,06 0,08 0,29 0,17 0,12 0,03 0,06 0,03 0,03 1,00v13 -0,13 0,31 -0,03 0,05 0,05 -0,08 0,02 -0,09 0,32 -0,08 0,08 1,00v14 0,02 -0,04 0,04 0,34 0,11 -0,05 0,00 0,02 0,11 -0,07 0,25 0,10 1,00v16 -0,05 0,28 0,03 0,08 0,00 -0,13 0,00 -0,04 0,36 -0,15 0,08 0,38 0,03 1,00v17 -0,01 0,28 0,01 -0,04 -0,01 -0,14 -0,06 -0,07 0,36 -0,18 0,04 0,33 -0,01 0,29 1,00v20 0,35 0,01 0,40 0,04 0,36 0,00 0,09 -0,06 -0,05 0,00 0,10 0,00 0,03 0,03 0,07 1,00

Let‘s first look at the correlations between the variables

nasseralajmi
Sticky Note
Correlation means when I answer the survey question similarly
nasseralajmi
Sticky Note
+ means I have similar answersExp. first question 5 then second is 5- means I have opposite answersExp. first question 5 then second 1
Nasser Alajmi

66Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Fa

Fb

x1

x2

x3

x4

Latent factors Manifest variables

a1

a2

b1

b4

X1 = a1*Fa + b1*Fb + u1

X2 = a2*Fa + b2*Fb + u2

etc

All relationships possible  Exploratory factor analysis:

Must specify either:- Number of factors (in line with your theory)- Minimum explanatory value of each factor (typically:

Eigenvalue > 1).

Two main “extraction methods“ for factors:- Principal component analysis: factors try to explain

entire variance of variables- Principal axis factoring: factors explain only shared

variance of variables

EFA: finding a good factor structure

Nasser Alajmi

67Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Illustration of factor structure

Factor 1

Factor 2

1

0.5

– 0.5

– 0.5 10.5

– 1

– 1

Variable x1 = 0.6 * F1 + 0.6 * F2 + u

Nasser Alajmi

68Quant. Methods 1 (M. Messner)

Exploratory factor analysis

SPSS: Analyze  Dimension Reduction  Factor

Nasser Alajmi

69Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Output from SPSS

nasseralajmi
Highlight
Nasser Alajmi

70Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Output from SPSS

nasseralajmi
Highlight
nasseralajmi
Sticky Note
Dimension reduction
Nasser Alajmi

71Quant. Methods 1 (M. Messner)

Exploratory factor analysis

• Eigenvalues: an indicator for how much variance in the variables the factor explains

• E.g., when Eigenvalue of each Factor = 1  each Factor explains the same amount of variance

• Mathematically: the sum of the squared loadings across all variables for a given factor (see next page)

• Variance Extracted: how much % of the shared variance in all variables is explained by the factor.

• E.g., when Eigenvalue (Factor 1) = 1, and 5 Factors  Factor 1 explains 20% of variance

Output from EFA

Nasser Alajmi

72Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Output from SPSS

Factor Matrixa

Factor1 2 3 4

v1 0,336 0,419 -0,350 -0,130v2 -0,281 0,292 0,207 -0,220v3 0,319 0,508 -0,280 -0,133v4 0,120 0,259 0,122 0,559v5 0,388 0,499 -0,135 -0,036v6 0,497 -0,018 0,330 -0,103v7 0,444 0,016 0,438 -0,032v8 0,387 -0,026 0,338 -0,049v9 -0,389 0,304 0,280 -0,044v11 0,622 -0,070 0,424 -0,126v12 0,107 0,303 0,100 0,308v13 -0,370 0,364 0,345 -0,048v14 -0,002 0,244 0,049 0,515v16 -0,363 0,380 0,270 -0,066v17 -0,388 0,342 0,175 -0,180v20 0,226 0,447 -0,257 -0,118Extraction Method: Principal Axis Factoring.a. 4 factors extracted. 12 iterations required.

nasseralajmi
Highlight
nasseralajmi
Sticky Note
The correlations between each factor
Nasser Alajmi

73Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Factor analysis first provides an “unrotated” solution.

• First factor is defined in such a way that it explains most of the shared variance of all variables  almost any variable will load on this factor

• Residual variance to be explained then by second, third, etc. factor

Rotation changes the factor structure such that variance is explained more equally among factors.

• The most frequently used method is “Varimax“, which defines factors in such a way that they have high or low loadings, but not medium ones.

Unrotated and rotated factor structure

nasseralajmi
Highlight
nasseralajmi
Highlight
Nasser Alajmi

74Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Unrotated and rotated factor structure

Hair et al., p. 111

Nasser Alajmi

75Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Output from SPSS

Nasser Alajmi

76Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Output from SPSS:

rotated solution

Rotated Factor Matrix

Factor1 2 3 4

v1 -0,002 -0,089 0,648 0,012v2 -0,019 0,496 0,045 -0,083v3 0,033 0,009 0,672 0,051v4 0,065 -0,007 0,040 0,635v5 0,166 0,013 0,604 0,165v6 0,593 -0,095 0,078 -0,007v7 0,619 -0,008 0,010 0,083v8 0,514 -0,047 0,006 0,031v9 -0,073 0,553 -0,073 0,084v11 0,750 -0,141 0,058 -0,022v12 0,082 0,083 0,140 0,418v13 -0,016 0,613 -0,051 0,114v14 -0,063 0,027 0,020 0,567v16 -0,060 0,584 0,003 0,093v17 -0,123 0,559 0,039 -0,042v20 -0,016 0,034 0,572 0,038Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.a. Rotation converged in 5 iterations.

Nasser Alajmi

77Quant. Methods 1 (M. Messner)

Exploratory factor analysis

Output from SPSS:

rotated solution

Rotated Factor Matrix

Factor1 2 3 4

v1 -0,002 -0,089 0,648 0,012v2 -0,019 0,496 0,045 -0,083v3 0,033 0,009 0,672 0,051v4 0,065 -0,007 0,040 0,635v5 0,166 0,013 0,604 0,165v6 0,593 -0,095 0,078 -0,007v7 0,619 -0,008 0,010 0,083v8 0,514 -0,047 0,006 0,031v9 -0,073 0,553 -0,073 0,084v11 0,750 -0,141 0,058 -0,022v12 0,082 0,083 0,140 0,418v13 -0,016 0,613 -0,051 0,114v14 -0,063 0,027 0,020 0,567v16 -0,060 0,584 0,003 0,093v17 -0,123 0,559 0,039 -0,042v20 -0,016 0,034 0,572 0,038Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.a. Rotation converged in 5 iterations.

Guide:

Nasser Alajmi

79Quant. Methods 1 (M. Messner)

Confirmatory factor analysis

Fa

Fb

x1

x2

x3

x4

Latent factors Manifest variables

a1

a2

b3

b4

X1 = a1*Fa + u1

X2 = a2*Fa + u2

X3 = b3*Fb + u3

X4 = b4*Fb + u4

Relationship specified  Confirmatory factor analysis:

CFA tests how well the specified model fits the data

Nasser Alajmi

80Quant. Methods 1 (M. Messner)

Confirmatory factor analysis

CFA produces one hypothesis test and various “fit indicators“

Indicator Description “good“ fit

GFI Goodness of fit GFI >= 0.95

RMSEA Root mean square error of approximation

RMSEA < 0.08

CFI Comparative fit index CFI >= 0.90

SRMR (Standardized) root meansquare residual

(S)RMR < 0.08

Hypothesis test (chi-squared):

H0: “The model fits perfectly.”P-value > 0.05 so as not to reject this H0

Typically these indicators are calculated for each factor separately and for the entire model as a whole

Nasser Alajmi

81Quant. Methods 1 (M. Messner)

Confirmatory factor analysis

Confirmatory factor analysis output

Model Test User Model:

Test statistic 95.847

Degrees of freedom 98

P-value (Chi-square) 0.543

cfi rmsea gfi rmr1.000 0.000 0.954 0.113

Nasser Alajmi

82Quant. Methods 1 (M. Messner)

Confirmatory factor analysis

Confirmatory factor analysis output

Latent Variables:Estimate Std.Err z-value P(>|z|) Std.lv Std.all

factor1 =~ v2 1.000 0.771 0.474v9 1.216 0.229 5.302 0.000 0.938 0.579v13 1.259 0.233 5.395 0.000 0.971 0.605v16 1.361 0.254 5.363 0.000 1.050 0.596v17 1.123 0.215 5.219 0.000 0.866 0.559

Nasser Alajmi

83Quant. Methods 1 (M. Messner)

Additional checks for convergent and discriminant validity

Convergent validity (for each factor separately):

• High Average Variance Extracted (guide: > 0.5)• How much of the variance in the items (averaged across all items of the given

factor) is explained by the given latent factor• E.g. AVE = 0.4  Factor explains 40% of the variation in item values

• High Composite reliability (guide: > 0.7)• Compares the variance extracted to the error variance (not explained).

• [Cronbach’s alpha / Coefficient alpha (guide: > 0.6 or 0.7)]

Nasser Alajmi

84Quant. Methods 1 (M. Messner)

AVE and Composite Reliability

Squared loading = % variance of this variable explained by the factor

Sum of squared loadings = 0.5792 + 0.6052 + 0.5962 + 0.5592 = 1.36

Average Variance extracted = 1.36 / 4 = 0.34

0.579 0.335 0.6650.605 0.366 0.6340.596 0.355 0.6450.559 0.312 0.6882.339 1.369 2.631

Composite reliability = 2.3392 / (2.3392 + 2.631) = 0.68

nasseralajmi
Highlight
nasseralajmi
Sticky Note
Less than 50%, not good
Nasser Alajmi

85Quant. Methods 1 (M. Messner)

Additional checks for convergent and discriminant validity

Discriminant validity of factors:

Is a given factor more strongly correlated with its own items than with other factors?

Criterion: √ AVE > cross-correlation with other factors (Fornell-Larcker test)

Why?

• Remember: correlation2 = R2 = % of variance explained

• AVE = average variance of items explained (by the given factor)

nasseralajmi
Sticky Note
Are they different from each other
Nasser Alajmi

86Quant. Methods 1 (M. Messner)

Concepts and their measurement

Typical procedure for technical reliability and construct validity (for reflective

constructs)

1. Exploratory factor analysis:

– Number of factors with Eigenvalue > 1: in line with theory?

– Possibly delete items that do not load correctly (and re-do EFA)

2. Confirmatory factor analysis:

– Test hypothesized factor structure: Different Goodness of fit criteria

3. Further criteria to ascertain convergent validity of factors: Composite Reliability, Average Variance Extracted

4. Further criterion to ascertain discriminant validity of factors: √ AVE > cross-correlation with other factors (Fornell-Larcker test)

Nasser Alajmi

87Quant. Methods 1 (M. Messner)

Example

Ramani & Kumar (2008), Journal of Marketing

“Composite reliability is an indicator of the shared variance among the set of observed variables used as indicators of a latent construct (Fornell and Larcker1981; Kandemir, Yaprak, and Cavusgil 2006). As Table 2, Panel A, shows,construct reliabilities for all the seven latent constructs ranged from .82 to .97, well above the recommended value. In addition, the coefficient alpha values were well above the threshold value of .7 that Nunnally (1978) recommends. The seven-factor CFA model exhibited a good fit with the data (χ2 = 253.95, d.f. = 131; CFI = .95; GFI = .82; TLI = .93; IFI = .95; and SRMR = .07). The standardized factor loadings ranged from .62 to greater than .90 and were statistically significant at the α = .95 level (see Table 2, Panel B). This provided the necessary evidence that all the constructs exhibited convergent validity. We examined discriminant validity using a procedure suggested by Fornell and Larcker (1981) and widely followed by other researchers (e.g., De Wulf, Odekerken- Schroder, and Iacobucci 2001; Kandemir, Yaprak, and Cavusgil 2006). We computed the average variance extracted by the indicators corresponding to each of the seven factors and compared it with the highest variance that each factor shared with the other factors in the model. The average variance extracted for each factor was always greater than the highest shared variance (see Table 2, Panel A).“

Nasser Alajmi

88Quant. Methods 1 (M. Messner)

Example (cont’d)

Ramani & Kumar (2008), Journal ofMarketing

Nasser Alajmi

89Quant. Methods 1 (M. Messner)

Concepts and their measurement

How to aggregate the item responses to a value of the construct?

• When used in further regression analysis: • Add up the values from the items (“summated scale”) or average

them• Use so-called “factor scores” (there are different ways to calculate

• Example: “We construct our composite variables using factor scores. The correlations between our variables using factor scores and summated scales range between 0.98 and 0.99 for our four composite variables (not tabulated).” (Abernethy, Bouwens, & Kroos, 2017)

• When part of a structural equations model:• Measurement model and structural model are estimated at the same

time  “weights” used depend on structural model

Nasser Alajmi

Is this the question you were looking for? Place your Order Here