Defining & Measuring Variables

G & F Chapter 3

I. Measurement

Two major measurement issues to consider when planning a study:

A. No one-to-one relationship b/tw variable and measurement

B. Measurement chosen can influence the measurements and the interpretation of the variables

Types of variables involved in research

–Well-defined, easily observed, and easily measured

Examples: height and weight

–Intangible, abstract attributes

Examples: motivation or self-esteem

Measurement is more complicated

II. Constructs and Operational Definitions

A. Theories and Constructs

Construct—a hypothetical attribute or mechanism that helps explain and predict behavior in a theory

External stimulus factors >>Construct>>Behavior

The construct itself cannot be directly observed or measured

Therefore, observe and measure the external factors and the behaviors that are associated theoretically with the construct.

Operational Definitions—specifies a measurement procedure for measuring external, observable behavior. The resulting measurements are used as a definition and a measurement of the construct.

When doing research, don't re-invent the wheel.

Examples

So if we are investigating the effect of watching violent television programs on children’s aggressive behavior:

We need to operationalize “violence” on television.

We need to operationalize “aggressive behavior.”

Another example:
Which of the following might be used as an operational definition of “assertiveness?”

nThe number of times a person makes requests or states his or her feelings over the course of a one-hour interaction.

An appearance of confidence and ease in social situations.

III. Validity of Measurement

How can we be sure that the measurements obtained from an operational definition actually represent the intangible construct?

Validity—Degree to which the measurement process measures the variable that it claims to measure?

Six commonly used definitions of validity

1. Face validity—Does the measurement technique look like it measures the variable that it claims to measure?

2. Concurrent Validity—Are the scores from a new measurement technique directly related to the scores obtained from another, better-established procedure for measuring the same variable?

3. Predictive Validity—Do the measurements of a construct accurately predict behavior according to the theory?

4. Construct Validity—Do measurements of a variable behave in exactly the same way as the variable itself?

5. Convergent Validity— A strong relationship between the scores obtained from two different methods of measuring the same construct.

6. Divergent Validity—Demonstrating that two two distinct constructs produce unrelated scores.

IV. Reliability of Measurement

Reliability is the stability or the consistency of measurement

Each individual measurement has an element of error. Measured Score=True Score + Error

The inconsistency in a measurement comes from error.

A. Common sources of error are:

Observer error

Environmental changes

Participant changes

B. Types and Measures of Reliability

Successive measurements--test-retest reliability

Simultaneous measurements--inter-rater reliability or inter-observer reliability

Internal consistency—split-half reliability

C. What is the Relationship between Reliability and Validity?

Partially related

Must a test be reliable in order to be valid?

Partially independent

Must a test be valid in order to be reliable?

V. Scales of Measurement

The process of measuring a variable requires a set of categories called a scale of measurement and a process that classifies each individual into one category.

Scale

Characteristics

Examples

Nominal

• Label and categorize

• No quantitative distinctions

• Gender

• Diagnosis

• Experimental or Control

Ordinal

• Categorizes observations

• Categories organized by size or magnitude

• Rank in class

• Clothing sizes (S, M, L, XL)

• Olympic medals

Interval

• Ordered categories

• Interval between categories
of equal size

• Arbitrary or absent zero point

• Temperature

• IQ

• Golf scores (above/below par)

Ratio

• Ordered categories

• Equal interval between categories

• Absolute zero point

• Number of correct answers

• Time to complete task

• Gain in height and/or weight since last year

VI. Modalities of Measurement

A. Self-Report Measures

Ask the participant a series of questions, i.e., administer a survey

Most direct way to assess a construct, but participants may distort responses

`B. Physiological Measures

Look at how the underlying construct affects physiology

Objective measure, but equipment may be expensive or setting may be unnatural

C. Behavioral Measures

Observe and measure overt behavior

Wide variety of options: e.g., "mental alertness" could be operationally defined by behaviors such as reaction time, reading comprehension, logical reasoning ability, or ability to focus attention.

Behavioral Observation--prepared set of behavioral categories is crucial

Observation without intervention

Naturalistic observation

Ethology

Observation with intervention

Why intervene?

Participant observation

Structured observation

Field experiments

VII. Other Aspects of Measurement

Whenever possible, use multiple measures of a construct

Desynchrony--lack of agreement between two measures; confuses the interpretation of the results

Sensitivity and Range Effects

The Dependent Variable

Must be sensitive enough to show variations in performance as a result of variations in the IV.

Ceiling effect: Performance high at all levels of the IV

Floor effect: Performance low at all levels of the IV

Participant Reactivity

Demand characteristics

Experimenter Bias

The tendency of an experimenter to unintentionally distort the procedures or results of an experiment based on the expected or desired outcome of the research.

Methods have been devised to help counteract these normal human tendencies that create bias:

Using blind observers who record data without knowing what the researcher is studying

Using a placebo control

Single-blind vs. Double-blind research