Project 2: Obtaining Accurate Self-Reports of Sensitive Behaviors

A variety of health conditions carry personal or social stigma and, as a result, may be subject to social desirability bias in survey data collection.  Social desirability biases are apparent in the over-reporting of positive behaviors (for example, voting in elections) and the underreporting of negative ones (such as illicit drug use).  Health surveys include many examples of both types: socially undesirable behaviors related to health (e.g., drug and alcohol use, smoking, risky sexual practices, abortion), and socially desirable behaviors (e.g., healthy eating, exercise, dental visits).

One particular data collection technique that may improve the quality of information obtained on sensitive topics audio computer assisted self-interviewing (audio-CASI).  In audio-CASI, a respondent hears the questions over headphones attached to a laptop computer from a recording of an interviewer reading the question and enters responses directly into the computer.

The technique shows substantial promise in reducing social desirability effects for behaviors of interest to health survey researchers (Johnston and Walton, 1995; O’Reilly et al., 1994).  Turner et al. (1998a; see also 1998b) embedded an experimental comparison of a paper and pencil self-administered questionnaire (SAQ) with audio-CASI for the most sensitive items on the National Survey of Adolescent Males (NSAM), and reported an almost fourfold increase in the number of respondents reporting male-male sexual contact, from 1.5 percent for paper and pencil to 5.5 percent for audio-CASI. Tourangeau and Smith (1996; see also 1998) compared computer assisted personal interviewing (CAPI), text-CASI, and audio-CASI, and found that across a wide variety of questions involving sexual behavior and drug use, audio-CASI and “text-CASI” (in which respondents recorded answers on paper rather directly into a computer) generally yielded higher levels of reporting than interviewer-administered computer assisted personal interviews.

While these initial studies have led to the widespread adoption of audio-CASI in large-scale national surveys on a variety of sensitive topics (e.g., the National Household Survey on Drug Abuse, National Survey of Family Growth, and National Survey of Adolescent Males), only one study (Tourangeau and Smith, 1996, 1998) has directly compared audio-CASI to text-CASI.  Audio-CASI has additional cost and effort relative to text-CASI for large-scale health surveys.  It is thus important to explore the relative advantages of different approaches to self-administration.

Survey-based studies of audio-CASI have focused on the increased privacy offered by audio-CASI with respect to other people present in the interview setting.  They have largely ignored the potentially biasing effect of the voice used in the audio-CASI device.  Audio-CASI may produce more, not less, social desirability bias than text-CASI, because additional social cues are provided to the respondent in the form of the interviewer’s voice.

Ongoing research is addressing human-computer interaction (HCI) issues with computer-assisted interviewing (CAI), but to date little or no research has focused on the design and implementation of CASI instruments.  Given the increasing use of methods such as audio-CASI in a wide variety of health survey and clinical settings, Project 2 addresses two key hypotheses that might explain the diverse reactions to CASI use in surveys: a) the dominant mode of delivery of the question (text or audio), and b) the presence of the interviewer.

One mechanism by which audio-CASI is hypothesized to improve the quality of self-report is the increased privacy it affords respondents with respect to other people who may be present during the interview.  A second mechanism may be the social presence of the live or virtual interviewer.

The effects of audio-CASI relative to text-CASI are hypothesized to be situation-dependent.  In a situation where other people cannot disturb the respondent’s privacy, text-CASI is expected to obtain reports that are at least as “good” (i.e., lower reports of socially desirable behaviors and higher reports of undesirable behaviors) as audio-CASI.  However, in a situation where other people do potentially intrude on the respondent’s privacy, it is expected that the effect of the virtual voice on audio-CASI will be negated by the enhanced privacy offered by this method.  Audio-CASI should therefore yield reports that are more accurate than text-CASI.

To test the relative effectiveness of audio-CASI and text-CASI in reducing social desirability effects, the project is implementing in a laboratory setting a 2 x 2 design crossing the degree of privacy (high versus low) and the mode of administration (text-CASI versus audio-CASI).  All conditions will be run in a laboratory setting to permit more carefully controlled measures of the hypothesized independent variables.  While this feature of the design may raise questions of external validity, it allows full control of the experiment treatment.  If the expected effects are found, later studies will attempt to replicate the findings in a field setting.

In the high privacy conditions, the interviewer will introduce the respondent to the audio- or text-CASI instrument and then leave the room.  The respondent will be left alone in a private room for the entire time he or she is completing the survey instrument.  The interviewer will be available outside the room should the respondent need assistance.  In the low privacy conditions, both the interviewer and a laboratory assistant will be present as the respondent is completing the interview.  The interviewer will remain in the room with the respondent during completion of the CASI questionnaire; in addition, a research assistant will enter and leave the room several times during the interview on the pretext of checking that the equipment is working correctly.

A variety of sensitive health-related items will be included in the instrument.  These will include questions about both socially undesirable behaviors (e.g., number of sex partners, unprotected sex, drug and alcohol use, smoking, etc.) and socially desirable behaviors (e.g., exercise, dental visits, diet).  To maximize power for items that may be rare in the general population, subjects will be recruited in high-risk categories on several of these variables.

Following completion of the experimental questionnaire, subjects will answer debriefing questions designed to assess (a) how sensitive they considered the different questions to be; (b) how concerned they were about other people finding out how they answered the questions; (c) how much privacy they felt the method of interviewing afforded them; (d) how comfortable they were with the interviewing method; and (e) what alternative method of interviewing they would have preferred.  In addition, a series of questions will be designed to elicit their awareness of the social presence of the virtual interviewer by asking about voice and other inferred characteristics of the interviewer.  These measures will be used to supplement the actual behavioral reports, and will serve in part as a check of the effectiveness of the experimental manipulation.

A total of 300 volunteer subjects will be recruited for the experiment, yielding 50 subjects per cell.  Data collection activities are ongoing; however, analysis of a preliminary data set is under way.

Publications: 

Social Desirability Effects on Self-Reports of Behavior: Understanding the Effects of Audio-CASI