Sampling
| researchmethodssampling.ppt |
Purposes of Monitoring the Future The Monitoring the Future (MTF) project, begun in 1975, has many purposes. Among them is to study changes in the beliefs, attitudes, and behavior of young people in the United States. In recent years, the U.S. has experienced tremendous changes in public opinion toward such diverse issues as government and politics, alcohol and other drug use, gender roles, and protection of the environment. Much of our current upheaval in attitudes is especially concentrated, and often first seen, in today's youth. This study focuses on youth because of their significant involvement in today's social changes and, most important, because youth in a very literal sense will constitute our future society.
The results of the study are useful to policymakers at all levels of government, for example, to monitor progress toward national health goals. Study results are also used to monitor trends in substance use and abuse among adolescents and young adults and are used routinely in the White House Strategy on Drug Abuse.
Design of Monitoring the Future The Monitoring the Future (MTF) project, also widely known for some years as the National High School Senior Survey, is a repeated series of surveys in which the same segments of the population (8th, 10th, and 12th graders; college students; and young adults) are presented with the same set of questions over a period of years to see how answers change over time.
The project has been conducted under a series of research grants from the National Institute on Drug Abuse, a part of the National Institutes of Health. Surveys have been carried out each year since 1975 by the University of Michigan Survey Research Center. MTF respondents are 8th, 10th, and 12th grade students who participate by completing self-administered, machine-readable questionnaires in their normal classrooms, administered by University personnel.
The survey began with senior classes in 1975, and each year about 16,000 students in approximately 133 public and private high schools nationwide participate. Beginning in 1991, similar surveys of nationally representative samples of 8th and 10th graders have been conducted annually; the 8th-grade samples contain about 18,000 students in about 150 schools, and the 10th-grade samples contain about 17,000 students in about 140 schools. In all, approximately 50,000 students in about 420 public and private secondary schools are surveyed annually.
Beginning with the class of 1976, a randomly selected sample from each senior class has been followed up biannually after high school on a continuing basis. These respondents receive a mail questionnaire at their home, which they complete and return to MTF.
The study's design permits the investigators to examine four kinds of change:
Sampling Procedures The data from students are collected during the spring of each year. Each year's data collection takes place in approximately 420 public and private high schools and middle schools selected to provide an accurate representative cross section of students throughout the coterminous United States at each grade level.
A multi-stage random sampling procedure is used for securing the nationwide sample of students each year at each grade level.
Stage 1: The selection of particular geographic areas.
Stage 2: The selection (with probability proportionate to size) of one or more schools in each area.
Stage 3: The selection of classes within each school. Within each school, up to 350 students may be included. In schools with fewer students, the usual procedure is to include all of them in the data collection. In larger schools, a subset of students is selected either by randomly sampling entire classrooms or by some other random method that is judged to be unbaised. Sampling weights are used when the data are analyzed to correct for unequal probabilities of selection that occurred at any stage of sampling.
Administration In-school Survey. About 10 days before the administration, the students are given flyers explaining the study. Also, advance letters to parents inform them about the study and provide them a handy means for declining their child's participation if they so desire. The actual questionnaire administrations are conducted by the local Institute for Social Research representatives and their assistants, following standardized procedures detailed in a project instruction manual. The questionnaires are group administered in classrooms during a normal class period whenever possible; however, circumstances in some schools require the use of larger group administrations.
Follow-up Survey. The questionnaires are mailed to respondents with a return, self-addressed, stamped envelope and a small monetary gift from the University of Michigan as a token of appreciation.
KEY TERMS:
bias - Any property of a question that encourages respondents to answer in a particular way.
closed-ended questions - Respondent is asked to select an answer from among a list provided by a researcher.
computer assisted interviewing - Interviewer use of laptops and handhelds to assist in the administration of the in-person interview.
focus group - 8 to 15 people are brought together in a room to engage in a guided group discussion of some topic.
interview survey - Using a questionnaire in a systematic way to interview a large number of people.
open-ended questions - Respondent is asked to provide his or her own answers.
questionnaire - Collection of questions and statements.
respondent – A person who responds to the questionnaire/survey.
response rate - The percent of people contacted who actually participate.
survey – Questionnaire or an instrument used to collect data.
Definitions
binomial variable
A variable that has only two attributes is binomial. Gender is an example; it has the attributes male and female.
cluster sample
A multistage sample in which natural groups (clusters) are sampled initially, with the members of each selected group being subsampled afterward. For example, you might select a sample of municipal police departments from a directory, get lists of the police officers at all the selected departments, then draw samples of officers from each.
confidence interval
The range of values within which a population parameter is estimated to lie. A survey, for instance, may show that 40 percent of a sample favor a ban on handguns. Although the best estimate of the support that exists among all people is also 40 percent, we do not expect it to be exactly that. We might, therefore, compute a confidence interval (for example, from 35 to 45 percent) within which the actual percentage of the population probably lies. Note that it is necessary to specify a confidence level in connection with every confidence interval.
confidence level
The estimated probability that a population parameter lies within a given confidence interval. Thus we might be 95 percent confident that between 35 and 45 percent of all residents of California favor an absolute ban on handguns.
disproportionate stratified sampling
Deliberately drawing a sample that overrepresents or underrepresents some characteristic of a population. We may do this to ensure that we obtain a sufficient number of uncommon cases in our sample. For example, believing violent crime to be more common in large cities, we might oversample urban residents to obtain a specific number of crime victims.
equal probability of selection method (EPSEM)
A sample design in which each member of a population has the same chance of being selected in the sample.
nonprobability sample
A sample selected in some fashion other than those suggested by probability theory. Examples are purposive, quota, and snowball samples.
population
All people, things, or other elements we wish to represent. Researchers often study only a subset or sample of a population, then generalize from the people, things, or other elements actually observed to the larger population of all people, things, or elements.
population parameter
The summary description of a particular variable in the population. For example, if the mean age of all professors at your college is 43.7, then 43.7 is the population parameter for professors’ mean age. Compare with sample statistic and sampling distribution.
probability sample
The general term for a sample selected in accord with probability theory, typically involving some random selection mechanism. Specific types of probability samples include area probability sample, equal probability of selection method (EPSEM), simple random sample, and systematic sample.
purposive sample
A type of nonprobability sample in which you select the units to be observed on the basis of your own judgment about which ones will be best suited to your research purpose. For example, if you were interested in studying community crime prevention groups affiliated with public schools and groups affiliated with religious organizations, you would probably want to select a purposive sample of school- and church-affiliated groups. Most television networks use purposive samples of voting precincts to project winners on election night; precincts that always vote for winners are sampled.
quota sample
A type of nonprobability sample in which units are selected in the sample on the basis of prespecified characteristics, so that the total sample will have the same distribution of characteristics as are assumed to exist in the population being studied.
sample
A subset of a population selected according to one or more criteria. Two general types are probability and nonprobability samples.
sample element
The unit about which information is collected and that provides the basis of analysis. Typically, in survey research, elements are people. Other kinds of units can be the elements for criminal justice research—correctional facilities, gangs, police beats, or court cases.
sample statistic
The summary description of a particular variable in a sample. For example, if the mean age of a sample of 100 professors on your campus is 41.1, then 41.1 is the sample statistic for professor age. We usually use sample statistics to estimate population parameters. Compare with sampling distribution.
sampling distribution
The range, or array, of sample statistics we would obtain if we drew a very large number of samples from a single population. With random sampling, we expect that the sampling distribution for a particular statistic (mean age, for example) will cluster around the population parameter for mean age. Furthermore, sampling distributions for larger sample sizes will cluster more tightly around the population parameter.
sampling frame
That list or quasi-list of units composing a population from which a sample is selected. If the sample is to be representative of the population, it is essential that the sampling frame include all (or nearly all) members of the population.
sampling units
Like sampling elements, these are things that may be selected in the process of sampling; often sampling units are people. In some types of sampling, however, we often begin by selecting large groupings of the eventual elements we will analyze. Sampling units is a generic term for things that are selected in some stage of sampling but are not necessarily the objects of our ultimate interest.
simple random sample
A type of probability sample in which the units composing a population are assigned numbers, a set of random numbers is then generated, and the units that have those numbers are included in the sample. Although probability theory and the calculations it provides assume this basic sampling method, it is seldom used for practical reasons. An alternative is the systematic sample (with a random start).
snowball sampling
A method for drawing a non-probability sample. Snowball samples are often used in field research. Each person interviewed is asked to suggest additional people for interviewing.
standard error
A measure of sampling error, the standard error gives us a statistical estimate of how much a member of a sample might differ from the population we are studying, solely by chance. Larger samples usually result in smaller standard errors.
stratification
The grouping of the units composing a population into homogeneous groups (or strata) before sampling. This procedure, which may be used in conjunction with simple random, systematic, or cluster sampling, improves the representativeness of a sample, at least in terms of the stratification variables.
systematic sampling
A method of probability sampling in which every kth unit in a list is selected for inclusion in the sample—for example, every 25th student in the college directory of students. We compute k (also called the sampling interval) by dividing the size of the population by the desired sample size. Within certain constraints, systematic sampling is a functional equivalent of simple random sampling and is usually easier to do. Typically the first unit is selected at random
The results of the study are useful to policymakers at all levels of government, for example, to monitor progress toward national health goals. Study results are also used to monitor trends in substance use and abuse among adolescents and young adults and are used routinely in the White House Strategy on Drug Abuse.
Design of Monitoring the Future The Monitoring the Future (MTF) project, also widely known for some years as the National High School Senior Survey, is a repeated series of surveys in which the same segments of the population (8th, 10th, and 12th graders; college students; and young adults) are presented with the same set of questions over a period of years to see how answers change over time.
The project has been conducted under a series of research grants from the National Institute on Drug Abuse, a part of the National Institutes of Health. Surveys have been carried out each year since 1975 by the University of Michigan Survey Research Center. MTF respondents are 8th, 10th, and 12th grade students who participate by completing self-administered, machine-readable questionnaires in their normal classrooms, administered by University personnel.
The survey began with senior classes in 1975, and each year about 16,000 students in approximately 133 public and private high schools nationwide participate. Beginning in 1991, similar surveys of nationally representative samples of 8th and 10th graders have been conducted annually; the 8th-grade samples contain about 18,000 students in about 150 schools, and the 10th-grade samples contain about 17,000 students in about 140 schools. In all, approximately 50,000 students in about 420 public and private secondary schools are surveyed annually.
Beginning with the class of 1976, a randomly selected sample from each senior class has been followed up biannually after high school on a continuing basis. These respondents receive a mail questionnaire at their home, which they complete and return to MTF.
The study's design permits the investigators to examine four kinds of change:
- Changes in particular years reflected across all age groups (secular trends or "period effects").
- Developmental changes that show up consistently for all panels ("age effects").
- Consistent differences among class cohorts through the life cycle ("cohort effects").
- Changes linked to different types of environments (high school, college, employment) or role transitions (leaving the parental home, marriage, parenthood, etc.).
Sampling Procedures The data from students are collected during the spring of each year. Each year's data collection takes place in approximately 420 public and private high schools and middle schools selected to provide an accurate representative cross section of students throughout the coterminous United States at each grade level.
A multi-stage random sampling procedure is used for securing the nationwide sample of students each year at each grade level.
Stage 1: The selection of particular geographic areas.
Stage 2: The selection (with probability proportionate to size) of one or more schools in each area.
Stage 3: The selection of classes within each school. Within each school, up to 350 students may be included. In schools with fewer students, the usual procedure is to include all of them in the data collection. In larger schools, a subset of students is selected either by randomly sampling entire classrooms or by some other random method that is judged to be unbaised. Sampling weights are used when the data are analyzed to correct for unequal probabilities of selection that occurred at any stage of sampling.
Administration In-school Survey. About 10 days before the administration, the students are given flyers explaining the study. Also, advance letters to parents inform them about the study and provide them a handy means for declining their child's participation if they so desire. The actual questionnaire administrations are conducted by the local Institute for Social Research representatives and their assistants, following standardized procedures detailed in a project instruction manual. The questionnaires are group administered in classrooms during a normal class period whenever possible; however, circumstances in some schools require the use of larger group administrations.
Follow-up Survey. The questionnaires are mailed to respondents with a return, self-addressed, stamped envelope and a small monetary gift from the University of Michigan as a token of appreciation.
KEY TERMS:
bias - Any property of a question that encourages respondents to answer in a particular way.
closed-ended questions - Respondent is asked to select an answer from among a list provided by a researcher.
computer assisted interviewing - Interviewer use of laptops and handhelds to assist in the administration of the in-person interview.
focus group - 8 to 15 people are brought together in a room to engage in a guided group discussion of some topic.
interview survey - Using a questionnaire in a systematic way to interview a large number of people.
open-ended questions - Respondent is asked to provide his or her own answers.
questionnaire - Collection of questions and statements.
respondent – A person who responds to the questionnaire/survey.
response rate - The percent of people contacted who actually participate.
survey – Questionnaire or an instrument used to collect data.
Definitions
binomial variable
A variable that has only two attributes is binomial. Gender is an example; it has the attributes male and female.
cluster sample
A multistage sample in which natural groups (clusters) are sampled initially, with the members of each selected group being subsampled afterward. For example, you might select a sample of municipal police departments from a directory, get lists of the police officers at all the selected departments, then draw samples of officers from each.
confidence interval
The range of values within which a population parameter is estimated to lie. A survey, for instance, may show that 40 percent of a sample favor a ban on handguns. Although the best estimate of the support that exists among all people is also 40 percent, we do not expect it to be exactly that. We might, therefore, compute a confidence interval (for example, from 35 to 45 percent) within which the actual percentage of the population probably lies. Note that it is necessary to specify a confidence level in connection with every confidence interval.
confidence level
The estimated probability that a population parameter lies within a given confidence interval. Thus we might be 95 percent confident that between 35 and 45 percent of all residents of California favor an absolute ban on handguns.
disproportionate stratified sampling
Deliberately drawing a sample that overrepresents or underrepresents some characteristic of a population. We may do this to ensure that we obtain a sufficient number of uncommon cases in our sample. For example, believing violent crime to be more common in large cities, we might oversample urban residents to obtain a specific number of crime victims.
equal probability of selection method (EPSEM)
A sample design in which each member of a population has the same chance of being selected in the sample.
nonprobability sample
A sample selected in some fashion other than those suggested by probability theory. Examples are purposive, quota, and snowball samples.
population
All people, things, or other elements we wish to represent. Researchers often study only a subset or sample of a population, then generalize from the people, things, or other elements actually observed to the larger population of all people, things, or elements.
population parameter
The summary description of a particular variable in the population. For example, if the mean age of all professors at your college is 43.7, then 43.7 is the population parameter for professors’ mean age. Compare with sample statistic and sampling distribution.
probability sample
The general term for a sample selected in accord with probability theory, typically involving some random selection mechanism. Specific types of probability samples include area probability sample, equal probability of selection method (EPSEM), simple random sample, and systematic sample.
purposive sample
A type of nonprobability sample in which you select the units to be observed on the basis of your own judgment about which ones will be best suited to your research purpose. For example, if you were interested in studying community crime prevention groups affiliated with public schools and groups affiliated with religious organizations, you would probably want to select a purposive sample of school- and church-affiliated groups. Most television networks use purposive samples of voting precincts to project winners on election night; precincts that always vote for winners are sampled.
quota sample
A type of nonprobability sample in which units are selected in the sample on the basis of prespecified characteristics, so that the total sample will have the same distribution of characteristics as are assumed to exist in the population being studied.
sample
A subset of a population selected according to one or more criteria. Two general types are probability and nonprobability samples.
sample element
The unit about which information is collected and that provides the basis of analysis. Typically, in survey research, elements are people. Other kinds of units can be the elements for criminal justice research—correctional facilities, gangs, police beats, or court cases.
sample statistic
The summary description of a particular variable in a sample. For example, if the mean age of a sample of 100 professors on your campus is 41.1, then 41.1 is the sample statistic for professor age. We usually use sample statistics to estimate population parameters. Compare with sampling distribution.
sampling distribution
The range, or array, of sample statistics we would obtain if we drew a very large number of samples from a single population. With random sampling, we expect that the sampling distribution for a particular statistic (mean age, for example) will cluster around the population parameter for mean age. Furthermore, sampling distributions for larger sample sizes will cluster more tightly around the population parameter.
sampling frame
That list or quasi-list of units composing a population from which a sample is selected. If the sample is to be representative of the population, it is essential that the sampling frame include all (or nearly all) members of the population.
sampling units
Like sampling elements, these are things that may be selected in the process of sampling; often sampling units are people. In some types of sampling, however, we often begin by selecting large groupings of the eventual elements we will analyze. Sampling units is a generic term for things that are selected in some stage of sampling but are not necessarily the objects of our ultimate interest.
simple random sample
A type of probability sample in which the units composing a population are assigned numbers, a set of random numbers is then generated, and the units that have those numbers are included in the sample. Although probability theory and the calculations it provides assume this basic sampling method, it is seldom used for practical reasons. An alternative is the systematic sample (with a random start).
snowball sampling
A method for drawing a non-probability sample. Snowball samples are often used in field research. Each person interviewed is asked to suggest additional people for interviewing.
standard error
A measure of sampling error, the standard error gives us a statistical estimate of how much a member of a sample might differ from the population we are studying, solely by chance. Larger samples usually result in smaller standard errors.
stratification
The grouping of the units composing a population into homogeneous groups (or strata) before sampling. This procedure, which may be used in conjunction with simple random, systematic, or cluster sampling, improves the representativeness of a sample, at least in terms of the stratification variables.
systematic sampling
A method of probability sampling in which every kth unit in a list is selected for inclusion in the sample—for example, every 25th student in the college directory of students. We compute k (also called the sampling interval) by dividing the size of the population by the desired sample size. Within certain constraints, systematic sampling is a functional equivalent of simple random sampling and is usually easier to do. Typically the first unit is selected at random
