There is not just a single research method in sociology. As we have seen in Chapter 1, sociologists use both quantitative and qualitative data. Both of these can be collected with a variety of methods. Sociologists sometimes disagree about what kinds of data collection strategies are appropriate for particular problems.

The Big Debate

As the discipline of sociology matured, two major methodological perspectives have formed. Most sociologists working today feel both have merit, even if they tend to be more in agreement with one or the other. Some prefer the positivist approach and some prefer the historicist approach.

The positivist position centers on the philosophical argument of empiricism: that the only means of knowing truth is through direct sense perception. Sense-perceived data are called empirical facts. These empirical facts lead to the discovery of universal laws, through the process of induction.

Most sociologists recognize the importance of empirical data. This does not make them positivists. An positivist would argue that only sense-perceived data (i.e., empirical data) are valid. Most sociologists, like most people in general, acknowledge that truth can be acquired through other means. Positivists take an extreme stand on the value of empirical data.

According to the positivist approach, empirical data are collected through the scientific method, a set of techniques that emphasize careful observation, consistent measurement and public reporting of data. To the positivist, this is the only way to conduct scientific research.

At the other end of the spectrum, the historicist believes that science is not a privileged method of discovering the truth, but rather, is a social practice. To the historicist, it doesn’t get us anywhere to talk about an external truth, universal laws, and so forth. Instead, the historicist believes that sociological knowledge is a social construction, that is, a body of statements and assertions (a discourse) that is considered truth by virtue of the consensus of sociological professionals. From this perspective, the truth is what we sociologists agree it is, based on our best efforts, our biases, the political context in which we work, and so forth. According to the historicists, the same social forces that operate on society also operate on sociology itself.

Historicists point out that humans, including scientists, are imperfect recording devices. We don’t always perceive the world accurately, even when we try our best to do so.

The Nature of Explanation

When sociologists explain a social phenomenon, just what is going on?

Sociologists disagree about what constitutes an explanation. Some argue that a thorough and accurate description of the phenomenon qualifies as an explanation of it. Others argue that description is not explanation. To explain something, they say, we must be able to predict under what conditions the phenomenon would occur, and under what conditions it would not. If we can’t predict a phenomenon, we can’t explain it.

When we construct mathematical models using quantitative data, we test relationships between variables by measuring covariance — the extent to which two variables tend to be systematically related. We infer causality between variables based on an empirical covariance between them.

We must be cautious, however. Sociologists recognize that covariance is a necessary condition for causality, but it is not sufficient. In other words, if two variables are causally related, then we should be able to find an empirical covariance between them. On the other hand, if we find an empirical covariance, we cannot conclude that they are causally related.

Another important issue regarding explanation has to do with our perspective as sociologists. Most of the time, when we study social phenomena, we observe the action from the outside. We are not ourselves participants in the phenomenon we want to examine. This allows us the opportunity to take a more neutral position, since we are not subject to the same social forces as the participants.

For example, if we are studying the effectiveness of a various styles of religious speech, we might visit a number of different religious organizations and observe their services and meetings. We could collect data about the way religious leaders communicate and then examine the extent to which followers believe the messages they hear. Because we are not members of these organizations, we are better situated to take a neutral perspective in evaluating them. Of course, being an outsider does not guarantee neutrality or the lack of personal bias. It is generally easier, though, to maintain neutrality when one is an observer and not a participant.

When we study phenomena from the outside, we run the risk of missing information and interpretations of experience that are known only to the participants themselves. We may not be looking for all the relevant variables. We may interpret the facts we observe in ways that are different from, and perhaps contrary to, those of the participants.

Sociology as a Collective Enterprise

Sociologists do not work alone. The discipline is based on a cumulative history of empirical facts and theoretical interpretations. When we do research, we refer to the work of others: we seek to confirm, clarify or dispute the findings and claims of our colleagues, past and present.

No single research project can answer all the questions the researchers would like. Good research, it is often said, raises more questions than it answers, because the new information it provides leads to new ways of looking at phenomena, and new relationships to be explored.

We can evaluate our research efforts by looking at the empirical data and seeking to identify all the plausible interpretations. We then try to weigh them against each other. We want to find the best theoretical explanation of our data that we can. When we think we have it, we make our research public, in sociological journals, at professional conferences, in correspondence with our colleagues.

This is the most important part of sociology: putting your data and interpretations before the community of social scientists. Some will agree with your interpretations and some will disagree. By discussing the claims of research, we come to gain a better understanding of the social world, and the ways in which we study it.

Before we begin discussing some of the important measurement issues, we have to be familiar with the vocabulary of quantitative analysis. Let’s look at some important contrasts:

Population:
A theoretically defined collection of elements to be studied.

Sample:
A subset of elements, taken from a population, from which measurements will be made.

Random (Probability) Sample:
A sampling technique in which each element in a population has an equal, non-zero, chance of being selected into the sample.

Convenience (Purposive) Sample:
A set of sampling techniques in which a researcher draws a sample by taking advantage of existing groups or easily accessed respondents. For example, a researcher might study the students in an Introductory Statistics course. A convenience sample is not necessarily representative of the intended population.

Parameter:
Numeric representation of a characteristic of a population.

Statistic:
Numeric representation of a characteristic of a sample.

Variable:
A characteristic that can take on more than one value.

Constant:
A characteristic that takes on only one value in a given sample or population.

When we speak of a variable, we are referring to a characteristic of the objects (people, organizations, movements, countries, etc.) we are studying. A variable consists of two or more values. Values are the particular qualities the variable may have. For example, gender is a variable with two values, male and female.

When we talk about statistical relationships, we mean relationships between variables, not just some of its values.

In order to understand the usefulness of quantitative analysis, we have to be familiar with the kinds of information that a quantitative variable may possess.

At the nominal level, a variable has no numeric qualities at all. The values of a nominal variable are simply labels. Religious affiliation, for example, is a nominal variable with values such as Protestant, Catholic, Jewish, or none. These are labels that describe kinds of religious organizations.

The values of nominal variables are unordered. For this reason, they may be presented in any order. No mathematical operations (such as adding or multiplying) are valid for nominal variables. They are, for this reason, of rather limited usefulness in quantitative analysis.

In ordinal variables, the values are ordered along some continuum; they represent rank. For example, the variable church attendance might have the values weekly, monthly, a few times a year, or never. These values are ordered from more frequent to less frequent attendance. We know that someone who says they attend weekly attends more frequently than someone who says a few times a year.

What we don’t know with ordinal data, however, is the amount of difference between the values. In this example, we don’t know how much more often a person who attends weekly attends than some one who attends a few times a year.

Ordinal variables have more numerical qualities than nominal variables, but we cannot treat the values as if they were numbers. Some mathematical operations are valid with ordinal data, but some are not.

Only interval level variables are fully numeric. The values of interval level variables are real numbers, with both order and magnitude information. Examples of interval variables include age, household income (in dollars), occupational prestige scores, number of foreign films seen last year, and so forth. Because mathematical operations are meaningful with interval level data, they are generally the most useful kind of data for quantitative analysis.

Quantitative data is useless if we have not measured our concepts in a reliable and valid way. Good measurement is the foundation of sociological research. When we design a research project, we usually begin by asking questions about the possible relationships among concepts of interest to us. For example, we might wonder if religious affiliation and religious commitment are related to political view. Religious affiliation, religious commitment and political view are concepts. Before we can collect data to try to answer this research question, we have to transform these concepts into measurable items. The process of transformation from concept to measurement is called operationalization.

When we try to operationalize a concept, we find it can be a surprisingly difficult task, even for some ordinary, familiar concepts. The first discovery we are likely to make is that concepts are multidimensional; that is, they have more than one kind of meaning. If we want to operationalize ideology, for example, we quickly realize that American ideology has more than one dimension. Ideology might include the dimensions of political liberalism, economic liberalism, foreign policy liberalism and social liberalism, among others.

The multidimensionality of concepts requires us to use more than one item to measure a concept. If we were going to measure ideology, we would need items measuring political liberalism, economic liberalism, foreign policy liberalism, social liberalism, and so forth.

When we measure concepts, we need to be able to assess how well we have operationalized them. This entails two related properties of a measurement: reliability and validity.

When we measure something, we want to be sure that, if we were to measure the thing a second time, we would end up with the same result. If we generate the same result upon repeated measurements of an item, we say it is reliable. Reliability is consistency in measurement.

Reliability is often inversely related to precision. Let’s consider an example. If we want to measure household income, if we ask our respondents to estimate their income to the nearest dollar, we would generate a relatively precise measurement. On the other hand, most people probably don’t know their household income that precisely, and so, our measurement would probably not have good reliability.

Instead, say we were to measure household income by asking respondents to pick one of the following categories: less than $25,000, $25,000-$40,000, $40,001-$75,000, $75,001 or more. Since most people can probably pick the appropriate category for their household income, we would have good reliability. This measurement, though, is much less precise.

When we measure a concept, we need to think about how much precision is useful. We don’t want to design items that are more precise than is necessary, because they might not be reliable.

We also have to assess the validity of our measurements. Validity concerns the accuracy of our measurements. We want to be sure that we are measuring what we think we are measuring. It is a challenge to create measurements that are fully valid.

We assess validity in four basic ways. The first is face validity. If a measure, on the surface, seems to be measuring what we want to measure, we say it has face validity. For example, if we want to measure church attendance, the following item has face validity:
How often do you typically attend religious services in a month?
but this item does not:
How often have you been to St Patrick’s Cathedral in New York?

If an item has face validity, we next assess its content validity. As we already noticed, our sociological concepts are usually multidimensional. To fully measure these concepts, we have to measure all the dimensions. If we succeed at this, we say the measurements have content validity. If we measure only some of the dimensions of a concept, we have failed to achieve content validity.

If our measure has content validity, we check to see if it is related to other concepts we’ve measured. Construct validity is determined by examining the relationship between our measurement and other measurements in our data that we expect, on the basis of theory, to be related. For example, if we wanted to measure religiosity, we would check to see if our measure of it is related to church attendance, since the literature in this area suggests that religiosity is related to attendance.

I. Discuss the following survey items in terms of their reliability:

Q1. Please indicate the extent to which you agree or disagree with the following statement: “I would agree to an increase in my taxes if the extra money is used to prevent environmental pollution.” Do you
1. strongly agree
2. agree
3. neither agree nor disagree
4. disagree
5. strongly disagree

Q2. All in all, how would you describe your state of health these days? Would you say it is
1. Very good
2. Good
3. Fair
4. Poor
5. Very poor

Q3. Generally speaking, would you say that most people can be trusted, or that you can’t be too careful in dealing with people?
1. Most people can be trusted
2. Can’t be too careful

Q4. Some people feel they have completely free choice and control over their lives, and other people feel that what they do has no real effect on what happens to them. Please use the scale to indicate how much freedom of choice and control you feel you have over the way your life turns out.
1. None at all
2.
3.
4.
5.
6.
7.
8.
9.
10. A great deal

Q5. How would you describe the place you currently live. Would you say it is
1. Large city
2. Small city
3. Suburb
4. Town
5. Rural area

Q6. There has been a lot of discussion these days about multiculturalism. Please indicate the extent to which you agree or disagree with the following statement: “English should be the official language of our government.” Do you
1. strongly agree
2. agree
3. neither agree nor disagree
4. disagree
5. strongly disagree

Q7. According to the U.S. tax code, corporations can write off interest payments for certain kinds of debt. The more money they borrow, the less they pay in taxes. Some economists argue that this encourages growth, and therefore, job creation. Others say that it an incentive to seek short-term profits through mergers and acquisitions rather than long-term growth through improvements in technology and productivity. On the following scale, which comes closest to your opinion about the effect of the tax write-off for corporate debt?
1. Encourages job creation
2.
3.
4.
5.
6.
7. Encourages short-term profit seeking

Let’s look at some examples from the Baylor Religion Survey.

We are going to learn about quantitative data analysis. Before we begin looking at some of the basic tools, let’s consider why sociologists quantify information in the first place. Much of what we’ll discuss in this class involves technical details, if you will, of data analysis. It is important to be comfortable following the analysis “recipes” we’ll cover (often in the form of Python notebooks) but these details are of the sort that you can look up what you don’t remember. (There are lots of good places to go for help with data analysis in Python.) More important, for our learning goals, is developing confidence in the logic of analysis, of being able to understand and explain why we do what we do when we do data analysis.

Quantification has some advantages and disadvantages. It is ideally suited for some kinds of research questions and badly suited for others. The popular perception of science is that it must be quantitative; this is because it is widely believed that quantitative information is more objective, and therefore, a more accurate reflection of the true state of affairs. We shall see that this is a mistaken belief, and that both quantitative and qualitative information can be scientific and part of a valid explanation of the social world.

The main reason that sociologists quantify information is to be able to construct and test mathematical models of the social world. We use quantitative variables as building blocks that fit together into a model of some social phenomenon. We build the model on the basis of sociological theory (a framework for understanding the social world). We can then test some of the features of the model by collecting empirical data and determining if the result of our analysis is consistent with what the model predicts.

Mathematical models offer an elegant economy of expression. We can concisely state a model of some phenomenon with a small number of variables and their relations. For example, we might build a model to explain attendance at religious services. We might argue that religiosity causes attendance, and express the model as:

where X is the score on a religiosity scale and Y is frequency of attendance.

Here’s what the Python code looks like for this kind of model.

On the other hand, quantification has a couple of disadvantages. The social world is very complex, and much of our experience and behavior can’t be easily quantified. As a result, quantitative information tends to simplify; the richness of detail is lost. This helps us understand relationships between variables in general, but at the cost of knowing in depth about particular cases. In this example, we might have an understanding of whether or not people who are more religious attend services more frequently, but we won’t be able to say anything about why a particular person attends services or doesn’t.

Mathematical models also tend to give the impression that social processes are static, that is, that they work one way, have always worked one way, and will always work one way. They tend to minimize the extent to which the social world is constantly changing. (There are ways to model time in quantitative models and, I believe, social scientists should be more familiar with these techniques, but that is a topic beyond the scope of this introductory course.)

Some topics lend themselves to quantitative analysis. When we want to describe social forces and structures–what we call macro-sociology–we need to study large groups, like social classes, countries or social institutions. We want to be able to understand the general, or typical, case. It is very difficult to study very large groups without quantification.

Sometimes we want to study the details of a particular social process; we are more interested in narrative detail and lived context than in a summary of the typical case. Quantitative information is not always very useful in this kind of research. Qualitative methods are usually employed.

Sociologists don’t have to employ only one method or the other. Frequently, both strategies are used to study a social problem or process, though one is usually emphasized. Most sociologists are pragmatists in practice. They use whatever tools are available to better understand their topic or question.