Posts

Factorial Analysis of Variance

When we want to compare more than two means, we use the F-test to determine if there is a difference among the groups.

This one-way analysis of variance test compares means for groups defined by one variable, such as class, religious affiliation, and so forth.

Sometimes, we want to examine the effect of more than one grouping variable. For example, we might want to look at the effects of social class and religious affiliation upon ideology.

Why not just do two separate one-way F-tests? This would increase the likelihood of a type-I error — remember our discussion of multiple t-tests.

In addition, the two-way ANOVA allows us to determine if the two group variables have a simultaneous effect, i.e., whether or not the effects of religious affiliation and social class build upon each other. Unless we look at both variables together, we cannot know if their effects are independent or simultaneous.

This kind of hypothesis test uses the same F-test we have been discussing. Instead of one F-score, though, the two-way analysis of variance requires four F-scores: one for the overall test — like the F-score in the one-way ANOVA; one for the interaction effect; and one for each main effect.

In my study of religious affiliation, commitment and ideology among elites, I conducted a series of two-way ANOVAs. I wanted to know if affiliation and commitment influenced the ideology dimensions I was measuring.

The two-way ANOVA is another kind of hypothesis test, and follows the same set of steps. To begin, we must state our hypotheses. Let’s use an example from the research article:

Null Hypothesis: There is no effect of religious affiliation and commitment upon expressive individualism.

Research Hypothesis: There is an effect of religious affiliation and commitment upon expressive individualism.

Now, let’s consider each of the F-tests.

The overall F-test
As with the one-way F-test, this test tells us whether there is any difference among the groups. In this case, there are 10 groups being compared: there are five affiliation groups and two commitment groups (5 x 2 = 10).

The first question to ask of this data is: are there any differences among these ten groups in terms of expressive individualism?

The two-way ANOVA begins with this test. If the probability of the F-score is greater than alpha (usually, 0.05), then the hypothesis test stops here. We conclude that there is no effect of religious affiliation or commitment on expressive individualism.

Let’s look at the results of this test from the paper:
The overall F-test in the formatted results.

As we can see, the overall F-test is statistically significant at the 95% confidence level.

The omnibus F-test tells us that there is some difference somewhere among these ten groups. This means that we should go on to test for the interaction and main effects. We want to try to pin down the nature of the effects of religious affiliation and commitment.

The Interaction Effect
One of the advantages of the two-way ANOVA is the interaction F-test. This tells us whether or not the effects of the two group variables, religious affiliation and commitment, are independent or build upon each other.

We might expect, for example, that the effect of commitment depends on the particular religious affiliation. In other words, the difference between more and less devout Jews might be different than the difference between more and less devout Conservative Protestants.

Let’s look at the results:
The interaction F-test from the formatted results.

In this case, there is a significant interaction effect. This tells us that, indeed, the effect of commitment depends on religious affiliation — or, similarly, the effect of affiliation depends on commitment.

Because the interaction is significant, it doesn’t make sense to talk about the main effects as such, because understanding the effect of commitment or affiliation on ideology depends on the other factor. For this reason, when the interaction is significant, we do not interpret the tests for the individual main effects.

Interactions are easily seen if we graph the group means:
Graph of expressive individualism by affiliation and commitment.

If there were no interaction effect, the two lines (more and less devout) would be parallel.

Interpreting the results
How do we explain the effects of religious affiliation and commitment, in light of the interaction effect?

Like with the post-hoc test in the one-way ANOVA, we are seeking an interpretation that describes the pattern of group differences.

Let’s look at the group means:
Table of means on expressive individualism by affiliation and commitment.

The first thing we observe is that the less devout of every affiliation group are more liberal on this dimension than the more devout of every affiliation.

There is a very large difference between the more and less devout among Catholics, Jews, and Conservative Protestants. There is a modest difference between more and less devout Moderate Protestants, and a small difference between more and less devout Liberal Protestants.

Introduction to hypothesis testing

With the confidence interval, we are able, for the first time, to say something about the population based on information in our sample. We use the confidence interval to estimate the population mean.

Often, we have ideas about the population mean before we collect data on our sample. We may have read other studies on the same topic, and they suggest that they population mean will be a particular value. Or, we may have an idea about the population mean based on the internal logic of the measurements we are using. Sometimes, we base our expectations on our experience of the particular social phenomena we are studying.

We need to formalize this testing of our expectations against our sample data. This process is called hypothesis testing. Each new statistic that we learn will be a tool for testing a different kind of hypothesis.

The Logic of Hypothesis Testing

At first, our approach to hypothesis testing will seem backwards. We will set up a certain hypothesis — the null hypothesis — and try to demonstrate that it is probably wrong, based on our sample data.

Why not just try to prove the hypothesis based on our expectations?

As it turns out, this is a difficult task. It is easier to use probability to show that the null hypothesis is probably wrong.

The null hypothesis always states that there is no effect. In contrast, the research hypothesis states that there is an effect — that is, our expectations about the population mean, for example.

The null and research hypotheses are always defined as logical opposites. They are mutually exclusive of one another — only one or the other can be true, not both.

Let’s consider a specific problem.

Say that we are interested in global social indicators and we compute the mean labor force participation for men. Knowing that inequality is gendered, we expect that labor force participation for women will be lower. (In explaining the hypothesis, we could cite literature which explains gender socializations regarding public life and household labor; men are more likely to work outside the home for wages but women do much more domestic labor, which is usually uncompensated.)

In the language of hypothesis testing, we start with the null hypothesis (H0) which always states no effect. In this case, that the mean labor force participation for women is not different than the computed value for men. The research hypothesis (H1) is that there is a difference — that is, that the mean for women is not equal to the mean for men.

Statement of null and research hypotheses.

We can calculate a t-score and test these hypotheses. The data from our sample will lend support to one or the other. By convention, we interpret our data from the point of view of the null hypothesis — that is, our data either supports or fails to support the null hypothesis.

Let’s think, for a moment, about the logic of hypothesis testing. When we make a decision about the null hypothesis, our decision is either correct or incorrect. If we were able to know reality directly, we could determine if the hypothesis is actually true or false.

Figure 1. The Logic of Inference
Table showing the kinds of errors we can make in hypothesis testing.
From Levin and Fox, Elementary Statistics in Social Research, 7th edition, 1997.

When we do hypothesis testing, we try to balance the risk of a type I error with the desire to correctly discover a real effect. Science, as a social practice, is conservative in this regard. We tend to favor a rather strict criterion — typically 95% certainty. Thus, we are relatively more likely to miss real effects than to mistakenly claim that there is an effect when, in fact, there isn’t.

The steps of hypothesis testing can be summarized:
A) State the research and null hypotheses;
B) set an alpha level (i.e., a level of confidence in the claim of an effect; this is almost always 0.05);
C) compute the appropriate significance test; and,
D) interpret the results.

Let’s see what this looks like in a Python notebook: hypothesis testing.

The Confidence Interval

We have just learned that we can estimate the likelihood of drawing a random sample with a particular mean from a population, when we know the population parameters. This is the z-test. With this, we take the first step in inferential statistics, decision-making about hypotheses based on probability.

The problem with the z-test is that we don’t usually know the population parameters. After all, this is usually why we are undertaking the research in the first place! What we need, then, is a way to estimate the parameters.

The z-score formula.

The standard error formula.

We usually don’t know the population standard deviation, sigma. As a result, we can’t calculate the standard error using the formula with which we are familiar.

When we collect data, we have information about the sample. Not knowing anything more, this is the best information we have about the population. We can use the sample standard deviation to estimate the population standard deviation, and therefore, the standard error.

We will use the following formula to estimate the standard error:
The formula for the estimated standard error.

We can rewrite the z-test, using the estimated standard error:
The formula for the t-test.

We will call this the t-ratio, instead of z, because the sampling distribution upon which it is based is ever-so-slightly different than the normal distribution.

With the t-ratio, we can generate an estimate of the population mean — remember, we usually don’t know this when we conduct a study. We refer to this as the confidence interval, and define it as:
Formula for the 95% confidence interval.

This is our best estimate of the interval into which the population mean, mu, falls.

At last! Our first example of how useful this inferential stuff (probability, sampling distributions, standard scores) is.

The Sampling Distribution

We have been discussing some concepts that imply a random sample, or a random selection from a distribution, and so forth. We need to be more precise, now, about what we mean by random in these cases.

We define a random sample as one in which every element in the population has an equal, nonzero, chance of being selected into the sample.

A random sample is the best way of generating a sample that is representative of its population. It does not guarantee such an outcome — we will discuss sampling error later. But, it is usually the best way to draw a sample.

It may seem counter-intuitive, at first, that random sampling is a better strategy than purposive sampling — where one goes out and looks for people representing the larger population. The key to its success is that every element has an equal chance of being selected.

In practice, we sometimes modify random sampling, combining it with other strategies designed to target specific groups in the population. When a population includes a small minority group, we often have to oversample for that group. Sometimes we identify important characteristics that define the population and stratify the sample along them. Both of these techniques, though, still rely on random sampling to select elements from subgroups.

Sampling allows the researcher to generalize characteristics of the sample to the population. This kind of inference is based on the well-studied sampling distribution.

Properties of the Sampling Distribution
The most important topic in the introductory statistics course is the logic of inference, and this begins with the sampling distribution.

Illustration of how probability samples are selected from a population.

Imagine some population with a particular mean, mu, and standard deviation, sigma, on some variable. We know that scores vary around the mean, some larger and some smaller. On average, scores differ from mu by sigma.

If we take a random sample, we can calculate its mean, x-bar.

Now, imagine that we take repeated random samples from this population, and calculate the mean for each.

Illustration of how a sampling distribution is derived.

We can take these sample means and generate a frequency distribution. We can define the mean of this distribution, X-bar sub-x-bar, and its standard deviation, sigma sub-x-bar.

Most of the sample means will be relatively close to the population mean. Some will be larger and some smaller. On average, sample means will differ from X-bar sub-x-bar by sigma sub-x-bar.

If we were to draw a large number of samples, the frequency distribution of the sample means would approximate the normal curve. This allows us to take advantage of the properties of the normal curve.

These are the things you want to remember about the sampling distribution:

A. for relatively large samples, the sampling distribution approximates the normal curve for a sufficiently large number of samples;

B. the mean of the sampling distribution equals the population mean;
Mean of sample means approaches the population mean.

C. the standard deviation of the sampling distribution is less than the standard deviation of the population. We call the standard deviation of the sampling distribution the standard error
The standard error is less than the population standard deviation.

In other words, samples are more alike than cases. Think about why this is. In the population standard deviation, the presence of extreme scores (above or below the mean) makes the standard deviation larger. But when we take probability samples from that population and compute the sample standard deviation, some of those extreme cases might be selected, but they are balanced in the sample by less extreme scores. The probability of a sample with only extreme cases (as we know from the multiplication rule) is very rare. FOr this reason, the standard deviation of the sampling distribution, the standard error, tends to be smaller. Samples tend to be more alike than cases.

The Z-Test
Since the sampling distribution has the characteristics of the normal curve, we can generalize the notion of a standard score. We can calculate a z-score for a sample mean.

Formula for the z-score.

This z-score tells us the distance between the sample mean and the population mean, in standard error units.

Formula for the standard error.

The denominator of this formula is defined as the standard error and is calculated thus.

We can mark off the area under the curve in standard error units. We can think of the area under the curve having a range of about six standard errors — just as the normal curve has a range of about six standard deviations.

We can use this knowledge to estimate the probability of drawing a sample from a population with a specific mean, or larger, for example.

The Normal Curve

The normal curve is a tool for understanding how probability helps us make inferences from sample statistics to population parameters. We could build a line graph to reflect the relative frequencies of some set of outcomes. This would show us the empirical probabilities in the distribution. We can extend this notion by constructing a frequency distribution based on theoretical probabilities instead of empirical results.

For example, if we have ten marbles, 6 are black, 3 are red and one green, we can build a corresponding probability distribution. There is a 60% chance that a marble selected at random would be black. There is a 30% chance that a marble selected at random would be red and a 10% chance that the marble would be green.

Now, imagine that we have four coins. We can build a probability distribution that reflects the likelihood of generating the possible number of tails.

Probability distribution

Probability distributions can be viewed as empirical frequency distributions for an infinite number of cases. Practically speaking, this means that empirical frequency distributions for very large data files will tend to approximate the theoretical distribution more than frequency distributions for small data files.

The Normal Curve
One very important probability distribution is the normal curve, sometimes called the bell-shaped curve. It plays a central role in the statistical decision making process.

The normal curve with the population mean (mu) and population standard deviation (sigma) illustrated.

The normal curve has a number of important properties.
A. It is symmetrical;
B. it is unimodal;
C. and, the area under the curve represents proportion, or probability.

Since the area under the curve represents proportion, we can calculate the percent of cases to be expected between some given point and the mean, for example.

We can, in effect, mark off the proportions on a scale of standard deviations. We can see the percent of cases between the mean and one standard deviation above the mean.

The normal curve with one standard deviation above the mean illustrated.

And likewise, for two standard deviations above the mean.

The normal curve with two standard devations above the mean illustrated.

Three standard deviations above is just about 50%.

The normal curve with three standard deviations above the mean illustrated.

Since the curve is symmetrical, we know that there are about 6 standard deviations under the curve–three above and three below. (Remember our use of R/6 to judge the relative size of the standard deviation? This is the explanation for the denominator.)

The proportions can be expressed as probabilities. The area under the curve represents the probability of drawing a score at random from the distribution at some point or below, for example.

The normal curve illustrating the probability of selecting a case one standard devation above the mean or less.

Using Google Drive in Your Colab Notebook

1. Open Google Drive and create a folder called “Data”.
Screen capture of Google Drive

2. CLick on the “New” icon on the upper left, and choose “new folder”
Screen capture of new folder selection

3. On your Drive, look for the folder “Shared with me”
Screen capture of Google Drive folder

4. You’ll see a folder called SOCY2112 if I’ve shared the folder with you. (If you don’t see SOCY2112 in the list, email me your user id for your Google account–the email you use with Google; it is usually a @gmail account but it could be any email address.) In this example, I am going to read into my Colab notebook a specific file, “ATP W89.sav”.

5. Right click on the filename, “ATP W89.sav” and select “add shortcut to Drive”.
Screen capture of creating a shortcut on Drive

6. A popup window will appear that will allow you to determine where to put the shortcut. Click on the “>” at the right of “My Drive” and you’ll a list of the folders on your drive, including the data folder you created in step 1. You can highlight “data” and click on “Add Shortcut”.
Screen capture showing how to copy the shortcut
Screen capture showing how to navigate folders to copy shortcut

7. In your Google Drive, in the Data directory, you should see a shortcut to the file “ATP W89.sav”.
Screen capture of the Data directory on Google Drive

8. In your Colab notebook, you’ll now the the file “ATP W89.sav” in the data directory. Right click on the filename, and in the popup window, select “copy path”. You can then paste the path information into the code block where you are reading in the file.
Screen capture of Colab notebook

Activity
Create a data folder in your Google Drive and create a link in it for the American Trends Panel survey data (ATP W89.sav). Copy the path in your Colab notebook to read the file into a DataFrame.

The questionnaire for this wave of the survey is here: https://drive.google.com/file/d/1Nu9sQ4Y6GIQCQH1YZfItRYNzK9ovWRY-/view?usp=sharing. You can find the demographic variables (included in all of the waves) here: https://www.pewresearch.org/wp-content/uploads/2018/05/Codebook-and-instructions-for-working-with-ATP-data.pdf

A. Formulate a research question that connects a demographic variable (such as gender, class, or race) and a belief or behavior (such as ideology or likelihood of purchasing an electric vehicle).

B. Create a table (with rp.crosstab).

C. Interpret the results.

D. What would be a good follow-up question based on your results?

Use this notebook as a guide for the code.

Applying Data Feminism

We’ll do a quick data biography for several common social science data sources. As we discussed, we want to answer the who, how, and why questions for the source. With that information, we’ll assess how the researchers positionality may affect the data, in both positive and negative ways.

We want to think about what information is present in the data source: what variables are available for analysis based on the why and how questions. For a couple of variables, we’ll reflect on reliability and validity.

But we also want to think about what is missing. Given the general aims of the project, whose experiences are omitted from the data, and how might these absences distort our understanding if we use the source for data analysis?

Activity, part 1
1. Exploring Strategies to Improve Health and Equity in Rural Communities – https://www.norc.org/Research/Projects/Pages/exploring-strategies-to-improve-health-and-equity-in-rural-communities.aspx

2. American National Election Study – https://electionstudies.org/

3. The American Trends Panel – https://www.pewresearch.org/our-methods/u-s-surveys/the-american-trends-panel/

4. General Social Survey – https://gss.norc.org/

Now, based on the data biography for the American Trends Panel survey, we can do some data analysis and apply some of the principles of data feminism in considering the sociological meaning of our results. Here’s a notebook: https://colab.research.google.com/drive/1gcWtIcJb1VFu-w4ZKVyybpIpxe5uel-i?usp=sharing

We’ll consider a table together, then each group will produce a different table and apply the principles in a similar manner.

Data Ethics

There are lots of ways to think about data ethics, but perhaps the most comprehensive in a sociological perspective is data feminism. We often refer to a diverse collection of social theories as “feminism” rather than “feminisms” but we should be mindful that there are many feminist perspectives in sociology. We’ll borrow some ideas from these perspectives to understand power and oppression so that we can ask questions about how data practices are implicated in both processes of oppression and, potentially, liberation.

I’m summarizing the argument of Catherine D’Ignazio and Lauren F. Klein in their book Data Feminism. I highly recommend it. We won’t need to go into the full detail of their argument in this class. If you are interested in feminist theory, it is a good resource. I teach with the book in both data analysis and sociology of science classes.

The key idea is that to understand data practices, we have to begin with the concept of power. One of the most significant contributions of feminist theory in sociology is the concept of intersectionality, a recognition that power operates in complex ways that are manifest in multiple dimensions simultaneously. So we talk about race and gender and class rather than just race or gender or class. This is a good place to begin because we want to view data practices in context: how they are connected to social processes and institutions. Recognizing that power is intersectional, we can avoid an overly simplistic account of the context of data practices.

Table 1 from Data Feminism by D'Ignazio and Klein

The second key idea is that knowledge is situated in social contexts. As we discussed with the data biography, everyone who is involved in a data practice is positioned with regard to the operations of power. Our positionality shapes how we experience the world, what we believe, and how we act. To understand the effects of a data practice–which is the essential question in any data ethics–we have to know how the positionality of the researchers. The kinds of data collected, or not collected, is partly determined by the positionality of the researchers.

The third key idea follows from an understanding of power as a social arrangement. In sociology, we often talk about inequality, but this concept obscures the operation of power. It appears (or presents itself) as a fact, that some groups have more and others less. But a feminist approach recognizes that the arrangements that produce inequality are socially constructed, so their operation produces inequality; there is agency in the process. Those who benefit from the arrangement may or may not recognized their privilege, but the operation of power that produces their privilege simultaneously produces oppression. Rather than think of data practices in terms of inequality, a more sociological perspective would ask about oppression. How do these data practices produce advantages for some and disadvantages, or harms, for others?

As D’Ignazio and Klein relate, “what counts gets counted.” That is, the things that are deemed important (by those with the power to determine what is important) are more likely to be incorporated into data practices. In many contexts today, the determination of what is important is a private decision by corporations, often related to their profit motives. So these data are collected, in order to be commodified, and other kinds of data are ignored. As data scientists advocate for data practices like “machine learning” to create algorithms that make important decisions within organizations, the availability of some kinds of data and the absence of others distorts the “learning” that guides these algorithms. The result is benefits go to some and harms are inflicted on others.

A data ethics would require of us to use data practices for good. Rather than think of a data practice in terms of its effect on inequality, which is an abstraction, we should frame our analysis in terms of resisting oppression. How do we use data practices for liberation?

This shift in the frame helps us to see the connection between data skills and activism. Instead of abstract notions like “data for good” we can use data practices for co-liberation. We speak of “co-liberation” because the positionality of the researchers and of those harmed by data practices are often not the same. As activists, we have to work together, not on behalf of those harmed, but as partners with those who are harmed.

By expanding the community involved in data practices we can extend the horizons of our viewpoint. More data democracy means greater insight. The limits of our understanding as a result of our positionality can be mediated by dialogue with others differently positioned. We are all experts in our own experience, and as researchers, we can share our expertise in data practices with those whose lives are affected by them.

Data Biography

WORLD HAPPINESS REPORT

URL: https://worldhappiness.report/ed/2022/#appendices-and-data

The first step in a thorough and reflexive data ethics is data biography. If we are considering using a data source, we need to know the basic facts about the project. We should always keep in mind that data are never “raw”–that is, are never a direct, unfiltered view of the social world. Every data source is the result of labor in a particular context.

So we begin by asking “who?”: who collected these data? A data source that doesn’t provide information about the personnel of the project is suspect. In academic settings, status and reputation are often taken to signify quality work. We should be skeptical of that perspective, but it is important to know if the people responsible for the project are trained in a relevant data field and stand behind the work by attaching their names to the project.

In a related way, we want to know the context for the data collection labor. Who, or what organizations, support the project? Were the data collective for a specific purpose, which might be different from our research question, so we should think about how those differences might shape what is in, and what is absent from, these data.

This aspect of the data biography can be complex, because there might be lots of labor involved. You want to be able to develop a brief narrative that you can use in your own work analyzing these data. What is the story you want to tell about who collected the data and why?

The other element of the biography is the “how?”: how were these data collected? Again, we should be suspicious of any data source that doesn’t explain its methods. Are these data from a survey? from official sources? from mass media? or, are these data from multiple sources? It might require us to track down individual sources if the data source is effectively an aggregate of different kinds and different sources of data.

With the World Happiness Report, we can consult the most recent report and investigate the who, how, and why questions.

https://worldhappiness.report/ed/2022/foreword/

As we create a narrative to explain the data source, keep in mind questions of power. The topics that inform the data collection are shaped by social processes and institutions, and these–we know as sociologists–are the result of the operation of power. That data are shaped by power is not a reason, itself, to dismiss these data. Rather, we need to be mindful that every social actor is positioned along various dimensions of power and that positionality limits what we experience and what we believe, so perspectives can be inclusive or exclusive depending on that positionality. We always want to ask what in the social world is visible to those who collect data (in a broad sense, from defining research questions and designing a survey to the labor of collecting responses) and what is invisible?

In sociology, it can be a challenge to ask what is absent. We are trained to see what is present and visible, and to take careful measurements. It is much harder to think about what we are missing.

In this process, we want to be mindful of our own positionality. Our view of the data source and of data practices more generally is also shaped by who we are and how our own horizons are limited.

As part of the data biography, you should get a copy of the codebook or data dictionary for the source you are examining. This is necessary for analysis, but we can use it in the biography also as we ask what is included and what is absent.

Activity.
Each group will sketch a data biography for a sample question by (a) identifying a data source, (b) getting a copy of the codebook or data dictionary, (c) investigating the who, how, and why questions from information included with the data source.

1. Are workers more dissatisfied with their work or workplace today than before the pandemic?
2. How much do states spend on public education?
3. Are voters more politically polarized than they were twenty years ago?
4. Are there gender differences in wages and are these differences affected by race?

Formulating sociological questions

The logic of analysis is revealed in the relationship between questions and approaches to answering them. It gets at the heart of the question, “what counts as evidence?” We use empirical data to test our ideas about the nature of the social world. Sometimes this takes the form of hypothesis testing (more about that later) and sometimes it is more exploratory.

In this exercise, we’ll discuss what would count as evidence of a relationship between variables suggested by a sociological question. To understand the logic of analysis, we need to think about (a) how to operationalize the concepts in the question and (b) where we might find relevant data (or how we might collect it) with those variables.

For example, let’s consider the question: is religiosity related to attendance at religious services?

First, we address operationalization. How do we define religiosity and attendance in measurable ways? In making decisions about measurement, we want to avoid creating a tautology. If, for instance, we defined religiosity as ‘people who go to services a lot’ then we’ve inadvertently created a circular question: does attendance cause attendance? So we need a way to measure religiosity in this instance that is conceptually independent of attendance. It could be a question such as “how religious are you? Are you (a) very religious, (b) somewhat religious, or (c) not at all religious?”

Next, we need to operationalize attendance. Again, a relatively straightforward way to do this would be to ask “how often do you attend religious services? Is it (a) weekly, (b) monthly, (c) occasionally, (d) rarely, or (e) never?”

(We’ll look at some examples of actual survey questions for these and other concepts. There are lots of important questions about how survey researchers ask questions, but we’ll hold off on a detailed discussion of question quality for the moment.)

Second, we address sources of data. Sometimes our question is embedded in a geographic and/or temporal location — eg., do Americans attend services less often than they used to? — and that shapes where we look for data. But often we ask a question, like the example above, that is general. As a practical matter, we still need to think about geographic and temporal limitations. In general, we should prefer newer data to older if we are not explicitly asking about a historical context; it is sort of implied in the general question that we mean “is religiosity related to attendance at religious services at this point in time?” We should not, however, assume that general questions are about the society in which we are currently located. In the history of social science, that kind of assumption was generally connected to a colonial worldview. We should be more reflexive and intentionally contextualize our question by being explicit about the limitations of our ability to generalize. We might have data from the US, and we can use that to make arguments about the contemporary US, but we should not assume that it reflects the general case. There is no such thing as a view from nowhere, and our current location is not, as such, privileged.

In this instance, we could certainly find data about religiosity in the contemporary US. But we could also easily find data about religiosity elsewhere. So we could pick a context and limit our generalizations to that context, or we could try to find data from multiple places that might allow us to generalize more broadly — but still not in a totalizing way. For example, the Eurobarometer survey collects data from multiple European nations. This would be somewhat broader than just the US but still very limited and would not allow us to make an argument about the general case because, as we contemplate not only what is in the data but also what is missing from the data, we are missing data from most of the regions of the world.

We can try to expand our coverage by looking for additional data sources. As we do this, if we look at different surveys, we need to be mindful of the fact that different variables (different survey questions) would also complicate our ability to generalize relationships from our results. Even small differences in question wording can be significant. This is not to say that we shouldn’t do this kind of analysis, but rather that we need to be careful in what we consider as evidence. Our goal is not to imagine that we can perfectly answer our question. What we are seeking is an approximation, or several, of an answer by recognizing the limitations of our data.

EXERCISE #1
1. Is political participation a function of social class?
2. Does working at large organizations cause alienation?
3. Does gender identity shape occupational aspirations?
4. Is the US more politically polarized now than it used to be?

EXERCISE #2 with data!
Look for variables in the 2017 Baylor Religion Survey codebook that would allow you to try to answer the following questions.
1. Are women more religious than men?
2. Are people more religious in rural places than cities?
3. Are religious people more likely to be married?
4. Is there a relationship between educational attainment and prayer?