The logic of analysis is revealed in the relationship between questions and approaches to answering them. It gets at the heart of the question, “what counts as evidence?” We use empirical data to test our ideas about the nature of the social world. Sometimes this takes the form of hypothesis testing (more about that later) and sometimes it is more exploratory.

In this exercise, we’ll discuss what would count as evidence of a relationship between variables suggested by a sociological question. To understand the logic of analysis, we need to think about (a) how to operationalize the concepts in the question and (b) where we might find relevant data (or how we might collect it) with those variables.

For example, let’s consider the question: is religiosity related to attendance at religious services?

First, we address operationalization. How do we define religiosity and attendance in measurable ways? In making decisions about measurement, we want to avoid creating a tautology. If, for instance, we defined religiosity as ‘people who go to services a lot’ then we’ve inadvertently created a circular question: does attendance cause attendance? So we need a way to measure religiosity in this instance that is conceptually independent of attendance. It could be a question such as “how religious are you? Are you (a) very religious, (b) somewhat religious, or (c) not at all religious?”

Next, we need to operationalize attendance. Again, a relatively straightforward way to do this would be to ask “how often do you attend religious services? Is it (a) weekly, (b) monthly, (c) occasionally, (d) rarely, or (e) never?”

(We’ll look at some examples of actual survey questions for these and other concepts. There are lots of important questions about how survey researchers ask questions, but we’ll hold off on a detailed discussion of question quality for the moment.)

Second, we address sources of data. Sometimes our question is embedded in a geographic and/or temporal location — eg., do Americans attend services less often than they used to? — and that shapes where we look for data. But often we ask a question, like the example above, that is general. As a practical matter, we still need to think about geographic and temporal limitations. In general, we should prefer newer data to older if we are not explicitly asking about a historical context; it is sort of implied in the general question that we mean “is religiosity related to attendance at religious services at this point in time?” We should not, however, assume that general questions are about the society in which we are currently located. In the history of social science, that kind of assumption was generally connected to a colonial worldview. We should be more reflexive and intentionally contextualize our question by being explicit about the limitations of our ability to generalize. We might have data from the US, and we can use that to make arguments about the contemporary US, but we should not assume that it reflects the general case. There is no such thing as a view from nowhere, and our current location is not, as such, privileged.

In this instance, we could certainly find data about religiosity in the contemporary US. But we could also easily find data about religiosity elsewhere. So we could pick a context and limit our generalizations to that context, or we could try to find data from multiple places that might allow us to generalize more broadly — but still not in a totalizing way. For example, the Eurobarometer survey collects data from multiple European nations. This would be somewhat broader than just the US but still very limited and would not allow us to make an argument about the general case because, as we contemplate not only what is in the data but also what is missing from the data, we are missing data from most of the regions of the world.

We can try to expand our coverage by looking for additional data sources. As we do this, if we look at different surveys, we need to be mindful of the fact that different variables (different survey questions) would also complicate our ability to generalize relationships from our results. Even small differences in question wording can be significant. This is not to say that we shouldn’t do this kind of analysis, but rather that we need to be careful in what we consider as evidence. Our goal is not to imagine that we can perfectly answer our question. What we are seeking is an approximation, or several, of an answer by recognizing the limitations of our data.

EXERCISE #1
1. Is political participation a function of social class?
2. Does working at large organizations cause alienation?
3. Does gender identity shape occupational aspirations?
4. Is the US more politically polarized now than it used to be?

EXERCISE #2 with data!
Look for variables in the 2017 Baylor Religion Survey codebook that would allow you to try to answer the following questions.
1. Are women more religious than men?
2. Are people more religious in rural places than cities?
3. Are religious people more likely to be married?
4. Is there a relationship between educational attainment and prayer?