Chapter 14 Reading and Understanding Social Research

14.1 Reading Reports of Sociological Research

By now you should have a good idea about the basic components of sociological research projects. You know how sociological research is designed, and you are familiar with how to frame a review of sociological literature. In Chapter 5 "Research Design", we discussed the various components of a research project and presented some tips on how to review literature as you design your own research project. But I hope that you’ll find the sociological literature to be of interest and relevance to you beyond figuring out how to summarize and critique it in relation to your research plans. We sociologists like to think the research we do matters, but it cannot matter if our research reports go unread or are not understandable. In this section we’ll review some material from Chapter 5 "Research Design" regarding sociological literature and we’ll consider some additional tips for how to read and understand reports of sociological research.

As mentioned in Chapter 5 "Research Design", reading the abstract that appears in most reports of scholarly research will provide you with an excellent, easily digestible review of a study’s major findings and of the framework the author is using to position her findings. Abstracts typically contain just a few hundred words, so reading them is a nice way to quickly familiarize yourself with a study. Another thing to look for as you set out to read and comprehend a research report is the author’s acknowledgments. Who supported the work by providing feedback or other assistance? If relevant, are you familiar with the research of those who provided feedback on the report you are about to read? Are any organizations mentioned as having supported the research in some way, either through funding or by providing other resources to the researcher? Familiarizing yourself with an author’s acknowledgments will give you additional contextual information within which to frame and understand what you are about to read.

Once you have read the abstract and acknowledgments, you could next peruse the discussion section near the end of the report, as suggested in Chapter 5 "Research Design". You might also take a look at any tables that are included in the article. A tableA tool used by researchers who wish to present large amounts of data in a succinct format. Tables are most commonly used in reports of quantitative research findings. provides a quick, condensed summary of the report’s key findings. The use of tables is not limited to one form or type of data, though they are used most commonly in quantitative research. Tables are a concise way to report large amounts of data. Some tables present descriptive information about a researcher’s sample. These tables will likely contain frequencies (N) and percentages (%). For example, if gender happened to be an important variable for the researcher’s analysis, a descriptive table would show how many and what percent of all study participants are women and how many/what percent are men. Frequencies, or “how many,” will probably be listed as N, while the percent symbol (%) might be used to indicate percentages.

In a table presenting a causal relationship, independent variable attributes are typically presented in the table’s columns, while dependent variable attributes are presented in rows. This allows the reader to scan across a table’s rows to see how values on the dependent variable attributes change as the independent variable attribute values change. Tables displaying results of quantitative analysis will also likely include some information about the strength and statistical significance of the relationships presented in the table. These details tell the reader how likely it is that the relationships presented will have occurred simply by chance.

Let’s look at a specific example. Table 14.1 "Percentage Reporting Harassing Behaviors at Work", based on data from my study of older workers, presents the causal relationship between gender and experiencing harassing behaviors at work. In this example, gender is the independent variable and the harassing behaviors listed are the dependent variables.It wouldn’t make any sense to say that people’s workplace experiences cause their gender, so in this example, the question of which is the independent variable and which are the dependent variables has a pretty obvious answer. I have therefore placed gender in the table’s columns and harassing behaviors in the table’s rows. Reading across the table’s top row, we see that 2.9% of women in the sample reported experiencing subtle or obvious threats to their safety at work, while 4.7% of men in the sample reported the same. We can read across each of the rows of the table in this way. Reading across the bottom row, we see that 9.4% of women in the sample reported experiencing staring or invasion of their personal space at work while just 2.3% of men in the sample reported having the same experience.

Of course, we cannot assume that these patterns didn’t simply occur by chance. How confident can we be that the findings presented in the table did not occur by chance? This is where tests of statistical significance come in handy. Statistical significanceA report of the likelihood that relationships observed could be caused by something other than chance. tells us the likelihood that the relationships we observe could be caused by something other than chance. While your statistics class will give you more specific details on tests of statistical significance and reading quantitative tables, the important thing to be aware of as a nonexpert reader of tables is that some of the relationships presented will be statistically significant and others may not be. Tables should provide information about the statistical significance of the relationships presented. When reading a researcher’s conclusions, be sure to pay attention to which relationships are statistically significant and which are not.

In Table 14.1 "Percentage Reporting Harassing Behaviors at Work", you’ll see that a p value is noted in the last very column of the table. A p valueA statistical measure of the probability that there is no relationship between the variables under study. is a statistical measure of the probability that there is no relationship between the variables under study. Another way of putting this is that the p value provides guidance on whether or not we should reject the null hypothesis. The null hypothesisThe assumption that no relationship exists between variables in question. is simply the assumption that no relationship exists between the variables in question. In Table 14.1 "Percentage Reporting Harassing Behaviors at Work", we see that for the first behavior listed, the p value is 0.623. This means that there is a 62.3% chance that the null hypothesis is correct in this case. In other words, it seems likely that any relationship between observed gender and experiencing threats to safety at work in this sample is simply due to chance.

In the final row of the table, however, we see that the p value is 0.039. In other words, there is a 3.9% chance that the null hypothesis is correct. Thus we can be somewhat more confident than in the preceding example that there may be some relationship between a person’s gender and his experiencing the behavior noted in this row. We might say that this finding is significant at the .05 level. This means that the probability that the relationship between gender and experiencing staring or invasion of personal space at work is due to sampling error alone is less than 5 in 100. Notice that I’m hedging my bets here by using words like somewhat and may be. When testing hypotheses, social scientists generally couch their findings in terms of rejecting the null hypothesis rather than making bold statements about the relationships observed in their tables. You can learn more about creating tables, reading tables, and tests of statistical significance in a class focused exclusively on statistical analysis. For now, I hope this brief introduction to reading tables will give you more confidence in your ability to read and understand the quantitative tables you encounter while reading reports of sociological research.

Having read the tables in a research report, along with the abstract, acknowledgments, and discussion in the report, you are finally ready to read the report in its entirety. As you read a research report, there are several questions you can ask yourself about each section, from abstract to conclusion. Those questions are summarized in Table 14.2 "Questions Worth Asking While Reading Research Reports". Keep in mind that the questions covered here are designed to help you, the reader, to think critically about the research you come across and to get a general understanding of the strengths, weaknesses, and key takeaways from a given study. I hope that by considering how you might respond to the following questions while reading research reports, you’ll feel confident that you could describe the report to others and discuss its meaning and impact with them.

14.2 Being a Responsible Consumer of Research

Being a responsible consumer of research requires that you take seriously your identity as a social scientist. Now that you are familiar with how to conduct research and how to read the results of others’ research, you have some responsibility to put your knowledge and skills to use. Doing so is in part a matter of being able to distinguish what you do know based on the information provided by research findings from what you do not know. It is also a matter of having some awareness about what you can and cannot reasonably know as you encounter research findings.

When assessing social scientific findings, think about what information has been provided to you. In a scholarly journal article, you will presumably be given a great deal of information about the researcher’s method of data collection, her or his sample, and information about how the researcher identified and recruited research participants. All these details provide important contextual information that can help you assess the researcher’s claims. If, on the other hand, you come across some discussion of social scientific research in a popular magazine or newspaper, chances are that you will not find the same level of detailed information that you would find in a scholarly journal article. In this case, what you do and do not know is more limited than in the case of a scholarly journal article.

Figure 14.1

Information provided on the same study will vary dramatically depending on the source reporting the study’s findings. In general, popular media and news sources will provide fewer details than scholarly sources such as academic journals.

© Thinkstock

Also take into account whatever information is provided about a study’s funding source. Most funders want, and in fact require, that recipients acknowledge them in publications. But more popular press may leave out a funding source. In this Internet age, it can be relatively easy to obtain information about how a study was funded. If this information is not provided in the source from which you learned about a study, it might behoove you to do a quick search on the web to see if you can learn more about a researcher’s funding. Findings that seem to support a particular political agenda, for example, might have more or less weight once you know whether and by whom a study was funded.

There is some information that even the most responsible consumer of research cannot know. Because researchers are ethically bound to protect the identities of their subjects, for example, we will never know exactly who participated in a given study. Researchers may also choose not to reveal any personal stakes they hold in the research they conduct. While researchers may “start where they are,” a process outlined in Chapter 4 "Beginning a Research Project", we cannot know for certain whether or how researchers are personally connected to their work unless they choose to share such details. Neither of these “unknowables” is necessarily problematic, but having some awareness of what you may never know about a study does provide important contextual information from which to assess what one can “take away” from a given report of findings.

14.3 Media Reports of Sociological Research

As you have probably already gathered, we are likely to encounter sociological research in the news and other media. For example, check out the American Sociological Association’s media coverage links (http://www.asanet.org/press/media_coverage_highlights.cfm). There you’ll see that for just one study, on the consequences of parental divorce for child development (Kim, 2011),Kim, H. S. (2011). Consequences of parental divorce for child development. American Sociological Review, 76, 487–511. there were 170 news articles describing the study and its findings over the course of one month, June 2011. This particular study provides a good example of the difference between the information provided about a study in a scholarly journal article and the media’s coverage of the same study.

Figure 14.2

A recent sociological study of the impact of parental divorce on children was covered in 170 popular media sources in one month alone.

© Thinkstock

Let’s look at some of the differences between the aforementioned study’s coverage in the media and its treatment in a scholarly journal. First, watch the following coverage from The View’s August 24, 2011, program: http://theview.abc.go.com/video/hot-topics-effects-divorce-kids. Once you have watched the clip, ask yourself what you have learned about the study. Who conducted the research described? What are the study’s key findings? How many people participated in the study? Who were those participants? What sorts of data were analyzed? Which findings were statistically significant? Also note what questions you still have about the study. Where might you go to get the answers to your questions?

After watching The View clip several times, I was able to gather that the study has two key findings: (a) a child is more negatively affected by losing a parent to divorce than by the tension that leads to the breakup, and (b) children’s math scores drop after a divorce but reading and “other skills” do not suffer. As far as who participated, I heard that “3-year-olds and so on” were the participants, though I am not certain how many of them participated. I also don’t know who conducted the study, who (if anyone) provided funding for the study, when the data were collected, and so on. But if you review the article published in the American Sociological Review (ASR) that reports results of the study, all these questions are answered.

You might be saying to yourself, “So what?” Perhaps you took note that The View coverage does mention that the study was published in the ASR. If you did notice this, then kudos to you. Because the ASR is a peer-reviewed publication of the American Sociological Association, we should have some confidence that the study is reputable. But we still don’t hear all the information that might shape what we choose to take away from this study. For example, a review of the ASR article will tell us that the data come from a sample of people who were in kindergarten from 1998 to 1999. Perhaps that is of little consequence, but we might wish to pause to consider whether or how our cultural social context has shifted since 1998 and how that might impact how kindergartners today respond to parental divorce. I am not at all suggesting that only studies whose data are seconds or days old hold value. (If that were the case, I’d say we can safely disregard any of my own publications.) Instead, I want to call your attention to some of the questions you might ask yourself as a responsible consumer of research.

In addition to all the times that sociological research does make the news, there are also instances when it does not but probably should. In June 2011, for example, an article on children’s gender nonconformity appeared in the New York Times (Hoffman).Hoffman, J. (2011, June 10). “Boys will be boys?” Not in these families. New York Times. Retrieved from http://www.nytimes.com/2011/06/12/fashion/new-challenge-for-parents-childrens-gender-roles.html?pagewanted=all The article took the perspective that children’s expressions of gender were natural and biologically ingrained. While we cannot say for certain that this isn’t true, we do know from many years of reputable and highly regarded research by sociologists of gender that gender norms and behaviors are in many cases constructed socially, not biologically. That the article omits this perspective and the voices of sociologists who do research in this area is unfortunate—both for New York Times’ readers and for sociology.

Keeping in mind your knowledge about sociology and sociological research the next time you come across descriptions of sociological research in various media outlets, ask yourself some questions about the research you encounter.

1. Where do you see sociological research described?
2. How is it described?
3. What information is present, and what is missing from the media account of sociological research?
4. How and where might you access the details that are missing?

Keep an eye out for the absence of sociological research as well and consider the following:

1. Are there programs or news stories that might be well served to incorporate sociological research findings?
2. How might the inclusion of sociological research shift the story?

By asking yourself these questions as you go about your daily routine, you will have integrated sociological research into your everyday life.

14.4 Sociological Research: It’s Everywhere

A few years ago, I was at home minding my own business and watching one of my favorite shows, Law & Order: Special Victims Unit, when sociology made an appearance. The episode, as I recall, centered on a child who was bullied at school because she had two mothers. In the show, the lawyers discuss research on parenting that was published in the American Sociological Review.While my search uncovered that the episode to which I’m referring originally aired on NBC on December 6, 2005, I have not been able to unearth the article to which the show’s characters refer. The American Sociological Association does note, however, Law & Order: Special Victims Unit’s mention of the journal on its website: http://www.asanet.org/news/2005.cfm.

It’s amazing where and how often you might discover sociology rearing its head when you begin to pay attention, look for it, and listen for it. The benefit of having knowledge about sociological research methods is that when sociology does appear in your everyday life, you’ll be better equipped to understand those brief mentions than you would be without some background in research methods.

Sometimes we might come across sociological research and not even realize it. As you’ve seen in the examples described throughout this chapter, there are opportunities every day to encounter sociological research or, at the very least, its effects. Remember our discussion of the Walmart case in Chapter 1 "Introduction"? As you may recall, Professor William Bielby testified as a sociologist on behalf of the plaintiffs in the case. The Walmart case is a great example of sociology playing a role in matters of everyday life even when we may not realize it. Sociologists have participated as expert witnesses in numerous other cases as well. As a sociologist who studies workplace harassment, I was once called upon to offer the sociological perspective in sexual harassment suit. Professor Emeritus Lewis Yablonsky (2002)You can read more about the cases that Professor Yablonsky has been involved in in the following article he wrote for the American Sociological Association’s newsletter in 2002: Yablonsky, L. (2002, January). Sociologists as expert witnesses in the criminal justice system. Footnotes. Retrieved from http://www.asanet.org/footnotes/jan02/fn17.html has been involved in more than 50 cases, providing his expert sociological opinion on cases involving homicide and other forms of violence.

In addition to offering their expert testimony in court cases and law suits, sociologists also play a role in shaping social policy. Professor Valerie Jenness, for example, has consulted with the state of California to help craft corrections policies there, particularly those focused on transgender inmates, sexual assault in correctional facilities, and hate crime statute implementation (http://www.asanet.org/about/awards/public/Jenness.cfm). Professor Diane Vaughn, an organizational sociologist, participated in the investigation following the space shuttle Columbia’s disintegration during reentry in 2003. Vaughn’s sociological perspective added a social dimension to the investigation and helped identify the social and cultural factors at NASA that contributed to the Columbia’s demise (http://www.asanet.org/about/awards/public/vaughan.cfm). Finally, Dr. Darlene Iskra’s research “had a dramatic impact on national policy” (http://www.asanet.org/about/awards/public/segal.cfm) when her work on gender discrimination in the military led to legislation that eliminated unequal requirements for men and women personnel serving in Saudi Arabia (“What is a trailblazer?,” 2011).What is a trailblazer? Dr. Darlene Iskra, adjunct instructor, sociology, is a Navy pioneer. (2011, July 15). Columbia College Spotlights. Retrieved from http://spotlight.ccis.edu/2011/07/what-is-trailblazer.html These are just a few of the many examples of how the sociological perspective and sociological researchers have played a role in shaping our policies.A useful source for additional examples is the American Sociological Association’s descriptions of past winners of its prestigious Public Understanding of Sociology Award. Those descriptions can be found at http://www.asanet.org/about/awards/public.cfm.

Figure 14.3

In some ways, we are all armchair sociologists. While each of us, at one time or another, may have made some unfounded assumptions about the way the world works, having a background in sociological research methods can help us question those kinds of assumptions.

© Thinkstock

Another way that we might inadvertently come across sociology is when we encounter the ever-popular armchair sociologist. Perhaps you’ve met some of these folks or even played the role yourself a time or two. Armchair sociologists tend to wax poetic about how society “is” or how various groups of people “are” without having anything more than anecdotal evidence (or perhaps no evidence at all) to support their sweeping claims. Remember the example from Chapter 1 "Introduction" about a friend who once proclaimed that “all men lie all the time?” That’s a perfect example of armchair sociology. Now that you are equipped with a better understanding of how we know what we know, and in particular how sociologists know what they know, you are well prepared to question the assumptions of the armchair sociologists you meet. And by sharing with others what you know about how we “know” things, perhaps you’ll even help others break the habit of making unfounded assumptions. Understanding sociological research methods is excellent preparation for questioning the everyday assumptions that others make. And let’s face it; we’ve all probably made some unfounded assumptions about the way the world works or about what “other” people are like at one time or another.