Wednesday, November 14, 2007

Qualifying and Quantifying the Physical Sciences

It's nice to know that it is not only in social sciences such as sociology and linguistics where there seems to be a qualitative/quantitative divide -- today, Female Science Professor, an anonymous female professor in the physical sciences, also discusses the debate between quantitative and non-quantitative methods in her field and the gendering that is associated with the assumptions of who is a qualitative and who is quantitative methodologist.

Her basic message, and one I hear often repeated by people who glorify statistics[1], is twofold. First, there is an assumption that quantitative methods arrive at some "true" answer. Now, this is, of course, true. Most statistics will give you a true answer in the sense that the computer program will give you a number best fit to the data of a given sample that is the most accurate answer to that problem. But, what is particularly important in what she says is where this runs into problems:

You can always get a number, but even if it means something.. what does it mean? (i.e., the number itself is not an end in itself, you have to think about it).

Just because one can get a number means that one has to be able to articulate A) what that number is, what it means, how it should be interpreted, etc. and B) what assumptions were used to produce that number. Now, those who glorify statistics, will indicate that this is the true number for which we should base all of our knowledge. The problem is that there are statistical assumptions made about the derivation of that number (i.e. sampling error and modeling error), not to mention the methodological concern over how the question was worded asked, primed by previous questions, etc. (i.e. measurement error; see my earlier post on that topic). Qualitative researchers often challenge quantitative researchers on what that number means and, in my experience, quantitative researchers often overly exaggerate the meaning of the number, particularly if it revolves around something like culture, attitudes or beliefs—things that are often complicated and nuanced.

On the other hand, I have seen qualitative researchers argue that, because of the assumptions required for quantitative methods, all of the findings are invalid. The problem here is that all analysis, by definition, requires that assumptions be made. Just because one has a more nuanced version of reality in qualitative methodology does not mean that it is not prone to errors as well. The sample might not be representative, informants might have the desire to be friends and not discuss difficult topics with the analyst, processes might be true for this subset, but it might be because of that subset's particular social position.

Although this is at this point dogma in sociology, the only way to appropriately address many questions is by some combination of quantitative and qualitative methods. But, here is where I see FSP's second issue arising: quantitative methods are seen as particularly rigorous because they require an understanding of math, equations, and statistics. These are also areas that are traditionally dominated by men. Qualitative methods are seen as feminine and not very difficult. The theory goes that anyone can talk to someone, where's the difficulty in that?[2] These comments, I find, often come from people—particularly when they find people in the media—who have a severe lack of understanding of statistics and don't realize the assumptions inherent in statistically analyses and blindly trust the all-holy "Number."

While there is beginning to be an appreciation of such topics, the understanding of statistics and math to be "difficult" (and, thus, by extension professors who are able to use that methodology indispensable) and qualitative methods "easy", that this is going to do a great disservice to both sociology as a whole. A large part of the problem, as I see it is the lack of cooperation between qualitative and quantitative sociologists. Grants are so scarce as it is, we are afraid to try and write anything beyond the proscribed standards to work with teams that might look at problems in both directions. Or, alternatively, individual researchers try and take on both components themselves many with great success. The problem I have found with this goal in my own development is that it is very difficult to be on top of methods in two very different methodological traditions and to produce anything that can pass muster in both qualitative and quantitative fields. I don't have enough time in my graduate career to do both well - maybe it might come when I have my first job or a postdoc, but it seems a severe disservice to the pursuit of knowledge to maintain these divisions. This division seems particularly severe for people associated with qualitative methods - particularly when that assumption (as is the case with FSP) is wrong and based solely upon one's gender!

[1] I am not positive that she is discussing statistics, this is the world in which I live, so I will go on that for my purposes.
[2] This is where I am sure that there is some difference in what FSP does and what qualitative sociologists do; unless there is some branch of the physical sciences wildly different than what I thought of the physical sciences -- talking to particles and all.


wobblie said...

I completely agree with your larger point re: the gendered nature of the quant/qual divide, and that for the most part, scholars who use different methodologies do a poor job of listening and learning from each other.

I did want to make the point, however, that there has been some interesting from folks who've found a way to bridge that particular divide. Randy Hodson at Ohio State immediately springs to mind - he's created a dataset from a number of ethnographies of work to use for quantitative analysis. You obviously have to know how to evaluate the quality of a particular ethnography and brush up on some issues involving coding, but I found his work to be a particularly novel way of bringing two seemingly disparate methodologies into some harmonious working relationship.

Mike3550 said...

Wobs, thanks for the advice - I'll definitely check out Hodson's work.

I guess what I find is that there is a lack of question-centered approaches as opposed to methodological approaches. Some really great work has been done looking at neighborhood effects trying to look at the same issue from both qualitative and quantitative perspectives. One example that springs to mind is the ethnographic Heat Wave by Klinenberg and the subsequent quantitative analysis by Browning, et al.

Being able to validate findings across methodologies or develop new theories by applying the methods from each other seems more productive than constantly bickering with each other about what can and cannot be said in any given methodological epistemology.

Post a Comment