Jump to content


Photo

Assumptions in qualitative research


  • This topic is locked This topic is locked
No replies to this topic

#1 Administrator

Administrator

    Administrator

  • Admin
  • PipPipPip
  • 422 posts
  • Gender:Male

Posted 11 October 2004 - 03:48 PM

From: Simon Lai
Posted: Mon 25/11/2002

I am a research student and I have been attended a departmental seminar hearing discussion about qualitative research which made me confusing. I am not sure if it is appropriate to present the issues for discussion here but would deeply appreciate any guidiance to books or literatures and responses. The issues are:

1. Qualitative research must not involve numbers (statistical analyses). Otherwise, they are not qualitative research because they violate the assumption of qualitative research (I am sorry that I did not no the assumption of qualitative research. Does it like the assumption in statistical analysis?)

2. Doing chi-square comparisons on the differences in the percentage of yes/no responses between groups (teenagers, parents, and teachers) is not "scientific" as the groups are not sampled in random. How should be the sampling issue be solved when doing statistical analysis in qualitative research? Or is it legitimate to do statistical analysis in qualitative research?

3. Most qualitative research (although what I have heard is "all qualitative research", but I think using "all" has a very high probability of being wrong) is about finding themes, and we must present the themes if we are doing qualitative research

From: Paula L. W. Sabloff
Posted: Tue 26/11/2002

As a professor of anthropology, I feel your questions give me an opportunity to give something back to the listserv group. Your questions are important so I want to answer each in turn.

1. Whoever said qualitative research must remain qualitative and not come into numerical analysis was just plain wrong. It is perfectly valid to report qualitative data in terms of percentages, counts, chi square, etc. See H. Russell Bernard's recent book on qualitative analysis for the social sciences. I use "Research Methods in Anthropology" but I know he has written a book for all the social sciences with examples taken from many fields. I've taught from the book for years and students at every level have loved it.

2. There are many kinds of sampling techniques. People have taught whole courses devoted just to that. Of course random sampling is the best method but it cannot always be used. Again, See H. Russell Bernard for the different types (stratified, quota, opportunity, representational, etc.). Check out his web page (he's at the Univ. of Florida-Gainesville) for leads also.

3. Not necessarily. However, that is the strong point of qualitative analysis. It also gets at 'why' better than statistical/quantitative analysis. i could go on but have to get back to work. perhaps your university has a research methods class in the anthropology department. I'm sure that would help you even if you audited it

From: R. Allan Reese
Posted: Tue 26/11/2002

I will endorse what Paula Sabloff wrote, and as a mainly quantitative data analyst add a comment on sampling.

Terms such as (not) "scientific" and "random sample" are often used academic putdowns by people reluctant to evaluate the evidence. One point is that "random sampling" may NOT be the best strategy; I quote W Edwards Deming as a convenient reference that the argument that a judgment sample may give more useful results, albeit that you cannot then apply probabilistic inference. A second point is that what you may actually require is an *unbiased* sample, and as long as you have no reason to expect your convenience sample to be biased *in respect of the quantity being tested*, then it may be used as a pseudo-random sample. The third point is that inferences are very rarely, if ever, based on the Platonic model described in statistics textbooks. If you truly believed the response of interest was unrelated to any variate you measured, why the hell did you choose to make those measurements?! Any test has to be interpreted in the light of previous knowledge or beliefs. This may be formalized in the Bayesian methodology, but is always implicit in the selection of what is "interesting".

Reference
Author: Deming, William Edwards
Title: Sample design in business research
Imprint: N.Y. : Wiley, 1960

From: Lyn Richards
Posted: Wed 27/11/2002

You've had wise words from members of this wise Forum, and I'd like to add a few thoughts. In years of working with new and experienced qualitative researchers, I've learned that the sorts of "rules" you outline are most destructive of qualitative research initiatives, since those always require flexibilty. I'd offer a couple of alternative "rules":

(1) All human thinking involves numbers. If numbers are the best way to express a discovery, they are not inappropriate. There's nothing "unqualitative" about numbers. For example, in my outer suburbia fieldwork for "Nobody's Home", it mattered hugely how many people came to the different residents' group meetings in different locations. If I hadn't counted them, I'd not have seen the social pressures at work.

But qualitative researchers properly avoid numbers when to express data numerically would pre-emptively reduce their data. If I want to understand how you see your research life, I'll listen to your words and record them, review and re-view them as I reflect on the meanings you put on research. If on the other hand I only want to know if research is "very important" = 1, "important" = 2 or "not important" = 3 to you, I can immediately and properly reduce your answer to a number. That's fine but I need to be sure that later I won't want to re-view what you said about its importance to you, and explore the many dimensions of the reasons for research being important. If that's what I'm asking about, I'd better not reduce your answer to a number.

(2) Other things being equal, "rules" of what qualitative research mustn't do are usually dysfunctional, since when it's good, qualitative analysis is flexible and adaptive to surprise and discovery. I can't think of any sort of data or analysis that hasn't turned up in some brilliant field research project or social policy contribution that was predominantly qualitative. And of course the increasingly rich field of mixed methods teaches new ways of combining the insights from different ways of asking and answering questions.

(3) Discovery in qualitative research is a process of making sense of complexity. That there are lots of ways of doing this is evidenced by the many methods available. Finding themes is one way of talking about discovery and making sense - but only one way. In "Readme First", my recent book with Jan Morse, I wrote about this, and about the need to have a *fit* between method and question, question and data, data and way of handling data. It's that fit that we use in evaluating the usefulness of different sorts of data and different ways of exploring data.

(4) Rule (4) is that there are no rules - not the inflexible, rigid sort of rules that restrict your enquiry. Qualitative research is, like commonsense understanding, best done by ways appropriate to the question and the goals.

(5) I'd add a final rule. Qualitative research should never be a default option. To work qualitatively is not better, finer, more enlightening or more moral. It *is* sometimes the best way of answering a questions. But you need to show this - you need a good reason for working qualitatively, since it will likely be more expensive of time and resources, and most importantly, it will be more ethically challenging, than other ways of working.

I hope this helps further, Simon. None of these are new thoughts, most are discussed and debated in qualitative groups, but those seem to be isolated often from "mainstream" research. Listening to discussions like those on this Forum will offer a hugely useful antidote to the more dogmatic and rulebound methodological statements.

From: Lioness Ayres
Posted: Thu 28/11/2002

This is lovely. In the spirit of the rule saying there are no rules, however, I'd like to point out that all of these rules that are not rules assume a pragmatic stance towards qualitative research. I would be first in line (or anyhow third in line behind Pat and Lyn) for pragmatism, but we need to recognize methods in which there ARE rules, in which emergent design flexibility is very limited, in which numbers are not allowed, and according to which qualitative research IS the default option.
I think the issue here is that there is no such thing as "qualitative research;" rather that the range of qualitative methods varies across these qualities. For this reason, generalizations across qualitative methods are difficult - they share a tendency not to use measurement theory as an essential part of the design, I suppose, but little else. Even traditional notions about small sample sizes don't seem to hold anymore based on information I have gathered from forum members.

Isn't that helpful?

From: Lyn Richards
Posted: Thu 28/11/2002

Mmm, Lioness, I agree, of course, that particular methods have rules - was responding to Simon's instructions about qualitative work generically.

Perhaps that's the most important response, Simon - that there is no generic "qualitative research". Which makes your original puzzle both simpler and more complex.

From: Simon Lai
Posted: Fri 29/11/2002

Many thanks for your reply and comments on having rules in qualitative research. I have attended a qualitative research method course 3-4 years ago and there were some quantitative people asking for decision-making rules in qualitative analyses, something like the confidence level of 0.05 to determine whether to accept or reject a hypothesis. But the speaker answered them that there is no such rule in qualitative research. Researchers always have to be conscious and careful in selecting the appropriate way to understand the data and information obtained. All the responsibility from making the choices of analyses are placed on the researhers instead of some authorities or rules. If researchers avoid this challenge by establishing some standards for all qualitative data analyses, I think the whole point of qualitative analyses is lost.

Just as suggested in Lyn's reply (thanks again), there are rules in doing good qualitative research but they must not be simple and mechanical ones like those suggested from a few "qualitative" researchers in my department (must not use number or statistical tests, must study and present theme, must have a random sample, etc.).

From: R. Allan Reese
Posted: Fri 29/11/2002

An over-emphasis on rigid methodology always worries me, and some qualitative regimes seem more about building the ego of the method-guru than promoting best practice. Tangentially, the reference to 0.05, still widely quoted as some sort of "gold standard" for hypothesis testing in statistics is, in my view, evidence of out-of-date and inadequate knowledge (but please don't anyone take this personally and flame me).

The basis for using a single value arose in the 1930s when calculations were laborious and the test statistic had to be interpreted by looking up a table. For practical reasons, statistics were tabulated at their 0.05 and 0.01 values. Any computer package now routinely displays a calculated p value, and that has to be interpreted in context. Not the least of the context is to examine the "textbook" assumptions under which the data were collected - ie look for patterns AND exceptions.

From: Lioness Ayres
Posted: Fri 29/11/2002

Speaking of not wanting to be flamed but . . . why rely on values of p at all when they are so vulnerable to influence by sample size? I would not hold myself up as (or even next to) a statistician but folks I rely on in these matters tell me that effect sizes are more informative and useful than p values in many circumstances. Some interesting reading coming out in the mixed methods literature about "quantizing" and "qualitizing" - probably many list members are thinking about statistics as another way to look at their data. Seems to me Allan is dead right about understanding our findings in context no matter what analytic tools we are using.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users