We’re midway through the research methods course and, this past weekend, I facilitated a workshop for students interested in ethnographic methods. As it turned out, most of the students who attended weren’t really intending to use ethnographic methods - some were just curious what ethnography was; others were intending to use interviews (including some relatively structured interviews) in their research, and weren’t sure where the boundary lay between interviews and ethnography. Since I’m only a sort of accidental ethnographer myself, it was no problem to adapt the session to cover issues of more general interest to qualitative researchers who might or might not ultimately decide to use ethnographic techniques.
As students shared their questions and research experiences to date, I was struck by some students’ accounts of arguably questionable advice about research design.
Several students, for example, wanted to know how many interviews they need to conduct to design a good qualitative study. This is a fairly common question for students tackling their first research project, and I always answer it the same way: it depends on what you are trying to prove. The basic rule of any kind of research - qualitative or quantitative - is that your claims need to be commensurate with your evidence. A single interview might be adequate for some research questions; a carefully normed sample of thousands of individuals might be necessary for other questions. No matter how many interviews you conduct, sheer volume won’t shield you from criticism if you make wild claims from inadequate evidence.
All of this is fairly obvious, of course - but it didn’t prevent one student from saying that their supervisor had indicated that around a dozen interviews would be sufficient for an undergraduate thesis. In other words, the level of the student, rather than the requirements of the research question, was taken to be the primary determining factor for the research design.
There is a rational core to this advice: it may in fact not be realistic to expect an undergradate thesis, which has to be completed by an inexperienced researcher within a fairly short period of time, and with little monetary or practical support, to involve more than a dozen detailed interviews. If this is the case, however, the supervisor needs to guide the student to a research question on which a dozen well-designed interviews can cast some meaningful light. This may, in fact, have been the spirit in which the supervisor picked a number out of the air - to caution the student to narrow and focus an overbroad research question. It was not, though, the spirit in which the advice was received - which is a pity, as understanding the connection between research questions and methodologies is one of the key insights students can gain from their research apprenticeship for their undergradate thesis.
The other issue that disturbed me - although I’ll have to admit that this one crops up well beyond introductory methods courses - was whether a particular method, or a particular research environment, can capture more “natural” behaviour than other methods or environments.
It’s not uncommon to see researchers earnestly claim in published works that quantitative research, or more structured qualitative research, is more “artificial” than less structured forms of qualitative social science. I am uneasy with this kind of claim on several levels.
First, I don’t personally have any problem with research that deliberately creates “artificial” conditions in order, e.g., to isolate and simplify variables and work more precisely on particular problems. Knowledge gained in this way is no less valid and useful, for the fact that we might never have achieved the same insights through observation in “naturalistic” settings. In fact, these sorts of “artificial” research settings will be the only way we could conceivably attain certain kinds of insights.
Second, though, I don’t believe it is valid or useful to try to rank qualitative research settings and techniques according to how “natural” these settings and techniques are, if the idea lurking in the background is that more “natural” data is somehow intrinsically “truer”, e.g., because the observer interferes less with the construction of the data.
In this light, one student, for example, reported that their supervisor had advised that it was best to interview people in the people’s own homes, because this was a more natural environment and would therefore yield better research results. Another reported that their supervisor recommended against any kind of interviews, because observation in “naturalistic” settings would provide better insight into how people naturally behave.
One obvious question is how “natural” it is to have one’s behaviour observed by a social scientist, regardless of how comfortable you might otherwise be in a particular environment - the presence of the observer necessarily alters the behaviour of the observed. More fundamentally, though, even if you could remove the observer from the equation more completely - place cameras and recording devices in a setting, and observe interactions among people who were not aware they were being filmed - you will still not get to a “natural” interaction, in the sense of gaining some kind of direct access to a person’s “true” self, because people behave differently in different company, with none of those different behaviours being more “natural” than others.
I don’t, for example, behave more “naturally” at home than I do in the classroom - in both settings, I occupy social roles, and continuously coordinate my behaviour with the behaviour and reactions of the other people around me - and I continue to make these kinds of behavioural adjustments in every context in which I find myself. The role I play with a researcher studying my behaviour, however occasional and atypical of my everyday life this role may be, is still another “natural” role, even if the occasion for acting out this role is consciously orchestrated by the researcher, and I or the researcher feels awkward, uncomfortable, or uncertain about the proper boundaries of our relationship.
I don’t want to take this too far: I do understand the rational core of the critique of “artificial” research - the curiosity of the researcher to try to observe what people would do if the research were not taking place, the desire to try to observe a community from the “inside”, etc. These are valid and important things to be curious about, and I have no objections to trying to work out research designs that will accommodate these interests.
My objections begin when some type of effectively moral ranking is attached to different research methodologies or settings, as though information gained in a particular way is somehow intrinsically superior, rather than being - like all forms of research - a particular avenue into a particular question, where both the methodology and the question have their strengths and weaknesses.
