All About Research Methodologies -- PIH 217D -- May 3, 1996
J. Chittick - School of Public Health
The magnitude of the task makes it essential for people at all levels of the
health and social system to have a common understanding of the approach to be
taken. Good intentions are not enough....information campaigns have potential
for harm as well as good. (Jonathan Mann)
A. How can the correct research methodologies improve the quality of
research studies?
1. Good research is a SCIENCE -- a major limitation to the usefulness
of behavioral measures is poor measurement quality.
2. A reliable way to gain a better understanding of what people really
think, believe, understand, and fear.
3. Verifiable research methods can uncover complex behavioral issues and
result in more effective interventions. Find trends -- answers puzzles
-- enlightens thinking -- benefits research data.
4. Research findings are built on top of previous research findings --
so each new set of data must be built on solid research methodologies.
II. How to organize your research study.
A. What do you need to know?
1. What prior research studies are in the literature? What methods
did they use?
2. Will it be original study? What do you
hope to discover?
3. Will your research be of value to other researchers as well as being
worthy of your own work?
4. Are you following-up on a past study to verify previous findings or
to measure temporal differences? What do you hope to confirm or refute?
5. Will your research benefit programs of HIV/AIDS
interventions?
B. State your hypothesis (your rationale for study): i.e., there are many reasons
why teenagers are vulnerable to HIV transmission. There are many reasons
why teens feel they are invulnerable to AIDS. By better understanding
the reasons for their risky behaviors, educators and public health officials
can design more effective behavioral interventions.
1. What is your objective? What are the research questions?
2. Who are the targeted sample populations?
3. What are your anticipated findings and their implications?
C. Making a realistic plan.
1. How much data do you need to collect?
2. What problems do you foresee?
Are prior approvals needed?
3. Can you reasonably expect to achieve
your research goals?
4. Are there special circumstances (legally, socially, politically, culturally)
that may affect your data gathering and subsequent analysis?
III. Choosing the correct research methodologies to realize your study's
objectives.
A. The most used methods in HIV/AIDS research are:
1. Quantitative (questionnaire surveys).
2. Qualitative (interviews, focus groups,
and case studies).
3. Combinations of quantitative and qualitative
methodologies.
B. Why Quantitative (surveys and questionnaires)?
Without a doubt our knowledge concerning the epidemic and our ability to plan
scientifically for strategies to prevent further spread of HIV-1 have benefited
enormously from quantitative estimates of knowledge, attitudes and behavior.
(Steven Gortmaker and Jose Antonio Izazola)
1. Types of Quantitative Methodologies:
a. Surveys -- on-street, marketing,
or telephone polling (oral).
b. Questionnaires
-- structured format (written).
1)
Mostly anonymous -- mailed and mail-in.
2) Quasi-anonymous -- passed out and collected.
3) Coded -- in class; observed and supervised.
4) Standard KABP (knowledge, attitude, belief, and practice).
2. Will you need additional data to supplement anticipated findings?
3. The advantages of quantitative (surveys and questionnaires)
methods:
a) A larger population can be tested for knowledge and attitudes than
with interviews.
b) A large sampling can discern trends and shifts in thinking and attitudes
that might not be apparent in smaller samplings.
c) Quantitative measures can provide data for future projections (and
these projections can play an important role in planning education campaigns).
d) A certain amount of "privacy"
is guaranteed with
anonymous surveys, especially when dealing with highly
personal, sexual issues (face-to-face interviews are more
intrusive).
e) Changes over time in populations
can be measured.
f) Different populations and subsets
can be measured.
g) Quantitative measures are easier to code if "yes," "no," and "not sure;"
answers can be compiled and analyzed using mathematical precision (e.g., statistically
significant tests).
h) Quantitative measurement can be easily replicated for validity; a key to
validation of scientific inference.
4. The disadvantages of quantitative (surveys and questionnaires) research:
a. Self-reported
measures can be self-serving.
b. Unreliability of information can be caused by a variety of factors
including:
1)
Limitations of short-term memory
2) Subjects afraid to report behavior perceived as deviant
3) Difficulty of characterizing complex behaviors in terms of simple responses.
c. Some research shows respondents may falsify answers to prevent supervisory
attention and avoid official disapproval.
C. Why Qualitative?
Qualitative research will be defined as the use of in-depth interviewing and
observational techniques on small samples of target groups to investigate the
attitudes, beliefs and social contexts associated with human behavior.
Qualitative approaches have evolved from several disparate fields of inquiry.
Anthropology has contributed techniques such as in-depth interviews, participant
observation and ethnographics, which were traditionally used to understand communities
and the context of social interaction. (William Smith and Mary Debus)
1. Why Interviews (Structured vs. Unstructured)?
a. Structured interviews can be coded and analyzed. Best qualitative
method for analytical cohesiveness. But sometimes structure limits scope
-- researcher is often not aware of original (or alternate) thinking
of subject.
b. Unstructured or open-ended answers are more difficult to code -- because
data is not uniform, it may be more difficult to compare (and time-consuming)
to analyze. But maybe more information.
c. Best for class: Interviews that are structured but with open-ended
probes: e.g., "one on one." Example: news programs on Sunday morning;
panelists have lists of prepared questions, but follow-up with new questions
if new avenues are opened for discussion or clarification.
1)
Additional interviewees can "snowball."
2) Establish friendly but formal communication
(comfort zone).
3) Important to decipher nuances -- helps explain survey results and may
lead to new information or future trends.
2. The advantages of qualitative (interview) research:
a. Qualitative research is inductive.
b. It imposes fewer concrete expectations on the study situation than
survey research and therefore increases the chance of finding unanticipated
outcomes.
c. Qualitative research is context-based. It uses observations and
in-depth interviews under conditions that are as natural as possible.
d. Qualitative analysis is narrative rather than numerical.
e. Qualitative methodologies give the researcher more leeway to follow
up on new avenues of information or clarify ambiguities. Also allows the
subject to express views in own words, not words of researcher.
f. Qualitative research can be used before a survey to develop hypotheses,
test concepts and identify precise vocabulary. It can be used after a
survey to illuminate seemingly contradictory results.
g. Qualitative research in AIDS prevention deepens understanding of the beliefs
and practices previously assigned to Knowledge and Practice (KCAP) surveys.
3. The disadvantages of qualitative (interview) research:
a. Researchers must take care when attempting to generalize conclusions
from qualitative data; generalizations may be skewed or false.
b. Interpretations of qualitative research data are subject to important
limitations such as researcher bias. (Where possible, using transcriptions
or videos will allow individual interpretation to be subject to the analysis
of other researchers).
c. Small
sample may increase interviewee bias or prejudices.
D. Why Focus groups?
1. The advantages of Focus Groups:
a. Especially useful in evaluating AIDS programs as it encourages group
participation (some people find it easier to open up as part of a group).
b. Focus
groups are informal yet structured at the same time.
c. Focus group data can provide relatively fast answers to questions,
thus proving useful when designing research projects, generating hypotheses
about a problem, or clarifying ambiguities from quantitative data.
d. Group discussions often make it easier for people to share experiences
thus helping the researcher make comparisons between participants.
e. Focus
groups can be similar to role-playing, in that people are made more
aware of other's opinions and feelings.
f. Groups can be less costly and time-consuming for researchers.
2. The disadvantages of Focus Groups:
a.
Some individuals can be distracted or intimidated by more
powerful
personalities and/or opinions, thus skewing the
observations elicited.
b. People may be more reluctant to discuss personal sexual practices in
front of strangers, as compared to one on one interviews.
c. An inexperienced
leader can elicit data ambiguities.
d. Focus group data can never be generalized to a larger population.
e. While focus group discussions can illustrate a range of thinking among
some people in a given population, this data cannot indicate the extent of the
thinking in the general population.
f. Problems can arise if participants are good friends (best if they are
not close confidants although this is not always possible within a school/class
setting).
g. Focus groups can be misused by researchers in a hurry to cut corners
with more time-consuming methodologies.
E. Why Case Studies?
Ethnography is a way of making a detailed description and analysis of human
behavior based on a long-term observational study on the spot....Another major
feature of ethnography is the goal of truth. On some philosophical level
it may be argued that reality cannot be truthfully representened [and] there
are some accepted conventional distortions of reality which occur in the translation
of the living act. (Karl G. Heider)
1. Case studies are often community ethnography. Three kinds are:
a. Strict observation; "fly on wall."
b.
Cultural reports with personal observation (Margaret Mead).
c. Culture
and personality research (Robert Levine).
2. Case studies may minimize cultural bias by explaining context of human
interaction and to best understand situation/locale.
3. The advantages of Case Studies:
a. Pure observations do not interfere with subjects' thinking or encourage
prepared or anticipated answers.
b. Observation can set mood, context, location; gives factual findings
local color; anecdotal information is part of findings and analysis.
c. Observation by outside researcher minimizes self-reporting biases of subjects.
4. The disadvantages of Case Studies:
a. It is difficult for a human researcher to be completely detached while
observing human interaction. There is always the opportunity for subtle
bias and observer slant. To some degree, video and tape records can assist
in measuring amount of researcher's personal opinions.
b. Case studies can be too dry, missing the nuances of personal interactions.
c.
There is very little interaction on specific questions asked of the
sample population.
F. Why Combination Methodologies (e.g., survey and interviews or interviews
and focus groups).
Qualitative and quantitative behavioral research are more than complementary.
The first allows us to gain deeper understanding of human behavior and its determinants,
while the latter permits us to assess that behavior and its consequences. To
be effective....AIDS programs should combine both quantitative and qualitative
research methods. (Jaime Sepulveda, Harvey Fineberg, and Jonathan Mann)
1. The advantages of Combination Methodologies:
a. The best of all available methods can be utilized
jointly.
b. More data can be double-checked
and verified.
c. Data may be more easily analyzed when comparing information gathered
by different methods.
d. Variety of methods can act as checks and balances.
2. The disadvantages of Combination Methodologies:
a. If different methods used are not narrowly focused on original hypotheses
and research questions, accumulated data may be contradictory and difficult
to analyze.
b. Using different methodologies can be time-consuming and more expensive
for the researcher.
c. Combination research may be more adaptable to proving original hypothesis,
while not being as conducive to verifying earlier suppositions and previous
research findings.
IV. Class mission: use methodologies to develop accurate research findings.
Specifically, discover more insight into vulnerability issues faced by students,
youth, teens so that you and others can understand, design, and implement better
HIV/AIDS intervention programs. The AIDS pandemic can be curbed by more
effective and intuitive thinking.
A. Ask important preliminary questions to see if you are on right track.
1. Are we
reconfirming or refuting earlier findings?
2. Are we trying to find new answers or clarify earlier ambiguities?
3. Are we striking out in uncharted territories? Is this original research?
B. Choose methodologies best suited to your research questions and hypothesis.
C. Be exact in your language. Leave no room for ambiguity.
Test your questions prior to meeting with subjects (i.e., on friends).
Remember: when we try to measure personal sexual behaviors and condom use, minor
variations in the wording of questions can skew the findings and hinder a correct
analysis of the data collected.
D. Be non-committal and without bias. Good research methods are
fair and impartial, not slanted or advocacy-oriented. We are not doing
political polling with directed answers. It is a SCIENCE.
1. Poor example:
"Isn't it true that you are worried about AIDS?"
a. Instead: "Do you consider yourself to be vulnerable to HIV/AIDS?
If yes, why do you think so? Do you think your fellow students are concerned
about getting AIDS? What reasons can you give? If no, why not?"
b. Start with general questions and become more specific. Keep the
conversation targeted to your original mission.
c. Prepare probing questions if the discussion indicates a willingness
to move faster than anticipated.
E. When writing analysis, state your original hypothesis, research methods
and the procedures used. Add anything important to say about your procedures
-- were there any problems that affected your data analysis (i.e., teacher intruded
in discussion; so students felt intimidated)?
1. Even studies that do not confirm researcher's original hypotheses,
or else refute earlier research data, are important and of value for what they
disprove or question.
F. Organize your data cohesively and coherently. Better to answer
a few questions well than mess with too much information that is garbled.
G. Suggest follow-up research to improve on your own efforts. Your
ideas and recommendations can be used by next year's class to expand on
these findings.
VI. Conclusion.
We must admit our own ignorance and limitations about the best way to combat
AIDS. It is now a commonplace to say that in the absences or drugs or
vaccines our only alternative is education. It is still valid and true,
yet easier said than done. A universally valid or accepted educational
model does notexist. (Jaime Sepulveda, Harvey Fineberg, and Jonathan Mann)
You can assist in curbing the AIDS pandemic by using the best research
methodologies at your disposal to better understand why people engage in high
risk behaviors, despite already being "educated" about the medical facts of
HIV/AIDS. Each population (and its subsets) that you find yourself dealing
with in the future needs to be specifically targeted with convincing arguments
in order to alter highly personal and pleasurable behaviors; generic aphorisms
such as "just say no!" do not work consistently or over the long term.
By understanding the variety and power of different research methodologies,
you and your professional colleagues can reasonably rely on the most accurate
and insightful data available -- information that you know how to verify, to
make the best decisions regarding AIDS prevention.
© 2002 JChittick