Data quality




The Social Networks Project has paid a great deal of attention to data quality. Several steps were taken during fieldwork to maximize the quality of the data collected by all three projects. In addition, post facto assessments of the validity and reliability of the data collected have been carried. In particular:

Fieldwork. At least one of the PIs has been present at every stage of fieldwork, unlike many other field projects. The presence of a PI has three advantages: 1) It has been possible to respond authoritatively and immediately to the inevitable problems that arise in the field; 2) Team morale is maintained; 3) Interactions with interviewers and supervisors aided in identifying potential issues of data quality and interpreting survey responses. We provide a myriad of examples in a document summarizing Watkins’ field notes for the first round of the Kenya Diffusion and Ideational Change Project (Watkins et al. 1995) and in the field report of the pre-test of the Family Transfers Project (Weinreb 1998).

Questionnaires. The same basic survey questionnaire has been used in all Social Networks surveys. The questionnaire was less developed in the first survey wave in Kenya but, since each round has served as a pretest for the subsequent rounds, successive refinements of the questionnaires have meant that there has been decreasing ambiguity in the questions.

Questionnaires' checking and data entry. To minimize the effects of interviewer error on the data, questionnaires were checked and data entered in each field site. Checking was done first by the supervisors and then on the next day by a PI and graduate student members of the field team. This permitted detecting interviewer errors while the team was still in the survey site, so that interviewers could return to respondents if there were missing data or other apparent errors. Questionnaires are stored at the University of Pennsylvania, which permits further checking in the course of new analyses. .

Quality assessments. The papers on the survey methodology of the Social Networks Project are numerous. Among them, there are various assessment of the quality of the data collected by the project's surveys:

  • The literature on interviewing in developing countries suggests that it is desirable to use same-gender interviewers who are strangers to the community. In some circumstances this may not be practical. Weinreb (2000) evaluated responses in cases where the interviewer knew the family of the respondent and cases where he/she did not, by gender.
  • Miller, Zulu and Watkins (2000) analyzed husband' and wives' responses on questions about the possession of household goods (e.g. a pit latrine and a mattress), and questions about current use of family planning and AIDS in th MDICP. They found that discrepant responses were systematic with respect to gender (i.e. when husbands and wives disagree, it is typically the husband who says "yes" and the wife who says "no") and with respect to region. Miller et al. found the same systematic patterns of husband-wife discrepancies in the Kenya Demographic and Health Survey and the Malawi Demographic and Health Survey, suggesting that these patterns are not particular to our surveys.
  • Bignami-Van Assche (2003) takes advantage of the panel nature of the study as well as of a set of re-interviews conducted in the MDICP-2, to examine the consistency of responses for respondents interviewed by the two waves of the Malawian survey.
  • Schatz (2003) compares the measurement of women's status and autonomy in the Malawi household survey with qualitative interviews she conducted with a sub-sample of the household sample in Malawi

A detailed summary of the literature on data quality produced in the context of the Social Networks Project, and a reasoned discussion of the more general problem of data quality assessment in demographic research in developing countries, can be found in the Introduction to the monographic volume on "Research on Demographic Aspects of HIV/AIDS in rural Malawi" (Watkins et al. 2003).

 

Quick links:

Kenya
Malawi
Transfers
Survey data
Qualitative Projects
People
Papers

last updated December 18, 2004

Please direct comments or questions about this site to pfleming@sas.upenn.edu