Washington Statistical Society on Meetup   Washington Statistical Society on LinkedIn

Curb-stoning Part III

Date and Time:June 9th, 2015 12:30pm to 2:30pm
Chair/Moderator:Nancy Bates, Census Bureau
Location:Offices of Mathematica-MPR (near L Street, north of Union Station)
1101 First Street NE, 12th Floor
Washington DC 20002
Sponsor:Methodology Section

Presentations

dajani_marquette-revised.pdf

IMPORTANT: If you would like to attend this seminar in-person, please RSVP to Ranjana Kohli at rkohli@mathematica-mpr.com or (202) 552-6456 at least a day in advance of the June 9th seminar to be placed on the seminar attendance list. Please provide your name and affiliation.

This Washington Statistical Society conference on curb-stoning is the third in a series of events on the same subject. The first event was sponsored by the WSS Metholodogy Section last December, and the second event was recently produced by the New England Chapter of the American Association for Public Opinion Research. Curb-stoning is a colloquialism for the practice of statistical enumerators and their supervisors to fabricate respondent data. Their techniques may be clever and even sophisticated. The speakers of this event will discuss the motives for curb-stoning, programs to detect its occurrence, and methods to discourage its practice.

WebEx event address for attendees: https://mprwebex.mathematica-mpr.com/orion/meeting/meetingInfo?MeetingKey=999482355&siteurl=mprwebex
Meeting Number:999 482 355
For only audio with no video:(609) 945-6996 (access code: 999 482 355)
Note:Particular computer configurations might not be compatible with WebEx.

Schedule

Time Speaker Affiliation Email
12:30 Mike Fleming WSS charles.fleming@bhox.com
12:35 Nancy Bates Census Nancy.A.Bates@census.gov
12:40 Regina Faranda Office of Opinion Research FarandaRD@State.gov
1:05 M. Rita Thissen, RTI International rthissen@rti.org
1:30 Rodrick J. Marquette Census Bureau Rodrick.J.Marquette@census.gov
  Aref N. Dajani Census Bureau Aref.N.Dajani@census.gov
2:00 Fritz Scheuren - Discussant NORC scheuren@aol.com

Title: The Cheater Problem Revisited: Lessons from Six Decades of State Department Polling

Abstract: Nearly 70 years after Leo Crespi's paper "The Cheater Problem in Polling," the challenge of preventing, detecting, and - in those rare instances - mitigating the damage from falsification remains. In the U.S. Department of State's Office of Opinion Research, which owes its founding to Dr. Crespi and other survey pioneers, we face this issue in some of the 200 surveys we conduct in nearly 100 countries yearly. Our office has faced institutional challenges in confronting falsification, including the instinct to shy away from tarnishing all of our research in the eyes of a sometimes skeptical audience, doubly so when the stakes of representing international publics are high. We have also grappled with limited capacity, both in terms of time for uncovering possible fraud and in the technical ability to systematically do so. This presentation will underscore the need to weave the ethos of confronting falsification into the fabric of survey research, and not just in an international context. It will focus specifically on the quality control process that State's Office of Opinion Research has developed over several years. The process includes a procedural checklist that our researchers must follow before reporting data, changes to our requirements for collecting paradata and metadata, and systematic guidance on approaching field firms to deal with cases of possible fraud.
— Regina Faranda, Acting Director, Office of Opinion Research,U.S. Department of State

Title: Technical Aids for Deterring and Detecting Falsification of Survey Data

Abstract: Quality assurance teams encounter ever-changing challenges in confirming the authenticity of interviewer-collected survey data. At RTI, several precautionary measures during and after data collection aid the monitoring of data collectors and review of data after receipt. When feasible, computer audio-recorded interviewing (CARI) allows detailed oversight of in-person surveys and augments live-monitoring of telephone interviews. Collection of global positioning system (GPS) coordinates can confirm the location of interviews conducted by field staff using mobile devices. For some surveys, on-location image-capture helps to validate authenticity. After data receipt at RTI, datasets undergo systematic review for high levels of unit or item non-response, paradata outliers, duplicate records and unexpectedly high levels of CARI refusals. The combination of these techniques provides an adaptable, multi-sourced, evidence-based process for quality assurance and control.
— M. Rita Thissen,
Department Manager, Center for Technology Solutions, Research Computing Division, RTI International, Research Triangle Park, NC 27709

Title: Falsification Detection and Prevention at Census: New Initiatives

Abstract: Falsification detection and prevention is one of several survey lifecycle quality assurance techniques conducted at Census. Recommendations from the Office of the Inspector General of the Department of Commerce, corroborated by the House Committee on Oversight and Government Reform, are being implemented Bureau-wide with the full backing of the Census Director. This presentation highlights new initiatives planned or currently underway to detect and prevent curbstoning at Census.

These operational initiatives include a centralized reinterview program, a redesign of the reinterview sample, implementation of Computer Audio Recorded Interviewing (CARI) pioneered by the Research Triangle Institute (RTI), and extensive use of paradata and administrative data. These curbstoning detection and prevention initiatives together will promote accurate, objective decisions made quickly and cost-effectively to ensure that our data and our employees adhere to our quality standards.
— Aref N. Dajani and Rodrick J. Marquette, U.S. Census