By Dana Milstein on April 23, 2018
Faculty across the disciplines persistently express concerns about their students’ skills in research collection, data management, and analysis of materials–particularly in fields where transparency of method and reproducibility of results are paramount. On March 12, 2018, the UCLA Data Archive offered a full-day workshop in conjunction with Project TIER on “Teaching Methods of Empirical Research.” Attendees comprised Social Sciences and Humanities faculty and staff, who participated in an active workshop in R Markdown and learned about the TIER Protocol for documenting the research process to make it transparent and reproducible by others. The workshop also included a tour of the OSF (Open Science Framework) as a platform for archiving projects, a presentation on HARKing and p-hacking, and a discussion about the challenges students face when they reconfigure their results to reach seemingly innovative (yet false) conclusions.
Professor Richard Ball and Associate Librarian Norm Medeiros of Haverford College developed the Project TIER (Teaching Integrity in Empirical Research) protocol to evolve professional norms of transparency and reproducibility of empirical research in the Social Sciences. Their goal is founded on the belief that replication documentation should be ubiquitous and accessible, and that mastering quantitative research begins in undergraduate education. The TIER Protocol was motivated by Professor Ball’s concerns about his inability to offer meaningful feedback on student projects due to: (a) absence of a documented, step-by-step research workflow taken by the students; (b) student inferences and analyses that seemed biased and unreproducible due to poor reading of their own data; and (c) a habit of students post-documenting their workflow as a discrete process at the project’s end rather than constructing it from the project’s start.
When following the TIER Protocol, a set of folders will include original data and metadata, a final paper with data appendix and Readme file, an analysis of data, and a folder for any code written to execute in statistical analysis software. The protocol can be adapted to any discipline where research needs to be collected and shared, and results reproduced or parsed for error during the collaboration and peer review processes.
For additional information, please contact Tim Dennis, the UCLA Director of the Data Archive (tdennis@library.ucla.edu) and see the Project TIER and OSF links below. Project TIER is supported by the Alfred P. Sloan Foundation.
Main photo (entitled binary-globe-null-one-earth-1317580) used courtesy of Pixabay user geralt – used under the CCO Creative Commons license. (image source) Image cropped for content.
Resources: