Designing a mixed-methods research instrument and scoring rubric to investigate individuals’ conceptions of plate tectonics

Resource category: 
Articles and Special Issues
Clark, S. K., & Libarkin, J. C. (2011). Designing a mixed-methods research instrument and scoring rubric to investigate individuals’ conceptions of plate tec- tonics. In Feig, A. D., & Stokes, A., (Eds.), Qualitative Inquiry in Geoscience Education Research. Geological Society of America Special Paper 474, 81–96. doi:10.1130/2011.2474(07)
Resource URL: 

Research methods and underlying theories for research designs that integrate quantitative and qualitative approaches (i.e., mixed methods) are well documented in the field of education research. What is missing in the literature is a nuts-and- bolts description of the actual practice that goes into creating a good mixed-methods survey instrument for research in the science education domain. This paper will detail the steps involved in designing, implementing, and scoring a valid and reliable mixed-methods survey instrument. This survey instrument was designed to inves- tigate experts’ and novices’ conceptual understanding of plate tectonics as inferred by their answers to a series of questions related to a modified version of a commonly used cross-section schematic published by the U.S. Geological Survey. Development of the instrument involved numerous revisions with iterative inputs from local and community-based experts. After integration of expert comments, the survey instru- ment was piloted to a physical science for nonscience majors course. This led to fur- ther revisions in the survey instrument to improve communication validity prior to widespread distribution. Development of scoring rubrics similarly required iterative modifications based on a thematic analysis of collected data. By outlining the steps involved in designing, validating, and analyzing this mixed-methods instrument, we believe that this paper can serve as a template for future survey instrument develop- ment. In particular, we hope to illustrate the iterative and time-intensive nature of mixed-methods inquiry, both in terms of pre-investigation design and postinvesti- gation analysis, and to offer our empirically based insights into the instrument and rubric development process.


This page last updated 19 May 2019 - 4:22pm