Which laboratory skills research-based assessment should I use in my class?
This recommendation initially appeared as an article in the American Journal of Physics: A. Madsen, S. B. McKagan, E. C. Sayre, and C. A. Paul, Resource Letter RBAI-2: Research-based assessment instruments: Beyond physics topics. Am. J. Phys, 87, 350 (2019).
Faculty often assume that during the laboratory portion of a physics course, students develop the ability to gather and evaluate data through experiments. Several assessments of different aspects of lab skills have been created to help instructors evaluate their students’ laboratory skills and critical thinking ability at the beginning and end of the course. There are four assessments of lab skills, the Physics Lab Inventory of Critical Thinking (PLIC) (Holmes and Wieman 2016), the Concise Data Processing Assessment (CDPA) (Day and Bonn 2011), the Physics Measurement Questionnaire (PMQ) (Lubben et al. 2001, Allie et al. 2007), and the Measurement Uncertainty Quiz (MUQ) (Deardorff 2001). There is also an assessment to gauge students’ attitudes about experimental physics (E-CLASS), which is discussed in this expert recommendation. The Data Handling Diagnostic (DHD) (Bates and Galloway 2010) is another assessment of laboratory skills, which will not be discussed further here because the authors did not finish the development and validation of this assessment and advise others to use the CDPA instead of the DHD.
| Title | Focus | Intended Population | Format | Research Validation | Purpose |
| Physics Lab Inventory of Critical Thinking (PLIC) |
Evaluating experimental methods, generating and evaluating conclusions based on data, comparing measurements with uncertainty, evaluating data fitted to a model |
Upper-level, intermediate, intro college |
Agree/disagree, multiple-choice, multiple-response, |
Silver |
To assess how students critically evaluate experimental methods, data, and models. |
| Concise Data Processing Assessment (CDPA) |
Measurement and uncertainty, relationships between functions, numbers, and graphs |
Graduate, upper-level, intermediate, intro college |
Multiple-choice |
Silver |
To probe student abilities related to the nature of measurement and uncertainty and to handling data. |
| Physics Measurement Questionnaire (PMQ) |
Measurement and uncertainty |
Intro college |
Open-ended |
Silver |
To probe students understanding of measurement and uncertainty using open-ended sample discussions. |
| Measurement Uncertainty Quiz (MUQ) |
Measurement and uncertainty |
Intro college |
Discussion of multiple-choice |
Bronze |
To discuss measurement and uncertainty concepts with students, and why one answer might be better than the others. |
Physics Lab Inventory of Critical Thinking (PLIC)
The Physics Lab Inventory of Critical Thinking (PLIC) (Holmes and Wieman 2016) assesses the way students critically evaluate experimental methods, data, and models and is the newest laboratory skills assessment. The PLIC includes an introduction that describes an experiment using masses and spring and sample laboratory notebook entries for two groups of physicists. The PLIC uses “choose many” multiple-choice questions and Likert-scale questions to assess students’ critical thinking around the laboratory notebook entries for this experiment. Students’ responses are compared to the “consensus expert response” and “consensus appropriate response” for each question. Because many of the multiple-choice questions allow students to “choose many,” the score for each question is between 0 and 4 points, depending on how many “consensus expert responses”, “consensus appropriate responses” and “novice responses” are given. The PLIC has been used in all levels of undergraduate laboratories. The PLIC is still under development. The questions on the PLIC were based on the series of questions an expert posed to himself or herself when conducting an introductory physics experiment.
Concise Data Processing Assessment (CDPA)
The Concise Data Processing Assessment (CDPA) (Day and Bonn 2011) is a 10 question multiple-choice pre-post assessment that measures students’ understanding of handling data with questions around uncertainty in measurements and the relationships between functions, graphs, and numbers. The CDPA is appropriate to use in any laboratory course with learning goals around data handling. The questions were based on established learning goals for an introductory laboratory course and iteratively refined using student interviews, expert review, and statistical analyses.
Both the PLIC and the CDPA assess students’ data analysis skills, but the PLIC also assesses other skills including how students critically evaluate experimental methods, data, and models. The CDPA has 10 multiple-choice questions, where each has its own context, whereas the PLIC has one rich experimental context outlined at the beginning of the assessment, to which all 16 questions refer. Both the PLIC and CDPA have strong research validation.
Physics Measurement Questionnaire (PMQ)
The Physics Measurement Questionnaire (PMQ) (Lubben et al. 2001, Allie et al. 2007) is an open-ended pre/post assessment of students’ understanding of experimental measurements, including data collection, data processing, and dataset comparison. There is an experimental situation described at the beginning of the assessment, and all the questions refer to this same experimental situation (similar to the PLIC). The questions ask students to reflect on how many measurements they should take, how to report the results of multiple measurements, how to compare sets of measurements, and how to fit a line to experimental data. Because the PMQ questions are open-ended, the answers and explanations are coded according to an established coding scheme, which can be time consuming. In each question or “probe,” there is a short conversation between several people, and students are asked to choose which they most agree with and then give a written explanation for their choice. The discussions in each probe are written with concise, simple language in order to be understandable for a wide range of English language levels. The developers use the PMQ results to look at their students’ paradigms of measurement as either “point” or “set.” A point paradigm would see each measurement as the possible true value, where differences between measurements are a result of environmental factors or experimenter mistakes. In the “set” paradigm, each measurement is an approximation of the true value, and deviations are random and always present. A set of measurements yields the best approximation of the true value, with an associated uncertainty. The questions on the PMQ were based on similar questions from the Procedural and Conceptual Knowledge in Science (PACKS) Project (Millar et al. 1994).
The PMQ has a unique format compared to the other laboratory skill assessments, where each question includes a conversation between students, with an open-ended question about the conversation. Furthermore, the scoring of the PMQ is different from the other assessments discussed, and instructors code the responses to understand their students’ results. The content and skills assessed on the PMQ are also included in the PLIC, though the PLIC goes into more depth in asking students to evaluate critically experimental methods, data, and models.
Measurement Uncertainty Quiz (MUQ)
The Measurement Uncertainty Quiz (MUQ) (Deardorff 2001) is a non-standard assessment that can be used as the basis of a discussion about precision, significant figures, accuracy, and error propagation with your introductory physics students. The developer explains that it is difficult to create a right/wrong test around the topics of measurement and uncertainty, because even experts may disagree on the correct answer. Because of this limitation, the MUQ questions are an opportunity to discuss with your students why one answer may be better than others. Because the MUQ is for discussion (and is not scored), it is not given as a pre/post-test. The 10 questions on the MUQ are a sample of the open-ended questions given to approximately 100 introductory physics students and 30 experts (graduate physics students and teachers). The most common responses were edited and turned into the multiple-choice options.
The MUQ focuses just on measurement uncertainty, whereas the CDPA also asks about fitting data and relating functions, graphs, and numbers. Both tests use the same question format and have the same number of questions, but the MUQ developers recommend using it to have a discus- sion with students, instead of using it as a pre/post-test and scoring it, as you would with the CDPA.
Recommendations for choosing laboratory skill assessment
Use the PLIC to get a rich understanding of your students’ skills around critically evaluating experimental methods, data, and models. The PLIC assesses the content covered on the MUQ, CDPA, and PMQ and additional content and skills related to critical thinking around experimentation. If you want a short, simple multiple-choice test of measurement uncertainty and relationships between functions, data, and graphs, use the CDPA. If you are interested in understanding your students’ open-ended responses about data collection, processing, and comparison or in looking at your students’ paradigms of the measurement as either “point” or “set,” use the PMQ. If you want to have a rich conversation about measurement uncertainty with your students, use the MUQ as the basis of the conversation.
Add a Comment
