Supporting physics teaching with research-based resources

The questions on the FORT came from a variety of other existing assessments including the Classroom Test of Scientific Reasoning, the online test bank for the American Association for the Advancement of Science and the published literature. The  created several versions of the assessment, gave them to students in introductory college courses and analyzed the results using item response theory (IRT), looking for questions with strong discrimination. They also tested several other versions of the FORT with students and used IRT to estimate the difficulty, discrimination, and guessing rate for each question. To determine the final version of the FORT, the developers looked at how individual questions influenced the reliability of the test and chose the best combination. During development, the FORT was given to over 9000 students in introductory science courses at a nine institutions. Student interviews were conducted and developers found students were properly understanding the questions and answer choices. The developers also performed a regression analysis to evaluate how well scores on the FORT predicted how well students learned natural selection and found it was a good predictor of student learning. The results of the FORT are published in one peer-reviewed publication.