Which electricity and magnetism research-based assessment should I use in my class?

posted April 10, 2021
by Adrian Madsen, Sarah B. McKagan, Eleanor C. Sayre

This recommendation initially appeared as an article in the American Journal of Physics:  A. Madsen, S. B. McKagan and E. C. Sayre, Resource Letter RBAI-1: Research-Based Assessment Instruments in Physics and Astronomy, Am. J. Phys. 85, 4 (2017).

RBAIs on electricity and magnetism (E&M) for introductory courses have been around since the late 1990’s. There are six research-based assessments that cover electrostatics and magnetism.  Four of these are for introductory courses: the Brief Electricity and Magnetism Assessment (BEMA) (Ding et al. 2006, Pollock 2008), the Conceptual Survey of Electricity and Magnetism (CSEM) (Pollock 2008, Maloney et al. 2001), the Diagnostic Exam for Introductory, Undergraduate Electricity and Magnetism (DEEM) (Marx 1998), and the Electricity and Magnetism Conceptual Assessment (EMCA) (McColgan 2017). There is one assessment specifically about symmetry and Gauss’s law: the Symmetry and Gauss’s Law Conceptual Evaluation (SGCE) (Singh 2006).  There is one assessment which covers just magnetism concepts: the Magnetism Conceptual Survey (MCS) (Li and Singh 2011). There is also the Electromagnetics Concept Inventory (EMCI) suite of assessments which includes EMCI-Waves, EMCI-Fields, and EMCI-Waves and Fields (Notaros 2002), which were developed for engineering courses and won’t be discussed further here.

For circuits, there are three RBAIs: the Determining and Interpreting Resistive Electric Circuits Concepts Test (DIRECT) (Engelhardt and Beichner 2004), the Electric Circuits Conceptual Evaluation (ECCE) (Sokoloff 1996), and the Inventory of Basic Conceptions- DC Circuits (IBCDC) (Halloun 2005). The CSEM also contains some questions about circuits, but this is not its main focus.

More recently, RBAIs for upper-level courses have been developed. We discuss three: the Colorado Upper Division Electrostatics Diagnostic-Free Response (CUE-FR) (Chasteen et al. 2012), the Colorado Upper Division Electrostatics Diagnostics-Coupled Multiple Response (CUE-CMR) (Wilcox and Pollock 2013, Wilcox and Pollock 2014) and the Colorado Upper-Division Electrodynamics Test (CURrENT) (Baily et al. 2013).

Introductory E&M RBAIs are summarized in Table 1; upper-level ones are in Table 2.

Table 1. Introductory electricity and magnetism assessments.

Title Content Intended Population Research Validation Purpose 

Introductory electricity and magnetism

Brief Electricity and Magnetism Assessment (BEMA)

Circuits, electrostatics, magnetic fields and forces

Intro college Gold

To assess students’ qualitative understand- ing of basic concepts in electricity and magnetism.

Conceptual Survey of Electricity and Magnetism (CSEM)  Electrostatics, magnetic fields and forces, Faraday’s law Intro college Silver

To assess students’ knowledge about topics in introductory electricity and magnetism.

Diagnostic Exam for Introductory, Undergraduate Electricity and Magnetism (DEEM)

Electric and magnetic fields and forces, electrostatic potential, Maxwell’s equations, induced currents

Intro college Bronze

To assess students’ understanding of basic concepts of electricity and magnetism.

Electricity and Magnetism Conceptual Assessment (EMCA)

Electrostatics, electric fields and forces, circuits, magnetism, induction

Intro college Bronze

To assess basic concepts in an introductory electromagnetism course, using terms that will feel familiar to students on the pre-test and without overly difficult questions that might discourage students from pursuing physics.

Symmetry and Gauss’s Law Conceptual Evaluation (SGCE)

Symmetry, electric field, electric flux

Intro college, upper-level graduate

Bronze

To assess students’ ability to identify situa- tions where Gauss’s Law is applicable and use it to calculate electric field strength.

Magnetism Conceptual Survey (MCS)

Magnetic fields and forces, Faraday’s law

Intro college Silver

To assess difficulties students have with magnetism concepts.

Circuits
Determining and Interpreting Resistive Electric Circuits Concepts Test (DIRECT)

DC circuits

Intro college

Gold

To evaluate students’ understanding of Direct Current (DC) resistive electric cir- cuits concepts.

DC and AC circuits

Intro college

Bronze

To assess students’ understanding of simple circuit concepts.

DC circuits

Intro college

 

Silver

To assess basic conceptions of DC circuits.

 

Introductory electrostatics and magnetism assessments

The two most commonly used RBAIs for introductory electricity and magnetism courses are the Brief Electricity and Magnetism Assessment (BEMA) and the Conceptual Survey of Electricity and Magnetism (CSEM). Both are multiple-choice and can be given both a pre- and post-test to measure student learning, or given as a post-test only, since students don’t have much initial knowledge of these topics before instruction and average pre-test scores are usually similar at different institutions

Brief Electricity and Magnetism Assessment (BEMA) 

The Brief Electricity and Magnetism Assessment (BEMA) (Ding et al. 2006, Pollock 2008) covers the main topics discussed in both the traditional calculus-based E&M physics curriculum and the Matter and Interactions curriculum including basic electrostatics, circuits, magnetic fields and forces, and induction. BEMA questions are mostly conceptual, but there are few questions that require simple calculations. The BEMA questions were developed based on student difficulties with relevant concepts. 

Conceptual Survey of Electricity and Magnetism (CSEM)

The Conceptual Survey of Electricity and Magnetism (CSEM) (Pollock 2008, Maloney et al. 2001) is an assessment of students’ knowledge of electricity and magnetism. It aims to assess a range of topics across the standard introductory course content, but without assessing every single topic covered in an introductory course. It is a combination of a test of alternative conceptions and knowledge.  It also has a combination of questions about the phenomena of electricity and magnetism and questions about the formalism explaining the phenomena. The questions on the CSEM are based on questions from two earlier tests, the Conceptual Survey of Electricity (CSE) and the Conceptual Survey for Magnetism (CSM). The questions on the CSE and CSM were developed by a group of college physics professors. 

Comparing the BEMA and CSEM

The BEMA and CSEM both cover basic topics covered in introductory electricity and magnetism courses. They share six questions that are identical or nearly identical. The topics covered on the BEMA and CSEM vary somewhat. The CSEM does not cover circuits, whereas the BEMA does (7 out of 31 questions). CSEM questions have only 5 answer choices, while BEMA questions have up to 10 possible answers choices on some questions. Both have similarly strong research validation. CSEM and BEMA scores were compared for one group of students, and on average both pre- and post-test CSEM scores were higher than BEMA scores by 5-6%, a statistically significant difference, with a moderate effect size (Pollock 2008). But the absolute and normalized gains were similar for the BEMA and CSEM, so for this group of students, both instruments measure learning in a similar way.

Diagnostic Exam for Introductory Undergraduate Electricity and Magnetism (DEEM)

The Diagnostic Exam for Introductory, Undergraduate Electricity and Magnetism (DEEM) (Marx 1998) is a less widely used and validated assessment of electricity and magnetism. The multiple-choice pre/post conceptual questions on the DEEM measure students’ understanding of basic concepts of electricity and magnetism including electric and magnetic fields and force, electrostatic potential and potential energy, Maxwell’s equations, and induced currents. The questions align well with the topics commonly taught in an introductory E&M course. The multiple-choice questions on the DEEM were developed based on student interviews, expert input, instructional objectives, literature review and observations of students

Comparing the DEEM to the CSEM and BEMA 

The DEEM is much longer than the CSEM or BEMA (66 questions versus 31 and 32 questions, respectively), so it covers topics much more thoroughly. The DEEM also contains follow-up questions, where students should answer a subsequent question only if they chose a certain answer(s) to a previous question. The DEEM, like the CSEM, does not cover circuits. It also does not cover graphical representations of vector fields, or conductors and insulators. About half the questions on the DEEM ask about the direction of the electric field, magnetic field, velocity, electric potential, or force for different situations. 

Electricity and Magnetism Conceptual Assessment (EMCA)

The Electricity and Magnetism Conceptual Assessment (EMCA) (McColgan 2017) is a multiple-choice assessment of standard second-semester introductory physics concepts including electrostatics, electric fields, circuits, magnetism, and induction. The authors developed the EMCA so that it aligned well with the topics taught in their course and so that it produced similar pre-test scores as the FCI for their student population. The EMCA is easier than the BEMA or CSEM. The authors designed the test this way so that on the pre-test students know the answers to some questions and gain confidence in the course (as opposed to the BEMA and CSEM, which many faculty give only as a post-test because students often score near guessing on the pre-test because they are not familiar with the material), but the post-test can still be used to show mastery at the end of the course. 

Symmetry and Gauss’s Law Conceptual Evaluation (SGCE) 

There is only one assessment specifically about symmetry and Gauss’s law: the Symmetry and Gauss’s Law Conceptual Evaluation (SGCE) (Singh 2006) which is designed for students in introductory calculus-based physics, but can also be challenging to upper-level students. The SGCE assesses students’ ability to identify situations where Gauss’s law is applicable and use it to calculate electric field strength. The SGCE questions are multiple-choice, and primarily conceptual, asking students about when and how to use Gauss’s law, but not to explicitly calculate values. The BEMA has one question on Gauss’s law, and the CUE-CMR and CUE-FR also ask questions which use Gauss’s law and that are aimed at upper-level students. 

Magnetism Conceptual Survey (MCS) 

The Magnetism Conceptual Survey (MCS) (Li and Singh 2011) was developed to help instructors assess difficulties their students have with magnetism concepts in introductory algebra-based and calculus-based courses. It assesses standard topics in introductory courses up to Faraday’s law. The MCS only covers magnetism and not electrostatics, so it follows that it has more questions about magnetism than the BEMA, CSEM, DEEM, or EMCA. The BEMA, CSEM, and MCS all cover charges in magnetic fields and magnetic field from current carrying wires.  

Recommendations for Choosing an Introductory Electricity and Magnetism Test 

When teaching an introductory electricity and magnetism (E&M) course, use either the CSEM, BEMA, or DEEM. If you would like to assess your students’ understanding of circuits in addition to other standard E&M topics, use the BEMA or DEEM. The CSEM and BEMA are used more commonly, so if having comparison data is important to you, use one of these tests (For a list of articles with BEMA or CSEM comparison data, see the research tab on the BEMA or CSEM assessment pages on PhysPort). If you want to assess your students’ understanding of magnetism separately from other introductory E&M topics, use the MCS.  Use the SGCE if you are particularly interested in introductory physics students’ understanding of Gauss’s law, or if you are making a change to your teaching about Gauss’s law and want to understand if that change helped your students.

Circuits assessments

Overview of Circuits Assessments

There are three RBAIs of circuits, the Determining and Interpreting Resistive Electric Circuits Concepts Test (DIRECT) (Engelhardt et al. 2004), the Electric Circuits Conceptual Evaluation (ECCE) (Sokoloff 1996), and the Inventory of Basic Conceptions-DC Circuits (IBCDC) (Halloun 2005). All three are multiple-choice pre/post assessments for introductory college classes. The CSEM also contains some questions about circuits, but this is not its main focus.

Determining and Interpreting Resistive Electric Circuits Concepts Test (DIRECT)

The Determining and Interpreting Resistive Electric Circuits Concepts Test (DIRECT) (Engelhardt et al. 2004) was developed to evaluate students’ understanding of direct current (DC) resistive electric circuits concepts. Most of the questions on the DIRECT apply well to a wide variety of introductory courses, though a couple of questions were designed to assess concepts around microscopic aspects of circuits, in a way that closely aligns with the way these concepts are taught in the Electric and Magnetic Interactions curriculum (part of the Matter and Interactions curriculum). The questions on the DIRECT were developed based on instructional objectives, literature review and expert input.

Electric Circuits Conceptual Evaluation (ECCE)

The Electric Circuits Conceptual Evaluation (ECCE) (Sokoloff 1996) assesses students’ understanding of both direct and alternating current circuits. About 80% of the questions are about DC circuits and cover standard concepts around current, voltage, resistance, and brightness of bulbs in circuits containing resistors and capacitors. The remaining 20% of the questions are about AC circuits and ask students to match current versus time graphs to different circuit configurations. The multiple-choice questions on the ECCE were developed based on open-ended questions about circuits published in the literature as well as the author’s personal experience with teaching circuits topics.

Comparing the DIRECT and ECCE

Both the DIRECT and the ECCE ask similar conceptual questions about standard introductory circuits topics, but also ask a couple of questions about non-standard topics: microscopic aspects of current on the DIRECT and AC circuits on the ECCE. Some of the questions on the ECCE have up to 10 answer choices. Also, some of the questions on the ECCE have boxes for students to explain their reasoning. If you grade these short answers, the ECCE could take longer to grade, but many instructors just skip grading these. The DIRECT has a higher level of research validation than the ECCE. 

Inventory of Basic Conceptions-DC Circuits (IBCDC)

The Inventory of Basic Conceptions-DC Circuits (IBCDC) (Halloun 2005) is a multiple-choice conceptual assessment of DC circuits developed in the US and Lebanon. The questions on the IBCDC were developed based on a taxonomy of relevant topics decided by experts in physics. The content on the DIRECT and IBCDC is very similar, though the IBCDC only covers circuit concepts that would be taught in a standard introductory level course. The DIRECT has a stronger research base and more comparison data than the IBCDC. (For a list of articles with DIRECT comparison data, see the research tab on DIRECT assessment page on PhysPort.) 

Recommendations for Choosing a Circuits Assessment

Use the DIRECT if you want to assess standard introductory DC circuit concepts because it has a stronger research base and is more commonly used, thus providing you with more comparison data. Use the ECCE if you cover AC circuits and DC circuits. Use the IBCDC if the content matches what you teach in your course more closely. 

Upper-level electricity and magnetism assessments

There are three RBAIs for electricity and magnetism for upper-level courses. These are: the Colorado Upper Division Electrostatics Diagnostic-Free Response (CUE-FR) (Chasteen et al. 2012), the Colorado Upper Division Electrostatics Diagnostics-Coupled Multiple Response (CUE-CMR) (Wilcox and Pollock 2013, Wilcox and Pollock 2014) and the Colorado Upper-Division Electrodynamics Test (CURrENT) (Baily et al. 2013).

Table 2. Upper-level electricity and magnetism assessments.

Title Content Intended Population Research Validation Purpose 

Upper-level electrostatics and magnetism

Circuits, electrostatics, magnetic fields and forces

Intro college Gold

To assess students’ qualitative understand- ing of basic concepts in electricity and magnetism.

Colorado Upper Division Electrostatics Diagnostics- Coupled Multiple Response (CUE-CMR)  Electrostatics, magnetic fields and forces, Faraday’s law Intro college Silver

To assess students’ knowledge about topics in introductory electricity and magnetism.

Upper-level electrodynamics

Electric and magnetic fields and forces, electrostatic potential, Maxwell’s equations, induced currents

Intro college Bronze

To assess students’ understanding of basic concepts of electricity and magnetism.

Colorado Upper Division Electrostatics Diagnostic- Free Response (CUE-FR)

The Colorado Upper Division Electrostatics Diagnostic- Free Response (CUE-FR) (Chasteen et al. 2012), contains open-ended, primarily conceptual questions that assess students’ understanding of electrostatics topics (15 out of 17 questions) commonly covered in the first half of a standard upper-division electricity and magnetism course. It also contains two questions about magnetostatics. In addition to assessing E&M content, the CUE-FR assesses several key skills such as the ability to choose a problem-solving method and defend that choice, visualize a problem, connect math to physics, and describe the limiting behavior. The CUE-FR has open-ended questions where students show work and explain their reasoning. There is an optional 20-min pre-test consisting of the subset of questions that incoming juniors could be expected to know. If you want to check your students’ knowledge coming into the course, and learning gains throughout the course, give the pre-test. The questions on the CUE-FR were developed based on previously established learning goals, expert input, and commonly observed student difficulties.

Colorado Upper Division Electrostatics Diagnostics- Coupled Multiple Response (CUE-CMR)

The Colorado Upper Division Electrostatics Diagnostics- Coupled Multiple Response (CUE-CMR) (Wilcox and Pollock 2013, Wilcox and Pollock 2014) was developed to cover the same content as the CUE-FR, but is easier to grade. The questions on the CUE-FR and CUE-CMR are almost identical, but the answer format is different. The CUE-CMR is a coupled multiple-response assessment where students can choose multiple-responses to a given question and are awarded partial credit depending on the accuracy and consistency of their answer. Students are first asked to select the correct answer or easiest method to solve a problem, and then select a “reasoning element” that supports their initial answer. Students get full credit for selecting all the correct reasoning elements (and only the correct elements). Students can also receive partial credit. A rubric is used to grade the free-responses to the CUE-FR. Partial credit is also granted here. The CUE-CMR also has similar an optional 20-min pre-test consisting of the subset of questions that incoming juniors could be expected to know. The CUE-FR has 17 questions, while the CUE-CMR has 16 questions (it is missing question 15 from the CUE-FR). On average, students score similarly on the multiple-response version of the test as compared to the free-response version of the test. The CUE-CMR was developed based on the CUE-FR, so it has a slightly lower level of research validation (as research has not yet been conducted using it at other institutions or published by other researchers).

Colorado Upper-Division Electrodynamics Test (CURrENT)

There is one assessment of upper-level electrodynamics: the Colorado Upper-Division Electrodynamics Test (CURrENT) (Baily et al. 2013). There is also the Electromagnetics Concept Inventory (EMCI) which includes questions about electrodynamics, but was created for engineering courses, so it will nzot be discussed further here.

The CURrENT is designed to assess fundamental skills and understanding of core topics in the second semester of junior- level undergraduate electrodynamics covering topics in chapters 7–9 of Griffiths (3rd Edition). The CURrENT is free-response in order to assess the ability of upper-level students to generate and justify their own answers. The CURrENT has a conceptual focus, though some mathematical manipulations are required. The CURrENT pre-test contains three questions, while the post-test contains six questions, as students do not have a-priori familiarity with many of the topics before taking the course. The CURrENT is graded with a rubric. The CURrENT questions were developed based on previously established learning goals, expert input, and common student difficulties. Use the CURrENT to assess your students’ understanding in second semester of junior-level undergraduate electrodynamics.

Recommendations for choosing an upper-division electricity and magnetism test

If you are teaching an upper-division E&M and want an assessment that is easy to grade and compare to others, use the CUE-CMR. Use the CUE-FR if you want a more in- depth look at the details of your students’ reasoning.