Expert Recommendations »Recommendation

Security FAQ for the PhysPort Data Explorer

posted January 22, 2016 and revised July 30, 2023
by Eleanor Sayre, Sarah McKagan, and Adrian Madsen

PhysPort’s Data Explorer is free, web-based software that transforms students’ multiple choice responses to research-based assessments into powerful data that instructors can use to gain insights into their students' learning and make decisions about their teaching. With just a few clicks, instructors can upload almost any format of CSV file with their students' responses. The Data Explorer matches, scores, analyzes, and interprets the results from research-based assessments. Instructors can save their data in their account, and explore it later, comparing their courses over time, tracking their students’ gains, and evaluating their students’ performance on specific content areas. A common question about the Data Explorer is: “This all sounds great, but can I really upload student data to your site? Is it safe and secure? Do I have to have approval?” We’ve compiled detailed responses to these Frequently Asked Questions and more about the Data Explorer’s safety and security.


For Faculty and Instructors:

For Physics Education Researchers:

Am I allowed to do this? Am I going to get in trouble?

The Protection of Human Subjects Committee at the American Center for Physics has reviewed this project and determined that this project is exempt from further human subjects review. Under the provisions of our approval, you are permitted to upload your data, view analyses, and download reports. We keep your data confidential, perform analyses, and share anonymized comparisons between users with you and with the research community.

Back to Top

What is a human subjects protection board?

Whenever someone does research on human subjects, they need to make sure that the benefits of the research outweigh the potential risks (beneficence), that research subjects know what's being asked of them and freely consent to it (respect for persons), and that some populations are not unduly stressed or badgered into participation just because it's easier for investigators (justice).

Human subjects protection boards (often called "Internal Review Boards" or "IRBs") ensure that these protections are in place for all research studies before the studies can start. The Protection of Human Subjects Committee at the American Center for Physics has reviewed this project.

Back to Top

Who can use the Data Explorer?

Our Protection of Human Subjects Committee has approved use of the Data Explorer by "faculty who teach physics classes in high schools or institutions of higher learning, or have a vested interest in assessment of physics classes (e.g. department chairs and course coordinators)." The Data Explorer has been designed for and tested with US college physics faculty. It may also be useful for international physics faculty and high school physics teachers, but it is not designed for, and has not been tested with, these users. The Data Explorer should not be used by students*, parents, tutors, home school teachers, middle school teachers, or elementary school teachers.

*Instructional staff who are also students (e.g. graduate TAs) are covered under our human subjects approval for faculty who teach physics classes.

Back to Top

Are there any risks for my students?

The risks to your students are minimal. We can't promise "zero" risk -- no one can promise zero risk -- but we promise that your students' identities are secure and no one can use our system track their performance back to them.

Actually, we promise something stronger than that. When we calculate "students like yours", we make sure that we combine data across a lot of instructors and a lot of schools. We only report aggregate information without any personally-identifying markers. No one can guess which schools, instructors, or students are aggregated together to generate "students like yours", so no one knows how your data compare except you and the people in your department that you've specifically shared it with (like your department head or course coordinator).

Back to Top

Are there benefits?

There can't be any direct benefits to your students because they are not interacting with our system. However, there can be indirect benefits: by educating yourself about how your students are doing, you arm yourself with data about how to teach better. If you use our recommendations and your data to teach better, your students will benefit.

Back to Top

Do you store personally identifiable information about my students?

We ask you to upload your students' IDs and/or names so that we can match your students' pre- and post-tests, but we don't keep this information in an identifiable form. As soon as you tell us which columns of your file contain student IDs and/or names, we encrypt them using one-way, cryptographically-secure transformations, the same way that we protect your password. This process relies on the difficulty of factoring large numbers to create identifiers that are unique for each student, but that can't be converted back to get the actual IDs or names you uploaded. If you upload a file with the same name or ID later, it will create the same unique identifier so we can match that student's data, but even the staff at PhysPort can't reverse the process.

Back to Top

Do I need permission from my institution to upload data?

No. Our human subjects protection committee says that you don't need to ask your institution if it's ok to upload your data or use our system. If you're worried about this, please contact us.

Back to Top

I'm a course coordinator or department chair, and I want to upload data for my faculty. Is that ok?

Yes, you can upload your department's data. You can ask the faculty first, but it's not required.

However, the other faculty in your department won't be able to see the analysis because only you, the uploader, can see the results of the data you upload. You can share it with them yourself, but we won't.

Back to Top

Do I need consent from my students to upload data?

No. Generally, your data is collected for the purposes of evaluating your teaching or the teaching in your department. Our human subjects protection committee says that means you are the research subject, not your students. We need your consent to use your data, but we don't need your students' consent. Because we don't need it, you don't need it either.

You can ask your students if you want, but it's not required.

Some people collect data specifically for research purposes, not to evaluate their teaching or to evaluate their department. This is pretty uncommon. To do their research, those people need to have approval from their human subjects protection board. If you are one of those researchers, the terms of their approval govern whether you can upload data to our system. Please contact us if you need help determining that. If you're not one of those people (and you would know it if you were), then you don't need to worry about this part.

Back to Top

What about students under 18?

We don't interact with students and they aren't our research subjects, so it doesn't matter how old they are. For the data explorer, our research subjects are faculty who are interested in evaluating their teaching. You may upload all your data to our system.

Back to Top

What about FERPA?

The Family Educational Rights and Privacy Act (FERPA) is a US law which protects the educational records of students enrolled in US institutions. Faculty members like you are permitted access to students' identifiable information, but students' parents and the general public are not. An exemption in FERPA (section 99.31.6C) allows access to data for organizations conducting research to improve instruction, which is what that Data Explorer does.

Our lawyers say that they are not your lawyers, so they cannot possibly give you legal advice. They also remind us that we are not lawyers, and thus should not give you legal advice either. If you're still worried about this, our lawyers think you should talk to your lawyers.

Back to Top

Is my data secure?

Yes.

We encrypt students' names or IDs using one-way, cryptographically-secure transformations, the same way that we protect your password. Even the staff at the PhysPort and our automated data explorer system cannot see your password or your students' identities.

Your account information and uploaded data are also encrypted. However, because we need to use this information to analyze your data and give you recommendations, PhysPort staff and the data explorer system can access this data.

Back to Top

How do you protect my data?

In addition to encrypting your data so that only you can see your personal details, we also protect you when we make comparisons to "students like yours" by only reporting information that has been combined with other instructors at other institutions. No one can guess which instructors or institutions are combined into "students like yours", so no one knows which parts of the combined data come from you, or even whether your data is included.

There are a few situations where other people can see your data and know that it is yours. If you give permission for your department chair or course coordinator to look at departmental data (as part of a departmental review, for example), they will know who taught what and when. If you print out some of your reports (for inclusion in your tenure file, for example, or a paper that you are writing), then the people you share your reports with will know which data are yours. It's also possible that your data could be subpoenaed for some kinds of lawsuits. That's incredibly rare and has never happened to us before.

Back to Top

What do you do with my data?

We analyze your data in four ways:

  1. We make recommendations to you about evaluation and teaching methods based on your data. That's why we need to know how and what you teach.
  2. We use your data, anonymously combined with others, to make recommendations for other users' "students like yours" comparisons. That's why we need to know where you teach and what kinds of students you are teaching.
  3. We use your data, anonymously combined with others, in order to do research on what kinds of teaching are best for which kinds of students and courses. When we do this analysis, we're not interested in you personally; we're interested in how trends in student learning are supported statistically.
  4. We use your data, together with the data from all other users, to make the PhysPort and the Data Explorer more helpful and robust.

To support all four of these purposes, we keep your data in a secure database forever.

Back to Top

Can others see that this is my data? How do you protect my identity?

We only report aggregate data where your data is combined with others' data. We make sure that the group of people who are combined together (the "anonymity set") is large enough that no one can guess the identities of anyone in it.

There are a few situations where other people can see your data and know that it is yours. If your department chair or course coordinator uploads departmental data (as part of a departmental review, for example) that includes yours, they will know who taught what and when. If you print out some of your reports (for inclusion in your tenure file, for example, or a paper that you are writing), then the people you share your reports with will know which data are yours. It's also possible that your data could be subpoenaed for some kinds of lawsuits. That's incredibly rare and has never happened to us before.

Back to Top

How do you protect my students' anonymity?

We don't store your students' names or IDs. Instead, we use one-way, cryptographically-secure transformations to convert this information into unique identifiers. We need this information so that we can match up students' pre- and post-scores, or to match up their scores on one assessment with their scores on another.

When we analyze the data, we only report it in aggregate. For you, that means that we report how your class as a whole is doing. Since these are your students, you might be able to guess how individual students are doing. That's ok, because they are your students and you already have access to the raw, identified data.

For other users, we combine data from many users at multiple institutions before we reporting the aggregate. No one can isolate one of your students from the data we report to others, which means that no one can figure out your students' identities.

Back to Top

Who has access to my data? Under what circumstances?

Mostly, only you have access to your data in a way that identifies it as yours.

There are a few situations where other people can see your data and know that it is yours. If your department chair or course coordinator uploads departmental data (as part of a departmental review, for example) that includes you, they will know who taught what and when. If you print out some of your reports (for inclusion in your tenure file, for example, or a paper that you are writing), then the people you share your reports with will know which data are yours. PhysPort staff have access to some of your data, but we only use it for the purposes outlined in the "what do you do with my data" section. It's also possible that your data could be subpoenaed for some kinds of lawsuits. That's incredibly rare and has never happened to us before.

There are other situations where people access your data after it has been automatically combined with others' data. For example, when we calculate "students like yours", we combine data across a lot of instructors and a lot of schools. We only report aggregate information without any personally-identifying markers. No one can guess which schools, instructors, or students are aggregated together to generate "students like yours", so no one knows how your data compare except you or the people in your department that uploaded it (like your department head or course coordinator).

Back to Top

What am I allowed to do with the analyses I get from the Data Explorer?

Comparisons of your data to national aggregate data:

  • You may use them for your own instructional purposes in your own classroom and department, for promoting your teaching in your tenure and promotion files, or in job applications.
  • You may not use them for commercial purposes, publish them, or post them on a publicly available website without our permission.

Analyses of your own data that do not include national aggregate data:

  • If you want to do research on your own data, you must secure human subjects permission from your institution.
  • If you publish analyses of your own data performed by the Data Explorer, you must cite the Data Explorer as follows: PhysPort Data Explorer, physport.org/dataexplorer, [year retrieved].

Back to Top

Do you have IRB approval for this?

Yes. This project was reviewed by the Protection of Human Subjects Committee at the American Center for Physics. They determined that our project is exempt under provisos B1 (educational settings) and B2 (educational tests) of 45 CFR 46.

To see our application and approval, contact us.

Back to Section Contents

Back to Top

Is the data in the database available for research?

Yes. To get research access to this data, you will need to apply for (and get) approval from your local IRB to use our dataset. Your application would probably be exempt under proviso B4 (existing data) of 45 CFR 46. Even though it will probably be exempt, you still need to apply to your IRB.

To help you be approved, we strongly recommend that you chat with us about the kinds of data that are available to be released before you apply to your IRB.

After you are approved, send a copy of your application and approval to us. We will review it, and release a spreadsheet with data that we can ethically share under our IRB approval and yours.

Back to Section Contents

Back to Top

I have some data that I already collected for a different research project. Can I upload it here?

Maybe. If you collected your data for normal educational purposes such as evaluating your teaching or the teaching in your department, and not for research purposes, then this section doesn't apply to you. Look at the section for faculty and instructors.

If your data was collected for research purposes, when you got permission from your IRB to collect that data, you needed to say which purposes you were going to use the data for. If one of those purposes includes uploading to our system or distributing the full data set, then yes, you can upload your data to our system. You might only have permission to share part of your data, in which case you should chat with us about how you should upload that part.

If your approval was ambiguous about that permission, or if you are confused about what you are allowed to upload, you should contact your institution's IRB for clarification.

If you don't already have permission to upload your research data, there are two ethically-available avenues, depending on what your previous approval says:

  1. We can apply to our IRB to use your data using proviso B4 (existing data) of 45 CFR 46.
  2. You can apply to have your existing approval retroactively increased to include participation in our database.

If you want to pursue either of those avenues, please contact us.

Back to Section Contents

Back to Top

Can you help me get IRB approval for a similar project?

Maybe. We're happy to discuss IRB issues with you in general and to provide copies of our forms, but every IRB will have slightly different forms and concerns.

If you want to do a similar project, contact us to talk about combining forces to make our projects better.

Back to Section Contents

Back to Top

My institution does not have an IRB. What should I do?

Generally, this happens because you are not in the United States, or because your institution does not do any research involving human subjects conducted, supported, or otherwise subject to regulation by any federal department or agency.

If this applies to you, your institution might already have procedures in place to protect human subjects. If you don't know what they are, the chair of the psychology department probably does.

If there are no procedures in place, or if you can't figure them out, please contact us.

Back to Section Contents

Back to Top

I am not affiliated with an institution. What should I do?

You need to find a sponsor at an institution which has an IRB. We do not make our data available to private researchers.

Back to Section Contents

Back to Top