In order to adequately evaluate our product, we would need to run two separate tests. One would test the usability of the product and the user experience. The other would test the effectiveness of the device as a learning instrument.
Test 1: User Interaction and Usability
For the first test we would create our earthquake simulation kit and bring it to an Earth Science class. The ideal group of students would be in middle school or high school and would be currently taking an Earth Science class. We would incorporate this product into their current curriculum and create lesson plans around it. In order to evaluate how the students react to the product, we will put them in small groups and give them a specific task to complete. We may then measure the success of the simulator based on student reactions at different points in the process, noting when students react well and when they become confused or unsure of where to move next in the process. If the process runs seamlessly from setup to engagement to cleanup, then we may confirm successful user interaction and usability.
Test 2: Educational Value
The goal of this test is to evaluate the educational value of our device and determine one of three possible outcomes:
- The device is beneficial to students’ understanding of the material
- The device is detrimental to students’ understanding of the material
- The device imparts no effect on the students’ understanding of the material
Before performing the test, we would need to determine the test’s significance level and the corresponding values for each of the three outcomes.
The test would ideally be performed using three earth science classes of the same grade and caliber (i.e. both “Honors” or both “Regular,” etc.). If this is not possible, then a single class will be used and the students will be randomly assigned to one of three groups. Before the normal lesson on earthquakes is taught, the students will take a pre-test to determine their existing knowledge of the subject. Then, the teacher will be instructed to proceed with the lesson(s) as per usual. Once the lesson is completed, or at a point just prior to assigning a lab/activity for the students to perform, the students will be given a second pre-test. This will be done to establish a baseline, as well as evaluate the effectiveness of the teaching prior to any lab/activity.
The students will then be assigned a lab/activity. One group will perform a traditional, non-simulated based lab, one will perform a simulation lab using a traditional earthquake simulator, and the third will perform a lab using our sound-based earthquake simulator. Once the lab is completed, the students will all take a post-test. The results of the post-test will be compared with those of the second pre-test across the three different groups in order to determine the effectiveness of the three different labs/activities.