Evaluation & Dissemination
Dec 8th, 2016 by mrugaa
To know if the ObservaStory achieved stated goals, the team evaluated its ability to withstand technical and user testing. Methods utilized include AB testing, surveys, and interviews. In short, the ObservaStory passed technical testing by creating a working prototype by December 1st. User testing was conducted on both December 1st and 7th when the team held informal user interviews, circulated a survey, and ran an AB test with users. With this evaluation, the team shaped a new direction for the ObservaStory project. They hope to disseminate their collective work to students in classes at RPI that may implement or at least advance the project in the future.
Method of Evaluation
Technological Verification, or “it works in-person starting on 12/01”
Technological verification ensures that a product will meet material demands that are pre-set by a certain group of people. The team determined that the ObservaStory must meet the following standards: View-Master technology shows information, the information is easy to see, the information is accurate.
Proof of this is shown in Figure 11. The final prototype is standing and information is displayed on the smartphone screen. However, the information is only fairly accurate.
Figure 11. The ObservaStory uses withstands technical and user testing on 12/01.
Validation, “or the team gets user feedback on 12/01 and 12/07”
User feedback generally affirmed the ObservaStory as visually immersive (e.g. visually appealing with interesting information) and easy to use. This validation was provided through the use of informal interviews after our class demo, separately conducted interviews with students from a Senior Thesis class at RPI, an AB Test run with students from the Senior Thesis class, and a survey circulated on our PDI Studio class Facebook page (Figure 12).
The ObservaStory design elicited sentiments like “the brightness is engaging” (Emily, Senior Thesis) while the survey results showed that some people were more interested in the urban ecology around the Sanctuary after using the ObservaStory kiosk. On the other hand, some users relayed that it hurt their eyes (Fariha’s Group) and that they would prefer the information be displayed on a poster because it could be used by more than one person at any given time (Heather, Senior Thesis). Our primary user, the Sanctuary for Independent Media, was represented (the team thought) by an RPI student who also interned in gardens at the Sanctuary during the Fall 2016 semester. An encouraging comment that she made in an interview was that the ObservaStory would be an important off-shoot of the Sanctuary’s North Troy People’s History Museum. This is because it takes personal and ecological information onto the streets, outside of the Sanctuary, and furthers their dedication to “tell a story” (Jessica, Senior Thesis).
Figure 12. The team created a poster that contained the same information as the ObservaStory kiosk to see if the physical design of the ObservaStory was truly effective. Some test users were shown the poster first while other test users were shown the ObservaStory first to maintain consistency across the experiment.
Dissemination
Based on the results of the evaluation, the team decided that the project should be revised but still should be fully implemented at the Sanctuary for Independent Media given cooperation from the organization. In hopes of doing so after the end of the Fall 2016 semester, the team can pitch the project to other public service and design classes at RPI. In particular, the team could make a flyer or poster asking PDI students to research and adopt the project in other studio classes. They could also communicate this information with other PDI students over Facebook or by word-of-mouth. The team could also frame the ObservaStory as an internship opportunity at the Sanctuary in a class known as Public Service Internship. The professor of this class is even asking his students to curate a collection of projects with which his future students can use to choose internships.
Future design considerations for this project include implementation of secondhand smart phones to display slide information, an accompanying website to further the interactive experience through themed scavenger hunts or original content submission capabilities, and continued dialogue with Sanctuary representatives to better tailor the included information to the Sanctuary’s needs.