• Home
  • 2017
  • 2016
  • 2015
  • 2014
  • 2013
  • 2012
  • 2011

PDI Studio 5

Design to make a difference

Feed on
Posts
comments

Evaluation and Assessment

Dec 7th, 2015 by jungg

When designing a product, it is important to understand the user and develop an analysis for your design choices. The best way to evaluate a design and determine if it was effective in the areas you wanted it to be is through implementation, testing, and user feedback. For our product it is important to evaluate each student’s role during the game and what they took away from the activity. It is also important to measure mathematical comprehension and abilities before and after using the device. To do this we could implement an after game survey and short mathematical exercises.

By carefully designing a survey that is unbiased and simple, we can create a table of information based on the experiences the children had using our device. Not only would the survey allow us to understand how we can improve the game for future applications, but it would also allow us to see what the students believed they were learning from playing the game. If we receive many responses that show the students perceive a different goal than intended, this would tell us that we must make revisions or remove parts of the original game that might be taking away from the main objective.

The short mathematical exercises could be designed to test many different understandings; this could include small comprehension problems, questions that test mathematical procedure (PEMDAS), and even a timed section testing how quickly the students can answer a number of equations. These exercises would need to be implemented before the students played the game and again after interacting with the game over the course of some number of uses. If these exercises could not be added to the curriculum it would also be possible to compare normal math exam scores over the course of their school year. However, the continued use of the device over the course of the year would be necessary. The game also makes it very easy for an instructor to increase difficulty as needed or necessary for continued improvement.

These evaluations would deal with threats of validity and statistical analysis by being balanced, fair, and neutral. The survey would ask questions that are written in a way that does not lead on a particular answer and would be purely designed to achieve an accurate understanding of what the children learned and what they liked or did not like about their experience. A survey such as this would not be meant to make our device look perfect but instead, and ideally, it would be designed for us to achieve optimal feedback that would in fact improve a particular aspect or process of the game. Certain questions could also be asked twice in different context to make sure a student was not only providing random answers and would further test the validity of a student’s responses.

The mathematical exercises would avoid threats by being carefully designed to test a particular understanding or skill in each question, or set of questions. By providing these exercises before and after continued use, we can measure a student’s performance in several mathematical comprehensions and any effect the device has had.  Ideally, the exercise before use and the exercise after use would need to be different, while still testing the same abilities and comprehension. Students would not be allowed to receive their first exercise back, further avoiding the possibility of memorization or preparation.

Practical problems carrying out these evaluations arise if students participate in the game as little as possible, or if the students choose to not be a part of the mathematical aspects of the game and instead become the “builders” and “shakers”.  Those students that choose to be purely uninvolved in the mathematical portions may have different survey answers and results from the exercises. To help clarify and organize the information from our evaluations, it is important to ask what the particular student’s roles were during the game. It is also possible for students to forget certain mathematical understandings that they learned from the game.

We believe that the combination of a carefully designed survey and mathematical exercises would help us to best evaluate the validity, success, and overall design of our device. By understanding each student’s individual role during the game, perceived goals, and mathematical comprehension, we can make improvements or alterations to the design, or prove the success of our game.

Comments Off on Evaluation and Assessment



Comments are closed.

  • Projects

    • 2017
      • Artistic Prosthetic
      • Cell Protobox Activity
        • Background
        • Overview and Lesson Plan
        • Pictures from the Activity
        • Final Prototypes from the Activity
        • Recount of the Activity
        • Demo Day with PDI Students
        • Final Protobox
        • Next Steps
        • Vision
      • Hue Harmony
        • Project Scope
          • HueHarmony2.ino
          • Sampler_12bit.ino
        • Future Development
        • User Feedback
        • Research
      • infinish.ed
        • Proposed Solution – Unfini-shed
        • Proposed Solution – infinish.ed
        • User Experience Mapping
        • Design Justification
        • Roadblocks/Challenges
        • User Feedback
        • Conclusion
      • Pop-Up Symphony
      • Saltwater Greenhouse
      • Smart Protection Brace
        • Background Research
        • Initial Prototype
        • Iterations and Second Prototype
        • Next Steps
        • Potential Methods
        • User Group
    • 2016
      • Creation Station: An Interactive Kiosk
        • User Group
        • Feedback #1: Oakwood Community Dinner-November 14th, 2016
        • Feedback #2: Uncle Sam’s Bus Stop-November 15th, 2016
        • Feedback #3: Farmer’s Market- November 19th, 2016
        • Prototype and Final Build of the Kiosk
        • Final Testing and Final Thoughts
      • ElektroTone
      • Furnäture
      • Media Sanctuary Mobile Experience
        • Prototype 1
        • Prototype 2
        • Prototype 3
        • Next Steps
        • Presentation Slides
      • ObservaStory
        • Background Research
        • First Iteration
        • Second Iteration
        • Evaluation & Dissemination
      • People Library
        • Target User Group
        • Primary Iteration: The Event
        • Secondary Iteration: The System
          • The Website
          • The App
        • User Feedback: The Process
        • The Final Product
      • Physics Plank
        • Physics Plank 1.0
          • Design
          • User Feedback
        • Physics Plank 2.0
          • Design
          • User Feedback
        • Next Steps
          • Ideal Code and Construction
      • Portable Foundation Finder (PFF)
        • Background Research
        • Customer Discovery
          • Personas
          • Survey Responses
        • Prototype
        • User Testing
        • Working Demo
    • 2015
      • Alien Adaptation
        • Target User Group
        • First School Visit (September 28, 2015) – Prototype #1
        • Second School Visit (October 15, 2015) – Prototype #2
        • Third School Visit (November 9, 2015) – Prototype #3
        • Fourth School Visit (December 10th, 2015) – Prototype #4
        • Background Research
        • Alien Cards
        • Methods of Organization
        • Other Resources
      • Zoe the Green Monkey
        • Background Research
        • Evaluation & Dissemination
        • Sources for Further Teaching
      • Recipe Ratios
        • School 2 Visits
        • Background Research and Information
        • Context
        • Evaluation & Assessment
      • Space Race
        • Background Research
        • Evaluation and Assessment
        • Field Work : 1st Visit to School 2
        • Field Work : 2nd Visit to School 2
        • Field Work : 3rd Visit to School 2
        • Field Work : Final Visit to School 2
      • Rhythmatic
        • Overview/Background Research
        • User Research & Design Specifications
        • Lesson Plan
        • User Testing
        • Ethnographic Observations
        • Final Interface Design
    • 2014
      • Ghanaian Solar Heat Collection
        • Solar Production of Adinkra Ink
          • Adinkra Ink Background Information
            • Interview with Mae-Ling Lokko
            • Production of Adinkra Ink
        • Solar Production of Biochar
          • Biochar Information
          • Heliostat Development
            • Codes and Hardware
            • Phase 1: Small setup
            • Phase 2: Medium Setup
            • Phase 3: Future Design Considerations
          • Interviews with Marianne Nyman and Alex Allen
          • Planning the process
          • Prototype Design Process Images
        • Prototype Iterations
        • Testing
        • Future Prospects
      • Cheap Sensors for Quality Education
        • Building the Sensor
        • CSnap Integration for Data Visualization
        • Future Work
        • Hardware
        • Previous Iterations
        • Professional Development Integration
          • Compost Computing PD Outline
        • Transferring Data from Arduino to CSnap
        • Code (Integration with SD card data collection)
          • SD Card Readout Example
        • Code (Temperature and Humidity sensors only)
      • Condom Vending Machine
        • Cultural Background
        • Why Open Source?
        • Screen Printing Vs. Stickers
          • Hints for stickers!
        • Intitial Design Ideas
          • Initial Designs
        • Prototypes and feedback
        • Final Design
        • Feedback
        • Future Plans
      • Music & Sound: Earthquake Simulator
        • Prototyping & Iterations
          • Rounds 1 & 2
          • Round 3: Final Prototype
        • Field-Testing: User Feedback
        • How-To/DIY Projects
        • Future Testing
        • Dissemination
    • 2013
      • Edu-Ponics
        • The Community Garden
        • The Tower
        • The Monitoring Unit
          • Code
          • Prototype
        • Dissemination
        • User Feedback
        • About
      • Fraction Bot
        • Future Iterations
        • Primary Market Research
        • Prototyping/Testing
        • Source Code–Please Use
      • Ghana Outreach Solar Reflector
        • Cultural Background
        • Partnership with KNUST
        • Prototyping
          • Initial Mock-up
          • Lamination
          • Plywood Model
          • Reaction Chamber
          • Stainless Steel Ribs
        • Testing
        • Future Iterations
      • iQube
      • Lean Green Clean Machine
        • Background
        • Redesign Proposal
      • Music Factory
        • Prototype
        • Field Testing
        • Dissemination
      • Nutrition Kitchen
        • The Game
        • Learning About Nutrition
        • Children’s Reactions and Results
      • Solar Tracker
    • 2012
      • 2 Fast, 2 Curious
        • Learning Objectives
        • Make It Yourself!
          • Hardware
          • The Code
        • Responses
        • The Game
        • Downloads
      • Apollo 2 1/4
        • Children’s Reactions
        • Final Trip Notes
        • Fraction Practice
        • Materials & Software Used
        • Source Code
      • Coordinate Kinection
        • The Game
          • How to Play
          • Screenshots
        • Student Reactions
        • Do It Yourself
          • main.cpp
          • Instructions
          • Download
      • Discover The Universe
      • Power Up!
        • Make Your Own
        • Platform Theory
          • Software Theory
        • Research and Development
        • Student Reaction
      • Squid Learn
        • Building the Lesson Plan
        • How It’s Made
        • Research and Iterations
          • 1st Visit
          • 2nd Visit
          • 3rd Visit
        • Student Reactions
    • 2011
      • Alien Frontier
        • Downloads
        • The Process
          • Ethnographic Description
          • Ethnographic Description 1
          • Ethnographic Description 2
      • Cookie Creations
        • Cookie Creation Game
        • Classroom Process
        • Assessments and Surveys Used
        • Research
      • Fast Track
        • DIY
        • Learning Objectives
        • Materials
        • Response
        • Components
      • Make ’em Fresh
        • Make ’em Fresh: How It’s Made
          • Team Contributions
        • Make ’em Fresh: Research and Iterations
          • Early Concepts
          • Images: First and Second Visits
          • Future Iterations
        • Make ’em Fresh: Student Response
          • Pre- and Post-Testing
      • Rap Star
        • The Game
          • What?
          • When & Where?
          • Who?
          • Why?
        • Field Testing and Iterations
          • First Visit
          • Second Visit
          • Final Visit
          • Ethnographic Descriptions
        • What Now?
          • Conclusion
          • Future Iterations
          • Additional Images
      • Rush More
      • School House of Pop
        • Educational Goals
        • Student Response
        • Design Evolution
        • Future Design Aspirations
        • Culture and Society
        • Pictures at School

Theme: MistyLook by Sadish. WPMU Theme pack by WPMU-DEV.