Prototype photos
Scenario tasks
Briefing
Demographics and Setting:
We tested people in a WVH class room. The room was empty except for us and spacious, so it was a healthy environment to conduct isolated testing in. We allotted each tester about 30 minutes to do the test and provide feedback; some users finished in only 10-15, others needed up to 40+ minutes. The equipment consisted of construction paper for the screens, post-its for the interactable objects on the widgets in said screens, and pen as a pointing device. We tested 3 white males of age 20-21, and one Asian male of age 21.
Observations:
Tester Alpha:
Task 1: Create a task
Task 3: Feedback widget
Create a task:
Create a task
Create a task
Interview and Feedback
Tester Alpha:
Create a Task
How did you feel about the drag and drop?
Task 2: Shop something
Any feedback for the shop?
Proposed revision: Have items stay in card, add buy item. When the user is finished shopping, they click on the buy item. Only one confirmation screen pops up.
Task 3: Feedback widget
How did you feel about the emojis?
Create a task:
How did you feel about the dragging?
How did you feel about the multiple confirmation windows?
-“Not sure why I would want drag and have a buy confirmation appear each time”
-he then compared it to amazon’s cart system
*We then propose our revision from earlier: allow users to drag multiple elements to the cart and hit buy once don*
What did you think about the first screen?
Tester Charlie:
Create a task
Any thoughts about the Date and Time selection?
What did you think about the shop and cart?
Thoughts about the first page?
Create a task
Any thoughts about the task creation? Particularly the drag and drop?
What were your thoughts about the shop?
What are your thoughts about the forced opening of the feedback widget? ( the modal nature of the widget)
Consolidated results:
The drag and drop nature of the ‘create a task’ is really not clear enough for first time users. Even with a small demonstration “gif” provided by the “Computer” users still did not understand how that they had to drag all elements to the center. A wording change might be needed. That said, once a user figured out they had to drag like that, it stuck with them.
The shop cart system should be revised to allow users to drag multiple desired items into the cart, and then click buy once they’ve got all the items they want.
The emoji sequence could be reworked such that we ask users to swipe their tasks first, and then ask for emoji feedback. The emoji system itself can have its functionality expanded as per Tester Charlie’s robust feedback.
We tested people in a WVH class room. The room was empty except for us and spacious, so it was a healthy environment to conduct isolated testing in. We allotted each tester about 30 minutes to do the test and provide feedback; some users finished in only 10-15, others needed up to 40+ minutes. The equipment consisted of construction paper for the screens, post-its for the interactable objects on the widgets in said screens, and pen as a pointing device. We tested 3 white males of age 20-21, and one Asian male of age 21.
- Tester Alpha is a 4th year Asian male who’s majoring in Financing
- Tester Bravo is a 4th year White male who’s majoring in CS
- Tester Charlie is a 5th year White male who’s majoring in MIS
- Tester Delta is a 3rd year White male who’s majoring in CS and ECE
Observations:
Tester Alpha:
Task 1: Create a task
- Found create a task decently fast
- Struggled to find create a new category, did not understand the plus symbol was for add new category
- Missed the drag and drop window, did not drag things to the window
- Got it after he finished completing all the tasks
- Completed time and date tasks fairly quickly and accurately
- Missed drag to card at first
- picked up on it quickly – good learnability
Task 3: Feedback widget
- Held onto the emoji as he completed the feedback session: different from what our design was
- Assumed he should swipe left for tasks completed
- He actually went through this very fast and superficially
- Didn’t really look at the wording of the prompts
Create a task:
- Found the create a task pretty quickly
- Figured out drag and drop as well
- Figured out date and time pretty fast
- Misc: People tend miss instructions. Maybe we should enforce the questions more. (can be difficult to diagnose design mistakes vs mistakes that arose because people skimmed through questions)
- “Oh I was supposed to drag” - again, learnability
- Finished the task pretty fast. Didn’t struggle much
- Took swiping and feedback seriously enough, though about each option
- Thought 2nd screen was fun
- Proceeded through it very naturally, as if he were really providing feedback on his day
Create a task
- found create a new category pretty quickly
- He skipped a few steps here almost.
- caught him an slowed him down
- slowed down when doing time and date selection
- asked about hour minute time selection
- Paused a bit to find skins
- Overall had no issue
- Finished first page competently
- dragged each emoji to its category without trouble
- finished second page VERY quickly
- started from the bottom
- swipe everything left and right in quick succession
- variable ordering should be a thing
Create a task
- wasn’t sure where classes came from
- prototype limitation?
- struggled on category creation a little
- fumbled with the confirmation window
- wasn’t sure how date would relate to task creation
- easily misclicked on dates
- struggles with the date concept of drag
- completes date slowly overall
- struggles with the concept of drag to make a task
- clicks on titles a lot
- once clarification was given he finished the other fields quicker
- “why can we add category but not class?” was something he wondered
- At this point is showing signs of high frustration level
- sarcasm evident in speech, clicking things in an annoyed manner
- asks questions that may or may not be legitimate (can’t tell if he’s actually confused or just wants to make fun of something)
- frustration in testing makes it harder to judge results and data
- still completes all tasks though
- gets stuck on the avatars button a little
- wording on instructions could be improved, but the interface seems solid
- completed first and second pages without incident
Interview and Feedback
Tester Alpha:
Create a Task
How did you feel about the drag and drop?
- “Drag and drop felt redundant, it could’ve been faster just to click on areas individually”
- “However from an aesthetic point of view it’s very nice”
- “The create a new category was ignored mentally because it was a different color”
- “It would’ve been better if it were the same color as everything else”
Task 2: Shop something
Any feedback for the shop?
- no notable feedback: was mostly good.
- See self feedback
Proposed revision: Have items stay in card, add buy item. When the user is finished shopping, they click on the buy item. Only one confirmation screen pops up.
Task 3: Feedback widget
How did you feel about the emojis?
- “I was not sure what the emojis meant, I feel like if some text was added to articulate what the emojis meant it would’ve been better. I was still able to discern their meanings though.”
- “How was your day: It sounds like a question eliciting an emoji response. It seemed like a very necessary prompt.”
- “I don’t really think the swipe direction makes a difference.”
Create a task:
How did you feel about the dragging?
- “The dragging isn’t that clear at first, but it’s pretty intuitive once I figured it out.”
- “The date and time seems fine but you could do a clock selection”
How did you feel about the multiple confirmation windows?
-“Not sure why I would want drag and have a buy confirmation appear each time”
-he then compared it to amazon’s cart system
*We then propose our revision from earlier: allow users to drag multiple elements to the cart and hit buy once don*
- “Yeah that seems like a much better approach.”
What did you think about the first screen?
- “It seemed… okay I guess.”
- “no not really.”
- “I thought 2nd screen was fun, you could slide everything away”
- “Hmm I don’t really think the order mattered.”
- “I thought message dialogues were okay.”
Tester Charlie:
Create a task
Any thoughts about the Date and Time selection?
- “ The date selection should be unified, and I think it’d be good to have the date default to today.”
- “The start and end should be bundled together as well, instead of having me fill out two of the same field”
- “While the drag and drop mechanic is somewhat obvious at first, it’s not obvious at all that all elements should be dragged into the middle. The wording should be improved on the box.”
What did you think about the shop and cart?
- “Definitely didn’t like the drag as much for this one. It made sense for the previous screen, but it’s, not as intuitive for this one.”
- “Like why is it asking are you sure twice? If I drag an item to cart I’m pretty sure I want to buy this one. The multiple confirmation screens are redundant as well.”
- “Also if I own something in the shop it should be greyed out or something.”
- “Yeah that seems like a much better way of using the drag and drop in a shop setting.”
Thoughts about the first page?
- “Don’t really like dragging for emojis, it has a higher chance for mistakes. I think it would be better if I clicked on each element, and then an emoji menu popped up for each one. That gives me more control.”
- “I liked the swipes. …. “
- … “is the swiping page branched off from the emojis?”
- “I think you should have the emoji screen be an overlay from the swipe screen. So like after you swipe a task, it’ll ask for your mood for that subject.”
- “I really didn’t like the thank you screens. They were obtrusive. Especially if it asks me for more feedback, I feel less inclined to participate.”
- “Does the emoji system do anything right now? “
- “Right, but if I’m doing [badly] in English right now I’ll definitely know it myself. I don’t need software to tell me that.”
- “How about instead of just showing the emoji levels, the app prioritizes tasks based on your emoji level in a class. For example if I’m super happy in Bio, and I have a big task coming up, and I also have a big task in English coming up but my emoji is very sad, then the app might say ‘hey should watch out for this English assignment’ because I feel really weak about English. prioritization in English
- He also mentioned something interesting: levels could indicate proficiency. Before we had just thought of them as a way to track reward. Now we might consider levels changing dynamically to indicate how proficient someone feels in a class.
Create a task
Any thoughts about the task creation? Particularly the drag and drop?
- “I would’ve never occurred to me that I need to add anything besides class and date. The drag and drop isn’t that clear.”
- “I kept clicking on titles, I feel like I should be able to click on them.”
- “The date input mechanisms are really weird. It’s really easy to misclick, and I didn’t know I could drag after I finished filling out a field.”
What were your thoughts about the shop?
- “Well first off your instructions sucked.”
- Not sure if he was being facetious or not but I decided to jot it down for future testing feedback
- “The shop was a lot more functional than the ‘create a task’ widget. Some things could be reworded or better articulated, like Avatars instead of Avatar. Avatar makes it sound like I’m buying stuff for my Avatar.”
- “That’s a much better design than what’s currently here.”
What are your thoughts about the forced opening of the feedback widget? ( the modal nature of the widget)
- “I think it’s absolutely necessary if you want people to provide their feedback.”
Consolidated results:
The drag and drop nature of the ‘create a task’ is really not clear enough for first time users. Even with a small demonstration “gif” provided by the “Computer” users still did not understand how that they had to drag all elements to the center. A wording change might be needed. That said, once a user figured out they had to drag like that, it stuck with them.
The shop cart system should be revised to allow users to drag multiple desired items into the cart, and then click buy once they’ve got all the items they want.
The emoji sequence could be reworked such that we ask users to swipe their tasks first, and then ask for emoji feedback. The emoji system itself can have its functionality expanded as per Tester Charlie’s robust feedback.