Software is only as good as it is discoverable. When you put Sugar in front of a new user, some will take to it and others will not. However, some of the parts of Sugar are not discoverable, for example, invoking the frame.
A selection of the screenshots displayed
I then sat down with people and watched as they used it. I tasked by test subjects to open and move between 2 activities running at the same time - something which happens via the frame. I also observed the way that they interacted with the software. I worked with 5 testers, all of whom where school age (Aust years 7-10) and how were very familiar with traditional computers.
It didn't help.
Not only did my thing not help people find the frame (or anything else), the added popups actually annoyed them. They didn't want to read the text and they didn't find it helpful. Even with pictures, some instructions where confusing for them. Really, it wasted their time.
So what would I do in the future? I would force them to read and interact with the frame. My design was too big, it added to much. Too much of the content was irrelevant, so people very quickly learnt to ignore it. I needed to choose 1 thing, and be forceful and evil to teach them it. That should have been forcefully teaching them to activate the frame, and activate palettes.
I also had some big takeaways about the palette system. The tooltip part of the palette system is great. Users find it very intuitive how fast the tooltips activate. They also seem to intrinsically know that there should be more there; they move their mouse over tooltips waiting for the secondary popdown. However this is the issue that they had with the palettes, the secondary popdown is too slow. In the time between the primary and secondary popdown, the users had mostly become confused and moved away. Maybe we could unify these popdowns and just always show the full palette?
Usability testing was the most fun thing to do. I need to make more friends so that I can do more of it. I learnt so much. You should give it a go too!
Comments, thoughts? Mail them to firstname.lastname@example.org. I would love to hear them!