How to Test Shortening the Tutorials

Continuing my theme from last week I'm working to make our tutorials much more enjoyable and effective for our users. I decided that I should try and start figuring out how to test the questions I had about the tutorials. So I'm working on some preliminary designs and also trying to figure out how to design some formative tests to answers my questions. What I've done is written a list of the questions I want answered so that I know that our designs are on a decent first pass when we can actually start testing them.

Basic Questions:

I think these are the probably the easy thing to start testing at the Science Center and I think very quickly I'll get a feel for what is reasonable and then I can start aiming my designs towards this mark.

• How many steps is too many? How many steps is just right?

• How much time is too much? How much time is just right?

 Auto-Advancing Questions:

I want to test whether auto-advancing is a reasonable approach to shortening the tutorial. It is the idea that I'm currently most keen on, but I'm not really sure if it will really be effective in practice. So I really want to test this early on so that I can start making drastic changes if this fails.

• Do users understand this auto-advancing step is being done for them by having a ghost complete it? I will be testing this one by not building the change but by using the Wizard of Oz technique. I think I'll need some additional onscreen information but that can come later if this idea seems like it works reasonably well. This should also help inform whether we need to have all of the setting the interface in the correct state steps.

• Can I remove all sub-scrolling steps and not impact the users. I think the scroll down or up sub-steps are really useless. I want to just set scroll panes so the component is already visible. Will this work?

Design Questions:

I've created several sketches of several designs that I think may help move forward my goal of shorten the tutorials. I'll test these several at time to get a feel for what works.

• Do users respond well to this tutorial progress component? I'll test several different ways to gauge progress through the tutorial.

• Do users respond well to this tutorial note instruction design? Here I'll test several designs that add progress and contain additional information for auto-advancing.

• Are users aware that they will be completing a tutorial to get their remix. I want to test some introduction screens that try to ground this process so they don't seem so surprised.

• If time permits I would also like to know whether my call-outs design for tutorial presentation could also shorten some steps for users. But this will be harder to test. So I will wait on this after I get more information about how well this is all working.