Diary

Internal Reports

Completed Papers

Papers in Progress

Conference Presentations

Site Visit and Quarterly Reports

Main DLI Page

Web Client- DeLIver

Summary of DLI Prototype Usability Tests

Prepared by Emily Ignacio for Site Visit, March 1996

I. Method

In consultation with the Testbed team, we prepared 10 different tasks which tested the functions of the interface. Nine were simple tasks designed to test the prototype's search capabilities, and one was a story problem involving retrieval and manipulation of a fulltext article with Panorama. They were encouraged to talk aloud as they worked, so that the notetakers would be able to record their thoughts. Their actions were recorded by a screen capture, and their comments were audiotaped. After each session, the participant was debriefed with general questions about the testbed and the usability session. We recruited 10 participants from several different Physics research teams. Virtually all were graduate students. Each session lasted one hour.

II. Major Findings

The usability sessions uncovered 31 specific problems with the DL prototype. Presented below are problems experienced by at least 70% of study participants, along with the solutions they suggested.

Problem: Difficult to find "And/Or/Erase All buttons
Solution: Put buttons above black line so they're grouped together
with search boxes

Problem: Scroll box next to "Add" button only contains "Or" option
Solution: Include "And" option as well

Problem:Icon next to figure captions in full text are not clear; two
participants thought globe would take them to the WWW
Solution: Use an icon that looks like a small graph to signify
retrieval of figure

Problem:Not clear how to change search fields when in full text search
mode; thought they could use pop-up menu to choose/change fields, but in
fact had to double-click to bring up window where fields could selected,
revised. Very tedious to unclick all unwanted fields
Solution: Have one step for displaying exhaustive list of fields;
indicate which are active

Problem: Font too small; difficult to read
Solution: Make font size customizable by user

Problem:"Exit" is too final; means you want to get out of the system.
Short entry window especially needs explicit clear
option
Solution: Add/replace with "done," "clear," "go back," "start
over," or "new
search"

Problem:General confusion about seeming inconsistencies in windows and
pop-up menus, e.g., pop-up menus didn't contain all options, windows
didn't seem to function as windows (with scroll bars, etc.).
Solution:Check consistency with standard design guidelines for use of
windows and pop-up menus

Problem: Color contrast needs to be changed; difficult to read
Solution:Change white on black highlighting, red on gray "Perform Search"
button, and aqua background, which is too bright

III. Wish List: What Users Were Most Eager to Change

Double clicking on title (in displayed results) should go directly to short entry (Mentioned by 90% of participants)

Double clicking on caption (in short entry) should go directly to figure (50%)

Hyperlink from "F1" in full text to figure (bypasses 'globe' and figure list) (50%)

Add box to check for case sensitivity (40%)

Figures embedded within text (40%)

Add ability to modify search in author search form (40%)

Highlight search term in bibliography and/or all fields where it appears (40%)

Tab to next entry box in author search form (30%)

Type whole author name on one line in author search form (30%)

Link from bibliography to other articles (30%)

IV. Best Things About the System

Ability to access full text ( Mentioned by 60% of participants)

Choices on top level for different search options (30%)

Ability to search in various ways (30%)

Responsiveness/quickness of system (20%)

Ability to access abstract (20%)

Print out capability (20%)

Ability to search components of article (e.g., figure captions) (10%)

Inclusion of figures in full text of articles (10%)

V. Places where Instructions/Examples would by Helpful

Many users had the same kind of uncertainties about their interaction with the testbed at particular stages. Based on these, we think the following are good junctures for further explanation or examples: