Diary

Internal Reports

Completed Papers

Papers in Progress

Conference Presentations

Site Visit Reports

Main DLI Page

Web Client- DeLIver

Report of First Usablity Test on Web Interface- December, 1996

prepared by Laura Neumann

15 December, 1996

I. Summary of Method:

We invited five people to come in and do a series of tasks, two of them are graduate students in engineering, two were masters students in Library and Information Science who work at Grainger, and one person was full time staff at Grainger. They were asked to "think aloud" as they went through the tasks, commenting on what they were seeing, doing, and thinking about the interface as they went. The two observers of the session took notes as the person talked. This summary is generated from those notes. At the end of the session, each person answered a set of general questions. Both the tasks that the people were asked to do and the general questions they answered are at the end of this document.

II. Comments:

A. Opening Search Screen: These are in order of importance and frequency:

• why is this here? should not exist at all
• why is the link to the second page buried at the very bottom of the screen in indistinct language?
• there is no "help" here--
• should be an explanation of syntax (L. Chen? Chen, L.? etc.)
• should explain what is possible esp. re: Boolean searching
• should explain the database that is being searched (range of dates, esp.)
• should explain what parts of articles are being searched
• list of journals problematic
• why can you only search one journal at a time?
• should be able to select journal by clicking anywhere on the title, not just in exact "0"
• why is the last selection of journals lumped together? Which are there specifically?
• reverse the order: journal title first, then society (order of importance to searchers!)
• journal title should not be in italics
• the journals and societies should be links to pages for those
• color and bold and italics seem to be used randomly here
• the information at the top of the screen is not very informative as to what the system is and what it does
• why is that one journal in particular up there?
• text is jargon ridden
• why is the window for text input so small?
• want the system to "remember" what has been typed here, so after you do a search you can go back and change what you typed in
• where does the cursor default to? Must always click somewhere to activate it.
• no one ever used the clear button

B. On the second search screen These are in order of importance and frequency: Almost all of the issues mentioned above apply here as well.

• Problems with the pop- up menu
• Why can't you select more than one thing from the pop- up menu?
• the order of items in that menu seems random. Author, title, and subject should be first
• is it a joke that you can select the title of the journal here?
• volume number, issue number, date of publication, date, repetitive and messy. In addition, they don't seem to work. (try task 5)
• if keeping the two screens, want system to carry through what was typed on the first one to the second, as well as the setting of the journal to be searched.

C. Displaying Results, step one These are in order of importance and frequency:

• Should highlight the found words (e.g. searched on ImP, that word should be highlighted in results, of what ever set.)
• Bibliographic information on journal articles is not displayed in a standard manner
• Author should be first
• Do not use italics or bold randomly here either
• For the rest, choose a standard format such as ANSE or MLA
• Why is the extraneous information here? (the number strings)
• Want to see more than 5 at a time, or have the option to set it, it takes too long for each set of 5 to come up and with only 5 you can't effectively use "find" (through Netscape) for your results.
• Should be able to sort the display in different ways, e.g. by journal, or have it ranked in some way (and tell what way!)
• Buttons giving options not clear-
• especially "View More Information" button-- what is that? Should say "abstract, figures, references"
• two other links also unclear as to what they are and that they are separate
• Should repeat at the top of each screen what has been searched, one person suggested allowing an input window there as well, "like Alta Vista"

D. Panorama, Acrobat See separate report

III. Things that should be addressed through examples or help

These are in order of importance and frequency:

• Boolean searching issues, syntax
• the system, if it could do Boolean searching, should tell you it searched X on Y and got Z hits, etc.
• how to enter author names "L. Chen; Chen, L.; L Chen;" etc.
• how to use punctuation
• case sensitivity
• contents of the system
• how the system deals with acronyms
• how the system deals with truncation (e.g. MIT as an affiliation- they get Mitsubishi too)
• needs as far as software goes: Panorama, Acrobat, and a JPEG viewer, make these links to go get those systems

IV. Suggestions for Improvement

These are in order of importance and frequency (also repeated from above):

• add Boolean search functionality, able to combine search terms as well as areas searched
• have only one search screen. First is pointless, frustrating to finally discover screen 2.
• give the user more information (see section on "things that should be addressed through examples or help")
• instead of "view more information" say "view a detailed reference"... i/f should borrow more from standard vocabulary of IBIS or what ever other system.
• paraphrase part of article that has keywords in it on first retrieval screen, right away.
• if keeping 2 screens, have text from screen one carry through to screen two.
• should give search statement at top of results screen in window there to modify... always there. ("like Alta Vista")
• put the results in some order and explain what that order is. Better yet, let the user choose.

V. Likes

• simplicity, speed
• interface is "streamlined," "uncluttered," and "simple"
• it has good potential for web access and through use of a known system (Netscape)

VI. What I Think You Should Do With This:

• dump first screen
• modify second screen
• put useful information at the top of the screen:
• what the system is, in plain English;
• explain what system can/ can't do, in plain English (esp. re: Boolean searching)
• A link to a list of all journal titles as well as dates covered for each;
• Each journal title should be linked to a list of each issue for browsing;
• an explanation that special software is necessary, with a link to a list of each program and those should be links to the places where you can get those
• a link to examples and general help as suggested above.
• just above text input window, put a line with some short examples
• input window should be larger
• options for searching areas of journal should be in a list down the right hand side of the screen with boxes to be "X"ed... with the capability of selecting more than one at a time
• journals to be searched should be listed below text input window as originally, however:
• journal title, underlined, should precede publisher (but that info is not really relevant for most people, perhaps this should be in the "info/ help" area)
• all journals should be listed (not "assorted titles")
• these should also be "X" boxes, not radio buttons... with the function of being able to search multiple titles
• draw a line under all this and then add text: "other useful resources:"
• O+ telnet link
• Los Alamos Server link:Los Alamos server
• AIP link:American Institute of Physics Page
• other links:
• JOURNAL DE PHYSIQUE I
• JOURNAL DE PHYSIQUE II
• JOURNAL DE PHYSIQUE III
• ASTRONOMY AND ASTROPHYSICS SUPPLEMENT SERIES
• EUROPHYSICS LETTERS
• ANNALES DE PHYSIQUE
• Yahoo Physics links page
• The Fermi Lab page
• Particle Physics
• ...etc...
• ... if there is too much text (looks like it!), write a sentence explaining what will be on the next page, make a link to a new page with this list of other links.
• On the displaying results screen, the first 20 hits should be displayed (thus making Netscape's "find" more useful)
• At the top of each screen of 20 hits should be a statement saying "X terms were searched in Y areas of Z journals"
• Place statements about software requirements for viewing at the top of each screen:
• Panorama, or some such viewer needed (and here is a link to go get that viewer) for SGML viewing
• Adobe Acrobat needed for PDF viewing
• JPEG viewer
• At the top of the screen should be a button getting the user back to "help"
• Article cites should be formatted in a way standard to the field
• There should be 3 buttons below each cite, not one button and 2 links:
• First one should say "abstract, figures, bibliography"
• Second one should say "full text in SGML"
• Third should say "full text in PDF"
• On every screen, there should be a link to an e-mail form for feedback
• On every screen, color and bold and underlining should be used consistently and meaningfully

Appendix A: The Instrument

1. Find this citation:

2. Find all the articles about "GaAs" and "InP" in the two physics journals in this database.

3. Find an article about "KrF excimer laser etching in the Applied Physics Letters.

4. Find work written by L. Chen in the journal from the Society of Electronical Engineers.

5. Final all the articles from Applied Physics Letters in March of 1995.

6. You met a guy named smith at a conference. He had an article of interest in Applied Physics Letters in May of 1996. The article was something about field screening in a multiple- quantum-well. Find the article and retrieve it.

7. You remember you needed a cite from a Holonyak paper, titled"Post fabrication space native- oxide improvement of the reliability of visible- spectrum AIGaAs-In(AIGa)Pp-n heterostructure diodes," you read in Applied Physics Letters. The cite was to Krames et al in 1995. Retrieve the Krames et al article.