Diary

Internal Reports

Completed Papers

Papers in Progress

Conference Presentations

Site Visit and Quarterly Reports

Main DLI Page

Web Client- DeLIver

User Evaluation

Social Science Research- Social Science Team

Quarterly Report

(work from July -October 1997)

Overall Activities

During the past quarter, the Social Science Team has been involved in the iterative design and testing of DeLIver; new analyses of data collected earlier in the project; the design and implementation of user registration, authentication, and transaction logging procedures for DeLIver; the development of a summative user survey instrument; ongoing research related to work practices in the digital era; and the redesign of the Teams web pages.

Specific Activities

Team members re-worked the registration form that all public users are required to fill out before using DeLIVer. Because people expect the web to operate on an open and, largely, anonymous basis, crafting a user registration form that would collect demographic data as non-intrusively and accurately as possible has become a delicate balancing act. Team members also worked with the Testbed Team to develop user authentication procedures for DeLIver and to set up analysis and reporting procedures for DeLIver transaction logs.

The Social Science Team implemented an online usability survey meant to provide user feedback during the trial rollout period from August to October. When the user pushed the submit survey button, email with the results of the survey was sent directly to the team member responsible for analysis. The email message held an attached MIME file with the survey results formatted for direct import into a database. Only several DeLIver users submitted surveys. We do not know how many people aside from DLI project members used DeLIver during this trial period, so we have no way of estimating a survey response rate. Nonetheless, we feel that we will need to be more inventive and careful if we expect to get good compliance with optional user surveys from our anonymous user base in the future.

The Social Science Team conducted usability tests of DeLIver, and participated in a radical redesign of the systems interface based on these results. The usability test represented a new approach -- subjects were instructed to use DeLIver on their own, in the course of their work over a two-week period. They kept a log of their impressions the first time they used the system, and participated in a group interview to discuss and demonstrate system strengths and weaknesses.

In addition, two researchers have begun their own user studies as independent affiliates of the DLI Social Science Team. Michael Twidale, a professor in the Graduate School of Library and Information Science and an acknowledged leader in the study of collaboration in online system use, has initiated a study that draws students from courses in which DeLIver has been demonstrated. Students are invited to sign up for individual help sessions with Twidale, during which they try using the system to retrieve literature for some current assignment. These usability sessions reveal basic problems with DeLIver, which are forwarded to DeLIver designers, but they will also support Twidale's own development of new interfaces that include visualizations of search. Najmuddin Shaik, a PhD student in Educational Psychology who previously has worked on evaluations of the Museum Educational Site Licensing (MESL) project, has begun designing a series of usability tests for the DLI Iodyne client. The aim of this study is to do a close task analysis of the search process in Iodyne in order to produce a new mock interface that explicitly represents typical search paths in the system.

Team members developed preliminary drafts of user survey instruments that will provide summative data on the extent and nature of testbed use, the level of user satisfaction with the system, and reasons for use and nonuse. The complexities of developing a reliable survey instrument for a system that is in flux and designing survey delivery mechanisms that will encourage response are proving challenging.

Finally, Social Science Team members continue work on their studies of work practices and the changing nature of digital information infrastructure. These include research on the use of document components, the organization of office workspaces, the manner in which people deal with new computer-based information systems, and information convergence.

Presentations

Bishop, Ann. "DLI Social Science Research," Fall Program for the Chicago chapter of the American Society for Information Science, University of Illinois at Urbana-Champaign, Sept. 27, 1997.

Sandusky, Robert. Organized a panel on DLI user studies for the Mid-Year Meeting of the American Society for Information Science.

Grants/ Awards

Ann Bishop received the Centennial Scholar award from the Graduate School of Library and Information Science at the University of Illinois. The award provides $4000 for continuing her study of document disaggregation and the use of journal components.