| Index
Site Visit and Quarterly Reports send comments or questions to: l-neuma1@uiuc.edu |
User EvaluationSocial Science Research- Social Science TeamQuarterly Report (work from January through March, 1996) During the previous quarter, the Social Science team continued its work in user needs assessment, research tool development, usability testing, targeted sociological studies, and fostering the development of a community of researchers committed to social studies of digital libraries. Specific contributions to the Digital Libraries Initiative are outlined below, as are plans for the next quarter. Particular analytic threads that link past and current Social Science team work include:
Needs Assessment In order to explore the user context for federated repositories of scientific literature, the Social Science team began an in-depth study of researchers in Physics. Beginning in late January, semi- structured interviews were conducted with five people belonging to a small campus research group working in the area of general relativity and the behavior of black holes. The physicists were asked to describe their work flows and practices, how they find information, why they need information, and how they use libraries and computer-based tools. They also described in detail one to two recent incidents in which they sought and used journal material; these data are being analyzed to develop a more specific picture of, on the one hand, the clues researchers bring to library systems and, on the other, the cues in documents and retrieval systems that researchers need to complete their searches. Research Tool Development The Social Science team has completed the development and pretesting of an online user registration form that will be used to limit testbed access to legitimate users and to collect demographic data from them (see http://anshar.grainger.uiuc.edu/dlisoc/rfpt.html). Also completed this quarter was the specification of the format for instrumenting the testbed to log all usage transactions that occur. The Social Science and Testbed teams are currently working together to integrate the registration form and instrumentation programming into the testbed itself. In addition, a survey instrument for collecting user needs data from additional members of the Physics research community at UIUC has been designed. Usability Testing In February, the Social Science team conducted cooperative evaluation sessions of the DLI prototype with ten Physics researchers. This series of usability tests uncovered 31 specific problems with prototype functionality, interface design, and ease of use. A summary report was prepared for and discussed with the Testbed team and a number of the user-identified problems have already been corrected. This first step also allowed the Social Science team to begin work to design strategies for collecting user feedback aimed specifically at the design and evaluation of mechanisms for user support. Targeted Sociological Studies Laura Neumann and Leigh Star have written a paper on the process of building infrastructure, based on their earlier study of DLI development at the University of Illinois, which they plan to submit for the upcoming Participatory Design conference. Star has also completed an informal prospectus for sociological studies that the Social Science team would like to undertake in the future, perhaps in collaboration with other researchers. This set of studies will focus on how people use digital material in their workplaces, what difference it makes to the work organization, and implications of "folk classifications" for creating and maintaining collections of digital document representations. Neumann has begun a targeted study of the information flows of the Physics research team by following up the interviews described above with ongoing observation at the Relavitity group's student office. Development of a Research Community The Social Science team has continued its commitment to fostering the development of a community of researchers committed to social studies of digital libraries. To encourage the sharing of knowledge across the six DLI projects during the upcoming synchronization meeting in Ann Arbor, Ann Bishop is organizing a preconference seminar, a plenary session, a working group breakout session, and a demo/discussion session in which all DLI social science researchers have been invited to participate. Bishop also helped to organize a workshop on User Needs Assessment at the ACM Conference on Digital Libraries in March 1996; Star served as a leader for this workshop, delivering the session on ethnography. Bishop submitted a proposal to NSF to sponsor the second Allerton Institute on user- centered design and evaluation in digital libraries, to be held in October 1996. Star and other members of the Social Science team have continued their work in planning for this conference, which will provide an interdisciplinary forum for researchers engaged in social studies of digital libraries. Star attended an NSF-Sponsored invitational workshop at the UCLA Department of Library and Information sciences in February, where she presented a discussion paper and worked on a national agenda for research in social contexts and impacts of digital libraries. Star and Bishop began working with Nancy Van House (UCLA) and Barbara Buttenfield (Santa Barbara) on the development of a monograph which will be devoted to user needs assessment for digital libraries. Plans for the Next Quarter During the next quarter, the Social Science team will conduct usability tests of the next version of the DLI prototype, including the online documentation portion, which is currently under development. The online user registration forms and transaction logs should become operational in the next quarter, allowing preliminary analysis of the resulting user and usage data. In addition, a new type of system instrumentation will be added: targeted surveys and user comment buttons. Analysis of user needs will continue with the study of user interaction logs, which are being manually maintained by librarians to record their interactions with patrons using the DLI testbed in the Grainger Library. A user needs survey for Physics researchers that will serve as a follow-up to the interviews conducted with members of the Relativity research group will be conducted; targeted study of the Relativity group, through ethnographic observations, will also continue. Finally, the Social Science team will continue planning for the 1996 Allerton Institute and for initiating a new set of sociological studies related to the organization and use of digital material in the workplace. Publications Bishop, Ann P. (Guest Editor). SIGOIS Bulletin, Special Issue on Digital Libraries, Volume 16, Number 2 (December 1995), 2-44. Ignacio, Emily, et al. Usability Report, February 1996. [http://anshar.grainger.uiuc.edu/dlisoc/usability.rept.html] Star, Susan Leigh. (Staff editor). Special double issue of Computer- Supported Cooperative Work on Computer Mediated Communication and Small Groups with guest editors Joseph McGrath and Holly Arrow. Appearing in Spring 1996. Presentations Bishop, Ann P. "Engineers and Scientists on the Net" [Presentation to the Manufacturing 2002 Colloquium Series sponsored by the University of Texas at Austin's Engineering School, Austin, Texas, March 1996]. Star, Susan Leigh. "Problems of Learning and Communication in Establishing an Information System for Scientists." [Colloquium presentation to Education Policy Studies, College of Education, UIUC] Star, Susan Leigh. "Slouchng toward Infrastructure." [Discussion paper presented at the NSF-Sponsored invitational workshop on Social Impacts of Digital Libraries at the UCLA Department of Library and Information Sciences, Feb. 1996] Star, Susan Leigh."User Needs Assessment: Ethnography." [Session presented at the User Needs Assessment Workshop at the ACM Conference on Digital Libraries, Bethesda, Maryland, March 1996]. Star, Susan Leigh. "To Classify is Human." [Keynote address, ACM Hypertext '96, University of Maryland, Bethesda, March 1996]. |