Usability evaluation of the South London and Maudsley
NHS Trust Library web site: http://slamlibrary.xiy.net
I. Introduction/background
I.1) The South London and Maudsley NHS Trust
I have recently built and publicised a web site for my library, the South London and
Maudsley NHS Trust (SLAM) Multidisciplinary Library in Stockwell, London SW9. The
South London and Maudsley Trust is a large, geographically dispersed organisation
covering four London boroughs (Lambeth, Southwark, Lewisham and Croydon), with
over 4500 staff based at at more than a hundred sites. I have been in post as the
manager of this library service for nearly six years; I maintain extensive personal
contact with the readers via user education and support, carrying out literature
searches, etc. It is not the only library serving the trust; staff also have access to all
the King’s College libraries and to the Lewisham Hospital library; however it is the only
library directly managed by it, and hence has a pivotal role in the provision of and
coordination of information services. The staff it serves vary widely in educational
level, in professional background, and in information and computer literacy.
There is a trust intranet, which holds a great deal of basic information regarding the
organisation and its policies; all clinical teams have access to this. The library has a
very limited presence on it at present. Significantly, Web connectivity within the trust is
very poor--only about 120 PCs currently have access to the NHSnet—therefore most
staff have to depend on access externally: at home, or via NHS, academic or local
authority libraries. The library also provides a service to the local community health
trust, Community Health South London NHS Trust (CHSL), in which Web connectivity
is much better, with most clinical teams having access at their team bases. Mental
health and social services are now fully integrated within the borough of Lambeth, and
the library is contracted to provide an information service to the staff of Lambeth
Social Services. Here, while most staff have access to email, Web connectivity is
almost non-existent.
I.2) The STELLA initiative
The library until recently was part of a consortium of NHS libraries, South Thames
Libraries (STLIS). In early 1998 STLIS adopted an overarching electronic information
strategy, known as STELLA, the principal aims of which were:
ξ to deliver to health care staff electronically, at their workplace, the full range of
services and resources they were receiving in print and paper form via their
local NHS libraries
ξ to deliver new information resources hitherto unavailable because of the
limitations of print and/or CD-ROM
ξ to treat the knowledge resources of the region as a single resource accessible
to all irrespective of location, affiliation or profession.
(South Thames Libraries 1998)
6
In terms of its content, the objectives were both to enhance access to existing library
services and to extend and improve their electronic content, through, e.g., the
enhancement of access to e-journals and bibliographic databases via consortium
purchasing schemes, and the publishing of the Regional union catalogues on the web.
The development of a STLIS web site, incorporating “satellite” sites for each of the
local libraries, was seen as central to these objectives.
For library users, the site was intended to:
ξ provide them with improved contact with their library
ξ give details of information resources available to them
ξ provide a gateway to the increasing quantity of health-related material on the
web
ξ offer online information and help, e.g. subject guides, online tutorials, current
awareness bulletins, briefings on current NHS issues
ξ provide news of new developments in electronic information
ξ reduce the cost of accessing library services, especially for community and
primary care staff
ξ publicise library services
The local library web sites were initially little more than single pages, providing contact
and resource information within a standardised format.
In September 1998, the Department of Health published its information strategy for the
National Health Service, Information for Health (IfH) (DoH 1998). Much of this related
to the management of patient-identifiable information; however, significant coverage
was given to information and library services, and it was proposed to set up a National
Electronic Library for Health (NeLH) to support existing library services. IfH required
each health authority to produce a full Local Implementation Strategy (LIS) for IfH
targets and objectives. Official guidance on the scope and content of LISs, which was
published in 1999, included the so-called Annex M which covered “library services and
access to evidence” (NHS Executive IPU, 1999). STELLA then effectively
amalgamated with LIS; librarians became involved with local implementation groups in
their areas and were able, within varying degrees to pursue a library agenda within
them. The provision of clinician desktop access to sources of information on evidence-
based practice was included as a key objective of the local LIS (Lambeth, Southwark
and Lewisham Health Authority, 2000).
The staff of the Regional Library Unit at South Thames Libraries envisaged initially
that the consortium web site would be used extensively as a gateway to information by
health professionals, as well as providing a resource for library staff. Activity statistics
are not available from the site host; however, it has become apparent from informal
observation within the libraries that, other than as a means of accessing the union
catalogue, users seemed to have little interest in it; the site is being used exclusively
by library staff themselves. As time went on it proved impossible, due to staffing
difficulties, for currency to be maintained across some of its key content areas.
Overall, the site was evidently not fulfilling its purposes as originally intended.
This was one of the factors leading me to consider the desirability of putting together a
more-than-vestigial web site for my own library.
7
I.3) Health professionals’ access to and use of the web
There is by now a large body of professional literature in the mental health, community
health and social welfare fields relating to web-based resources and their importance
for the development of evidence-based practice. In psychiatry, these include: Bremer
and Beresein 2000; Huang and Alessi 1996; Huang and Alessi 1999, Kotak and Butler
2001, Kramer and Kennedy 1998, Lim 1996, Senior et al. 1997; in psychotherapy:
Tantam 2001, Laszig 2000; in nursing: Fleck and Levy 1999, Ribbons 1998, Ward and
Haines 1998; in psychology: Barak 1999; in social work: Holden et al. 2000. In
accordance with current e-government strategy, the Department of Health and other
statutory bodies use the Web extensively as a vehicle for official communications.
There are also several journals devoted to health informatics and to health information
on the Internet. One would expect all clinical staff, therefore, to have some awareness
of the web and its potential uses for locating information of professional relevance (cf.
Appendix 2).
However, a web site can obviously only be useful insofar as access to it is available.
Anecdotal evidence (conversations between library staff and readers) suggests that
many staff (estimated >50% and increasing steadily) have Internet access at home.
One frequently discovers, however, particularly with staff at lower grades, that they
have made little use of it, and have no idea how to conduct a Web search; one reader
reported to me asking her daughter to carry out searches for her on a regular basis! It
can be useful also only insofar as it fits with and relates to existing patterns of
information seeking and use among its intended users; appropriate training and
support also needs to be provided to them in respect of information sources and
services (cf. Yeoman et al. 2001).
While Web access among the group of test participants was surveyed (see below,
Table 9), it did not prove feasible to carry out a detailed survey of Web access and
use among members of the library or among staff of the local NHS trusts generally;
one has to assume, therefore, that the local situation is reasonably typical of findings
elsewhere. A number of surveys have been undertaken in recent years, worldwide,
across the UK and locally, on access to and use of the Internet by health professionals
(see Appendix 1) These have focused largely on doctors or on nurses; there is little
information available for other health professions. There have also been a number of
significant studies of information needs and use within health professions in relation to
the web (Appendix 2).
The NHS is now under instruction that 25% of clinicians should must have browser
access to NHSnet and the Internet by 31 March 2001; by March 2002 this should be
100%. (NHS Executive Information Policy Unit, 2001, page 39). Accordingly, access to
the Web will then be universal within the NHS. It remains to be seen whether this
target will be achieved. I am informed by the director of information management and
technology that, within SLAM, the position will depends on the outcome of a number of
current bids for funding to improve network infrastructure; the figure could be as low
as 10% by March 2002.
8
I.4) The library web site: origins and rationale
Palmer, in her study of information for community mental health workers (Palmer
1999), suggested the development of “signposts” to information as a method of
improving awareness and use of information resources. A signpost, she says, should
aim to function as:
ξ a guide to the availability of sources and services of learning resource, library
and information providers
ξ a way to inform people of their rights to access to these sources and services
ξ a channel through which users could be directed to the appropriate initial point
of access to resources
ξ a device to educate and develop users as independent learning resource users.
I envisaged that a full-scale library web site might be able to function as sort of
“electronic signpost” in this way, and to address a number of problems in the existing
provision of information services, by:
ξ publicising the library and its services beyond its historical user base and
immediate geographical catchment area (Lambeth and North Southwark);
ξ providing considerably enhanced access to library services and other electronic
sources from home for staff who are geographically isolated and who have
difficulty accessing conventional library services during their working day;
ξ providing a window to selected, quality-filtered information geared to the
specific needs and interests of practitioners working in mental health, and in
community and primary care, which is not available on the STLIS web site;
ξ offering an effective, customised means of access to the large variety of
electronic journals which the library provides;
ξ making available internal resources and products, such as bibliographies and
search guides (cf. Diaz 1998, Stover 1997).
No formal methodology was employed in the initial design of the SLAM library site,
which was based largely on intuition and first-hand knowledge of the intended user
group, with some reference to the web sites of similar libraries. I used Microsoft
FrontPage 2000 as my main HTML editor, as the trust had standardised on this;
Hester (1999) was my main source on its use. (This subsequently proved problematic,
both in terms of the accessibility evaluation (IV.5)--due to the non-standard nature of
Microsoft HTML (Christiansen 2000)--and of the site’s functionality. Initially I used the
FrontPage forms template and site search; when I discovered that the STLIS web host
did not support Microsoft Extensions, I was obliged to use externally-hosted services.)
drew some of the content from existing material, such as guides to services, forms,
new book announcements, and lists of web-based resources; the remainder was
created de novo (see Appendix 3).
According to Fowler (1998) “usability is the degree to which a user can successfully
learn and use a product to achieve a goal.” It is often assessed in terms of a range of
aspects: ease of learning, retention of learning over time, speed of task completion,
error rate, and subjective user satisfaction (Levi and Conrad 1998).
9
A web designer aims to create a site that is useful (enables users to achieve their
particular ends and meets their needs), easy to use (enables users move around the
site rapidly and with few errors), visually attractive, and popular (Veldof 1999). A site
should be user-centred in that it is based on knowledge of the site’s users, in particular
their technological and physical capacities, their cultural context, and their information
needs (Monash 2001).
Library web sites are complex applications “integrating access to and interaction with
a diverse set of information products and services and with people” (McGillis and
Toms 2001). These would seem to be truisms, yet the extant studies of the
development, evaluation and subsequent revision of library web sites suggest that
major problems of usability are often encountered, requiring radical restructuring of the
site. Some of the sites documented in the literature have undergone an iterative
design process involving prototyping, usability testing and rebuilding, analogous to
that of other major software projects. Normally, usability testing would be carried out
during the development of a web site as well as after it. However, it seemed
appropriate, with an initial version of the site live and available, to use the opportunity
presented by the M.Sc. project to carry out some usability testing as a means of
evaluating the appropriateness of its design and content, and establishing what
modifications might be necessary.
Matylonek (1999) identifies four typical sources of bias for the creators of web sites:
ξ discipline jargon---professional terminology not understood by users
ξ hierarchical bias---confusion of the organisation’s structure with customer
services
ξ expert proficiency: design of layout that presumes comfort in a web
environment
ξ “folk classification”: naturally preferred terminology sets among various cultures
ξ preferred and novel services: developers often use certain services and over-
emphasise them in their designs.
It is suggested by Veldof (2000) and Marmion (2001) that information on library web
sites is typically identified and structured in a librarian-orientated fashion which does
not accord with the users’ needs, preconceptions and mental maps; their organising
principle is inappropriate (Gullikson et al.1999).
The SLAM library site had originally been put together early in 2001 as an M.Sc. web
publishing project. It was officially launched in late June, being publicised to readers
via internal email, library stationery, flyers and posters, and promoted generally via
submission to medical library listings and to the main general search engines and
directories. Outside the scope of the present study, there has been no user feedback
to highlight particular problems. I anticipated when implementing the initial design that
that users might encounter problems with:
10
1) navigation using the nested menus and journal lists
2) general comprehension of library terminology (“holdings”, “classification
scheme”, “subscriptions”, etc.)
3) determining how information about library services is organised
4) determining the most appropriate resources to meet their information needs
5) distinguishing appropriately between locally served and web based sources
(free and fee-based)
6) accessing electronic journals
7) identifying journal information (current titles, print journals, electronic journals)
This project aimed, via a combination of appropriate usability testing methodologies
and a corroborative analysis of transaction logs, to answer the questions:
ξ Is the site:
o readily intelligible, i.e. not confusing to the reader
o intuitive and easy to navigate (with respect to overall structure,
navigation, labelling, searching/browsing, general features)
o visually attractive
o consistent in design and terminology?
ξ Are the readers able readily to locate information about library services?
ξ Are the readers readily able to locate through it the sources of mental health
and community health information they need? Is its scope and content, as far
as is possible, adequate and appropriate to the needs of the readership?
ξ Does the way in which information about library resources is presented accord
with the reader’s mental maps? In particular, is the division clear between
locally served and web-based resources?
Overall, my aim was to generate a set of evidence-based proposals for redesigning or
modifying the site (Guenther 2000). I sought not to focus narrowly on design and
navigation aspects of usability, but to go some way toward assessing the usefulness,
value and appropriateness of the site content in relation to the perceived role of the
library within the trust (cf. Halub 1999, and Appendix 2). One question that was
uppermost in my mind (cf. McCready 1997), particularly in relation to web resource
guides, was “how much is too much?”
Depending on one’s user population, accessibility can become an important aspect of
usability. Web accessibility is a major subject in itself; while my primary focus was
elsewhere, I attempted to bear in mind basic accessibility issues, given that the site
might be accessed by health professionals with visual impairments (e.g.
physiotherapists, dieticians) for whom the library has a legal duty under the Disability
Discrimination Act 1995 to provide information in an accessible form.
11
II. Related literature
II.1) Web site design
As I developed the site, I drew on material on design and management of web sites
for libraries and information services (Clyde 2000, Cottrell and Eisenberg 1997, Fourie
1999, Gullikson et al. 1999, Houghton 2000, Kalbach 2000, Klein 2000, McCauley
2001, McCready 1997, Marques 1999, Nicotera 1999, Orna 1992, Quintana and
Bardyn 1996, Rosic-Hristovski et al.1999, Schnell 2001, Tennant 1999), general
literature and guidelines on web site design and navigation (e.g. Degener, Eskins
2000, Farkas and Farkas 2000, Lynch and Horton 1997, Neilsen 1997, Nielsen 2000a,
Nielsen AlertBox, passim, Raymond 2001, SitePoint passim, Usability.gov (2001a),
Web Developer’s Virtual Library passim, Web Design Group, WebReview.com
passim, WebTechniques), and heuristic checklists (Davis 2000, Gaffney 1998, Instone
1997b, Kaiser 2001, Nielsen 1994a, 1994b, Tognazzini).
The NHS Web site identity guidelines (Department of Health 2000) were borne in mind
but not rigidly adhered to, as I perceived them to be too limiting (see Appendix 3).
II.2) Web site usability: methodologies
Overviews of web site usability issues are provided by Davis (2000), Head (1999), Lee
(1998) Murray and Costanzo (1999), Nielsen (1997a, 2000) and Spool (1998).
Monash (2001) provides a concise and clear overview of typically occurring usability
problems. Usability testing methodologies in general, their applicability, uses and
limitations are described by Anthony and Formidoni (2001), Campbell (1999), Fichter
(2000), Fowler (1997), Instone passim, Levi and Conrad (1998), Usability.gov.
(2001b), UserDesign.com passim and WebWord passim. A review and critique of web
heuristics is offered by de Jong and van der Geest (2000). Card sorting is described in
detail by Frederickson-Mele et al. (1998), Fuccella and Pizzolato (1998), Gaffney
(2000) IBM (2001), StepTwo (2001) and Robertson (2001). Matylonek (1999) provides
a detailed discussion of card sorting and cluster analysis as a tool for the initial design
of library web sites. Formal observation is described in detail by Dickstein and Mills
(2000). Category membership testing is described by Frederickson-Mele et al. (1998).
The use of focus groups in usability testing in general is described by Nielsen
(1997b).The library literature includes several accounts of the use of focus groups in
library service evaluation: Young (1993), Valentine (1993), Widdows et al. (1991),
Meltzer et al. (1995) and Glitz (1997).
II.3) Web site evaluation: library and information services
A number of studies have focused on evaluating the content and design of library web
sites. Whalen (1996) carried out a brief qualitative study of academic and public library
site features. Stover and Zink (1996) developed an evaluation tool for the design of
library home pages based on contemporary guidelines, and evaluated the home
pages of 40 American and Canadian higher education libraries. Clyde (1996) carried
out a content analysis of school and public and library web sites in 13 countries.
12
Quintana (1996) developed a set of content and design principles for health web sites
and used them to evaluate ten well-known services. Cottrell and Eisenberg (1997)
developed a six-factor framework for web site evaluation and organisation. Cohen and
Still (1999) carried out a comparison of the web sites of research university and two-
year college library web sites, evaluating them in terms of their provision of information
about their services and their support for reference, research and instruction, and of
particular features of their design and functionality. Misic and Johnson (1999) carried
out a comprehensive benchmarking exercise comparing web sites of business
schools. Dewey (1999) studied the findability of links on the library web pages of
members of an American university consortium. Sowards (1998), in a highly detailed
study, established a theoretical typology for library guides to web resources in terms of
their depth, organisation and design features. Osorio (2001) provides a useful
literature review and an account of a content and design evaluation carried out on 45
science and engineering libraries. The only work relating to health libraries that I have
been able to find was that of Tannery (1998) who analysed American academic
medical centre libraries, establishing a method for characterising and describing their
design and content.
II.4) Web site usability: library case studies
Although usability evaluation is an important component, together with content
analysis and web log analysis, of web site evaluation and redesign, there are relatively
few published studies of re-design projects relating to library web sites that make
extensive use of formal usability methods, and those that do all relate to large
American or Canadian academic libraries. These include Clairmont et al. (1998),
Dickstein and Mills (2000) and Veldof (1999a, 2000) at the University of Arizona,
Hennig et al. (2001a, 2001b) at MIT, McMullen (2000, 2001) at Roger Williams
University, Chisman et al. (1999) relating to the WebPAC and library web site at
Washington State University, Prown (1999) at Yale University, Cardwell and Hunker
(2000) at Bowling Green State University, France et al. (1998), of the MARIAN
experimental OPAC at Virginia Tech University, Dean et al. at University of Wisconsin-
Madison (1998, 1999), Smart et al. (1999) at the University of California, San Diego,
and Mitchell et al. (2001) at Georgia Southern and Appalachian State universities. An
important new study by McGillis and Toms (2001) of the Memorial University of
Newfoundland library web site has just been published. Veldof (1999b) offers a
general overview of usability testing of electronic library services. McCready (1997)
describes a library web site redesign at Marquette University that depended
exclusively on focus groups. There is little of specific relevance to health libraries other
than Guenther’s study (1999), which describes a medical library portal design project
that made extensive use of focus groups.
13
Ricevi informazioni sui nostri servizi, sulle offerte e non perdere news e consigli su università e lavoro.
