The requirement is originally defined as:
"Blankchester is a cathedral city in the centre
of England. It is an historical
city with tourists visiting but also has modern industry. As part of the
"Blankchester - a New Millennium" project, Blankchester City Council
have commissioned a web site to raise the profile of Blankchester. The project
will also contribute to the "Blankchester - access to all" equal
opportunities project. It is anticipated that it will be used by tourists and
by residents for information about Blankchester, and by local tourist
information officers for finding information when answering telephone
enquiries. The web site may be accessed via Internet browsers, or for local
authority employees via the proposed Intranet. In the Intranet the local
authority employees will have access to additional information and pages. The
site may be used simply to browse information and additionally to make
bookings for tourist accommodation and events. The website shall be designed
with reference to the requirements of the Disability Discrimination Act, and
shall therefore comply to WAI and RNIB guidelines."
A particular difficulty with setting requirements for a
web site is correctly identifying the tasks. The only safe way is to find out
what users actually want to do with the site (perhaps by investigating how
people use other similar sites, or by user-based evaluation of very early
paper prototypes). The IT
department visits two other Tourist Offices, at Dashmouth-on-Sea and Cipher
Hills, to see how the tourist officers used the site and to look at their site
feedback. This information is fed into the validation and review
process. The validation and review of these requirements gives rise to a
context checklist, part of which is shown below:
Figure 1
context checklist:
|
Blankchester
City web site - Context Checklist
|
|
Goals for this
system: raise the profile of Blankchester, provide access to
information ("access to all" campaign), provide booking system
for accommodation and events in the City.
|
|
Users
|
|
|
|
|
Type
|
Tourist
|
Resident
|
Tourist
Information officer
|
|
Skills and knowledge
No
/ low / good Web/IT
knowledge
No
/ low / good Local knowledge
No
/ low/high previous
experience of Hotel booking
|
ü
ü
ü
|
ü
ü
ü
|
ü
ü
ü
|
|
Attributes (people)
with
sight / hearing / language / mobility
difficulties /
with
other special needs / without special needs
|
ü
|
ü
|
ü
|
|
Smoking
/ non smoking
With
children / without children
With
pets / without pets
|
ü
ü
ü
|
ü
ü
ü
|
ü
ü
ü
|
|
List of tasks
Browsing
general information / tourist attractions / events
Hotel/accommodation/events
searches and bookings
Browse
additional intranet pages
(for
space in the example this is not expanded)
|
ü
ü
N/A
|
ü
ü
N/A
|
ü
ü
ü
|
|
Task checklist
|
Browsing
|
Searches
|
Bookings
|
|
Goal
User
Blankchester
City
|
Finds information
Profile raised
|
Finds information
More rooms let
|
Room
/ event ticket
More
rooms let
Larger
attendance at events
|
|
Output
|
List of general
information meeting personal criteria
|
List of hotels
meeting personal criteria
|
Booked and confirmed
accommodation
|
|
Frequency
|
5000
hits per day
|
5000
hits per day
|
500
per month
|
|
Duration
|
4
clicks max traverse.
|
Hotel
search 1 click from home
page
|
book
hotel less than 1 minute from selection
|
|
Environment
in which the software/system is to be used
|
Tourist
office - Tourist Information Officer's PC - IE5
Tourist
office / library public PC IE5
Other
user - could be a range of browsers - test for IE5 + IE4 + Netscape4
Design
note: ensure text only version of the site available.
(Note: this is a newly identified requirement and must be fed
back to the designers)
|
| |
|
|
|
|
|
Note: A context checklist
may also consider side effects, flexibility, physical/mental demands,
dependencies, safety, and criticality.
The requirements, context checklist and paper
layout of the screens is inspected, using heuristic evaluation checklists. The
context checklist is then used to draw up tests using equivalence partitioning
and boundary value analysis for users against environment.
Figure 2
Partitions
The user partitions are:
1.
Tourist
2.
Resident
3.
Tourist officer
4.
No / low Web/IT knowledge
5.
No / low Local knowledge
6.
No / low previous experience of Hotel booking
7.
good Web/IT knowledge
8.
good Local knowledge
9.
previous experience of Hotel booking
10.
with sight difficulties
11.
without sight difficulties
12.
with hearing difficulties
13.
without hearing difficulties
14.
with language difficulties
15.
without language difficulties
16.
with mobility difficulties
17.
without mobility difficulties
18.
with other special needs
19.
without other special needs
20.
smoker
21.
non smoker
22.
with children
23.
without children
24.
with pet
25.
without pet
The environment partitions are:
26.
Tourist office - TI's PC - IE5
27.
Library public PC IE5
28.
User's own equipment - IE 5
29.
User's own equipment - IE 4
30.
User's own equipment - Netscape 4
31.
Tourist officer uninterrupted
32.
Tourist officer also answering phone enquiries
The tasks will also be
partitioned, from the context checklist, but for ease of explanation we will
consider only one task: hotel booking. BS
7925-2 says that more than one partition can be checked in a single test case,
so we combine the partitions to make a smaller number of tests.
The table below shows 7 tests:
Figure
3 Action/task list – test cases
|
ACTION/TASK:
Book a hotel room
|
|
Test case
|
New Partitions
tested
|
user type / skill /
Attribute (people) / environment
|
Actions
|
Expected outcome
|
|
1
|
1, 4, 5, 14, 16, 20,
23, 24, 27
|
A tourist with sight
difficulties and a guide dog, little web experience, and no local
knowledge, no children, uses the library public PC. She wants a smoking
room. English is not her first language.
|
Browses for a 4 star
hotel and books it
|
Able to book a room
(smokers) at the 4 star Jupiter Hotel, which welcomes pets and guide dogs.
Site / PC has tools to offer information in audio and in languages other
than English
|
|
2
|
2, 6, 7, 8,
10, 11, 13, 15,17,19, 30
|
A local person with
good local knowledge and good IT /web skills but no experience of hotel
booking uses their own PC (Netscape 4) to book hotel rooms for a visiting
party of friends. No Special needs
|
Browses for hotel in
the £80-100 per night range and books hotel
|
Able to find and book
Mars hotel, able to find and print map to Mars hotel
|
|
3
|
3, 9, 26, 32
|
A tourist officer with
little web experience, good local knowledge, previous experience of hotel
booking, no special needs, at own PC, takes enquiry and booking from
tourist in office, while being interrupted by phone calls
|
Browses for hotel in
the £40-60 per night range and books hotel
|
Able to find and book
Saturn hotel, able to answer additional questions, provide map to Saturn
hotel. No timeouts.
|
|
4
|
12, 28
|
A local person with
good local knowledge and good IT /web skills but no experience of hotel
booking uses their own PC to book hotel rooms for a visiting party of
friends. Hearing difficulties, own PC (IE5)
|
Browses for hotel in
the £80-100 per night range and books hotel
|
Able to find and book
Mars hotel, able to find and print map to Mars hotel. Major information given visually.
|
|
5
|
16
|
A local person with
good local knowledge and good IT /web skills but no experience of hotel
booking. Person with mobility difficulties (wheelchair and arthritis in
hands and wrists) Uses library PC to book a hotel room for a friend
|
Browses for hotel in
the £80-100 per night range and books hotel
|
Able to find and book
Gaia hotel, able to find and print map to Gaia hotel. Height and angle of
PC suits needs. Keyboard and pointer device easy to handle and control
without pain.
|
|
6
|
31
|
A tourist officer with
little web experience, good local knowledge, previous experience of hotel
booking, no special needs, at own PC, takes enquiry and booking from
tourist in office, without being interrupted by phone calls
|
Browses for hotel in
the £40-60 per night range and books hotel
|
Able to find and book
Saturn hotel, able to answer additional questions, provide map to Saturn
hotel.
|
|
7
|
18, 21, 22, 25, 29
|
A tourist whose child
has an allergy to animals, no pets, little web experience, and no local
knowledge, her own PC (IE4) to book a no-smoking room
|
Browses for a 4 star
hotel and books it
|
Able to book a
no-smokers room for self and child at the 4 star Cathedral Hotel which has
a no pets policy and check for allergy aware hotels
|
|
Scenario
name:
|
Enquiry
for hotel
|
|
System:
|
Blankchester
web site
|
V0.9
|
Date:
12-1-2000
|
|
Goal
of test
|
The
goal of this use case is to test the basic enquiry job, where a
(potential) tourist is searching for accommodation
|
|
|
Description
|
|
Context
|
This
is core functionality, so this is an important test of
a frequent usage function
|
|
Actors
|
Any
user may use this function :
Tourist
- low web user, Tourist - high web user, BCC Tourist officer (trained),
Local person - low web user, Local person - high web user
|
|
|
|
Checked
|
Comments
|
|
Preconditions
|
The
PC is on and functional
The
user is connected via their normal ISP
The
user is online and looking at the web site home page and has chosen to use
the drop down selections
|
|
|
|
Description of main
flow
|
2
star in walking distance of cathedral
1.
the user clicks on Hotels
2.
the user clicks on 2 star
3.
the user clicks on location - cathedral
4.
the user clicks on Find
|
|
|
|
Expected
results
|
the
system displays 2 hotels details - "Wardens House" and
"Deans House" and the user can print or save the information
|
|
|
|
Alternative
flows
|
The
user wants to see 3 star hotels - all in Blankchester
1.
the user clicks on Hotels
2.
the user clicks on 3 star
3.
the user clicks on find
|
|
|
|
Expected
results
|
the
system displays a drop down list with all 3 star hotels (15 on test
database)
|
|
|
|
Failure
conditions
|
No
booking conformation number displayed
Wrong
rating hotels displayed
|
|
|
|
Additional
attention points
|
Note
response times
|
|
|
With the constraints on local authority budgets, it is
decided not to use a full usability lab test, but instead to use the
"Thinking Aloud" protocol with an observer logging what the user does,
using the use cases and selecting the user group from the equivalence
partitioning tests already defined. After
the tests are run, the users will be asked to complete a SUMI questionnaire.
Representative users will be asked to complete typical tasks, and
measures taken of effectiveness, efficiency and satisfaction.
It was expected that all Tourist Office users could successfully complete
a hotel booking in an average time of less than 5 minutes. All SUMI scores should be above the industry average of 50.
Context used for the Tourist Office test: For this
part of the test, eight typical users from the Tourist Office were selected who
had the key characteristics and capabilities, but no previous experience of the
web site. The other characteristics
of the participants that might influence usability were recorded, together with
the age group and gender (see Figure 5 on page 4):
Figure 5 User group characteristics
*1=prefer
to use a computer as much as possible, 7=prefer to use a computer as little as
possible
The intended context of use for this test was the
Tourist Office, and so the tests will be carried out there, but it is arranged
so that participants could work alone without any interruptions. They are to be
observed by an observer/tester. Rules for carrying out the tests are agreed:
participants would be shown the evaluation suite, informed that their
interaction would be recorded, they would be asked to sign a release form, asked
to confirm the information they had provided about, and to score their attitude
towards use of computers. Tasks
were to be timed using a stopwatch. Sessions
would not be videotaped but the user asked to use "Thinking aloud" so
that the observer/tester could take notes.
At the end of the sessions, participants would complete the SUMI
satisfaction questionnaire.
Instructions for user testers woul