How can we best use landmarks to support older people in navigation?

Joy Goodman, Stephen A. Brewster and Philip Gray

Glasgow Interactive Systems Group
Department of Computing Science
University of Glasgow
GLASGOW G12 8RZ, UK
Tel: +44 (0)141 330 3541 / Fax: +44 (0)141 330 4913
{joy, stephen, pdg}@dcs.gla.ac.uk
www.dcs.gla.ac.uk/utopia

Abstract

Although landmarks are an integral part of navigation, they have rarely been used explicitly within electronic pedestrian navigation aids. We describe a two-part study into the use of landmarks in such aids, using a set of field experiments. The first part investigated whether such devices can be effective for older adults (over 60 years old), who might particularly benefit from them due to declines in sensory, cognitive and motor abilities. The second part compared the effectiveness of different methods of presenting landmark information. We show that a pedestrian navigation aid based around landmarks is particularly useful for older people and demonstrate that text, speech and photographs are all effective ways of presenting landmark information, although speech on its own has some drawbacks. We found that different people prefer information to be presented in different modalities, indicating a need for personalisation, although multimodality may also help to address this issue.

1. Introduction
The proportion of older people in developed countries is rapidly increasing. According to estimates from the US Census Bureau’s International Database (2004), the proportion of those in the UK who are over 60 is expected to increase from 20% in the year 2000 to 27% by 2025. There is therefore an urgent need to provide greater support for this section of the population in new and innovative ways. Support is needed not only for basic needs and activities of daily living but also for aspects of life that enhance independence and quality of life. One of these aspects is the ability to stay mobile, which can play an important part in maintaining social connectivity, accessing local facilities and staying independent.

In particular, navigation is an important activity, key for maintaining mobility and independence. However, many older people find increasing difficulties with it due to declines in their cognitive, perceptual and motor abilities (Kirasic 2000). Hunt and Waller (1999) note that spatial disorientation is frequently observed in individuals aged 70 and over who show no other sign of mental deterioration and Wilkniss et al (1997) showed that older people commonly have greater difficulty retracing routes and memorising maps. They also have greater difficulty maintaining extrapersonal orientation (i.e. spatial orientation with respect to external objects) (Aubrey and Dobbs 1989), and in making distance and direction judgements about novel environments (Kirasic 1991). Difficulties can also arise as sensory difficulties make it harder to perceive information from the surrounding environment. Physical difficulties can also cause problems as older people have to concentrate harder on physical aspects of mobility and have less energy and resources to spare for finding their way around.

Navigation is therefore one area where support could make a positive difference to many older people’s lives. Technology offers one way of providing such support. The increase in use of mobile phones and other handheld computers, coupled with the increase in accuracy of location-sensing technology, such as GPS (Global Positioning System), has led to the production of a variety of computerised navigation guides and aids (e.g., Garmin 2004).

However, the design of current pedestrian navigation aids does not usually take the needs of the older user into consideration. The requirements of someone with reduced sensory and cognitive abilities have not been adequately addressed. For example, mobile devices often have small displays and buttons and complicated menu structures. In addition, it is particularly important that navigation aids for older people place minimum cognitive demands on the user to avoid causing confusion and distraction. Yet, such aids often use methods of navigation that do not match those in common use and they do not provide the types of navigation information and cues that people usually use.

May et al (2003) investigated the navigation information needed by pedestrians in a town centre and found that landmarks formed by far the most popular cue type. Other studies by Bradley and Dunlop (2002) and Burns (1997) have investigated the changes in navigation cues with age and found that landmarks still form a key part of the cue set for older people, although there were some indications that their importance may decline slightly with age. This also agrees with seminal work by Lynch (1960) on cognitive representations of city environments, which listed landmarks as one of the key elements of such representations.

However, landmarks are not used widely in navigation aids. Current navigation aids usually support navigation by guiding the user along a given route, using turn-by-turn arrow-based directions, or by presenting maps (see, for example, Garmin 2004). Some research projects have tried other methods, such as overlaying information on a detailed first-person view of an area (e.g. Piekarski et al 1999, Laakso et al 2003) and describing a route through a sequence of video clips (Tyler et al 2001). These methods allow reference to specific environmental information, such as landmarks, but they do not use landmarks explicitly. Other systems, designed for the visually impaired, provide spoken information that includes descriptions of landmarks (e.g. Hine et al 2000, Loomis 2001). However, this information is designed to orient users and give them an idea of their surroundings rather than to help them navigate per se. It seems that the use of landmarks in navigation aids and the ways in which they are best presented have rarely been examined explicitly.

There are, however, some exceptions. Burnett (2000) has shown that the use of landmarks in vehicle navigation systems can greatly improve their effectiveness. More recently, Ross et al (2004) carried out a preliminary study investigating the use of landmarks in pedestrian navigation aids, simulating a navigation aid by presenting navigation instructions on flip cards. They demonstrated that the inclusion of landmarks does raise user confidence and reduce errors.

This paper builds on such work by examining the role of age and of different ways of presenting navigation information on the effectiveness of landmark-based systems. To do this, it uses a set of experiments in the field using an operational prototype of a navigation aid. Theoretically, from the above discussion, it would appear that older people would particularly benefit from the inclusion of landmark information and the work described in this paper examines this hypothesis. It further investigates how this information can be best presented to older people in view of reduced sight and hearing, by examining different modalities. Some of the results from the first part of this study were previously presented in (Goodman et al 2004b).

Section 2 of this paper describes the design of the navigation aid and the different versions of its user interface. It also discusses how the needs of the older population were considered in the development of its design. The use of the prototype to investigate the effectiveness of landmarks for different age groups is discussed in Sections 3 to 5, which describe the experimental hypotheses and methods involved in the first part of the study and the results obtained from it. Sections 6 to 8 then describe the second part of the study, focusing on different methods of presenting landmark information. The results as a whole are discussed in Section 9.

2. Design of the device

2.1 Requirements gathering
The design of the navigation aid was informed by preliminary requirements gathering in the form of focus groups with older people (Goodman et al 2004a). Focus groups were chosen to obtain the opinions of a variety of people in a relatively short space of time and to receive feedback on specific suggestions for device design. In (Goodman et al 2004a) we discuss how the methods used in the focus groups were carefully chosen to overcome the disadvantages of using stationary focus groups to investigate mobile settings. These methods included the use of drawings, photographs and slide presentations to illustrate locations and routes and to help participants to imagine scenarios; discussion sessions to obtain in-depth responses; and specific tasks, such as giving directions to each other and choosing a preferred version of a navigation instruction, to focus the discussion and obtain feedback on specific ideas.

The design was particularly influenced by the results of one final focus group on navigation involving seven participants over the age of 60. Among other results from this group, we observed a generally positive response to the idea of a navigation aid. Some, though not all, of the participants reported difficulties finding their way around, indicating a need for more assistance in navigation. In navigation instructions, participants appreciated information about landmarks and liked to be given a visual indication of what these landmarks looked like. There was also a positive response to the idea of presenting navigation information through audio, provided that it was in an easily understood form, such as speech.

2.2 Basic design
Due to the importance of landmarks in navigation in general and the responses of participants in focus groups, we based the design of the navigation aid around the concept of landmarks. The aid guides the user along a route by presenting a sequence of descriptions of landmarks. At each point, the user should head towards the location described and then press a button labelled “Next” or “Next Image” to receive the next description. The descriptions themselves are explained in more detail in the following subsection. The initial version of the device used photographs of the landmarks in order to give the visual indication preferred by focus group participants. However, versions using text and speech were also developed to investigate the contribution of different modalities towards the effectiveness of the device. Figure 1 illustrates the device in use in practice.

Figure 1. Participants using the device during an experiment.

The application was written in C# and deployed on a handheld computer (i.e. PDA). It does not use GPS or any other positioning technology due to their unreliability in built-up environments and in order to keep the application simple. Instead, location is determined by the user’s interaction with the application. The interface and its different versions are described in more detail in the following subsection.

Ultimately, we expect that such an aid would be incorporated into a larger system, allowing users to explore an area and select different start and end points, adapting routes to different personal requirements and coping gracefully when users wander away from the route. It could also be included as part of a larger system that provides, for example, information about the surroundings and places of interest. However, this study focused on the core aspect of such a device (the navigation aid itself) and the methods by which navigation assistance can be best provided.

2.3 Different versions of the user interface
Four different versions of the user interface were created, using different modalities to present information about landmarks, so that these could be compared against each other. This subsection describes each of these versions in turn.

2.3.1 Images, text and speech
The initial version of the device was designed to test the basic concept of using landmarks in a navigation aid and so it provided landmark information in several modalities – photographs, text and speech. This was done so that participants’ reactions to the use of landmarks would not be determined by any one particular method of presentation.

An example screen from the initial version of the device is shown in figure 2(i). It displays a photograph (58x43mm) of a landmark that can be seen from the start of the route. Once the user reaches that location, he or she should press the button labelled “Next Image” to progress to the next instruction and receive a photograph of a new landmark or location to head towards.

As well as the photograph, a brief text instruction is shown and a longer speech instruction can be heard when the “Audio” button is pressed. For example, the speech instruction for the screen in Figure 2(i) is “You’ll see a large gateway directly ahead of you. Please go through it and turn left at the end.” Audio is presented using the device’s built-in speaker. The “Restart” button returns the user to first screen at the beginning of the route. The “View Map” button shows a simplified map of the route as shown in Figure 2(ii). This was included to give the user an overall impression of the route and of the contextual position of the current landmark. This position is marked on the map with a red dot (circled in Figure 2(ii) and dark grey in the greyscale version).

2.3.2 Later versions of the interface
The first version of the device was used to test the feasibility of using landmarks in navigation aids. After it was evaluated (as described in Section 3), three further versions of the interface were developed to investigate and compare different methods of presenting the information about landmarks. These versions were based on the initial version described above but they focused on providing landmark information through text and speech, in order to compare these methods with each other and with the initial version.

The design of these versions also benefited from our experiences in testing the initial version of the device. Some sample screens can be seen in figure 4. The map was removed due to lack of use and to remove extra variables from the experiment. In addition, the “Restart” button was replaced with a “Previous” button that returned to the previous instruction. This allowed easier reversal of actions and responded to several direct requests by users for a “back” or “previous” button. As these versions focused on providing information through text and speech, the photograph was removed. This made extra screen space available, allowing the buttons and fonts to be increased in size.

In addition, the audio instructions were provided through a single earphone worn over or in one ear rather than through the device’s speaker. This was because later versions of the handheld computer contained a speaker with lower volume and audio quality. A single earphone was used so that only one ear was covered and users could still hear noises from the surrounding environment. Participants were offered a choice between an earphone worn over the ear, as pictured in figure 3, or one that fitted into the ear.

Three different versions of the interface were created, using different modalities in order to compare their effectiveness in conveying navigation information. One version provided information in both written and spoken form, while one provided written instructions only and the last provided only spoken instructions.

2.3.3 Text and speech
An example screen from the text and speech version is shown in Figure 4(i). A written instruction occupies the top half of the screen. This instruction is longer than the brief written instruction in the initial version of the device and is exactly the same as the spoken version. As before, when the user completes this instruction, he or she should press the button labelled “Next” to receive the next instruction. When this button is pressed, the next written instruction is shown and the spoken version of this instruction is played automatically. Similarly, when the button labelled “Previous” is pressed, the previous written instruction is shown and its spoken version played. The speech version of the current instruction is also played when the “Start Speech” button is pressed and any audio being played is stopped on activation of the button labelled “Stop Speech”.

2.3.3 Text only
An example screen from the text only interface is shown in figure 4(ii). This interface operates in the same way as the text and speech version except without the audio. Spoken instructions are not played automatically when the “Next” or “Previous” buttons are pressed and no buttons are provided for requesting or stopping the speech.

2.3.4 Speech only
Similarly, a sample screen from the speech only version is shown in figure 4(iii). This interface operates in a similar way to the text and speech version except that no written instructions are displayed. Otherwise, each button works as before.

2.4 Taking older people’s needs into consideration
These interfaces were designed with guidelines for the design of desktop applications for older adults in mind (e.g. Hawthorn 2000, Fisk et al 2004). These guidelines were used because little work on mobile design guidelines for older people has been carried out. Although these guidelines had to be adapted to take account of the limited screen size and different interaction techniques on a mobile device, they provided a useful starting point for the design.

For example, the design was kept simple. Clutter, unnecessary features and irrelevant information were avoided as they can distract attention and increase confusion especially in older people (Zec 1995). This is particularly important in a mobile device with a limited screen size on which clutter is more apparent and easier to create. In a fully functional system, more attention could be paid to enhancing the aesthetic appeal of the device. However, this must not be done at the cost of compromising the device’s usability by introducing distracting and unnecessary screen elements.

Scrollbars were avoided by sizing images and text to fit into a single screen. Scrollbars can be difficult to use for people with low dexterity (Hawthorn 2000) and this effect is likely to be even more significant on handheld devices where scrollbars must be manipulated by selecting and moving tiny screen elements while on the move. There are also some indications that scrolling can cause confusion to older people because the partial view of the screen must be related to the whole text or graphic and because the act of scrolling can disrupt the user’s attention (Hawthorn 2000).

Simple text descriptions rather than icons were used on the navigation buttons as many older people are not familiar with standard computing icons. For example, McGee et al (2003) found that users of their system (many of whom were older adults) did not attempt to use “Back” and “Next” navigational arrows until these were annotated with text descriptions.

The text size used in the interface was also chosen carefully. A slightly smaller size than that recommended by Hawthorn (2000) and Fisk et al (2004) (11pt instead of 12pt) had to be used due to the limited screen size available. To compensate for this, a bold typeface was used. A sans-serif font was chosen as such fonts tend to be easier for older people to read (Hawthorn 2000).

The audio instructions were spoken using a natural male voice, in line with findings from Lines and Hone (2002), who found that natural voices were easier for older people to understand than synthetic ones and male than female. A fairly neutral educated Scottish accent was used as the device was to be tested in Scotland.

3. First phase of the experiment
The four versions of the device described in the previous section were designed to be used to investigate the effectiveness and presentation of landmarks in navigation aids. This section describes how the first version was used to examine the basic effectiveness of a landmark-based aid and the role of age in its use. The investigation of different methods of presenting information using the remaining versions of the device is described in Section 6.

3.1 Hypotheses
First of all, it is important to ascertain whether a landmark-based pedestrian navigation aid can be effective in aiding navigation. We were particularly interested in whether it is effective for the older population at which it is targeted. The first part of the experiment focused on these issues, with the following hypotheses:

1. An electronic pedestrian navigation aid based on landmarks can be used effectively by older people, improving performance (measured by the time taken and the number of errors), over that obtained by using a paper map.
   

2. Such an aid is more effective for older than for younger people.

3.2 Participants
The initial version of the navigation aid was tested with 32 able-bodied users. In order to examine age differences in its use, these were divided into two groups: 16 of the participants were aged between 63 and 77 and 16 between 19 and 34. Each group was balanced with respect to gender.

In addition, four “backup” participants were recruited and run through the experiment in case any of the data from the main participants was confounded by large changes in the external environment such as changes in weather conditions. Only data from one backup participant was actually used. This was because one of the main participants arrived late and so did part of the experiment in the dark. To avoid over-familiarity with the area (part of the campus of the University of Glasgow), no participant was either a student or staff member at that university.

All but one of the older participants had never used a hand-held computer before and the remaining participant had only used one a few times. The majority of younger participants (11 out of 16) had also never used a hand-held computer and only one was a regular user. All participants had used a map before with 10 of the younger participants and 11 of the older participants rating themselves as regular map users.

3.3 Field experiments
The hypotheses were investigated using a set of field experiments, or quantitative experimental evaluations in the field. The advantages of this method and ways of carrying it out are discussed in Goodman et al (2004c). In this case, field studies were necessary because a navigation aid is highly dependent upon the surrounding environment and cannot be tested realistically in a laboratory setting. An experimental setup was used to obtain a quantitative comparison of our device against a paper map and of usage by different age groups.

We choose to use an experimental evaluation rather than an ethnographic field study to focus on a single aspect of the device (the aiding of navigation) and to obtain an objective evaluation of the performance of different methods and groups of users.

However, field experiments do present the experimenter with several challenges, primarily that of limiting the effect of possibly confounding variables, such as light and noise levels, weather conditions and traffic. When the levels of such variables could not be kept consistent, their effects on results were reduced by varying them across conditions. We believe this to be an acceptable approach because variation in such variables is an integral part of real-world usage. Removing all variation would therefore produce unrealistic results. Using real locations and realistic environmental conditions gives real data on how the device is used in practice, and the advantages outweigh the difficulties and make the extra effort worthwhile.

3.4 Experiment design
All participants were asked to navigate along two distinct (but similar) routes, one of them using the device and the other using the standard paper map for the area. The order of the two routes and the two methods were counterbalanced, creating four conditions. Equal numbers from each age group and gender were assigned to each condition.

The experiment was conducted on the campus of the University of Glasgow, a common tourist destination within the city of Glasgow in Scotland. This location was chosen because it has a large number of junctions and decision points in a small area, allowing a sufficiently complicated route to be tested while limiting the length of the routes (and therefore of the experiment) to avoid tiring participants, particularly the older ones. It also has a low volume of traffic, creating a relatively safe environment, and thus conforming to ethical guidelines.

Routes were chosen within this area with 13-16 waypoints and taking about 10 minutes to walk if walked directly. The sequence of photographs in figure 5 illustrates a segment of one of the routes used.

3.4 Experiment procedure
After an initial briefing, the use of the map or device was explained to participants and they then used this method to find their way along the route. After completing the first route, the other method was explained to them and they then used that method to navigate the second route. Each participant navigated both routes, using a different navigation method on each one. The order of the route and the method were fully counter-balanced.

On the routes, the experimenter walked with the participants, a few steps behind them to avoid influencing navigation. He or she made written observations on navigation behaviour, as well as providing help when participants got lost. Such help was only provided when it was necessary. Help was given to prevent distress and conform to ethical guidelines, and was noted by the experimenter.

After each route, participants filled in a questionnaire on the device or map. This incorporated the NASA Task Load Index (TLX) scale (Hart and Staveland 1988), which measures perceived workload, as an indication of how difficult participants found these methods and how they felt about using them. We modified the TLX response scales slightly to provide only five possible responses to make them simpler and less daunting for older participants.

The time taken and the number of times that participants got lost on each route were also measured. A participant was defined as lost if the experimenter had to intervene.

3.5 Map condition
The map used was one of the standard paper maps provided to visitors to the University of Glasgow. It was a greyscale version of that available at http://www.gla.ac.uk/general/maps/colourmap.pdf, covering a slightly smaller area and with a shorter list of buildings. A route was indicated on the map using a sequence of numbered circles, highlighted in yellow. Part of this map is shown in figure 6.

Participants were asked to navigate along the indicated route, visiting each of the numbered locations in turn. The equivalent route when using the device also passed through each of these locations in the same order.

4. Results of the first phase
4.1 Timings and frequency of getting lost
The mean times taken to navigate the routes with the map and the device are shown in figure 7. A two-way ANOVA on age and method showed a significant main effect of both the navigation method (map or device) and the age group and a significant interaction between age and method (all p<0.001).

An analysis of this interaction showed that the younger group took significantly less time to navigate the routes than did the older group – but only when using the map. There was no significant difference between the age groups when using the device. In addition, the older sample navigated the routes significantly faster when using the device than when using the map, but the younger group had no significant difference in the time taken with the two methods (all significances p<0.001, t-tests).

Participants also got lost significantly less often with the device (p<0.001, t-test), where “lost” is defined as in Section 3.4. In fact, no participants got lost when using the device, compared to a mean of 1.9 times per route for older users and 0.4 times for younger users when using the map.

4.2 TLX Scores
Raw TLX (RTLX) scores can be used as a measure of overall workload (Byers et al 1989). They are calculated by summing the individual components of the TLX workload and dividing by six. In this case, the RTLX scores were significantly lower for the device than for the map (p<0.001, Mann-Whitney) and there was no significant effect of age group (p>0.05). The TLX scores can be further investigated by analysing their individual components as shown in figure 8.

Using Mann-Whitney tests, significant differences between the map and the device were found for mental demand, effort and frustration (p<0.001), as well as for performance (p<0.005) and physical demand (p<0.05). There was no significant difference in temporal demand (p>0.05).

4.3 Responses to different methods of presentation
After using the navigation aid, participants were asked some questions about it and about the different methods of information presentation used within it. In particular, they were presented with a series of statements about the aid and asked to indicate the extent to which they agreed with them using a Likert scale. Their responses are summarised in the graph in figure 9.

In general, the responses were positive with all means above 3, where 3 indicates a neutral response and numbers greater than 3 indicate a positive one. In fact, all of the aspects except map usefulness had mean scores above 4 and few dissenters. Opinions on the usefulness of the map provided by the device (see figure 2(ii)) were more varied. This can be seen in the greater standard deviation for this aspect in figure 9. Several people from both the younger and older groups disagreed that this aspect was useful.

There were no significant differences in the responses with respect to age, except as regards the audio instructions. Both age groups liked these instructions but the older group rated them significantly higher than the younger group (p<0.05, Mann-Whitney).

When given the opportunity to make more general comments, several people mentioned that they liked the provision of audio instructions or of images. Several said that they liked the combination of two of the methods of presentation (e.g. images and audio) and two of the older group appreciated the combination of images, audio and text.

4.4 General preferences and comments
After trying both methods, participants were asked to indicate on a 5-point scale which method they found most useful. The results are shown in figure 10. Only one person (an older user) indicated a preference for the map, explaining that she was “accustomed to using maps and feels comfortable with them”.

Reasons given for preferring the device were varied, with some people giving multiple reasons. Most commonly mentioned was the provision of images of locations, which some said helped to confirm where they were or to determine more easily where they should go. Some participants liked having step-by-step directions and one user said this was like “walking with a guide who knew each and every corner”.

Other reasons focused on the shortcomings of maps. Both difficulties with maps in general and specific shortcomings of the map provided were mentioned. One participant explained that “using maps in general is quite difficult”, while another described how he “would have to turn and try and work out which way using the map”. A few people mentioned problems with the particular map used; for example, one person complained that there was no indication that a grey area represented a car park.

Despite the preferences for the device, some of the participants did have some reservations about it. Some felt that a map would be better for longer routes; others that the device gives less freedom and control over the route and a poorer idea of the route as a whole. Nevertheless, the majority felt the device to be more useful.

5. Discussion of the first phase
The results from the first phase confirm both of the hypotheses listed in Section 3.1:

1. An electronic pedestrian navigation aid based on landmarks can indeed be used effectively by older people, improving performance over that obtained by using a paper map. In our study, such an aid improved the time taken to navigate a route and reduced the number of times when people got lost, compared to a paper map.
   

2. Such an aid is indeed more effective for older people than for younger. Although both older and younger users found the device useful, only older participants navigated the routes faster using the device. In fact, the device reduced the time taken by the older users to a level comparable with that of the younger age group.

In addition, the users’ questionnaire responses and comments indicate that they appreciated the provision of information in different modalities. Both groups appreciated the combination of different methods of information presentation and found the provision of photographs very useful. The older group, in particular, liked being given audio instructions. This paves the way for a more detailed study of different methods of presenting information about landmarks, which forms the basis of the second phase of the experiment.

The results of the first phase of the experiment are discussed in more detail in Section 9 which discusses the results of the experiment as a whole.

6. Second phase of the experiment
Once the basic effectiveness of a landmark-based navigation aid is established, it is important to investigate how the landmark information can be best presented. The second phase of the experiment therefore compared different versions of the device’s interface, shown in figure 4. The particular versions used were chosen based on the results from the first phase of the experiment.

Participants had indicated that they found photographs to be very useful but photographs take up a lot of storage memory, which is limited in a mobile device. We therefore wanted to examine their contribution more quantitatively. Photographs were therefore removed from the new versions of the device to determine if the device can still be effective without them. The results from these versions were then compared with those from the first phase of the experiment.

In addition, several participants in the first phase indicated that they appreciated the combination of several modalities. Therefore, a combination of text and speech was used and evaluated against both text on its own and speech on its own.

The experiment design and method were very similar to those used in the first phase of the experiment to enable the data obtained in both phases to be compared with each other.

6.1 Hypotheses
In choosing to investigate text and speech on their own as well as together, we wanted to ascertain the effect of combining different modalities. Our hypotheses were

6.2 Participants
24 able-bodied participants were involved in this part of the experiment. None of them had previously taken part in the experiment’s first phase. As the second phase did not investigate the effects of age and as the first phase had already determined that such a device was more useful to older people, all participants were aged between 60 and 78 years old. Equal numbers of men and women were involved and all but one had never used a handheld computer before. As before, no participant was either a student or staff member at the university, in order to avoid over-familiarity with the test area.

6.3 Experiment design
The design of this phase of the experiment was heavily based on that of the first phase. As before, each participant was asked to find his or her way along two distinct routes on the campus of the University of Glasgow. The same routes as in the first phase of the experiment were used to enable the data from the two phases to be compared.

However, this phase of the experiment involved three methods of navigation instead of two – the three versions of the navigation aid shown in figure 4. Each participant only used two of these three methods in order to limit the length of the experiment and to allow the use of the same routes as in the first phase. This therefore necessitated a mixed between- and within-groups design.

The participants were divided into three main groups, each investigating a different pair of methods (e.g. the text version and text and speech version). Within each group, the order of the two methods and of the routes were counterbalanced, creating four conditions. Equal numbers of men and women were assigned to each condition.

6.4 Experiment procedure
This phase of the experiment followed the same procedure as described in Section 3.4, with the following exceptions:

7. Results of the second phase
7.1 Effect of combining modalities on performance
The mean times taken to navigate the routes with the text only, speech only and combined text and speech interface are shown in figure 11 (along with the times for other conditions for comparison). T-tests show no significant differences between these conditions. Similarly, the mean numbers of times that participants got lost are shown in figure 12 and there are no significant differences between the conditions (Mann-Whitney). It may be interesting to note that the variation in the number of times help was needed was very large (more than the mean) in all of the conditions.

7.2 Effect of combining modalities on user response
There was no significant effect of modality (text, speech or both) on the RTLX measure of overall workload or on any of the individual TLX scores (using Kruskal-Wallis). These scores are shown in the graph in figure 13, from which it can be seen that all of the scores were low, with the highest mean being 2.125, which is below the neutral response of 3.

Another measure that can be used to illuminate the user’s interaction with the device is the number of times he or she pressed the “Previous” button. This is shown in figure 14. A Kruskal-Wallis test showed that there was a significant effect of modality on the number of times this button was pressed (p<0.001). Individual Mann-Whitney tests showed that the back button was pressed significantly more often in the speech only mode than in either of the other two modes (p<0.001) and that there were no other significant differences.


After using two versions of the device, participants were asked to compare the two, using a Likert scale, and to say which one they found to be most useful and why. Responses were very varied, with at least one person choosing each extreme of the scale for each pair of modalities. However, there was a slight tendency to prefer the text only and text and speech versions to the speech only version as shown in figure 15.

Figure 15. Average (mean) preference in pairwise comparison of different modalities. A rating closer to one end indicates a preference for that modality. Error bars show standard deviation.

Comments explaining these choices indicate that text was often preferred because it stays on the screen and can be re-read without pressing buttons. It also eliminates problems with audio instructions due to background noise and mishearing instructions. A few people preferred speech because it allows you to move while listening to the instructions, removes the need to look at the device and eliminates the problem of putting on and off reading glasses in order to be able to read the text. Interestingly, those people who said they preferred text on its own to text and speech together gave reasons that only indicate a preference for text over speech, and similarly for speech over text and speech together.

7.3 Comparison with data from first phase
As the same routes were used for the two phases of the experiment, data can be compared across these two parts. These comparisons are only made with the data for the older group from the first phase as all participants in the second phase fell into the older age category.

7.3.1 Comparison with the paper map
A one-way ANOVA on the times taken with the new versions of the device and with the paper map from the first phase showed a significant effect of navigation method (p<0.001). In fact, the times taken with all the versions of the device are significantly less than that taken with the map (all p<0.001, Bonferroni simultaneous tests with the map as the control).

Similarly, there is a significant effect of navigation method on the frequency of getting lost (p<0.001, ANOVA) and participants got lost significantly less often with all versions of the device than with the map (all p<0.005, Bonferroni simultaneous tests). RTLX scores are also significantly lower for all versions of the device than for the map (p<0.005, Bonferroni simultaneous tests).

7.3.2 Comparison with the photographic version of the device
A similar analysis was carried out, comparing the new versions of the device (the ones involving text or speech) with the version employing photographs from the first phase of the experiment. No significant differences were observed between the conditions with regard to the time taken to navigate the routes (ANOVA) or the RTLX scores (Kruskal-Wallis).

However, a Kruskal-Wallis test on the number of times participants got lost (see figure 12) did show a significant effect of device interface (p<0.01). Post-hoc Bonferroni tests comparing the newer versions with the photographic version showed that the speech only version performed significantly worse than the photographic version (p<0.01) and there were no other significant differences.

7.4 Envisioned use in practice
At the end of the experiment, participants were asked if they would consider using such a navigation aid in practice. The responses are summarised in figure 16. Only two (out of 24) participants disagreed. One explained that he did not currently need such a device and the other that she liked maps and was “sure following a printed plan would be quicker”.

However, the majority of the participants (20 out of 24) agreed or strongly agreed with this statement. There were several different reasons given for this response. The ease of use of the device, particularly when compared with a map, was often cited and some people also considered that the device could be useful in unfamiliar territory. Some additional reasons, given by just one person, are shown in table 1.

7.5 General comments
Comments on the device as a whole and on the individual versions of the interface mostly fall into three categories: complaints, praise and suggestions for changes.

The most common complaint was about ambiguous or unclear instructions, for example, in the descriptions of buildings or the direction to turn. Four people also had difficulty understanding the voice used for the spoken instructions (e.g. because of its accent and/or diction). However, it should be noted that two people particularly commented on how easy the accent was to understand. Other positive comments referred to the ‘user-friendliness’ of the device and users’ enjoyment of the experience.

Suggestions for changes and additions to the device include using a different voice for the spoken instructions, providing a simple means of altering the volume and using raised buttons so that the user would not have to look at the device and could have it in a pocket.

8. Discussion of the second phase
The results are discussed in more detail in the following section, Section 9, which examines the results of the experiment as a whole. This section simply examines the hypotheses listed in Section 6.1.

1. There is insufficient evidence that presenting landmark information in more than one modality at once improves older people’s navigation performance. Therefore, this hypothesis was rejected.
   

2. There is little evidence that older people respond better to a navigation aid that provides information multimodally. However, there are some indications that older people do respond better to both text on its own and text and speech together than to an interface that only provides speech instructions.
   

3. A navigation aid that presents landmark information in simpler modalities such as text and speech does indeed still prove effective in aiding older people in navigation. It also does improve their performance over that of a paper map.
   

4. There is little evidence that providing landmark information in text only or text and speech form is not as effective as providing it in photographic form. However, participants did get lost more often when using a speech only interface than when using the photographic version.

9. Overall discussion
9.1 Effectiveness of landmarks
This work demonstrates that landmarks can be used effectively to support navigation through a handheld device. Such a device can improve the time taken to navigate a route and reduce the number of times when people get lost, compared to a paper map. It can also reduce workload and users agree that such a device is more useful than the map.

The use of the device can be investigated further by examining the components of the TLX scores, shown in figure 8 in Section 4.2. As well as increasing efficiency, the device decreases mental and physical demand, the effort expended and the frustration experienced. Users were also aware of an increase in their performance level.

There are a variety of possible reasons for this increase in performance and decrease in workload. While this study cannot give any definitive answers, some indications can be gathered from participants’ comments. When asked to explain why they preferred the device, several participants (both older and younger) explained that it gave a visual identification or confirmation of locations on the route. Several also liked being given a set of directions and being told which direction to turn rather than having to figure it out from a map.

All of this does not mean that there are no difficulties with a landmark-based navigation aid. The step-by-step nature of such an aid reduces the user’s freedom and control and provides a poorer overall idea of the route. There is also a degree of natural resistance to new methods. Research is needed into ways to overcome these challenges, e.g. by providing support for the user to change the route.

9.2 Effects of age
The first phase of the experiment shows that a pedestrian navigation aid based on landmarks has a greater potential benefit for older users than for younger. Although both older and younger users found the device useful (they got lost less often with the device, found that it produced a smaller workload and felt that it was more useful than the map), only older participants completed routes faster when using the device.

It may have been expected that older participants would have difficulty using a handheld device, particularly because all but one had never used one before. However, participants had little difficulty using the navigation aid and gave it low ratings on all aspects of TLX workload. We found that if the interface on a handheld computer is carefully designed with older people in mind, then it can be used without difficulty by this age group. This agrees with results from other studies. For example, McGee et al (2003) found that when the interface for a handheld application for cancer patients was redesigned based on the results of pilot studies, it was used without difficulty by the user group, many of whom were in the older age category.

We also expected older participants to be slower than younger ones, due to reduced walking speed, but this effect was only observed with the map. A landmark-based device has the potential to improve older people’s performance to a level comparable with a younger age group.

This does not mean that such a device would not be useful for any younger person. Although the younger group as a whole did not experience significant time improvement with the device, individuals did. We cite the example of one of the backup participants (one of the younger group) who took over three times as long and got lost seven times with the map, as opposed to once at the start when using the device.

9.3 Effects of different presentation modes
9.3.1 Effectiveness of text and speech instructions
The results from the second phase of the experiment show that a landmark-based navigation aid continues to be effective for older people when the information is presented through spoken instructions and/or written instructions rather than primarily through photographs. Participants using these methods navigated the routes significantly faster and got lost significantly less often than participants using a paper map. These methods were also rated significantly lower on perceived workload.

9.3.2 Comparison of text and speech to photographic instructions
What is more, there was little difference observed in performance or preference between the different methods of presenting the information, although the speech only interface did perform worse on some of the measures. It is hard to say whether there is actually no difference or whether the difference was simply not large enough to be significant with the number of participants used.

In particular, no participants got lost using the initial version of the device (which used photographs) whereas some participants got lost using all of the other versions (see figure 12). Although not statistically significant, the difference between “none” and “some” is nevertheless an important distinction, particularly for this measure. Because of the serious consequences of getting lost and its subsequent impact on users’ confidence in the device, this is one measure which we should attempt to reduce to zero, if possible.

The users’ comments help to shed light on the reasons for the reduced frequency of getting lost. With the text and/or speech interfaces, several people complained about ambiguous or unclear instructions. In actual fact, these instructions had already been improved from the initial versions used in the photographic interface, through a series of pilot studies. When only text and/or speech are used, it appears that much more effort must be invested in getting the instructions right. The provision of a photograph removes a lot of the ambiguity that is otherwise present.

9.3.3 Effectiveness of speech on its own
There were some significant differences between the speech only interface and the other interfaces. As indicated in Section 7.3.2, participants got lost significantly more often when using only spoken instructions than when using the original version of the device which also provided photographs and brief text instructions. They also pressed the “Previous” button to obtain the previous navigation instruction significantly more often than when using interfaces involving text. This is a tentative indication of greater navigational confusion in using the device. In addition, figure 15 indicates a slight user preference for interfaces involving text, although a few people did prefer the speech only version.

All this indicates that caution should be exercised when designing navigation aids that provide instructions in spoken form only. More research is needed to understand the extent to which this is true.

9.3.4 User preferences
Preferences between the text and/or speech versions were widely varied, with at least one person choosing each extreme of the preference scale for each pair of modalities. However, the reasons given for preferring other methods over text and speech together were simply reasons for preferring text on its own to speech on its own and vice-versa. This may indicate that these users would actually be happy enough with both text and speech together – they simply dislike one of the modalities. Providing information in multiple modalities may therefore be the best solution for satisfying the largest number of older users. However, further investigation is needed and a facility for customising the interface could also prove very successful.

Reasons given for preferring text centred on its persistence on the screen and the ease with which it can be re-read, highlighting shortcomings in the use of speech. Similarly, the preferences for speech instructions identify some difficulties with written information – some participants highlighted that speech did not require them to look at the screen, freeing them up to look around more and releasing them from having to put on and off reading glasses. It is important to consider how these shortcomings can be overcome, by increasing the persistence of speech and the freedom of using text.

10. Further Work
Due to the scale of the experiments described in this paper, some of the results are inconclusive. Further investigation is needed to gain more conclusive evidence on the use of speech only interfaces, the provision of photographs and the combination of several modalities. More work is also needed to understand how these different interfaces can be best designed and to develop guidelines for describing landmarks textually and verbally. In addition, this work has focused on just a few methods of presenting navigation information – text, speech and photographs. Other methods may also prove useful, such as the use of vibration (e.g. Bosman et al 2003) and non-speech sounds such as Earcons (Blattner et al 1989).

The results also highlight some changes and improvements which could be made to the navigation aid. While some of these are relatively straightforward, others require more in-depth investigation into alternative methods of presenting information and dealing with issues.

Some of the more straightforward changes which could be made include the provision of a simple means of altering the volume and using raised buttons so that the user does not have to look at the device to operate it. However, investigation is still required into the best ways to do these. The volume can currently be altered through a physical switch on the side of the device and a soft volume control on the screen. Users could be trained to operate these or a simpler volume control could be incorporated into the navigation aid interface itself. The device used, a Compaq iPAQ, also provides four raised buttons that can be mapped to different software controls. These could be used rather than soft buttons to control the navigation aid. Alternatively, an overlay with holes for the soft buttons could be used or haptic or audio feedback added to soft buttons to aid in the device’s non-visual use.

In addition, some users felt that the aid gave less freedom and control over the route and a poorer idea of the route as a whole than a paper map does. It is therefore important to investigate methods of changing the route to accommodate individual user preferences and to deal gracefully with detours and on-the-cuff changes to the route. It is also important to investigate how such a navigation aid can help users to gain a better overall idea of a route and area. As Wilson (2003) says, this is important to reduce user misconceptions about their location and to increase their confidence in the device. Her initial work suggests the importance of salient landmark information in doing so. This makes a landmark-based navigation aid like that described here particularly promising in this regard. However, more work needs to be done on how landmarks can be used most effectively to build up a mental map of an area.

We are also investigating how handheld applications in general can be designed so that they are easy for older people to use. We aim to produce some guidelines for design. As mentioned in Section 2.4, such guidelines exist for desktop applications but do not always apply to handheld computers. With the increase in popularity of handheld devices, such as mobile phones and personal organisers, the development of such guidelines is important to ensure that older people’s needs are considered in the development of such products.

11. Conclusions
Landmarks are a key part of navigation and this study has shown that they can be used effectively within electronic pedestrian navigation aids. A device that bases its navigation guidance around landmarks can significantly outperform a paper-based map, as well as reducing subjective workload and eliciting a positive response from users.

Various modalities can be used effectively to provide information about landmarks, including photographs, text and speech. However, the use of speech instructions on their own can be less effective than using text or photographs and using photographs can reduce the effort that needs to be put into removing ambiguity from written and spoken instructions. Combining modalities may be useful for meeting the preferences of a variety of users. However, more investigation of different methods of providing the information is warranted. We found that different people prefer the information to be presented in different modalities, indicating the need for some degree of personalisation of such devices, although presenting the same information in different modalities may go some way towards addressing this issue.

In addition, we have found that older people derive substantially more benefit from such a device than do younger users, with a large reduction in the time taken to navigate routes. The use of handheld technology does not prevent them from using the navigation aid successfully. Such aids could, therefore, provide key support to older people in maintaining their mobility and independence.

Acknowledgements
This work was funded by SHEFC through the UTOPIA project (grant number: HR01002), investigating the design of usable technology for older people (Eisma et al 2004). We would like to thank Kartik Khammampad who programmed the initial version of the navigation aid and carried out several of the experiments in the first phase of the evaluation. A full account of his work can be found in his Masters thesis (Khammampad 2003). We would also like to thank Morag Stark, Sarah Baillie and Lorna Brown for helping to conduct some of the experiments. We are grateful to Dundee University for providing its facilities for the key focus group on navigation and to Anna Dickinson and Audrey Syme for helping to run this focus group. Many thanks are also due to all those who took part in the focus groups and experiments.

References
Aubrey, J. B. and Dobbs, A. R., 1989, Age differences in extrapersonal orientation as measured by performance on the Locomotor Maze. Canadian Journal on Aging, 8(4), 333-342.

Blattner, M., Sumikawa, D. and Greenberg, R., 1989, Earcons and icons: their structure and common design principles. Human Computer Interaction, 4(1), 11-44.

Bosman, S., Groenendaal, B., Findlater, J. W., Visser, T., de Graaf, M. and Markopoulos, P., 2003, GentleGuide: an exploration of haptic output for indoors pedestrian guidance. In L. Chittaro L. (ed), Proceedings of Mobile HCI 2003 (Springer, LNCS 2795), pp. 358-362.

Bradley, N. A. and Dunlop, M. D., 2002, Understanding contextual interactions to design navigational context-aware applications. In F. Paternò (ed), Proceedings of Mobile HCI 02 (Springer, LNCS 2411), pp. 349-353.

Burnett, G. E., 2000, “Turn right at the traffic lights” The requirement for landmarks in vehicle navigation systems. The Journal of Navigation, 53(3), 499-510.

Burns, P.C., 1997, Navigation and the older driver. PhD Thesis, Loughborough University, UK.

Byers, J. C., Bittner, Jr, A. C. and Hill, S.G., 1989, Traditional and Raw Task Load Index (TLX) correlations: are paired comparisons necessary? In Advances in Industrial Ergonomics and Safety I, edited by A. Mital (Taylor and Francis), pp. 481-485

Eisma, R., Dickinson, A., Goodman, J., Syme, A., Tiwari, L., Newell, A. F., 2004, Early user involvement in the development of information technology-related products for older people. Universal Access in the Information Society, 3(2).

Fisk, A. D., Rogers, W. A., Charness, N., Czaja, S. J. and Sharit, J., 2004, Designing for Older Adults: Principles and Creative Human Factors Approaches (CRC Press).
Garmin, 2004, http://www.garmin.com, accessed June 2004.

Goodman, J., Dickinson, A., Syme, A., 2004a, Gathering requirements for mobile devices using focus groups with older people. In Designing a More Inclusive World, edited by S. Keates, J. Clarkson, P. Langdon and P. Robinson (Springer), pp. 81-90.

Goodman, J., Gray, P., Khammampad, K. and Brewster, S., 2004b, Using landmarks to support older people in navigation. In S. Brewster (ed), Proceedings of Mobile HCI 2004 (Springer, LNCS series).

Goodman, J., Brewster, S. and Gray, P., 2004c, Using field experiments to evaluate mobile guides. In K. Cheverst and B. Schmidt-Belz (eds), Proceedings of Mobile Guides, Workshop at Mobile HCI 2004.

Hart, S. G., Staveland, L. E., 1988, Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In Human Mental Workload, edited by P. Hancock, N. Meshkati (Amsterdam: North Holland B.V.), pp. 139-183.

Hawthorn, D., 2000, Possible implications of aging for interface designers. Interacting with Computers, 12(5), 507-528.

Hine, J., Swan, D., Scott, J., Binnie, D. and Sharp, J., 2000, Using technology to overcome the tyranny of space: information provision and wayfinding. Urban Studies, 37(10), 1757-1770.

Hunt, E. and Waller, D., 1999, Orientation and wayfinding: A review. Office of Naval Research: Arlington, VA.

Khammampad, K. R., 2003, Use of audio enhanced images for location sensing for navigation aids for older people. MSc Thesis, University of Glasgow, UK.

Kirasic, K. C., 1991, Spatial cognition and behavior in young and elderly adults: implications for learning new environments. Psychology and Aging, 6(1), 10-18.

Kirasic, K. C., 2000, Ageing and spatial behaviour in the elderly adult. In Cognitive Mapping: Past, present and future, edited by R. Kitchin and S. Freundschuh (Routledge, Routledge Frontiers of Cognitive Science 4), Chapter 10.

Laakso, K., Gjesdal, O. and Sulebak, J. R., 2003, Tourist information and navigation support by using 3D maps displayed on mobile devices. In B. Schmidt-Belz and K. Cheverst (eds), Proceedings of HCI in Mobile Guides, Workshop at Mobile HCI 2003, pp. 34-39.

Lines, L. and Hone, K. S., 2002, Older adults' evaluations of speech output. In Jacko, J. (ed), Proceedings of Assets 2002 (ACM Press), pp. 170-177.

Loomis, J. M., Golledge, R. G. and Klatzky, R. L., 2001, GPS-based navigation systems for the visually impaired. In Fundamentals of Wearable Computers and Augmented Reality, edited by W. Barfield and T. Caudell (Mahwah NJ: Lawrence Erlbaum Associates), Chapter 13, pp. 429-446.

Lynch, K., 1960, The Image of the City (Cambridge, Mass: MIT Press).
May, A. J., Ross, T., Bayer, S. H. and Tarkianainen, M. J., 2003, Pedestrian navigation aids: information requirements and design implications. Personal and Ubiquitous Computing, 7(6), 331-338.

McGee, M., Gray, P., Muir, L and Hargan, I, 2003, Designing a chemotherapy symptom management system for PDAs: emerging design guidelines from early patient pilot studies. Technical report TR-2003-146, Department of Computing Science, University of Glasgow.

Piekarski, W., Hepworth, D., Demczuk, V., Thomas, B. and Gunther, B., 1999, A mobile augmented reality user interface for terrestrial navigation. In Proceedings of 22nd Australasian Computer Science Conference, Jan 1999 (Auckland, NZ), pp. 122-133.

Ross, T., May, A. and Thompson, S., 2004, The use of landmarks in pedestrian navigation instructions and the effects of context. In S. Brewster (ed), Proceedings of Mobile HCI 2004 (Springer, LNCS series).

Tyler, N. A., Caiaffa, M. M. and Wainstein, M., 2001, Transport information for people with learning difficulties. In Proceedings of Include 2001, International Conference on Inclusive Design, April 2001 (London, UK).

U.S. Census Bureau, 2004, International Data Base (IDB). http://www.census.gov/ipc/www/ idbnew.html, updated 30th April 2004.

Wilkniss, S. M., Jones, M. G., Korol, D. L., Gold, P. E. and Manningal, C. A., 1997, Age-related differences in an ecologically based study of route learning. Psychology and Aging, 12(2), 372-375.

Wilson, J. Computer support for a person's cognitive map in a navigational domain. In P. Gray, H. Johnson and E. O'Neill (eds), Proceedings of HCI 2003: Designing for Society, Vol 2 (British HCI Group), pp. 135-136.

Zajicek, M. and Brewster, S. (eds), 2004, Design principles to support older adults, Special issue of Universal Access in the Information Society, 3(2).

Zec, R. F., 1995, The neuropsychology of aging. Experimental Gerontology, 30(3-4), 431-442.