10
1 Introduction
1 Introduction
1.1 Augmented Reality
Augmented reality (AR) is a term for a live direct or an indirect view of
a physical, real-world environment whose elements are augmented by
computer-generated sensory input, such as sound or graphics.
Augmented reality is related to a more general concept called
mediated reality, in which a view of reality is modified (possibly even
diminished rather than augmented) by a computer. As a result, the
technology functions by enhancing one’s current perception of reality.
By contrast, virtual reality replaces the real world with a simulated one
(Augmented Reality).
For many years, augmented reality has been topic of study for
researchers from all over the world who tried to enhance the real
world by wearing sophisticated devices into augmented reality labs.
After the smartphones revolution, research focused to implement
mobile applications which allow to enhance and augment the reality
no more just in the labs, but also “on the move”.
Last years have been crucial in AR development. Many companies
invested in developing reality augmenting software for many
purposes, from industry to medicine, from games to e-learning
platforms. The shared idea was to make the interaction experience
between user and software as natural as possible.
1.2 Prototyping
Someone claims that prototyping is an “art”.
“The word “prototype” derives from the Greek πρωτότυπον
(prototypon), "primitive form", neutral of πρωτότυπος
(prototypos), "original, primitive", from πρῶτος (protos), "first"
and τφπος (typos), "impression".”
(Online etymology dictionary)
So, essentially, prototyping is the art of producing prototypes, which
are early models of the product that is going to be built.
In many fields, there is great uncertainty as to whether a new design
will actually do what is desired. New designs often have unexpected
11
1 Introduction
problems. A prototype is often used as part of the product design
process to allow engineers and designers the ability to explore design
alternatives, test theories and confirm performance prior to starting
the production of a new product. (Prototype)
In general, an iterative series of prototypes will be designed,
constructed and tested as the final design emerges and is prepared for
production. With rare exceptions, multiple iterations of prototypes are
used to progressively refine the design.
A common strategy is to design, test, evaluate and then modify the
design based on analysis of the prototype.
The word "prototype" has sometimes the meaning of the word
"model" and this can be confusing. Prototypes can be mainly classified
into four categories:
Proof-of-Principle Prototypes:
Proof-of-principle prototypes are built to test the design without
exactly simulate the visual appearance, the right materials or the final
process. They can be used to try a potential design approach and they
are often used to check whether a design options will work or not and
if further development and testing is necessary. (Prototype)
Form Study Prototypes:
Form study prototypes are the kind of prototypes which allows
designers to “explore the basic size, look and feel of a product without
simulating the actual function or exact visual appearance of the
product”. (Prototype)
They are used to explore the product’s ergonomics and the visual
aspect of the product's final form. This kind of prototypes is often
“not-durable”.
Visual Prototypes:
Visual prototypes’ purpose is to “capture the intended design aesthetic
and simulate the appearance, colour and surface textures of the
intended product” (Prototype), but they don’t have the functionalities
of the final product. They are generally used in market research.
Functional Prototypes (also called “ working prototypes”):
12
1 Introduction
This type of prototypes’ aim is to try to simulate the final design, look
and feel, materials and functionalities of the product. They can have a
different size from the final product. Engineers usually adopt
functional prototypes to check for design flaws and to do last-minute
improvements industrialize the product. (Prototype)
In software engineering, prototypes have a key role in the software
development process, first of all because they allow to get a valuable
feedback from the users in the early stages of the project. Moreover,
they allow to check if the design specifications are met. They also
allow the software engineers to do some insights into the accuracy of
initial project estimates and whether the deadlines and milestones
proposed can be successfully met.
The original purpose of a prototype is to allow users of the software to
evaluate developers' proposals for the design of the eventual product
by actually trying them out, rather than having to interpret and
evaluate the design based on descriptions. Prototyping can also be
used by end users to describe and prove requirements that developers
have not considered, and that can be a key factor in the commercial
relationship between developers and their clients. Prototyping can also
avoid the great expense and difficulty of changing a finished software
product. (Software prototyping)
The process of prototyping involves the following steps: (Software
prototyping)
1. Identify basic requirements : Determine basic requirements
including the input and output information desired. Details,
such as security, can typically be ignored.
2. Develop Initial Prototype: The initial prototype is developed
that includes only user interfaces.
3. Review: The customers, including end-users, examine the
prototype and provide feedback on additions or changes.
4. Revise and Enhance the Prototype: Using the feedback both
the specifications and the prototype can be improved.
Negotiation about what is within the scope of the
contract/product may be necessary. If changes are introduced
then a repeat of steps n° 3 and n° 4 may be needed.
13
2 State of the Art
2 State of the Art
2.1 Prototyping general software applications
In software engineering, prototyping is a crucial stage of software
development. A software prototype is something comparable to
prototypes as we know them in other fields, such as mechanical
prototypes of engine’s components in automotive industry or buildings
prototypes in manufacturing industry. In these terms a prototype
typically simulates only some crucial aspects of the definitive final
solution and may be totally different from it.
It’s very important to create prototypes in software engineering as it
gives immediate feedback already in the early stages of the projects.
This is not a one way benefit on the developers’ hand: both customers
and contractors get significant value from a prototype as it makes
possible to check whether software requirements are met or not.
It also allows the software engineer some insight into the accuracy of
initial project estimates and whether the deadlines and milestones
proposed can be successfully met. The degree of completeness and
the techniques used in the prototyping have been in development and
debate since its proposal in the early 1970s. (Software_prototyping)
2.1.1 Prototyping strategies
A prototyping strategy is a set of principles, guidelines, plans of action
aimed to achieve the final goal of building a useful system prototype.
Different strategies gives different views on the same problem and
looking at the same situation with different perspectives is a good way
to discover usability problems.
According to the scientific community and literature, there are several
prototyping strategies which can be adopted to prototype software
(but also other products). In this chapter we’ll take a look at the most
used and well-known worldwide.
Further technical and implementation specifications can be found in
Jakob Nielsen’s book “Usability Engineering”.
Horizontal prototypes
A common term for a user interface prototype is the horizontal
prototype. It provides a broad view of an entire system or subsystem,
14
2 State of the Art
focusing on user interaction more than low-level system functionality,
such as database access. Horizontal prototypes are useful for:
Confirmation of user interface requirements and system scope
Demonstration version of the system to obtain buy-in from the
business
Develop preliminary estimates of development time, cost and
effort.
Vertical prototypes
A vertical prototype is a more complete elaboration of a single
subsystem or function. It is useful for obtaining detailed requirements
for a given function, with the following benefits:
Refinement database design
Obtain information on data volumes and system interface
needs, for network sizing and performance engineering
Clarifies complex requirements by drilling down to actual
system functionality
Task-oriented prototypes
Task-oriented prototypes are focused on the tasks the user is going to
perform on the system. The prototype, differently from the vertical
prototypes, is not intended as the elaboration of a single system’s
subset, rather it represents the bunch of functionalities needed in
order to perform a specific task.
Scenario-based prototypes
Scenario-based prototypes are prototypes based on a scenario. A
scenario is the representation of the user’s behaviour in performing a
task. A task differs from a scenario because the scenario is specific for
a certain kind of design, and in that “it shows how a task would be
performed if you adopt a particular design, while the task itself is
design-independent: it's something the user wants to do regardless of
what design is chosen” (Lewis & Rieman, 1993, p. 20)
2.1.2 Prototyping methods
Prototyping methods are the physical ways of building prototypes.
According to the design stage there are methods that better fits. Each
prototyping method differs from the other in terms of time needed to
build it, of costs, of realism, of interaction with the user, of tools
needed to implement it, of technical skills of the designer. It doesn’t
15
2 State of the Art
exist the “best” method. Each one has pros and cons, and it’s up to the
designer to balance them and find the best trade-off for the specific
case.
Most of the methods’ descriptions and specifications of this chapter
comes from the referenced book “Effective prototyping for software
makers”. (Arnowitz, Arent, & Berger, 2006)
Paper and pencil
Paper and pencil prototyping is probably one of the fastest, easiest and
cheapest way of prototyping. This method of design, refine and test
user interfaces is particularly adapt in the earliest stages of design,
when many design patterns are still opened and the designer is just
exploring the many alternatives.
Carolin Snyder has defined it as a
“variation of usability testing where representative users
perform realistic tasks by interacting with a paper version of the
interface that is manipulate by a person “playing computer,”
who doesn’t explain how the interface is intended to work”.
(Snyder, 2003, p. 4)
The above definition represents quite concisely this method, in which
all is needed to start prototyping are some sheets of paper, pens,
pencils and crayons.
The way paper prototyping works is quite simple but not obvious. First
of all it is necessary to identify the typical representative interface’s
users who will run the tests on the paper prototype. Afterwards, the
typical tasks must be determined and according with them the
interfaces’ screen-shots and hand-sketches must be drawn. These
includes windows, menus, buttons, pages, messages and all that is
necessary to re-create a realistic setting of the application. When all
the material is ready it is possible to run the usability test with the
representative users. This is what differentiate paper and pencils
prototypes from other kinds of prototypes, also made with paper: the
interaction with the audience. Even though all the prototypes are just
made of paper, there is already a possibility of interaction with the
computer (that is actually played by the usability test administrator).
This is not true with other kinds of prototypes also made with paper,
like for example Storyboards and Compositions, in which there is no
16
2 State of the Art
interaction with the user. The main pros of paper prototyping are the
following (Snyder, 2003):
Fast way to mock up an interface — no coding required.
Finds a wide variety of problems in an interface, including many
of the serious ones.
Allows an interface to be refined based on user feedback
before implementation begins.
A multidisciplinary team can participate.
Encourages creativity from the product team and users alike.
On the other hand, cons are:
Does not produce any code.
Does not find all classes of problems with an interface.
Can affect the way users interact with the interface.
Makes some development teams nervous because they fear
users will think it unprofessional.
Has stronger benefits in some situations than in others.
Storyboards
Storyboards are essentially a set of sketches, images or pictures
displayed in sequence, with the purpose of illustrating the sequence of
actions and interactions that happen during the utilization of a
software. Storyboards are settled milestones in movies and animations
industry, but are very effective in usability and design science too.
Their main benefit comes from the possibility of “visual thinking”
instead of “mental thinking”, e.g. the possibility to follow the evolution
of the interaction of the user with the software in a visual way, like a
comic. In this way, many people can focus on the same problem at the
same time and they can physically see the interaction experience
evolving. Storyboards are mainly used in the early stages of the project
because they make brainstorming easier, as it gets easier to develop
new ideas and immediately place them in the story. Just like in paper
prototyping, they just need paper and pen/pencils to be created (but
sometimes also digital illustration software are used) but they differ
from them for the lack of interactivity between the user and the
prototype.
It’s quite easy to create a storyboard and, it’s important to highlight,
that creating storyboards it’s not an artistic task. Sophisticated
graphical elements are not required, rather they can be elements of
17
2 State of the Art
distraction. In fact, one important rule to bear in mind in the
storyboards’ creation process is to “keep them as simple as possible”.
It is not relevant to create a “nice-looking” or a “realistic” storyboard,
instead it should be effective, with all the significant elements
considered, so that the interaction sequence can be clear and easily
understandable.
Storyboards’ audience is both internal than external to the
development team. Storyboards are used essentially in the first and
mid stages of the project and their longevity is medium, because they
are used along all the project as guidelines. They look narrative,
medium fidelity and with conceptual expression. They are presented
both on physical and digital media.
Storyboards pros:
Helps to think “linear”
Useful brainstorming tool
Fast
Cheap
Storyboards cons:
Not interactive
Prototyping technique only for the early stages of the project
Cannot show many usability problems
Mock ups
In manufacturing and design, a mock up, or mock-up, “is a scale or full-
size model of a design or device, used for teaching, demonstration,
design evaluation, promotion, and other purposes. A mock up is called
a prototype if it provides at least part of the functionality of a system
and enables testing of a design.” (Mockup)
In software engineering Mock ups are used to create user interfaces,
to show to the final users the aspect of the product without the needs
of creating software and routines. Mock ups can be very simple hand
sketches, but also very high definition images, and partially working
user interfaces.
Mock ups are often used to create Unit tests, to be able to test one
part of a software system (a unit) without having to use dependent
modules. The function of these dependencies is then "faked" using
mock objects. (Mockup)
18
2 State of the Art
Wizard of Oz
One important tool for developing complex interactive applications is
the “Wizard of Oz” simulation. Wizard of Oz simulation allows design
concepts, content and partially completed applications to be tested on
users without the need to first create a completely working system.
(Steven, Jaemin, Christopher, Blair, Jay, & Maribeth, 2005).
Created in the 80’s by J.Kelley, Wizard of Oz is a testing method used
before having the system fully working. Wizard of Oz works having
someone behind the scene who controls the actions, and the user who
is testing the interface does not know that the responses are not
automatic, but generated by a human (Wizard of Oz).
How does it work:
human simulates system intelligence and interacts with user
uses real or mock interface
o “Pay no attention to the man behind the curtain”
user works with system as if it were real
the “wizard” (sometimes hidden):
o interprets input according to an algorithm
o controls the user’s screen appropriately
wizard of oz trials are good for:
o adding simulated and complex vertical functionalities
o testing futuristic ideas
o systems that would be hard to implement or change
Wizard of Oz technique has an external audience, it is not intended to
be used within the development team but to demonstrate the
interface to the user. It is used essentially in the mid-term stage and
has a quite rapid development speed. Its longevity is medium-long, it
can be reused in several iterations and has both conceptual and
experiential expression. The style is interactive and the fidelity
medium-high.
Video prototyping
Video prototyping is defined as “a technique for visualizing the
interactive behaviour of a system using video animation.” (Video
prototyping)
Video Prototyping can be a very powerful tool to communicate a
design. However, it can also be expensive and time consuming in order
to be produced. Video prototyping can be a very compelling way in
19
2 State of the Art
which to show a design. It allows stakeholders, users, developers and
others to quickly understand the deign experience. Some major
drawbacks are that it costs (time and resources) a bit more than other
forms of prototyping. (video-prototyping)
Video prototyping has an audience both internal than external to the
development team. It is used in the early and midterm stage and has a
medium-long longevity. It is expression of both conceptual than
experimental issues and the style can be either narrative or interactive.
It is presented on digital media and it is a medium-high fidelity
prototype.
Interactive simulations
Interactive simulation is a computer based kind of prototype used to
virtually simulate and approximate realistic behaviours of physical
phenomenon. Interactive simulations are particularly helpful in design
process because they allow to interact real time with the prototype
and to edit and modify it just acting on a software, that is why they are
commonly considered a kind of cheap prototypes even though the
fidelity can be medium-high.
Coded prototypes
Coded prototypes can be created in several forms including a
programming language, a mark-up language, or in a visual
programming environment in conjunction with a scripting language. All
these forms are digital interactive expressions of program code, each
meant to portray a software product or one of its components as
similar to the finished coded result as possible. (Arnowitz, Arent, &
Berger, 2006)
A coded prototype is usually developed in a programming language or
scripting code and should be considered as a later iteration of other
forms of low and medium-fidelity prototyping. A coded prototype has
the advantage of allowing the software team to productively reuse
code for an actual finished product. (Arnowitz, Arent, & Berger, 2006)
Coded prototypes often comes late in the project. They are intended
both for internal than external inspection and they require more time
to be built than other prototypes. Their longevity is long, most of the
time a coded prototype evolves into a working application/interface
and they have a pretty experiential expression. The style is interactive,
the fidelity is high and the medium on which they are presented is
digital.
20
2 State of the Art
All the prototyping methods explained above are only a limited subset
out of all the methods available in literature, but they are the most
common and used by designers all over the world for general purposes
applications. Of course, the more specific is the application’s domain,
the more particular and specific will be the prototype itself. Sometimes
designers prefer to adopt a mix among different methods or to use an
“ad-hoc” method for the specific purpose of their application.
2.2 Prototyping mobile applications: fidelity
Mobile devices are pocket-sized computing devices typically having a
display screen and an input system that can be a reduced keyboard, an
extended keyboard, a touch-screen, a voice control and more
peripherals. Mobile devices are nowadays part of our daily life; we call
them mobile phones, smartphones, handheld, PDAs, tablets, and we
use them every day for a broad set of activities, from communicating
(voice calls, text messages, emails) to organize our activities and our
businesses (agenda, to-do list, synchronizer), from taking pictures and
videos with the embedded camera to creating and editing documents
and, why not, to also play videogames.
Mobile devices look like small computers, and most of them can do
what traditional computers can. But, compared to modern laptops and
desktop computers, they all have a limited computing capability, a
limited amount of memory and different operating systems, with
different environments and peripherals. These differences constraint
how the applications running on these mobile devices are built.
Moreover, the pace of mobile devices’ evolution and changing is
incredibly fast and often technology, development methodologies,
environments and tools get old and inadequate in few months.
There are many references in literature about guidelines and
methodologies to develop mobile application. As in “not-mobile”
applications, e.g. desktop and laptop computer applications, an
important aspect is reserved to usability evaluation. At this stage the
existence of a good prototype can be essential and perhaps even more
important than in not-mobile applications development, given the
heterogeneous audience and the ubiquitous and pervasive
characteristics of the mobile software itself.
As in not mobile applications, prototypes can belong to a wide range of
categories, according to their level of fidelity, which represents how
21
2 State of the Art
closely they resemble the final product: “low fidelity”, “medium
fidelity” or “high fidelity”. Moreover they can be mixed together to get
a further level called “mixed-fidelity”, which mixes some aspects of the
former three. Let’s step through them, one by one.
Prototyping mobile applications at low-fidelity
In the 2006 paper “Low-Fi Prototyping for Mobile Devices” the authors
claim that still “low-fidelity prototypes are scarcely used for mobile
devices” (Sá & Carriço, 2006, p. 696), notwithstanding their
effectiveness especially in the early evaluation stages of the project
and the proved successes. Instead, still many teams prefer to adopt
working software prototypes or specific equipment specifically
developed and consequentially more expensive, even where not
needed or justified. Two important aspects to keep in mind when
prototyping mobile applications are the context and the settings of
use (Sá & Carriço, 2006). In fact, besides evidencing some design
problems that could occur, they promote participatory design,
allowing users to engage actively on the process. In fact, most of the
innovations appeared when testing the prototypes on real situations
and scenarios. The contextual evaluation highlight the advantages of
the mobile devices, but also provide a deeper insight on their
problems for those particular applications. (Sá & Carriço, 2006)
Low fidelity prototypes in mobile applications can be as effective and
efficient as some higher fidelity ones. They allow developers to build
them fast and in a cheap way, making easy to test some early design
ideas and preventing other posterior design errors happening. But it is
also important to deep analyze the trade-off between a quick and
cheap prototype, its pros and cons, and the same of having a medium
or high fidelity one. When the lack of precision, the rough and raw
interface and the fake interaction are balanced with a quick and cheap
available prototype it can be a successful strategy to adopt a low
fidelity prototype.
Some important guidelines when prototyping at low fidelity can be
summarized in the next four points: (Sá & Carriço, 2006)
Using rigid materials on the prototypes allows users to carry
them, keep them in their pockets, take them home and
participate on the sketches’ design directly on the device.
Context of use is essential for effective usability testing.
Evaluation should be done in several possible scenarios.
22
2 State of the Art
Users should be allowed to participate on the sketching process
during contextual sessions.
Tasks that users will complete should be defined previously,
but the possibility of creating new features and test them on
the spot should be provided.
All in all, the following list of hints should be kept in mind when using
low fidelity prototypes in mobile applications: (Sá & Carriço, 2006)
Low-fidelity prototypes can be used for mobile applications as
effectively as on desktop settings.
A careful distinction between the device prototypes and the
application prototype has to be made.
The device prototype should be created using rigid materials in
order to be used in real-life settings. It should have
approximately the same size and weight of an actual device.
A slot where cards can be easily and quickly inserted and
removed needs to be available.
The sketches must be drawn with the same size of the device’s
screen using similar components and fonts to those available
for a real device.
The interaction type of the actual device should also be
emulated
Prototyping mobile applications at medium-fidelity
Medium-Fidelity Prototypes are:
“a sort of a best-of-both-worlds implementation that allows for
a combination of both high-level and detailed views; rapid,
iterative changes and the ability to conduct meaningful user
tests to evaluate complex functionality and to help determine
system specifics.” (Prototypes)
The medium fidelity prototypes key capabilities are:
higher level of interactivity than available with paper.
little or no coding required.
more accurate look & feel than low-fidelity prototypes.
sufficiently “unpolished” to solicit user feedback and criticism.
In the mobile devices context, a medium fidelity prototype can be an
adequate compromise to give the users a more realistic feedback
about the look and feel of the application. This strategy can be