Sustained Participatory Design:
Extending the Iterative Approach
Jesper Simonsen, Morten Hertzum
Introduction
In 2005, Shapiro described how many large-scale systems develop-
ment projects are highly troubled.1 Attempts to introduce ambi-
tious information systems in the public sector have been especial y
notorious for being late, over budget, or functional y inadequate; in
addition, “the situation in the private industry may be no better,
but commercial confidentiality and the lack of public accountabil-
ity may make it less visible.”2 For Participatory Design approaches
to lead to the best and most effective systems that support the
work they are used for, “Participatory Design as a community of
practitioners should seriously consider claiming an engagement in
the development of large-scale systems.”3

Participatory Design undoubtedly has a lot to offer. Benefits
can accrue in terms of clarifying goals and needs, designing coher-
ent visions for change, combining business-oriented and social y
sensitive perspectives, initiating participation and partnerships
with different stakeholders, using ethnographic analyses in the
design process, establishing mutual learning processes with users
from the work domains in question, conducting iterative experi-
ments aimed at organizational change, managing stepwise imple-
mentation based on comprehensive evaluations, and providing a
large toolbox of different practical techniques.

Participatory Design is characterized by the intention of
1 Dan Shapiro, “Participatory Design: The
establishing mutual learning situations between users and design-
Will to Succeed,” in Proceedings of the
ers.4 A sustained Participatory Design approach al ows an organi-
4th Decennial Conference on Critical
zation to experiment and learn—not only as part of the initial
Computing: Between Sense and
Sensibility (New York: ACM Press, 2005),
design, but also as part of the organizational implementation and
29-38.
use of a technology. The overall design process that includes, and
2 Ibid., 30.
transcends, the technical development of a technology has
3 Ibid., 32.
been identified by Markus as “technochange” management and, in
4 Keld Bødker et al., Participatory IT
particular, as a technochange prototyping approach.5 Techno-
Design. Designing for Business and
change combines large information technology (IT) projects with
Workplace Realities (Cambridge, MA:
MIT Press, 2004).
organizational change programs to produce technology-driven
5 Lynne Markus, “Technochange
organizational change: “Here, what is to be prototyped is not just a
Management: Using IT to Drive
technical solution or just an organizational change, but both
Organizational Change,” Journal of
together.”6 The technochange prototyping approach uses the tradi-
Information Technology 19, no. 1 (2004):
tional iterative prototyping approach as an overall model for orga-
4-20.
6 Ibid., 17.
nizational change.
© 2012 Massachusetts Institute of Technology
10
DesignIssues: Volume 28, Number 3 Summer 2012

---

Iterative Participatory Design experiments using various
sorts of mock-ups and prototypes have been conducted for
decades.7 However, most experiments have been restricted either
to smal -scale systems (often driven by researchers), or to the ini-
tial stages of larger scale information systems development, fol-
lowed by a conventional contractual bid.8 Recently, however, a
growing number of Participatory Design experiments includes
both initial design and real-use evaluation.9

Active engagement in—and documentation of results
with—large-scale information systems represents a major goal
for Participatory Design. In this article, we pursue Shapiro’s cal
for a col ective approach by extending the iterative prototyping
7 For a more elaborated review of related
approach into a sustained Participatory Design approach, includ-
literature, see Jesper Simonsen and
ing large-scale Participatory Design experiments. We do this by
Morten Hertzum, “Participatory Design
and the Challenges of Large-Scale
means of an exemplary reflection: What are the chal enges that Partic-
Systems: Extending the Iterative PD
ipatory Design must face when engaging in design and implementation of
Approach,” in Proceedings of the Tenth
large-scale information systems? We describe and reflect on a Danish
Anniversary Conference on Participatory
Participatory Design initiative in the healthcare sector involving a
Design 2008 (New York: ACM Press,
Participatory Design experiment with an Electronic Patient Record
2008), 1-10.
(EPR) system. The experiment was conducted by the authors in
8 For the former, see Andrew Clement and
Peter van den Besselaar, “A
close col aboration with the vendor, CSC Scandihealth (CSC), and
Retrospective Look at PD Projects,”
the customer, the region of Zealand, one of Denmark’s five health-
Communications of the ACM 36, no. 6
care regions—and in particular, the region’s EPR unit and the
(1993): 29-37; Anne-Marie Oostveen and
neurological stroke unit at Roskilde Hospital. We describe the
Peter van den Besselaar, “From Small
experiment and our experiences and present the chal enges that
Scale to Large Scale User Participation: A
Case Study of Participatory Design in
the Participatory Design paradigm must address to succeed in fil -
E-Government Systems,” in PDC 04:
ing a greater role in large-scale information-systems projects.
Proceedings of the Eighth Conference on
Participatory Design: Artful Integration:
A Sustained Participatory Design Approach
Interweaving Media, Materials, and
Our sustained Participatory Design approach introduces iterations
Practices 1 (New York: ACM Press, 2004),
of design and implementation and emphasizes improvisation,
173-82. For the latter, see Keld Bødker et
al., Participatory IT Design.
experimentation, and learning. This approach chal enges conven-
9 See, e.g., Monika Büscher et al., “Ways
tional plan-driven approaches that maintain a clear distinction
of Grounding Imagination” in PDC 04:
between design and organizational implementation.10 Orlikowski
Proceedings of the Eighth Conference on
and Hofman suggest that, as an alternative model for managing
Participatory Design, 193-203; Thomas
technological change, improvisational change management be
Riisgaard Hansen et al., “Moving Out of
the Laboratory: Deploying Pervasive
defined as “a way of thinking about change that reflects the
Technologies in a Hospital,” IEEE
unprecedented, uncertain, open-ended, complex, and flexible
Pervasive Computing 5, no. 3 (2006):
nature of the technologies and organizational initiatives… [where]
24-31.
managing change would accommodate—indeed, encourage—
10 Barry Boehm and Richard Turner,
ongoing and iterative experimentation, use, and learning.”11
Balancing Agility and Discipline: A Guide

Orlikowski and Hofman characterize improvisational
for the Perplexed (Boston: Addison-
Wesley, 2004).
change management by distinguishing between three kinds of
11 Wanda Orlikowski and Debra Hofman,
organizational change: anticipated, emergent, and opportunity-
“An Improvisational Model for Change
based.12 Anticipated change is planned ahead and occurs as intended
Management: The Case of Groupware
by the originators of the change. Emergent change is defined as local
Technologies,” Sloan Management
and spontaneous changes, neither originally anticipated nor
Review 38, no. 2 (1997): 12.
12 Ibid.
intended. Such change does not involve deliberate actions but
DesignIssues: Volume 28, Number 3 Summer 2012
11

---

Figure 1
Outline of our sustained Participatory
Design approach.
grows out of practice. Opportunity-based changes are purposefully
introduced changes resulting from unexpected opportunities,
events, or breakdowns that have occurred after the introduction of
a new information system.

Emergent and opportunity-based changes are widely noted
in Participatory Design projects,13 but there has been surprisingly
little focus on managing and learning from such changes over lon-
ger periods of time. A sustained Participatory Design approach in
large-scale information-systems projects entails the integration of
design and development with organizational implementation. This
integration is necessary to obtain data and experiences from real
use during design and development and thereby to move itera-
tively through the three change perspectives: (1) evaluate progress
on planned changes, (2) become aware of emergent changes, and
(3) turn selected emergent changes into opportunity-based
changes. Charting progress on planned changes is a means to
ensure that system possibilities get integrated into actual work
practices, while turning emergent changes into opportunity-based
changes is a means to ensure that work practices are changed in
relevant ways.

Our sustained Participatory Design approach—outlined in
Figure 1—is an extension of the traditional iterative approach. It
emphasizes the evaluation of systems by exposing them to real—
13 See, e.g., Gro Bjerknes and Tone
situated14—work practices. The anticipated and intended changes
Bratteteig, “The Memoirs of Two
are the starting point of an iteration. These desired changes are
Survivors: Or Evaluation of a Computer
System for Cooperative Work” in
further specified, for example, in terms of the effects of using the
Proceedings of the 1988 ACM Conference
system. The system (or a part/prototype of it) is then implemented
on Computer-Supported Cooperative
and tried out under conditions as close as possible to real use.
Work, Irene Grief, ed., (New York: ACM
Actual use of the system al ows for unanticipated changes (both
Press, 1988): 167-77; Susanne Bødker,
emergent and opportunity-based) to occur. Final y, evaluation of
and Jacob Buur, “The Design
using the system informs subsequent iterations. Thus, selected
Collaboratorium—A Place for Usability
Design,” ACM Transactions on Computer-
emergent changes are turned into opportunity-based and new
Human Interaction 9, no. 2 (2002):
desired changes, thereby forming the starting point for the next
152-69; Pelle Ehn, Work-Oriented Design
iteration.
of Computer Artifacts (Stockholm,

In the fol owing sections, we describe the sustained Partici-
Sweden: Arbetslivcentrum, 1988).
patory Design approach we propose by presenting an experiment
14 Lucy A. Suchman, Human-Machine
Reconfigurations: Plans and Situated
that exemplifies the four elements depicted in Figure 1. The exper-
Action, 2nd Edition (New York: Cambridge
iment involved the clinical-process module of an EPR system. This
University Press, 2007).
EPR module supports clinical documentation and decision-making
12
DesignIssues: Volume 28, Number 3 Summer 2012

---

Figure 2
Results from three iterative Participatory
Design workshops: mock-up, non-interactive
PowerPoint prototype, and running prototype
of screen to be used during nursing handover.
DesignIssues: Volume 28, Number 3 Summer 2012
13

---

and comprises the ongoing documentation of medical patient
information made by the clinical staff. Today, the majority of clini-
cal documentation is still paper-based. To initiate the development
of this EPR module, a large-scale Participatory Design experiment
was conducted during the fal of 2005, involving a close col abora-
tion between CSC, the region of Zealand, the stroke unit at
Roskilde Hospital, and the authors. The stroke unit is an acute in-
patient clinic with nine beds, and it treats approximately 850
patients a year. The experiment involved one iteration of the sus-
tained Participatory Design approach.
Step 1: Identify the Desired Change
The overall desired change that the experiment aimed for was to
implement a ful y IT-integrated EPR system that included support
for the clinical process and replaced all paper-based patient
records. The clinicians at the stroke unit specifically requested
improvements in obtaining a patient overview and support of their
mutual coordination. On a national level, another long-term aim
was to increase the structuring and standardization of the content
of patient records as part of the development of the EPR system.15
In response to this overal political objective, the EPR unit wanted
to introduce and evaluate a new structure of the nurses’ narrative
recordings by dividing it into 14 categories of basic nursing care.16
Step 2: Specify and Implement
The desired changes were specified in the first part of the experi-
ment (August to October) through five full-day Participatory
Design workshops where clinical staff, in cooperation with design-
ers from CSC and project managers from the EPR unit, designed
and configured the EPR system. The main parts of the system were
designed and configured in three steps, as depicted in Figure 2: At
one workshop, mock-ups were drawn on flip-over charts. At the
following workshop, a preliminary, non-interactive PowerPoint
prototype was discussed. At a third workshop, a running proto-
type was demonstrated and discussed. In articulating their
requirements, the physicians and nurses focused on two aspects
central to their work: their continual creation and re-creation of an
overview of the status of the patients and the coordination among
the clinicians. The overview and coordination are particularly
15 Claus Bossen, “Participation, Power,
prominent in relation to three clinical activities:
Critique: Constructing a Standard for
• Team conferences. Every morning on weekdays, the
Electronic Patient Records” in
physicians, nurses, and therapists meet for about 15
Proceedings of the Ninth Conference
on Participatory Design: Expanding
minutes to go through the admitted patients.
Boundaries in Design 1 (New
• Ward rounds. After the team conference, the chief
York: ACM Press, 2006), 95-104.
physician starts the ward round, which consists of
16 Virginian Henderson, “Virginia
medically assessing each patient and adjusting the
Henderson International Nursing Library,”
treatment and care accordingly.
www.nursinglibrary.org (accessed
January 22, 2012).
14
DesignIssues: Volume 28, Number 3 Summer 2012

---

• Nursing handovers. At the start of every nursing shift,
the nurses meet for about 45 minutes to go through the
admitted patients and coordinate activities.
Through the Participatory Design workshops, the clinicians speci-
fied a number of desired effects. For example, they requested coor-
dination support during the three activities described. The chief
physician wanted to be able to complete the daily ward rounds as a
“one-man show” (i.e., without an escorting nurse), where all shar-
ing of information and coordination with other clinical staff were
done through the EPR system. This effect was given high priority
because the nurses are busy, and time taken to escort the chief
physician during the lengthy ward round takes away from patient
care. Improved patient overviews were also identified as a desired
effect, especial y in relation to the team conferences and nursing
handovers. In addition, the EPR unit needed the nurses’ recordings
to fol ow a more consistent structure and needed prompt system
response times to evaluate the performance capabilities from
CSC’s new configurable development platform.

CSC undertook the technical development of the EPR sys-
tem in November and December, which included ensuring appro-
priate interfaces to various systems currently used at the hospital.
Five years of patient data were migrated to the EPR system to
enable access to previous patient records, as well as to records of
patients that would be hospitalized during the experiment. The
amount of data also provided a data load that enabled a realistic
evaluation of system performance.
Step 3: Real Use Enabling Unanticipated Change
The trial period, where the EPR system was in real use, took place
in December and lasted five days. During this trial period, al cli-
nicians at the stroke unit used the EPR system 24 hours a day, and
the system replaced all paper records for all patients. The system
involved stationary and portable computers, PDAs for bedside
measurement of patient parameters, and a large shared display
projected on the wall during team conferences and nursing hando-
vers (see Figure 3). Transactions involving other wards not
involved in the experiment were simulated by a back office staffed
Figure 3
Photos from the five-day period of real use.
DesignIssues: Volume 28, Number 3 Summer 2012
15

---

24 hours a day. Patient-record entries that involved paper transac-
tions with other wards were initiated in the EPR system by the cli-
nicians. The back office continuously monitored the system,
identified such entries, mailed them in the conventional fashion,
waited for the results to arrive, and immediately typed them into
the EPR system. Thus, the clinicians at the stroke unit experienced
the EPR system as if all transactions were ful y IT supported. To
safeguard against troubles and misunderstandings, which might
have entailed risk to patient health, the clinicians were supported
by “shadows” who had detailed knowledge of the EPR system and
were present 24 hours a day.

The five-day trial period made it possible to test the EPR
system in real use and to identify unanticipated changes.
Although the trial period was short, we observed both emergent
and opportunity-based changes. Emergent changes included that
the traditional oral way of communicating about patient status
changed to collectively reading the information on the large
shared display used for team conferences and nursing handovers.
As a result of the clinicians’ ability to read the patient record on
the shared display, we further observed that the clinicians initi-
ated col ective investigations of the patient record during these
activities.17 We observed that at the nursing handovers before the
trial period, the patient record was seen only by the nurse team
leader, who held the patient record in her or his hand and con-
veyed the status of the patient by reading key information out
loud. During the trial period, the patient record was projected on
the wall and repeatedly inspected by all nurses present at the
handovers, and they col ectively participated in interpreting the
status of the patient.

As an example of an opportunity-based change, the nurses
were able to make their observations more visible at the team con-
ferences: Halfway through the trial period, they initiated a change
in the team conference screen by having CSC add a panel with
nursing observations relevant for the team conference. In this way,
the nurses’ observations became more salient to the clinicians as
they were forming their overview of the patients’ status.
Step 4: Evaluate
17 For a detailed ethnographic study of this
The evaluation of the desired changes included a quantitative anal-
behavior, see Jesper Simonsen, and
ysis that verified a number of positive effects.18 For example, the
Morten Hertzum, “Iterative Participatory
chief physician was able to complete his daily ward rounds with-
Design,” in Design Research: Synergies
out a nurse escort. This result was important to the clinicians. To
from Interdisciplinary Perspectives, ed.
Jesper Simonsen et al. (Boston:
CSC, the major result of the experiment was the implementation of
Routledge, 2010): 16-32.
a ful y integrated EPR module that performed well throughout the
18 Morten Hertzum, and Jesper Simonsen,
trial period. Thus, CSC received a valuable reference in proving
“Positive Effects of Electronic Patient
that it has a highly configurable EPR platform that can deliver sat-
Records on Three Clinical Activities,”
isfying response times. However, the experiment also fostered sev-
International Journal of Medical
Informatics 77, no. 12 (2008): 809-17.
eral new desired changes that were unanticipated and significant.
16
DesignIssues: Volume 28, Number 3 Summer 2012

---

To summarize, using the large shared display during the
team conferences and nursing handovers resulted in various unan-
ticipated changes: the change from oral presentation to col ective
reading of patient records, initiation of col ective investigations of
patient records, and inclusion of nurses’ observations as a promi-
nent part of the shared agenda during team conferences. As a
direct consequence of the clinicians’ requests for coordination sup-
port, CSC initiated the design of a completely new EPR module
supporting task al ocation and management. After the experiment,
the nurses requested the addition of more structure to the nursing
record. This request resulted from their experiences of how struc-
tured nursing observations became part of the agenda during team
conferences. This request came as a surprise to the members of the
EPR unit, who expected that the nurses would resist rather than
request increased structure in their documentation.
Challenges for Participatory Design
We argue that the Participatory Design community should think
big by applying a sustained Participatory Design approach to large
information systems. Extending the iterative Participatory Design
approach beyond initial design (as outlined in Figure 1) raises the
overall challenge of how to manage this improvisational and
relatively open-ended process. We identify in the following
sections four major challenges in managing such a sustained,
iterative process.
Creating Appropriate Conditions for Participatory Design
Both customer and vendor need to be motivated and interested
in committing to a Participatory Design approach. An initial chal-
lenge, thus, is to obtain the appropriate conditions for Participatory
Design. This necessity might presuppose, for example, earlier expe-
riences and previous collaborations motivating Participatory
Design; access to mature, configurable development platforms; and
knowledge of other successful Participatory Design projects. In our
experiment the customer (the EPR unit) had become ready for a
Participatory Design approach through earlier experiences with a
drug administration module. The manager of the EPR unit (who
had a background as a physician) was further aware that the EPR
system supporting the clinical process could not be designed as a
one-size-fits-al standard system. The vendor (CSC), on the other
hand, had a new and highly configurable EPR platform and an
urgent need to prove its ability and to obtain a valuable reference.
Final y, the customer and the vendor knew each other from the
development and deployment of the drug administration module.
This mutual knowledge laid the foundation for the close partner-
ship and col aboration required by the experiment.
DesignIssues: Volume 28, Number 3 Summer 2012
17

---

Managing a Multitude of Stakeholders
Large-scale information-systems projects are characterized by the
involvement of a number of different actors spanning different
organizations and different organizational levels. Thus, a second
major chal enge is to manage and align the motivations and interests of
this multitude of stakeholders. Traditional y, the focus of Participatory
Design projects has been restricted to the relationship between
designer and end-users.19 As a result of our experiment, we can
identify the fol owing, broader range of stakeholders:

• Politicians and strategists engaged in health care


at a national level (requesting increased structuring

and standardization of the EPR content).

• The vendor (needing a reference for another

contractual bid).

• The EPR unit (requesting an initial structuring

of the nursing record and proof of system performance).

• The management of the stroke unit (requesting

improved quality of the reporting to a national

clinical research database).

• The physicians (striving to obtain a more autonomous

and efficient ward round).

• The nurses (wanting improved overview and

coordination during nursing handovers).
The challenge is to comply with the premises and goals set at
the national and political levels and by high-level organizational
strategists; to align these premises and goals at the different levels
represented by the stakeholders; and to argue how Participatory
Design, with its direct involvement of end-users, is an effective
means to manage, mesh, and meet the needs of these different
interests.

Navigating and managing this complex set of multiple
19 Clement and van den Besselaar, “A
stakeholders in a political environment is a major challenge to
Retrospective Look At PD Projects;”
Participatory Design approaches, as noted in other large-scale
Oostveen, and van den Besselaar,
projects.20 In our research, we experiment with using means-end
“From Small Scale to Large Scale
hierarchies, known from cognitive systems engineering as part of
User Participation.”
20 Bødker et al., Participatory IT Design;
a strategic analysis to identify and relate different stakeholders’
Oostveen, and van den Besselaar,
interests.21 Using such means-end hierarchies, we might, for exam-
“From Small Scale to Large Scale
ple, make the following argument: (1) A national and political
User Participation.”
demand for increased structure in the EPR system (2) can be met
21 Jens Rasmussen et al., Cognitive
by a stepwise change and incremental increase of the EPR struc-
Systems Engineering (New York:
ture, which again (3) can be initiated by introducing structure to
John Wiley and Sons, Inc., 1994); K. J.
Vicente, Cognitive Work Analysis:
the narrative part of nursing records, which (4) can only succeed
Towards Safe, Productive, and Healthy
if the categories fit the nurses’ documentation practice, (5) all of
Computer-Based Work (London:
which ultimately cal s for a Participatory Design approach focus-
Lawrence Erlbaum Associates, 1999);
ing on the nurses’ work practices.
Bødker et al., Participatory IT Design,
(especially chapter 5: In-Line Analysis
Phase: Strategic Alignment Analysis).
18
DesignIssues: Volume 28, Number 3 Summer 2012

---

Managing a Stepwise Implementation Process
A third major challenge is to effectively manage sustained, large-
scale, iterative Participatory Design experiments that form an overal
stepwise implementation process. This process includes managing
individual Participatory Design experiments, as well as an overal
stepwise implementation process that involves a series of such
experiments. The latter introduces an important problem of repre-
sentation: Our experiment was carried out in close col aboration
with one clinical specialty. How wel the results are transferable to
similar specialties at other hospitals remains an open question.

Our Participatory Design approach entails conducting a
series of experiments where functional prototypes are evaluated
during real use, resulting in a stepwise implementation process
similar to the technochange prototyping suggested by Markus.22 A
stepwise implementation process stands in contrast to the tradi-
tional way of managing large IT projects as a “design first then
implement” process;23 no iterations or improvisations are incorpo-
rated into the prevailing way of conducting competitive bids and
formulating IT contracts. The argument for a stepwise process
includes emphasizing the problems in the traditional process
while pointing to the less risky aspects in a phased implementa-
tion process. However, phased implementation also introduces the
chal enge of managing an implementation process that acknowl-
edges the need for improvisation.24

Participatory Design needs a strategy for managing this
challenge. In our research, we investigate how to manage a
stepwise design and implementation process on the basis of identi-
fying and measuring the effects of using a system.25 The sustained
22 Markus, “Technochange Management:
Participatory Design approach facilitates an iterative process
Using IT to Drive Organizational Change.”
managed on the basis of the effects of using a system: The desired
23 Ibid., 17.
24 Ibid., 18.
changes can be specified in terms of the effects of the system’s
25 Morten Hertzum, and Jesper Simonsen,
use, focusing on the work domain in question (e.g., to be able to
“Effects-Driven IT Development:
complete the ward round alone). We have been successful in con-
Specifying, Realizing, and Assessing
vincing managers in both the customer and vendor environment
Usage Effects,” Scandinavian Journal of
that such a sustained focus on effects is a promising idea and that
Information Systems 23, no. 1 (2011):
it potential y leads to an effects-based commercial contract model,
3-28. Hertzum and Simonsen, “Effects-
Driven IT Development: A Strategy for
where the customer’s payments depend on the effects that come
Sustained Participatory Design and
from using the vendor’s system.26 This research, however, is a work
Implementation,” in eds. K. Bødker et al.,
in progress, and many questions are still unresolved.
Proceedings of the 11th Biennial
Conference on Participatory Design:
Conducting Realistic, Large-Scale Participatory Design Experiments
Participation – the Challenge (New York:
ACM Press, 2010): 61-70.
A fourth major chal enge concerns the methodological question of
26 Morten Hertzum and Jesper Simonsen,
how to conduct realistic, large-scale Participatory Design experiments to
“Effects-Driven IT Development: Status
evaluate prototype systems during real work. Our experiment
2004-2011” in Balancing Sourcing and
raises two issues in respect to this chal enge: the restricted time-
Innovation in Information Systems
frame for evaluations and the need to safeguard against errors.
Development, ed. Morten Hertzum and
Carsten Jørgensen (Trondheim, NO: Tapir
Academic Publishers, 2011): 165-92.
DesignIssues: Volume 28, Number 3 Summer 2012
19

---

The timing of real-life experiments is a trade-off between
two perspectives:

• Evaluating early and quickly to acknowledge project

deadlines, save resources, and curtail the diffusion of

ineffective systems.

• Evaluating after a longer period of time to al ow

system errors to be corrected, users to gain proficiency,

work practices to stabilize, use situations to reach

their true level of heterogeneity, emergent and

opportunity-based changes to develop, and

long-term outcomes to emerge.
If a Participatory Design experiment is biased toward early and
brief evaluation to honor the realities of IT projects, the conse-
quences of various learning effects become critical to the interpre-
tation of the experiment.

In our experiment, the trial period was five days. In this
short period of time, none of the clinicians gained proficiency in
using the EPR system, and their ways of working were thus in
flux; meanwhile, their prior effective use of paper records was
facilitated by long-standing work practices. The encouraging ele-
ment is that some improvements could be identified after using the
EPR system for only five days. However, longer trial periods are
highly desirable because they, among other things, provide a
means of getting beyond the goodwill that can be a factor in trying
something new for a restricted period of time.

Special precautions against errors may be necessary to eval-
uate systems during real use. Participatory Design experiments
involve a balancing of the benefits of evaluating prototype systems
during real use against the confounding elements introduced
because of the necessity of taking special precautions to safeguard
against unacceptable errors. While experiments with real use
increases validity and the possibility of unanticipated discoveries,
special precautions may reduce validity. For safety- or security-
critical systems, leaving users to a process of trial and error when
they encounter situations not covered by training might not be
acceptable. Thus, users must have immediate access to appropriate
support during the entire real-use experiment.

In our experiment, the clinicians were supported by shad-
ows, and certain parts of the EPR system were simulated by the
back office using Wizard of Oz techniques,27 where designers from
the vendor played the “wizard” by simulating the system’s trans-
actions with other wards. These precautions were necessary
because troubles and misunderstandings in using the system
27 David Maulsby et al., “Prototyping an
could have entailed risk to patient health. However, with these
Intelligent Agent Through Wizard of Oz,”
precautions in place, the EPR system could replace paper records
in Proceedings of the SIGCHI Conference
for the duration of the trial period.
on Human Factors in Computing Systems
(New York: ACM Press, 1993), 277-84.
20
DesignIssues: Volume 28, Number 3 Summer 2012

---

Conclusion
Participatory Design has achieved an international reputation and
application. Nevertheless, its proponents still seem reluctant to
engage it in the development of large-scale information systems.
Participatory Design undoubtedly has a lot to offer; but as an
approach, it also faces considerable chal enges in claiming a seri-
ous influence on the design and implementation of large-scale
information systems.

We have suggested an ambitious and sustained Partici-
patory Design approach, emphasizing that mutual learning
situations should be provided during the organizational imple-
mentation of large-scale systems. This approach acknowledges the
uncertainties of technology-driven organizational change and at
the same time poses the chal enge of treating the entire design and
implementation process as a process of genuine development. Our
sustained Participatory Design approach incorporates anticipated
changes, as well as emergent and opportunity-based changes, as
identified by Orlikowski and Hofman.28 We argue for large-scale
Participatory Design experiments that transcend traditional proto-
typing tests to evaluate systems as they are exposed to real
work situations.

We have reflected on our experiences leveraging Partici-
pated Design in the Danish healthcare sector and have reviewed
the important lessons learned. Four major chal enges have been
discussed: the establishment of appropriate conditions for
Participatory Design; the handling of the different interests of a
multitude of stakeholders; the management of an ongoing and
stepwise implementation process, guided by a series of large-scale
Participatory Design experiments; and the conduct of experiments
during which the system is in real use, although it is still being
designed as opposed to deployed.

So far, this approach has yielded promising results in the
Danish healthcare sector. However, applying it also forces us to
face the chal enges described. It thereby raises a number of how-to
questions that cannot be satisfactorily answered with general
methodological guidelines. What we need is research—preferably
28 Orlikowski and Hofman, “An
action research—that refines this Participatory Design approach
Improvisational Model for Change
by applying it in a number of cases and thus stimulating the
Management.”
mutual creation and sharing of knowledge and experiences.
DesignIssues: Volume 28, Number 3 Summer 2012
21

---