Music performance with delayed auditory feedback: an embodied perspective

  
16 
back in music performance, reporting the results coming from investigations on mu-
sic performance without auditory feedback (Subparagraph 1.3.1) and under DAF 
(Subparagraph 1.3.2). Subparagraph 1.3.3 resumes these findings. Paragraph 1.4 
shows how latency affects many electronic instruments and interfaces, so that DAF 
contexts are frequently found in the field of music production. Paragraph 1.5 presents 
the adopted ecological, embodied approach to the study of DAF affected music per-
formance. Lastly, Paragraph 1.6 describes the organization of the next chapters.   
 
1.1 Music performance  
 
Music performance is a highly skilled activity that involves both cognitive and motor 
capabilities and demands for a strict connection between them. The ability of skilled 
musicians to coordinate fine body movements to produce complex meaningful se-
quences is often considered as one of the ultimate examples of human motor skills 
(Bernstein, 1967; Lashley, 1951). Some musicians are capable of carrying into action 
extreme tasks, showing impressive abilities in the control of hand and finger move-
ments. Skilled pianists, for example, can produce movements at rates that exceed 
visual reaction times (e.g., in the execution of trills; Lashley, 1951), playing up to 30 
sequential notes per second for sustained periods (Rumelhart & Norman, 1982). 
High-level playing is based on long-term and intensive rehearsal of motor patterns, 
that has the aim of forming an inner space of automatic motor trajectories to be re-
called and generated without paying too much conscious attention to them (Leman, 
2007). Decades of regular practise are necessary to completely automate the motor 
patterns: the hours of training needed can be roughly estimated at 10,000 (Ericsson, 
Krampe, & Tesch-Römer, 1993; Howe, Davisdon, & Sloboda, 1998). The acquired 
experience permits good players to focus on musicality - the transmission of the ex-
pressive intentions - rather than on movements and technique. Thoughts, emotions, 
aesthetic forms and ideas can thus be communicated to the audience through sounds. 
 
1.1.1 Music performance and sensory feedback 
In music performance, a strict correspondence between player’s intentions and gen-
erated sound is necessary for the transmission of the correct musical message. There-
fore, musicians continuously monitor their performance through sensory feedback, 
  
17 
also when they’re playing solo. Some musical activities like group playing (e.g., or-
chestral music) require in addition the player to coordinate with other musicians, so 
that visual and auditory feedback plays a more central role (for the interdependency 
between musicians during ensemble performances see Rasch, 1979; 1988).  In some 
playing styles based on group improvisation (e.g., jazz ensembles, see Figure 1.1) 
feedback is absolutely necessary: musicians are often called to decide in real-time 
what and how to play basing themselves on the information they collect about what 
their colleagues are playing at the same time (e.g., in standard jazz improvisation, 
“interdependent routines such as call and response, propagating motifs, supporting 
and contrasting dialogs, and a higher level of leader/follower dynamics”; Weinberg, 
2002, p. 21). These observations point out the necessity of investigations of the role 
of feedback in music performance: is feedback always necessary for playing? Is 
feedback used for error correction? Are musicians able to cope with distorted sensory 
feedback? In the following paragraphs, relying on the results in literature, I will try to 
answer these questions limiting to the case of solo music performance, in which 
feedback is not used for inter-subjects coordination.   
 
 
 
Figure 1.1 A jazz quartet performance. In improvised group music auditory and vis-
ual feedback have a very central role.  
  
18 
1.1.2 Music performance as a timed sequence of motor acts 
A common approach to the study of human serial behaviours like music performance 
or speaking implies a focus on the role of feedback. This kind of serial behaviours 
can be viewed as examples of timed sequences of motor acts: research on feedback 
investigates how such timed sequences are actually produced, which is indicated by 
Lashley (1951) as one of the central problems of cognition. The manner in which 
perceived consequences of actions influence the production of subsequent ones is 
studied, with the aim of clarifying both planning and production mechanisms. In 
general, understanding the role of feedback is crucial for the comprehension of the 
relationships between action and perception (see Pfordresher, 2006): this two com-
ponents of human behaviour are simultaneously involved in sequence production, 
and require a certain degree of congruency to keep the production fluent and correct. 
In the last analysis, understanding the role of feedback is important to clarify the 
complex relationships between the inner subjective world of human beings, ex-
pressed through action, and the external reality (even if, as we will see, such a strict 
subdivision between inner and outer is somewhat outdated).  
 
1.2 The role of feedback in movement execution  
 
1.2.1 Closed-loop vs open-loop models 
The control of motor acts such as musicians’ movements can be  explained by two 
main theoretical models: the closed-loop model, in which, during a movement, feed-
back is used to control if the goal is being achieved, and the open-loop model, in 
which feedback is not used for correctness control. In the closed-loop model, sensory 
feedback is necessary since movement control is totally depending on the peripheral 
information (feedback control hypothesis): the execution and the completion of an 
action is guided by a centralized comparison between the intended movement and the 
feedback information. Vice versa, in the open-loop model, execution is centrally 
leaded by an abstract representation of the motor sequence stored in memory (motor 
program), and feedback can have a role in determining or triggering possible re-
sponses, but not in the guidance of the current movement. An early variant of the 
open-loop theory is the response-chaining hypothesis (James, 1890), in which 
  
19 
movement is composed by a chain of muscular contractions, and the feedback from 
one contraction (response-produced feedback) serves as a stimulus for the next con-
traction in the chain. In this approach, sensory feedback, as a trigger, is necessary for 
the execution of the movement, in contrast with other open-loop theories according 
to which feedback doesn’t have a role at all in the execution phase. It may be of in-
terest for the present discussion to notice that the response-chaining hypothesis pro-
vides an account for the timing among the contractions, very important in skilled ac-
tivities like music performance: such timing, called relative timing, would be deter-
mined by the temporal delays in the various sensory processes (i.e., responses propa-
gation).  
Each motor control theory relies on some experimental evidences and is opposed by 
others. In general, closed-loop theories give the better account for longer duration 
movements, such as driving a car, in which the possibility of error correction is evi-
dent, whereas open-loop theories seem more suitable for explaining rapid move-
ments. The hypothesis of the cohabitation of different types of motor control relies 
on the limited velocity of feedback transmission. In fact, though some studies 
(Bowman & Combs, 1969; Cohen, Goto, Shanzer & Weiss, 1965; Fuchs & Korn-
huber, 1968) have shown that some kinds of very fast responses – 5 to 10 ms – are 
possible, the elapsed time between the error detection and the start of the correction 
is estimated on average at 200 ms. During the execution of fast movements, a re-
sponse time of 200 ms is too long to permit feedback to have an active role in error 
correction, as stated by closed-loop theories: open-loop theories seem therefore 
likely, at least for this class of movements.  
 
1.3 Auditory feedback in music performance  
 
In order to investigate the role of feedback in sound production tasks like music and 
speech, audition is the most studied feedback channel. In these kind of tasks, in fact, 
the output of the system is sound: therefore, though in music performance visual, tac-
tile and proprioceptive feedback are very important (for the importance of vision in 
music performance see Sloboda, 1982; Banton, 1995), audition is the only feedback 
channel which consents a direct comparison between the produced action (e.g., a 
keypress) and the desired goal. Experimenting with auditory feedback is therefore an 
  
20 
appropriate solution to test whether open-loop or closed-loop systems are involved in 
music production.  
The role of auditory feedback in sound sequence production has been investigated 
mainly through two experimental approaches. The first one implies a study of the ef-
fects of feedback removal: an impairment of production, in the absence of feedback, 
would indicate the necessity of auditory feedback, at least for a correct realization of 
the production aspects disrupted by such absence. A second approach consists in 
studying the effects of feedback alteration: each sound resulting from a produced ac-
tion is altered, so that the coordination of the auditory feedback with actions is modi-
fied. This experimental paradigm is known as altered auditory feedback (AAF). Al-
terations can occur in the dimensions of time and pitch, where timing alterations in-
troduce an asynchrony between the onset of a produced action and the onset of the 
corresponding feedback, whereas in pitch alterations the pitch associated with a pro-
duced action is not the pitch that is normally associated with that action. Both kinds 
of alteration can be simultaneously present, giving rise to a combination of onset 
asynchrony and unexpected feedback content. The AAF paradigm means to investi-
gate the relationships between action and perception, and, in particular, to answer the 
questions about how action and perception are bounded together.  
 
1.3.1 Feedback deprivation 
Studies on feedback deprivation seem to show that auditory feedback is not strictly 
necessary in music sequence production. In fact, though auditory feedback is shown 
to be important in the learning phase (Finney & Palmer, 2003; Highben & Palmer, 
2004), other researches indicate that its absence doesn’t significantly impair the pro-
duction of learned sequences (Gates & Bradshaw, 1974; Banton, 1995; Finney, 1997; 
Repp, 1999), even for untrained performers (Pfordresher, 2005). However, auditory 
feedback may still be necessary in some kinds of fine control, since Repp (1999) re-
ported small effects of its absence on expressive parameters of production. The fact 
that auditory feedback doesn’t appear to be necessary in music performance supports 
Lashley’s open-loop theory (Lashley, 1951), that, based on the trilling speed of con-
cert pianists, argued for the impossibility of a role of feedback in motor control dur-
ing very fast movements (see also Keele, 1968). However, in the above mentioned 
studies on auditory feedback deprivation, the remaining feedback channels were not 
  
21 
inhibited, so that it could be argued that visual, tactile and proprioceptive feedback 
can still guide players’ movement for a correct execution. Nevertheless, feedback 
does not have much time to affect execution, which testifies against this hypothesis, 
as well as the fact that some kind of activities seem to be executable in absence of 
kinesthetic feedback (Keele & Summers, 1976; Lashley, 1951). 
To sum up, relying on the results in literature, it seems likely that motor acts in music 
performance are memorized through sensing during training, to form an inner space 
of motor trajectories (Leman, 2007). These trajectories can be recalled without the 
aid of auditory feedback, even if, in this case, a small degradation of fine perform-
ance parameters is possible. For what concerns motor control, a motor program 
(open-loop) model is adopted, at least for fast movements: comparison between 
feedback and intended results may be used to adapt subsequent actions, but not to 
guide the current. On the other hand, closed-loop models can be applied to slower 
movements, for which error correction is possible. 
 
1.3.2 Delayed auditory feedback (DAF) 
The most extensively studied AAF paradigm consists of introducing a certain delay 
between the onset of a produced action and the onset of the corresponding feedback: 
this experimental condition is called delayed auditory feedback (DAF). It is well in-
vestigated how DAF strongly disrupts sequence production in many tasks including 
music performance, speech, and rhythmic tapping. Two early studies on DAF speech 
(Lee, 1950; Black, 1951) reported significant slowing of production rate, increased 
sound level, and increased articulatory errors, with a predominance of insertions and 
repetitions. Many studies confirmed these and other negative effects of DAF on the 
various kind of sequence production tasks. A review of the studies on music per-
formance under DAF is given in Chapter 2.  
The introduction of a delay between note onsets and feedback onsets may lead to 
three different situations, depending on the relationship between the amount of delay 
(delay length) and the inter-onset-intervals (IOIs) duration. First, when the delay 
length is shorter than the IOIs duration, the feedback onset of a produced event i oc-
curs before the produced event  1i : in this case, only the timing of production is 
altered. The second case occurs when the delay length is equal to the IOIs duration, 
so that the feedback onset of a produced event i is simultaneous to the produced 
  
22 
event  1i : in such situation, only the pitch contents are altered. The third case oc-
curs when the delay length is longer than the IOIs duration, and the feedback onset of 
a produced event i succeeds the produced event  1i : in this case, both timing and 
pitch alterations are present. 
In general, DAF disrupts both temporal relationships between note onsets (timing) 
and notes correctness (accuracy). Recently, Pfordresher (2003) showed that altera-
tions of feedback timing disrupt the timing of sequencing more than the accuracy, 
whereas alterations of feedback content without asynchrony (e.g., pitch alterations 
that occur with DAF when the delay time is equal to the IOIs duration) disrupt accu-
racy but do not influence much timing. These findings suggest that a strict connec-
tion exists between the disrupted aspects of the performance (i.e., timing or accuracy) 
and the kind of feedback alteration (i.e., timing or pitch alterations).  
Critical interval vs relative time hypothesis 
A frequently discussed topic regarding DAF disruption is the kind of dependency 
from the amount of delay, and, in particular, the amount of delay that causes maxi-
mal disruption. In the early studies, disruption caused by DAF was found to increase 
with the amount of delay up to a certain point, called critical (delay) interval, or de-
lay of maximal impairment, and then to reach asymptote (e.g., in music, Gates, Brad-
shaw, & Nettleton, 1974, found an asymptote around 270 ms,) or to decrease (e.g., in 
speech, Fairbanks & Guttman, 1958). These findings have given credit to the so 
called absolute time hypothesis, according to which the delay of maximal impairment 
occurs when the absolute temporal separation between a produced action ant its 
feedback onset falls within a certain temporal window, regardless of the production 
rate. Anyway, all experiments on music performance supporting this view have been 
using  fixed delay lengths, so that, in case of tempo variations or of non-isochronous 
pieces, the phase relationships between the timing of produced actions and their rela-
tive feedback onsets would not be constant. Moreover, production rate was usually 
not controlled. Recently, Pfordresher & Benitez (2007), using a kind of delay called 
adjustable delay, which roughly preserve the phase relationships between actions and 
feedback onsets, found that disruption is best predicted by the relative phase location 
rather than by the absolute position of feedback onsets. This result gives support to 
  
23 
the relative time hypothesis, which states that perception and action are coordinated 
according to the rhythmic cycles formed by IOIs (cf. Jones, 1976; Robinson, 1972).  
Break-point interval 
Another discussed issue concerning DAF is which is the minimum delay length 
which causes auditory feedback to be actually perceived as delayed and DAF disrup-
tion to become significant. This topic is related to the investigation of temporal 
thresholds in perception and cognition. In a classical study on vision, Card, Moran & 
Newell (1983) showed that, if two events are connected by an immediate causality 
relationship and the perception of the second is progressively delayed, degradation of 
immediate causality starts for some subjects as early as 50 ms. Moreover, they found 
that, while the perception of immediate causality ends around 100 ms, perception of 
delayed causality begins at 50 ms, reaches a peak around 100 ms, and terminates 
around 160 ms, threshold after which the events are recognized as independent. Re-
search on the temporal ordering of two distinct stimuli showed that the events re-
quired a minimum of 30 ms to be perceived as successive, regardless of sensory mo-
dality (Poppel, 1997). In the field of music perception, Rasch (1979) observed that 
listeners often judge ensemble performance as synchronous despite asynchronies of 
30-50 ms.  
For what concerns playing an instrument under DAF, Finney (1997) reported that 
professional pianists may perceive delay lengths of under 10ms, so that this threshold 
is often suggested as the maximum latency for a music controller (Finney, 1997; 
Freed, Chaudhary, & Davila, 1997). However, a certain degree of tolerance to higher 
latencies is well-documented. Dahl & Bresin (2001), in a study on synchronization 
under DAF, individuated between 40 and 55 ms a possible break-point at which DAF 
begins to make the performance increasingly difficult. Mäki-Patola & Hämäläinen 
(2004), testing the threshold of latency tolerance of subjects playing a theremin (a 
continuous sound instrument without tactile feedback), reported that latency started 
to be perceived at 30 ms, when comparing to a reference with zero latency. With this 
delay length, subjects perceived latency with a high degree of uncertainty; conscious 
detection was found to start at 60 ms.  
Other studies addressed the break-point interval problem for network duet perform-
ances. In this situation, differently from solo playing, inter-subjects coordination is 
  
24 
required, so that the effect of DAF is supposed to be even stronger. However, results 
do not differ from what found as for solo playing. Chew, Sawchuk & colleagues 
(Sawchuk et al., 2003; Chew et al., 2004) found latency tolerance in network playing 
to be dependent on both the piece and the instrument played. In general, in such 
situations, they suggested a general threshold for latency tolerance at 50 ms. Chafe,  
Gurevich, Leslie, & Tyan (2004), quantifying the effects of latency on rhythmic 
clapping network performance, showed that the performance is at its best when the 
round-trip bi-directional latency is comprised between 20 and 30 ms, and degrades 
with higher latencies.  
Basing on these empirical results, the interval of delay lengths in the range of 30-60 
ms seems therefore to be a plausible break-point for the correct execution of music 
performance under DAF. A certain variability is to be taken into account, mostly due 
to the kind of instrument and the piece played.    
Theoretical implications 
In a recent review, Pfordersher (2006) resumed the empirical results on music per-
formance under AAF, and DAF in particular, discussing their theoretical implica-
tions. What clearly emerges from studies on DAF is that, despite the fact that audi-
tory feedback is not necessary for musical sequence production, a certain match be-
tween auditory feedback and produced actions is required when feedback is present. 
In other words, congruency between action and perception is needed: miscoordina-
tion between them causes disruption of performance. In particular, it seems that high 
impairment from DAF is due to the fact that the feedback sequence is equal in struc-
ture to the planned events sequence, but it is also in conflict with it, since each feed-
back onset occurs simultaneously to the production of events with a different posi-
tion in the planned sequence. In other words, it is likely that DAF disrupts production 
by virtue of the interfering effect of perception of events planned for the past on the 
current activation of subsequent events for production. This view is strengthened by 
research on AAF paradigms different from DAF: manipulations of feedback contents 
resulting in a sequence of events highly dissimilar to the planned sequence (extrane-
ous feedback) do not disrupt performance as much as DAF does (Gates & Bradshaw, 
1974; Finney, 1997; see Finney 1999 for a review of similar results in speech pro-
duction). Further support comes from the reduced disruption reported by Finney