The musical composition process has been studied so far mostly from an individual perspective (e.g. Donin07). Alternatively some studies have addressed the issue of collaborative composition (Donin06). In this study, we are interested in a somewhat intermediary activity: composition with peer feedback. This type of composition is becoming prevalent with the increase of on-line music composition systems and pedagogical tools (e.g. Nuernberg13). To which extent is feedback useful, relevant, or creativity enhancing? To which extent peer feedback can affect, positively or negatively, the quality of a musical composition? To which extent the impact of a feedback is related to the musical skills of the commentator? (i.e. do better composers produce better feedback?).
In this study we propose a quantitative study to address these questions (and others) through an experiment in leadsheet composition in popular music genres and we report on the findings obtained so far. Leadsheets are monophonic melodies associated with chord labels. Leadsheets are routinely used in many styles of popular music such as songwriting, jazz, or bossa nova. In order to study the composition process, we consider the case where a composer creates an initial version of a leadsheet, and then tries to improve it based on feedback received by commentators. The feedback we consider here is modeled as a set of suggestions to modify certain notes or chord labels.
The first question addressed here concerns the effect of feedback. Peer feedback, in general, has been claimed to yield beneficial effects (Rollinson05). However, this effect depends on many factors (valence, source, timing, etc.). Therefore we can expect that peer feedback has a beneficial effect, e.g through an enhancement of the subjective appreciation of the composed song.
Another question is to understand what makes a commentator a good one, i.e. who makes good musical suggestions. We can distinguish here composition skills, which can be evaluated from judgments of the original compositions, from the commenting skills which can be evaluated from comparisons of different versions of songs (before/after taking suggestions into account). We can expect only a weak link between these two skills, in spite of their musicality proximity, because composing is a creative activity with low constraints compared to suggesting which implies strong constraints from the original composition (style, tempo, rhythm, key, etc.). Constraints are known to have a large impact on final production (e.g. Smith93), and might explain some of the differences between the composition skill and commenting skills.

Protocol
In a first phase, a group of subjects are asked to 1) compose an original leadsheet and 2) suggest modifications to leadsheets composed by others. In a second phase, each subject of a group reviews the suggestions received for his composition and has the possibility to modify his leadsheet (with the goal of improving it). All subjects are asked to answer an initial questionnaire about their musical composition skills. The experiment is carried out by means of a web-based software for collaborative score edition. The subjects are divided in two groups:
Group 1 is the control group. Subjects write a composition and later, without receiving suggestions, may improve it by themselves.
Subjects of the experimental group (group 2) write a composition and later receive suggestions on their composition from two other subjects. Then, they compose an improved version after reviewing those suggestions.
In this experiment, modifications are anonymous and subjects cannot discuss with each other, even though subjects can explain their musical suggestions when providing feedback. To quantify the impact of feedback we evaluate the quality of a composition as well as that of the improved version. The quality of a composition is estimated from a social consensus, obtained by asking the opinion of the participants. They evaluate the quality of each pair of original and improved compositions. We track how many times they listen to each original song. Rating is an integer in [1, 5] obtained from a consensual evaluation method (Baer04).

Results and discussion
Preliminary results confirm our expectations about the general impact of feedback: the
experimental group does show a larger improvement in quality between their first version and their final version, as well as on the subjective evaluation of the composer. These findings suggest that anonymous peer feedback can produce an improvement on both the consensued quality of a composition and on the composers’ satisfaction.
Further, we have checked whether the experience of a subject can influence his evaluation when judging other subjects’ compositions. We have found that subjects tend to give higher marks to compositions of other subjects with a similar musical experience. This finding is in line with Bonnardel05, who showed the influence of experience in evocation processes in creative tasks. Other questions are investigated. For example, we can identify which suggestions are useful to a composer by, on one hand, tracking which ones the composer used when trying to improve the song (in the second phase), and on the other hand, by computing automatically the similarity between the final version and the ones suggested using various MIR-based distances.

Conclusion
Music composition is a creative activity which, as such, might benefit from inspiration
from peers. Feedback provided by peers can be seen as inspiring examples and seems to enhance the quality of the final production. More generally, this study is a step towards a quantitative analysis of the impact of peer feedback on the music creation process. More sophisticated forms of suggestions could be taken into account (e.g. not only on local structures). Ideally, insights about optimal groups of commentators (in terms of skill level or variety for instance) could have direct benefit to the design of on-line musical composition and training systems.