Sonification, Auditory Display, Embodied Cognition, Human-computer Interaction, Sound & Music Computing, Multi-modal Data Analytics
Auditory display is the use of sound to present information to a listener. Sonification is aparticular type of auditory display technique in which data is mapped to non-speech sound tocommunicate information about its source to a listener. Sonification generally aims to leveragethe temporal and frequency resolution of the human ear and is a useful technique for representingdata that cannot be represented by visual means alone. Taking this perspective as our point ofdeparture, we believe that sonification may benefit from being informed by aestheticexplorations and academic developments within the wider fields of music technology, electronicmusic and sonic arts. In this paper, we will seek to explore areas of common ground betweensonification and electronic music/sonic arts using unifying frameworks derived from musicalaesthetics and embodied cognitive science (Kendall, 2014; Lakoff & Johnson, 1999).Sonification techniques have been applied across a wide range of contexts including thepresentation of information to the visually impaired (Yoshida et al., 2011), process monitoringfor business and industry (Vickers, 2011), medical applications (Ballora et al., 2004), humancomputer interfaces (Brewster, 1994), to supplement or replace visual displays (Fitch & Kramer,1994), exploratory data analysis (Hermann & Ritter, 1999) and, most importantly for the currentmilieu, to reveal the invisible data flows of smart cities and the internet of things (Rimland et al.,2013; Lockton et al., 2014). The use of sonification as a broad and inclusive aesthetic practiceand cultural medium for sharing, using and enjoying information is discussed by Barrass (2012).As networked smart societies grow in size and becomes increasingly complex the ubiquitousinvisible data flows upon which these societies run are becoming hard to monitor and understandby visual means alone. Sonification might provide a means by which these invisible data flowscan be monitored and understood.In order to achieve this type of usage, sonification solutions need to be applicable to andintelligible to an audience of general listeners. This requires a universal shared context by whichsonifications can be interpreted. Embodied cognition researchers argue that the shared physicalfeatures of the human body, and the capacities and actions which our bodies afford us, defineand specify mid-level structures of human cognitive processing, providing shared contexts bywhich people can interpret meaning in and assign meaning to their worlds (Lakoff and Johnson1980; 1999; Varela et al., 1991). At present, embodied perspectives on cognition are infrequentlyexplored in auditory display research, which tends to focus on either higher level processing interms of language and semiotics (Vickers, 2012) or lower level processing in terms ofpsychoacoustics and Auditory Scene Analysis (Carlile, 2011).
(PDF) Sounding Human with Data: The Role of Embodied Conceptual Metaphors and Aesthetics in Representing and Exploring Data Sets.