An Online Networking Event on Acoustic Ecology & Bioacoustics
Photo: From left-right top-down, Mark Plumbley, Anna Xambó, Pete Batchelor, Frederic Font, Jinhua Liang, Emmanouil Benetos, Panagiota Anastasopoulou, Amaia Sagasti, Lia Mazzari, Nicolas Farrugia, Błażej Kotowski, Ilyass Moummad, Antonella Maria Cristina Torrisi, and Inês De Almeida Nolasco.
On Monday 13th May, 13:30 to 15:00 BST, it took place an informal gathering across C4DM-QMUL, MTG-UPF and the AHRC Sensing the Forest project team members & friends with the common goal of promoting knowledge exchange and networking on acoustic ecology & bioacoustics.
The rationale for this meeting emerged from: 1) discussions with Emmanouil Benetos about potential synergies between the aspect of acoustic ecology of the Sensing the Forest project and the work of some current PhD students at C4DM - not to say the pioneering work done by Emmanouil in acoustic ecology and Dan Stowell in bioacoustics for many years in the centre; 2) discussions with Frederic Font about the collaboration of Sensing the Forest with Freesound and the subsequent relevant research that MTG-UPF is doing; and 3) discussions in the 2nd Advisory Board meeting of the project, with some highlights raised by the advisory board member Mark Plumbley on porting acoustic ecology to embedded systems and what can Sensing the Forest do about it, with connections with the UK Acoustics Network Plus (UKAN+)'s proposal for a Noise Network Plus.
The meeting was hybrid: we met on Zoom but with a group of six C4DM members on-site located in the same meeting room. To allow time for both individual presentations and group discussions, we organised the session as follows:
- Part 1 (40-45 min) - 13:30-14:15 - Welcome & Quick round of 3-minute presentations about your work (feel free to use 1-2 slides or other media).
- Part 2a (20 min) - 14:15-14:35 - Small group discussions
- Part 2b (20 min) - 14:35-14:55 - General group discussion
- Part 2c (5 min) - 14:55-15:00 - Closing
Next, a summary of how the meeting went is presented.
3-minute presentations
The brief for this part was that each of the participants should be ready to tell:
- Who you are
- What is your institution and role
- How does your work relate to acoustic ecology/bioacoustics and from what disciplinary perspective(s)
- Why are you interested in acoustic ecology/bioacoustics
We decided on the running order using an algorithmic tool. Next, we give a brief overview of the 14 presenters in terms of role, research topic, and how this connects to acoustic ecology and/or bioacoustics:
Frederic Font
Frederic Font leads the Freesound project and all Freesound-related research at the Music Technology Group (MTG), Universitat Pompeu Fabra (UPF). From 2016-2019, Frederic was the coordinator of the EU H2020-funded project Audio Commons, which also involved Mark Plumbley and Anna Xambó, among others. Frederic mentions that Freesound links well (and has potential) to connect even more with topics related to bioacoustics, a direction that is of current interest at the MTG.
Jinhua Liang
Jinhua Liang is pursuing his doctoral degree at Queen Mary University of London (QMUL), co-supervised by Emmanouil Benetos, Huy Phan, and Mark Sandler on the topic of AI for everyday sounds. Jinhua is a member of the Machine listening lab at C4DM, QMUL. His research interests mainly focus on everyday sound perception, including sound event detection, acoustic scene classification, and audio captioning. His connection with bioacoustics relates to collaborations with colleagues such as Inês Nolasco with the Few-shot Bioacoustic Event Detection.
Nicolas Farrugia
Nicolas Farrugia is an assistant professor ([HdR]) at IMT Atlantique. Nicolas is in the BRAIn team (Better Representations for Artificial Intelligence) together with Vincent Gripon, Giulia Lioi, Bastien Pasdeloup, Axel Marmoret and Mathieu Leonardon. His research interests include developing innovative methods to better understand Sounds and the Brain using modern machine learning and deep learning. Nicolas presented three relevant projects:
- Nocturnal environment observatory - Field work on (nocturnal) soundscapes of socio-ecosystems.
- Silent Cities - A dataset of acoustic measurements from soundscapes collected worldwide during the COVID-19 pandemic.
- LATITUDE - long-term monitoring of Arctic biodiversity.
Mark Plumbley
Mark Plumbley is a Professor of Signal Processing and EPSRC Fellow in “AI for Sound” (University of Surrey) as well as an Advisory Board member of the Sensing the Forest project. Mark’s research concerns AI for Sound: using machine learning and signal processing for analysis and recognition of sounds. Mark’s focus is on the detection, classification and separation of acoustic scenes and events, particularly real-world sounds, using methods such as deep learning, sparse representations and probabilistic models. Mark presented the related work conducted at the University of Surrey involving several centres, such as the Centre for Vision, Speech and Signal Processing (CVSSP), the Institute of Sound Recording (IlSR), the Digital World Research Centre (DWRC), and the Centre for Digital Economy (CoDE), among others. Mark mentioned a range of related projects (e.g. Audio Commons, AI4ME, making sense of sounds, S3A Future Spatial Audio, AI for sound) and gave an overview of potential tasks and applications.
Anna Xambó
Anna Xambó is a Senior Lecturer in Sound and Music Computing at C4DM and the PI of the Sensing the Forest project. Anna presented her perspective on acoustic ecology as two-fold: as a practitioner and as a human-computer interaction (HCI) researcher. As a practitioner, Anna has been using everyday sounds in her practice, especially involving Freesound. As an HCI researcher, her work focuses on creating systems and interfaces to interact with these types of sounds. The Sensing the Forest project aims to record natural soundscapes for further analysis in collaboration with Freesound. Hence, acoustic ecology and bioacoustics will be relevant for sound recognition, sound processing and sound analysis.
Ilyass Moummad
Ilyass Moummad is a PHD Student (last year) at IMT Atlantique, working under the supervision of Nicolas Farrugia and co-supervised by Romain Serizel. Currently, Ilyass is a Visiting Researcher at C4DM, QMUL, working under the supervision of Emmanouil Benetos. Ilyass’ PhD topic is Deep Learning for Bioacoustics with a special interest in representation learning of animal sounds and few-shot learning (species sound classification and detection). His current research focus is on invariant learning: how to learn bird sound representations without human annotations? (see Self-Supervised Learning for Few-Shot Bird Sound Classification). The goal is to come up with general-purpose representations of bird songs (e.g. extend to Xeno-Canto, a website dedicated to sharing wildlife sounds from all over the world with more than 10.000 bird species), and evaluate the learned representations.
Inês Nolasco
Inês Nolasco is a PhD student (last year) at the Machine listening lab, C4DM, QMUL. Inês’ PhD topic is the automatic identification of individual animals for understanding animal behaviour and improving population monitoring. Information used to recognise individuals includes call-response and acoustic signatures. Inês’ work focuses on building on these methods to distinguish these individuals automatically. Inês is currently working on the Few-shot Bioacoustic Event Detection with other colleagues such as Jinhua Liang. This task focuses on sound event detection in a few-shot learning setting for animal (mammal and bird) vocalisations. Participants will be expected to create a method that can extract information from five exemplar vocalisations (shots) of mammals or birds and detect and classify sounds in field recordings.
Antonella Maria Cristina Torrisi
Antonella Maria Cristina Torrisi is a 2nd-year PhD student at the Machine listening lab, C4DM, QMUL, working under the supervision of Emmanouil Benetos and Elisabetta Versace. Antonella’s research interests include animal behaviour, behavioural analysis and sound analysis. Antonella’s PhD topic focuses on enhancing the affective state of chicks through communication with an artificial agent. The aim is to develop a new technology to detect and enhance animal emotional states and welfare. The research question revolves around whether the interaction with an artificial agent, used as a vicarious hen, can improve the chick’s welfare. Antonella presented the project outline to develop such a system and the research done so far.
Peter Batchelor
Peter Batchelor is a Senior Lecturer and Programme Leader in Music Technology, and a member of the Music, Technology and Innovation - Institute of Sonic Creativity (MTI2) at De Montfort University, Leicester, and a Co-I of the Sensing the Forest project. Pete presented his work in acousmatic composition (composition with recorded sound), typically presented in a multichannel format. Pete introduced his motivation to move from the concert hall to multichannel installations. His research interests include exploring ecological relationships within and between sound environments in an artistic outcome and sonifying it in an accessible way e.g. using Raspberry Pis. Lately, he has become interested in representing the sounds in natural environments rather than appropriating them, which connects with acoustic ecology, through that intervention.
Lia Mazzari
Lia Mazzari is a sound artist and researcher based in Bristol. Lia is currently pursuing her practice-based PhD at the Geography Department of Royal Holloway, University of London. Lia works with performance, composition, installation, and curation, among others. Her research focuses on looking at the potential of live audio streaming (linked to the Soundcamp project and Luigi Marino’s work with the streamers in the Sensing the Forest project), which can connect with sonic activism, asking research questions such as: How can soundscapes be understood through live streaming? What forms of cultural and geographical knowledge might emerge from that activity? Do they transfer invisible narratives of place? How do artists compose with live audio streamings? Lia talks about her involvement with the Hydrophiles project, which explores how live audio streaming engenders different modalities of being and listening with environments, in this case, the waterways of Amsterdam.
Panagiota Anastasopoulou
Panagiota (Penny) Anastasopoulou is a 2nd year PhD student at MTG, UPF. Penny’s PhD topic is characterisation and retrieval in large and diverse sound collections. Freesound is the main use case of her PhD topic. Penny is looking at a range of aspects, including taxonomical structures and classification; data accessibility for a broad audience; music creation and live coding (use case for artistic practice); and soundscape analysis. One of the main categories in her work is soundscapes, which are relevant to bioacoustics. Penny’s work aims to propose better ways of accessing online the data (e.g. soundscapes, sounds recorded).
Amaia Sagasti Martinez
Amaia Sagasti Martinez is an Acoustics Engineer and Researcher at MTG, UPF. Her research position is on acoustic monitoring to develop acoustic sensors based on embedded platforms capable of running machine learning algorithms for the analysis of urban acoustic environments. The work is carried out in the context of the SoundLights project. Amaia talked about the development of a device for estimation of acoustic “pleasantness” and “eventfulness” in urban soundscapes. The project team is trying to create a device that can predict pleasantness and eventulness using machine learning and deep learning. The project goals are to raise awareness of noise, control and reduce noise, question the current legislation, and generate a device in the form of a traffic light with the signs “too noisy!” (red), "be careful with noise! (ambar), and “nice sound environment!” (green).
Emmanouil Benetos
Emmanouil Benetos is currently Reader in Machine Listening at the School of Electronic Engineering and Computer Science (EECS) of QMUL, Royal Academy of Engineering / Leverhulme Trust Research Fellow in resource-efficient machine listening, and Turing Fellow at The Alan Turing Institute. Within QMUL, Emmanouil is member of C4DM, Centre for Multimodal AI, Centre for Intelligent Sensing, and Digital Environment Research Institute, and he co-leads the School’s Machine Listening Lab. Emmanouil mentioned that more than 10 years ago, he co-founded with Mark Plumbley, Dan Stowell and a few others the DCASE challenges, which has taken off as an annual event. His main research topic is computational audio analysis, also referred to as machine listening or computer audition - applied to music, urban, everyday and nature sounds. Over the years, Emmanouil has worked with several PhD students on different aspects of machine listening. Currently, Emmanouil is working on machine learning and signal processing methods for sound understanding, representation learning, multimodal approaches, and so on.
Błażej Kotowski
Błażej Kotowski is an artist and researcher born and raised in Poland, currently based in Barcelona. Błażej is pursuing his PhD (1st year) at the MTG, UPF, examining the intricacies of AI’s architectural design and working around Freesound. Błażej is interested in representations because they encode patterns that might be relevant. Błażej gave an overview of his interests in latent representations and the importance of keeping the human-in-the-loop. He has an artistic practice as a musician, composer, and field recordist, as well as working with collaborators from different disciplines and creating tools for them. Błażej’s interest in acoustic ecology connects with his experience as a field recordist, with questions related to understanding the ecosystem as a complex system, enabling forms of non-semantic mediation, as well as understanding soundscape as a “space for multi-species semiosis”, among others.
Small group discussions
The brief for this part was that 3 small groups would be created, which should be ready to:
- Find common interests among the group
- Explore how this connects with present/future challenges & opportunities in acoustic ecology/bioacoustics
- Identify potential new research areas/interests that emerged from the discussion that can inform your work
We created 3 breakout rooms with 4-5 attendees. Each breakout group had a Chair. The three Chairs were Emmanouil, Frederic and Mark. The chair was in charge of facilitating the discussion and also assigning a voluntary “scribe” who could take notes. We used a padlet, a collaborative writing platform.
General group discussions
After the small group discussions, we convened back to the general room. The three Chairs shared a 2-minute brief of the discussion when the groups (we had less time than originally planned due to the presentations taking longer than expected).
In sum:
Group 1 - Chair: Mark Plumbley. Members: Lia Mazzari (RHUL), Błażej Kotowski (MTG-UPF), Jinhua Liang (C4DM-QMUL), Antonella Maria Cristina Torrisi (C4DM-QMUL). Mark summarised that the group mainly discussed their common interests. Representation was a shared interest, highlighting the difference between real-time and non-real-time representations. As challenges, the group agreed that it should be approached as a complex system approach rather than simple systems or individual elements.
Group 2 - Chair: Emmanouil Benetos. Members: Panagiota Anastasopoulou (MTG-UPF), Amaia Sagasti Martinez (MTG-UPF), Nicolas Farrugia (IMT Atlantique). The group discussed potential connections among their projects and research interests. The group talked about how automatic labelling works, as well as various analysers/features and psychoacoustic indicators which can be used to predict pleasantness and eventfulness. This connects with state-of-the-art descriptors that can be used for predicting psychoacoustic descriptors.
Taxonomies can also be used to predict psychoacoustic descriptors and add them as analysers on Freesound to enhance search and retrieval.
Group 3 - Frederic Font. Members: Peter Batchelor (DMU), Ilyass Moummad (C4DM-QMUL/IMT Atlantique), Inês Nolasco (C4DM-QMUL). The group commented on the commonalities of using the same datasets for both artistic and scientific purposes. The team members also discussed what is the bias in their collections with the open question of whether the use of supervised learning and labels to tackle specific problems is biasing their algorithms and how this affects different practices. The group also talked about the relevance of explainability and the understanding of learnt spaces linked with the perceptual/semantic/cultural meaning of soundscapes.
Take-away message & what’s next
An overall take-away message is that acoustic ecology and bioacoustics can be studied from different angles, namely engineering/computing/acoustics and humanities/music/geography/practice-based research, with some shared methods, but different research questions. We found that the research questions in engineering/computing/acoustics tend to be technical, narrow and specific, whereas in humanities/music/geography/practice-based research, the research questions tend to be open and exploratory. Can we combine both approaches to solve more complex problems?
A second salient aspect is that especially in the fields of engineering/computing/acoustics, the role of the machine and AI (machine learning, deep learning) is taking over manual tasks such as tagging or classifying sound events. Questions about data bias were highlighted. How can the state-of-the-art algorithms used in engineering/computing/acoustics inform humanities/music/geography/practice-based research disciplines?
A third prominent aspect is the qualitative aspect of the “listening” experience and keeping the human-in-the-loop, which is especially advocated by the humanities/artistic disciplines. How can we make sure that we avoid data bias and keep the human voice with more large-scale automatic processes related to engineering/computing/acoustics disciplines?
We all agreed that this meeting was a good start, it was too short and that it might be the beginning of an interdisciplinary conversation. Thanks to all the participants for their contributions and for their patience with a hybrid meeting that is still a technical challenge per se! Special thanks to Emmanouil, Frederic and Mark for chairing the sessions and for promoting the need for this inter/multi/cross disciplinary conversation.