Conditioned source separation for musical instrument performances
In music source separation, the number of sources may vary for each piece and some of the sources may belong to the same family of instruments, thus sharing timbral characteristics and making the sources more correlated. This leads to additional challenges in the source separation problem. This pape...
| Authors: | ; ; |
|---|---|
| Format: | Article |
| Status: | Accepted version |
| Publication date: | 2021 |
| Country: | España |
| Institution: | Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| Repository: | Recercat. Dipósit de la Recerca de Catalunya |
| OAI Identifier: | oai:recercat.cat:10230/47693 |
| Online Access: | http://hdl.handle.net/10230/47693 http://dx.doi.org/10.1109/TASLP.2021.3082331 |
| Access Level: | Open access |
| Keyword: | Single Channel Source Separation Audio-Visual Analysis Conditioned Neural Networks |
| _version_ | 1878742474536517632 |
|---|---|
| author | Slizovskaia, Olga |
| author2 | Haro Ortega, Gloria Gómez Gutiérrez, Emilia, 1975- |
| author2_role | author author |
| author_browse | Gómez Gutiérrez, Emilia, 1975- Haro Ortega, Gloria Slizovskaia, Olga |
| author_facet | Slizovskaia, Olga Haro Ortega, Gloria Gómez Gutiérrez, Emilia, 1975- |
| author_role | author |
| collection | Recercat. Dipósit de la Recerca de Catalunya |
| description | In music source separation, the number of sources may vary for each piece and some of the sources may belong to the same family of instruments, thus sharing timbral characteristics and making the sources more correlated. This leads to additional challenges in the source separation problem. This paper proposes a source separation method for multiple musical instruments sounding simultaneously and explores how much additional information apart from the audio stream can lift the quality of source separation. We explore conditioning techniques at different levels of a primary source separation network and utilize two extra modalities of data, namely presence or absence of instruments in the mixture, and the corresponding video stream data. |
| eu_rights_str_mv | openAccess |
| format | article |
| id | ES_f6950f8a81e7e4b6e983de013eaefbb7 |
| institution | Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| instname_str | Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| network_acronym_str | ES |
| network_name_str | España |
| oai_identifier_str | oai:recercat.cat:10230/47693 |
| publishDate | 2021 |
| publishDateSort | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| publisherStr | Institute of Electrical and Electronics Engineers (IEEE) |
| reponame_str | Recercat. Dipósit de la Recerca de Catalunya |
| spelling | Conditioned source separation for musical instrument performances Slizovskaia, Olga Haro Ortega, Gloria Gómez Gutiérrez, Emilia, 1975- Single Channel Source Separation Audio-Visual Analysis Conditioned Neural Networks In music source separation, the number of sources may vary for each piece and some of the sources may belong to the same family of instruments, thus sharing timbral characteristics and making the sources more correlated. This leads to additional challenges in the source separation problem. This paper proposes a source separation method for multiple musical instruments sounding simultaneously and explores how much additional information apart from the audio stream can lift the quality of source separation. We explore conditioning techniques at different levels of a primary source separation network and utilize two extra modalities of data, namely presence or absence of instruments in the mixture, and the corresponding video stream data. This work was funded in part by ERC Innovation Programme (grant 770376, TROMPA); Spanish Ministry of Economy and Competitiveness under the Mar´ıa de Maeztu Units of Excellence Program (MDM-2015-0502) and the Social European Funds; the MICINN/FEDER UE project with reference PGC2018-098625-B-I00; and the H2020-MSCARISE-2017 project with reference 777826 NoMADS. We gratefully acknowledge NVIDIA for the donation of GPUs used for the experiments. Institute of Electrical and Electronics Engineers (IEEE) http://hdl.handle.net/10230/47693 http://dx.doi.org/10.1109/TASLP.2021.3082331 |
| spellingShingle | Conditioned source separation for musical instrument performances Slizovskaia, Olga Single Channel Source Separation Audio-Visual Analysis Conditioned Neural Networks |
| status_str | acceptedVersion |
| title | Conditioned source separation for musical instrument performances |
| title_full | Conditioned source separation for musical instrument performances |
| title_fullStr | Conditioned source separation for musical instrument performances |
| title_full_unstemmed | Conditioned source separation for musical instrument performances |
| title_short | Conditioned source separation for musical instrument performances |
| title_sort | Conditioned source separation for musical instrument performances |
| topic | Single Channel Source Separation Audio-Visual Analysis Conditioned Neural Networks |
| topic_facet | Single Channel Source Separation Audio-Visual Analysis Conditioned Neural Networks |
| url | http://hdl.handle.net/10230/47693 http://dx.doi.org/10.1109/TASLP.2021.3082331 |