今日论文合集：cs.SD语音3篇，eess.AS音频处理3篇。

cs.SD语音

【1】 Language Models are Drummers: Drum Composition with Natural Language  Pre-Training

标题：语言模型是鼓手：自然语言预训练的鼓乐作曲

链接：https://arxiv.org/abs/2301.01162

作者：Li Zhang,Chris Callison-Burch
机构：University of Pennsylvania
备注：Accepted to the 1st workshop on Creative AI across Modalities in AAAI 2023
摘要：利用人工智能的自动音乐生成通常需要大量数据，而对于许多不太常见的流派和乐器来说，这些数据是难以获得的。为了解决这个问题，我们介绍了正在进行的工作和初步发现，通过微调在数百个鼓演奏MIDI文件的海量文本语料库上预先训练的大型语言模型，深度模型将知识从语言转移到音乐的可能性。通过这样做，我们证明了最大的、最先进的模型之一（GPT3）能够生成合理的鼓槽，而未经预训练的模型（Transformer）除了简单重复之外，没有显示出这种能力。评价产生的音乐是一项具有挑战性的任务，更是评价鼓槽几乎没有优先权的文学。因此，我们提出了一种量身定制的结构评估方法，并将GPT3产生的鼓槽与人类专业人士演奏的鼓槽进行了比较分析，揭示了通过语言到音乐迁移产生鼓槽的优势和劣势。我们的研究结果表明，使用大型语言模型进行语言到音乐的迁移学习是可行的，并且有前途。
摘要：Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.


【2】 Supervised Acoustic Embeddings And Their Transferability Across  Languages

标题：有监督的声学嵌入及其跨语言可移植

链接：https://arxiv.org/abs/2301.01020

作者：Sreepratha Ram,Hanan Aldarmaki
机构：UAE University, MBZUAI
备注：Presented at ICNLSP 2022
摘要：在语音识别中，必须对输入信号的语音内容进行建模，同时丢弃诸如说话者变化和噪声等不相关因素，这在低资源设置中是具有挑战性的。自监督预训练（Self-supervised pre-training）被提出作为一种改进监督和无监督语音识别的方法，包括帧级特征表示和用于可变长度片段的声学词嵌入（AWE）。然而，自监督模型本身不能学习语言内容的完美分离，因为它们被训练成优化间接目标。在这项工作中，我们用不同的预训练自监督特征作为AWE模型的输入进行实验，并表明它们在监督框架内工作得最好。在英语上训练的模型可以不经调整地转移到其他语言，并且优于仅在目标语言上训练的自监督模型。
摘要：In speech recognition, it is essential to model the phonetic content of the input signal while discarding irrelevant factors such as speaker variations and noise, which is challenging in low-resource settings. Self-supervised pre-training has been proposed as a way to improve both supervised and unsupervised speech recognition, including frame-level feature representations and Acoustic Word Embeddings (AWE) for variable-length segments. However, self-supervised models alone cannot learn perfect separation of the linguistic content as they are trained to optimize indirect objectives. In this work, we experiment with different pre-trained self-supervised features as input to AWE models and show that they work best within a supervised framework. Models trained on English can be transferred to other languages with no adaptation and outperform self-supervised models trained solely on the target languages.


【3】 Hyperuniform disordered parametric loudspeaker array

标题：超均匀无序参数扬声器阵列

链接：https://arxiv.org/abs/2301.00833

作者：Kun Tang,Yuqi Wang,Shaobo Wang,Da Gao,Haojie Li,Xindong Liang,Patrick Sebbah,Jin Zhang,Junhui Shi
机构：Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou , China.,  Taiji Laboratory for Gravitational Wave Universe, School of Physics and Optoelectronic Engineering, Hangzhou Institute for Advanced Study, Hangzhou, China and
摘要：可控参量扬声器阵列以其方向性和窄波束宽度而闻名。然而，由于阵列的周期性分布，它常常受到栅瓣的影响。本文提出了超均匀无序阵列结构，这种结构具有短程随机性，但在大尺度下具有相关性，是相控阵声学天线的一种有前途的替代分布。角分辨测量结果表明，该阵列抑制了栅瓣，并在主瓣附近保持了最小的辐射区域。这些独特的发射特征有利于次级频率波消除栅瓣，而不管初级波的频率如何。此外，超均匀无序阵列是可复制的，这有利于超大阵列的设计，而不需要任何额外的计算工作。
摘要：A steerable parametric loudspeaker array is known for its directivity and narrow beam width. However, it often suffers from the grating lobes due to periodic array distributions. Here we propose the array configuration of hyperuniform disorder, which is short-range random while correlated at large scales, as a promising alternative distribution of acoustic antennas in phased arrays. Angle-resolved measurements reveal that the proposed array suppresses grating lobes and maintains a minimal radiation region in the vicinity of the main lobe for the primary frequency waves. These distinctive emission features benefit the secondary frequency wave in canceling the grating lobes regardless of the frequencies of the primary waves. Besides that, the hyperuniform disordered array is duplicatable, which facilitates extra-large array design without any additional computational efforts.
eess.AS音频处理

【1】 Hyperuniform disordered parametric loudspeaker array

标题：超均匀无序参数扬声器阵列

链接：https://arxiv.org/abs/2301.00833

*与cs.SD语音【3】为同一篇

作者：Kun Tang,Yuqi Wang,Shaobo Wang,Da Gao,Haojie Li,Xindong Liang,Patrick Sebbah,Jin Zhang,Junhui Shi
机构：Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou , China.,  Taiji Laboratory for Gravitational Wave Universe, School of Physics and Optoelectronic Engineering, Hangzhou Institute for Advanced Study, Hangzhou, China and
摘要：可控参量扬声器阵列以其方向性和窄波束宽度而闻名。然而，由于阵列的周期性分布，它常常受到栅瓣的影响。本文提出了超均匀无序阵列结构，这种结构具有短程随机性，但在大尺度下具有相关性，是相控阵声学天线的一种有前途的替代分布。角分辨测量结果表明，该阵列抑制了栅瓣，并在主瓣附近保持了最小的辐射区域。这些独特的发射特征有利于次级频率波消除栅瓣，而不管初级波的频率如何。此外，超均匀无序阵列是可复制的，这有利于超大阵列的设计，而不需要任何额外的计算工作。
摘要：A steerable parametric loudspeaker array is known for its directivity and narrow beam width. However, it often suffers from the grating lobes due to periodic array distributions. Here we propose the array configuration of hyperuniform disorder, which is short-range random while correlated at large scales, as a promising alternative distribution of acoustic antennas in phased arrays. Angle-resolved measurements reveal that the proposed array suppresses grating lobes and maintains a minimal radiation region in the vicinity of the main lobe for the primary frequency waves. These distinctive emission features benefit the secondary frequency wave in canceling the grating lobes regardless of the frequencies of the primary waves. Besides that, the hyperuniform disordered array is duplicatable, which facilitates extra-large array design without any additional computational efforts.


【2】 Language Models are Drummers: Drum Composition with Natural Language  Pre-Training

标题：语言模型是鼓手：自然语言预训练的鼓乐作曲

链接：https://arxiv.org/abs/2301.01162

*与cs.SD语音【1】为同一篇

作者：Li Zhang,Chris Callison-Burch
机构：University of Pennsylvania
备注：Accepted to the 1st workshop on Creative AI across Modalities in AAAI 2023
摘要：利用人工智能的自动音乐生成通常需要大量数据，而对于许多不太常见的流派和乐器来说，这些数据是难以获得的。为了解决这个问题，我们介绍了正在进行的工作和初步发现，通过微调在数百个鼓演奏MIDI文件的海量文本语料库上预先训练的大型语言模型，深度模型将知识从语言转移到音乐的可能性。通过这样做，我们证明了最大的、最先进的模型之一（GPT3）能够生成合理的鼓槽，而未经预训练的模型（Transformer）除了简单重复之外，没有显示出这种能力。评价产生的音乐是一项具有挑战性的任务，更是评价鼓槽几乎没有优先权的文学。因此，我们提出了一种量身定制的结构评估方法，并将GPT3产生的鼓槽与人类专业人士演奏的鼓槽进行了比较分析，揭示了通过语言到音乐迁移产生鼓槽的优势和劣势。我们的研究结果表明，使用大型语言模型进行语言到音乐的迁移学习是可行的，并且有前途。
摘要：Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.


【3】 Supervised Acoustic Embeddings And Their Transferability Across  Languages

标题：有监督的声学嵌入及其跨语言可移植

链接：https://arxiv.org/abs/2301.01020

*与cs.SD语音【2】为同一篇

作者：Sreepratha Ram,Hanan Aldarmaki
机构：UAE University, MBZUAI
备注：Presented at ICNLSP 2022
摘要：在语音识别中，必须对输入信号的语音内容进行建模，同时丢弃诸如说话者变化和噪声等不相关因素，这在低资源设置中是具有挑战性的。自监督预训练（Self-supervised pre-training）被提出作为一种改进监督和无监督语音识别的方法，包括帧级特征表示和用于可变长度片段的声学词嵌入（AWE）。然而，自监督模型本身不能学习语言内容的完美分离，因为它们被训练成优化间接目标。在这项工作中，我们用不同的预训练自监督特征作为AWE模型的输入进行实验，并表明它们在监督框架内工作得最好。在英语上训练的模型可以不经调整地转移到其他语言，并且优于仅在目标语言上训练的自监督模型。
摘要：In speech recognition, it is essential to model the phonetic content of the input signal while discarding irrelevant factors such as speaker variations and noise, which is challenging in low-resource settings. Self-supervised pre-training has been proposed as a way to improve both supervised and unsupervised speech recognition, including frame-level feature representations and Acoustic Word Embeddings (AWE) for variable-length segments. However, self-supervised models alone cannot learn perfect separation of the linguistic content as they are trained to optimize indirect objectives. In this work, we experiment with different pre-trained self-supervised features as input to AWE models and show that they work best within a supervised framework. Models trained on English can be transferred to other languages with no adaptation and outperform self-supervised models trained solely on the target languages.

机器翻译，仅供参考