今天跟大家分享一篇语音相关的论文合集：cs.SD语音7篇，eess.AS音频处理7篇。
cs.SD语音
【1】 Multiple-hypothesis RNN-T Loss for Unsupervised Fine-tuning and Self-training of Neural Transducer
标题：多假设RNN-T损失下神经传感器的无监督微调与自训练
链接：https://arxiv.org/abs/2207.14736
作者：Cong-Thanh Do,Mohan Li,Rama Doddipatla备注：Accepted to Interspeech 2022摘要：本文提出了一种利用未标记语音数据对递归神经网络进行无监督微调和自训练的新方法（RNN）—传感器（RNN-T）端到端（E2E）自动语音识别当使用未标记的音频数据时，传统系统使用ASR假设作为目标来执行微调/自训练，并且容易受到基础模型的ASR性能的影响。在这里，为了减轻ASR错误对使用未标记数据的影响，我们提出了一种多假设RNN-T损失，它将多个ASR 1-best假设加入到损失函数中，在Librispeech上的ASR实验表明，多假设方法的误词率相对降低了14.2对于自训练任务，ASR模型使用来自华尔街日报的监督数据进行训练（《华尔街日报》）、Aurora-4和CHiME-4真实噪声数据作为未标记数据，多假设方法在CHiME-4单假设数据的基础上产生了3.3%的WER相对减少。信道真实噪声评估集。摘要：This paper proposes a new approach to perform unsupervised fine-tuning and self-training using unlabeled speech data for recurrent neural network (RNN)-Transducer (RNN-T) end-to-end (E2E) automatic speech recognition (ASR) systems. Conventional systems perform fine-tuning/self-training using ASR hypothesis as the targets when using unlabeled audio data and are susceptible to the ASR performance of the base model. Here in order to alleviate the influence of ASR errors while using unlabeled data, we propose a multiple-hypothesis RNN-T loss that incorporates multiple ASR 1-best hypotheses into the loss function. For the fine-tuning task, ASR experiments on Librispeech show that the multiple-hypothesis approach achieves a relative reduction of 14.2% word error rate (WER) when compared to the single-hypothesis approach, on the test_other set. For the self-training task, ASR models are trained using supervised data from Wall Street Journal (WSJ), Aurora-4 along with CHiME-4 real noisy data as unlabeled data. The multiple-hypothesis approach yields a relative reduction of 3.3% WER on the CHiME-4's single-channel real noisy evaluation set when compared with the single-hypothesis approach.
【2】 Towards Unconstrained Audio Splicing Detection and Localization with Neural Networks
标题：基于神经网络的无约束音频拼接检测与定位
链接：https://arxiv.org/abs/2207.14682
作者：Denise Moussa,Germans Hirsch,Christian Riess备注：Accepted at MMFORWILD 2022, ICPR Workshops - Code will be published after the presentation at the conference摘要：免费提供且易于使用的音频编辑工具使得执行音频拼接变得简单明了。通过组合来自同一个人的各种语音样本，可以创建令人信服的伪造品。在公共部门考虑错误信息时，以及在法律环境中验证证据的完整性时，检测此类拼接都是重要的。不幸的是，大多数现有的用于音频拼接的检测算法使用手工制作的特征并做出特定的假设，然而，刑事调查人员经常面对来自不受约束的源的具有未知特性的音频样本，这增加了对更普遍适用的方法的需求。 本文的目标是为实现无约束音频拼接检测迈出第一步，以后处理操作的形式模拟各种可能伪装拼接的攻击场景，提出一种Transformer序列到序列的算法，该算法在Transformer过程中引入了一个新的检测机制，并对该算法进行了改进，提高了检测效率我们的广泛评估表明所提出的方法优于用于剪接检测的现有专用方法[3，10]以及通用网络EfficientNet [28]和RegNet [25]。摘要：Freely available and easy-to-use audio editing tools make it straightforward to perform audio splicing. Convincing forgeries can be created by combining various speech samples from the same person. Detection of such splices is important both in the public sector when considering misinformation, and in a legal context to verify the integrity of evidence. Unfortunately, most existing detection algorithms for audio splicing use handcrafted features and make specific assumptions. However, criminal investigators are often faced with audio samples from unconstrained sources with unknown characteristics, which raises the need for more generally applicable methods. With this work, we aim to take a first step towards unconstrained audio splicing detection to address this need. We simulate various attack scenarios in the form of post-processing operations that may disguise splicing. We propose a Transformer sequence-to-sequence (seq2seq) network for splicing detection and localization. Our extensive evaluation shows that the proposed method outperforms existing dedicated approaches for splicing detection [3, 10] as well as the general-purpose networks EfficientNet [28] and RegNet [25].
【3】 Pronunciation-aware unique character encoding for RNN Transducer-based Mandarin speech recognition
标题：基于RNN变换器的汉语语音识别中语音感知的唯一字符编码
链接：https://arxiv.org/abs/2207.14578
作者：Peng Shen,Xugang Lu,Hisashi Kawai摘要：适用于普通话端到端（E2E）自动语音识别（ASR）任务中，与基于字符的建模单元相比，基于发音的建模单元在模型训练中可以提高建模单元的共享性，但会遇到同音字问题，我们提出了一种新的语音感知的唯一字符编码方法来构建基于E2E RNN-T的普通话ASR系统，所提出的编码方法是语音-基本音节字索引通过引入CI，RNN-T模型克服了同音字问题，同时利用发音信息提取建模单元，通过一对一的映射，将模型输出转换为最终的识别结果.在Aishell和MagicData数据集上进行了实验，实验结果表明了该方法的有效性。摘要：For Mandarin end-to-end (E2E) automatic speech recognition (ASR) tasks, compared to character-based modeling units, pronunciation-based modeling units could improve the sharing of modeling units in model training but meet homophone problems. In this study, we propose to use a novel pronunciation-aware unique character encoding for building E2E RNN-T-based Mandarin ASR systems. The proposed encoding is a combination of pronunciation-base syllable and character index (CI). By introducing the CI, the RNN-T model can overcome the homophone problem while utilizing the pronunciation information for extracting modeling units. With the proposed encoding, the model outputs can be converted into the final recognition result through a one-to-one mapping. We conducted experiments on Aishell and MagicData datasets, and the experimental results showed the effectiveness of the proposed method.
【4】 Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network
标题：基于生成式对抗网络的非成对语音序列音素识别
链接：https://arxiv.org/abs/2207.14568
作者：Da-rong Liu,Po-chun Hsu,Yi-chen Chen,Sung-feng Huang,Shun-po Chuang,Da-yi Wu,Hung-yi Lee摘要：近年来，ASR已被证明具有很好的性能，但大多数ASR都依赖于大量的成对数据，这对于低资源语言来说是不可行的.本文研究了如何直接从不成对的音素序列和语音中进行学习.我们设计了一个两阶段的迭代框架.第一阶段采用GAN训练来寻找不成对的语音和音素序列之间的映射关系.第二阶段采用GAN训练来寻找音素序列和语音之间的映射关系.在第二阶段，引入另一个HMM模型，从生成器的输出进行训练，这提高了性能，并为下一次迭代提供了更好的分割。在实验中，我们首先研究了不同的模型设计选择。然后我们将该框架与不同类型的基线进行了比较：（i）有监督的方法（ii）基于声学单元发现的方法（iii）从不成对数据学习的方法.我们的框架在TIMIT数据集上的性能一致地优于所有声学单元发现方法和从不成对数据学习的先前方法.摘要：ASR has been shown to achieve great performance recently. However, most of them rely on massive paired data, which is not feasible for low-resource languages worldwide. This paper investigates how to learn directly from unpaired phone sequences and speech utterances. We design a two-stage iterative framework. GAN training is adopted in the first stage to find the mapping relationship between unpaired speech and phone sequence. In the second stage, another HMM model is introduced to train from the generator's output, which boosts the performance and provides a better segmentation for the next iteration. In the experiment, we first investigate different choices of model designs. Then we compare the framework to different types of baselines: (i) supervised methods (ii) acoustic unit discovery based methods (iii) methods learning from unpaired data. Our framework performs consistently better than all acoustic unit discovery methods and previous methods learning from unpaired data based on the TIMIT dataset.
【5】 Domain Specific Wav2vec 2.0 Fine-tuning For The SE&R 2022 Challenge
标题：针对SE&R 2022挑战的特定领域Wav2vec 2.0微调
链接：https://arxiv.org/abs/2207.14418
作者：Alef Iury Siqueira Ferreira,Gustavo dos Reis Oliveira备注：Proceedings of the First Workshop on Automatic Speech Recognition for Spontaneous and Prepared Speech & Speech Emotion Recognition in Portuguese (SE&R 2022), co-located with PROPOR 2022摘要：本文介绍了我们的努力，建立一个鲁棒的ASR模型，为共享任务自动语音识别自发和准备语音&语音情感识别在葡萄牙语（SE&R 2022）。挑战的目标是推进葡萄牙语的ASR研究，考虑不同方言中的准备语音和自发语音。我们的方法包括在特定领域的方法中微调ASR模型，该方法在4个可用的声道中有3个声道的测试集上提供了强基线，改进了强基线摘要：This paper presents our efforts to build a robust ASR model for the shared task Automatic Speech Recognition for spontaneous and prepared speech & Speech Emotion Recognition in Portuguese (SE&R 2022). The goal of the challenge is to advance the ASR research for the Portuguese language, considering prepared and spontaneous speech in different dialects. Our method consist on fine-tuning an ASR model in a domain-specific approach, applying gain normalization and selective noise insertion. The proposed method improved over the strong baseline provided on the test set in 3 of the 4 tracks available
【6】 Low-data? No problem: low-resource, language-agnostic conversational text-to-speech via F0-conditioned data augmentation
标题：低数据量？没问题：经由F0条件数据扩充低资源、语言不可知的会话式文本到语音
链接：https://arxiv.org/abs/2207.14607
作者：Giulia Comini,Goeric Huybrechts,Manuel Sam Ribeiro,Adam Gabrys,Jaime Lorenzo-Trueba备注：Accepted for presentation at Interspeech 2022摘要：为了克服这些问题，我们演示了如何构建低资源的神经文本—语音转换系统（TTS）语音，当没有相同语言的其他会话数据可用时，假设该语言的非表达性语音数据可用，我们提出了一种3步技术：1）我们训练F0条件语音转换（VC）模型作为数据增强技术; 2）我们训练F0预测器以控制语音转换的合成数据的会话风格; 3）我们训练了一个使用增强数据的TTS系统。我们证明了我们的技术能够实现F0可控性，在说话人和语言之间是可扩展的，并且在自然度方面比最先进的基线模型具有竞争力，基线模型是另一种不使用F0信息的增强方法。摘要：The availability of data in expressive styles across languages is limited, and recording sessions are costly and time consuming. To overcome these issues, we demonstrate how to build low-resource, neural text-to-speech (TTS) voices with only 1 hour of conversational speech, when no other conversational data are available in the same language. Assuming the availability of non-expressive speech data in that language, we propose a 3-step technology: 1) we train an F0-conditioned voice conversion (VC) model as data augmentation technique; 2) we train an F0 predictor to control the conversational flavour of the voice-converted synthetic data; 3) we train a TTS system that consumes the augmented data. We prove that our technology enables F0 controllability, is scalable across speakers and languages and is competitive in terms of naturalness over a state-of-the-art baseline model, another augmented method which does not make use of F0 information.
【7】 Predicting Global Head-Related Transfer Functions From Scanned Head Geometry Using Deep Learning and Compact Representations
标题：使用深度学习和紧凑表示从扫描的头部几何结构预测全局头部相关传递函数
链接：https://arxiv.org/abs/2207.14352
作者：Yuxiang Wang,You Zhang,Zhiyao Duan,Mark Bocko摘要：在日益增长的虚拟听觉显示领域中，个性化的头部相关传递函数本文提出了一种基于卷积神经网络的HRTF个性化方法为了使CNN模型的训练更容易，我们使用了一种新的训练方法，我们提出了一种新的预处理方法，用于头部扫描和HRTF数据，以获得紧凑的表示。对于头部扫描，我们使用截断球冠谐波（SCH）系数来表示耳廓面积，这在声散射过程中是重要的。我们使用截断球谐函数一个CNN模型被训练成根据扫描的耳朵几何形状的SCH系数和头部的其他人体测量来预测HRTF幅度的SH系数。另一个CNN模型被训练成仅根据耳朵、头部和躯干的人体测量来预测HRTF开始的SH系数。我们的方法能够预测完整的和全局的HRTF数据。（LSD）度量用于客观评价，结果表明，与地面相比，该方法在空间和时间维度上都具有较好的LSD水平。基于听觉模型的定位仿真结果也与客观评价指标相一致，表明本文预测的HRTF定位响应明显优于BEM计算的HRTF定位响应。摘要：In the growing field of virtual auditory display, personalized head-related transfer functions (HRTFs) play a vital role in establishing an accurate sound image. In this work, we propose an HRTF personalization method employing convolutional neural networks (CNN) to predict a subject's HRTFs for all directions from their scanned head geometry. To ease the training of the CNN models, we propose novel pre-processing methods for both the head scans and HRTF data to achieve compact representations. For the head scan, we use truncated spherical cap harmonic (SCH) coefficients to represent the pinna area, which is important in the acoustic scattering process. For the HRTF data, we use truncated spherical harmonic (SH) coefficients to represent the HRTF magnitudes and onsets. One CNN model is trained to predict the SH coefficients of the HRTF magnitudes from the SCH coefficients of the scanned ear geometry and other anthropometric measurements of the head. The other CNN model is trained to predict SH coefficients of the HRTF onsets from only the anthropometric measurements of the ear, head, and torso. Combining the magnitude and onset predictions, our method is able to predict the complete and global HRTF data. A leave-one-out validation with the log-spectral distortion (LSD) metric is used for objective evaluation. The results show a decent LSD level at both spatial \& temporal dimensions compared to the ground-truth HRTFs and a lower LSD than the boundary element method (BEM) simulation of HRTFs that the database provides. The localization simulation results with an auditory model are also consistent with the objective evaluation metrics, showing the localization responses with our predicted HRTFs are significantly better than with the BEM calculated ones.
【1】 Low-data? No problem: low-resource, language-agnostic conversational text-to-speech via F0-conditioned data augmentation
标题：低数据量？没问题：经由F0条件数据扩充低资源、语言不可知的会话式文本到语音
链接：https://arxiv.org/abs/2207.14607
* 与cs.SD语音【6】为同一篇
作者：Giulia Comini,Goeric Huybrechts,Manuel Sam Ribeiro,Adam Gabrys,Jaime Lorenzo-Trueba备注：Accepted for presentation at Interspeech 2022摘要：为了克服这些问题，我们演示了如何构建低资源的神经文本—语音转换系统（TTS）语音，当没有相同语言的其他会话数据可用时，假设该语言的非表达性语音数据可用，我们提出了一种3步技术：1）我们训练F0条件语音转换（VC）模型作为数据增强技术; 2）我们训练F0预测器以控制语音转换的合成数据的会话风格; 3）我们训练了一个使用增强数据的TTS系统。我们证明了我们的技术能够实现F0可控性，在说话人和语言之间是可扩展的，并且在自然度方面比最先进的基线模型具有竞争力，基线模型是另一种不使用F0信息的增强方法。摘要：The availability of data in expressive styles across languages is limited, and recording sessions are costly and time consuming. To overcome these issues, we demonstrate how to build low-resource, neural text-to-speech (TTS) voices with only 1 hour of conversational speech, when no other conversational data are available in the same language. Assuming the availability of non-expressive speech data in that language, we propose a 3-step technology: 1) we train an F0-conditioned voice conversion (VC) model as data augmentation technique; 2) we train an F0 predictor to control the conversational flavour of the voice-converted synthetic data; 3) we train a TTS system that consumes the augmented data. We prove that our technology enables F0 controllability, is scalable across speakers and languages and is competitive in terms of naturalness over a state-of-the-art baseline model, another augmented method which does not make use of F0 information.
【2】 Predicting Global Head-Related Transfer Functions From Scanned Head Geometry Using Deep Learning and Compact Representations
标题：使用深度学习和紧凑表示从扫描的头部几何结构预测全局头部相关传递函数
链接：https://arxiv.org/abs/2207.14352
* 与cs.SD语音【7】为同一篇
作者：Yuxiang Wang,You Zhang,Zhiyao Duan,Mark Bocko摘要：在日益增长的虚拟听觉显示领域中，个性化的头部相关传递函数本文提出了一种基于卷积神经网络的HRTF个性化方法为了使CNN模型的训练更容易，我们使用了一种新的训练方法，我们提出了一种新的预处理方法，用于头部扫描和HRTF数据，以获得紧凑的表示。对于头部扫描，我们使用截断球冠谐波（SCH）系数来表示耳廓面积，这在声散射过程中是重要的。我们使用截断球谐函数一个CNN模型被训练成根据扫描的耳朵几何形状的SCH系数和头部的其他人体测量来预测HRTF幅度的SH系数。另一个CNN模型被训练成仅根据耳朵、头部和躯干的人体测量来预测HRTF开始的SH系数。我们的方法能够预测完整的和全局的HRTF数据。（LSD）度量用于客观评价，结果表明，与地面相比，该方法在空间和时间维度上都具有较好的LSD水平。基于听觉模型的定位仿真结果也与客观评价指标相一致，表明本文预测的HRTF定位响应明显优于BEM计算的HRTF定位响应。摘要：In the growing field of virtual auditory display, personalized head-related transfer functions (HRTFs) play a vital role in establishing an accurate sound image. In this work, we propose an HRTF personalization method employing convolutional neural networks (CNN) to predict a subject's HRTFs for all directions from their scanned head geometry. To ease the training of the CNN models, we propose novel pre-processing methods for both the head scans and HRTF data to achieve compact representations. For the head scan, we use truncated spherical cap harmonic (SCH) coefficients to represent the pinna area, which is important in the acoustic scattering process. For the HRTF data, we use truncated spherical harmonic (SH) coefficients to represent the HRTF magnitudes and onsets. One CNN model is trained to predict the SH coefficients of the HRTF magnitudes from the SCH coefficients of the scanned ear geometry and other anthropometric measurements of the head. The other CNN model is trained to predict SH coefficients of the HRTF onsets from only the anthropometric measurements of the ear, head, and torso. Combining the magnitude and onset predictions, our method is able to predict the complete and global HRTF data. A leave-one-out validation with the log-spectral distortion (LSD) metric is used for objective evaluation. The results show a decent LSD level at both spatial \& temporal dimensions compared to the ground-truth HRTFs and a lower LSD than the boundary element method (BEM) simulation of HRTFs that the database provides. The localization simulation results with an auditory model are also consistent with the objective evaluation metrics, showing the localization responses with our predicted HRTFs are significantly better than with the BEM calculated ones.
【3】 Multiple-hypothesis RNN-T Loss for Unsupervised Fine-tuning and Self-training of Neural Transducer
标题：多假设RNN-T损失下神经传感器的无监督微调与自训练
链接：https://arxiv.org/abs/2207.14736
* 与cs.SD语音【1】为同一篇
作者：Cong-Thanh Do,Mohan Li,Rama Doddipatla备注：Accepted to Interspeech 2022摘要：本文提出了一种利用未标记语音数据对递归神经网络进行无监督微调和自训练的新方法（RNN）—传感器（RNN-T）端到端（E2E）自动语音识别当使用未标记的音频数据时，传统系统使用ASR假设作为目标来执行微调/自训练，并且容易受到基础模型的ASR性能的影响。在这里，为了减轻ASR错误对使用未标记数据的影响，我们提出了一种多假设RNN-T损失，它将多个ASR 1-best假设加入到损失函数中，在Librispeech上的ASR实验表明，多假设方法的误词率相对降低了14.2对于自训练任务，ASR模型使用来自华尔街日报的监督数据进行训练（《华尔街日报》）、Aurora-4和CHiME-4真实噪声数据作为未标记数据，多假设方法在CHiME-4单假设数据的基础上产生了3.3%的WER相对减少。信道真实噪声评估集。摘要：This paper proposes a new approach to perform unsupervised fine-tuning and self-training using unlabeled speech data for recurrent neural network (RNN)-Transducer (RNN-T) end-to-end (E2E) automatic speech recognition (ASR) systems. Conventional systems perform fine-tuning/self-training using ASR hypothesis as the targets when using unlabeled audio data and are susceptible to the ASR performance of the base model. Here in order to alleviate the influence of ASR errors while using unlabeled data, we propose a multiple-hypothesis RNN-T loss that incorporates multiple ASR 1-best hypotheses into the loss function. For the fine-tuning task, ASR experiments on Librispeech show that the multiple-hypothesis approach achieves a relative reduction of 14.2% word error rate (WER) when compared to the single-hypothesis approach, on the test_other set. For the self-training task, ASR models are trained using supervised data from Wall Street Journal (WSJ), Aurora-4 along with CHiME-4 real noisy data as unlabeled data. The multiple-hypothesis approach yields a relative reduction of 3.3% WER on the CHiME-4's single-channel real noisy evaluation set when compared with the single-hypothesis approach.
【4】 Towards Unconstrained Audio Splicing Detection and Localization with Neural Networks
标题：基于神经网络的无约束音频拼接检测与定位
链接：https://arxiv.org/abs/2207.14682
* 与cs.SD语音【2】为同一篇
作者：Denise Moussa,Germans Hirsch,Christian Riess备注：Accepted at MMFORWILD 2022, ICPR Workshops - Code will be published after the presentation at the conference摘要：免费提供且易于使用的音频编辑工具使得执行音频拼接变得简单明了。通过组合来自同一个人的各种语音样本，可以创建令人信服的伪造品。在公共部门考虑错误信息时，以及在法律环境中验证证据的完整性时，检测此类拼接都是重要的。不幸的是，大多数现有的用于音频拼接的检测算法使用手工制作的特征并做出特定的假设，然而，刑事调查人员经常面对来自不受约束的源的具有未知特性的音频样本，这增加了对更普遍适用的方法的需求。 本文的目标是为实现无约束音频拼接检测迈出第一步，以后处理操作的形式模拟各种可能伪装拼接的攻击场景，提出一种Transformer序列到序列的算法，该算法在Transformer过程中引入了一个新的检测机制，并对该算法进行了改进，提高了检测效率我们的广泛评估表明所提出的方法优于用于剪接检测的现有专用方法[3，10]以及通用网络EfficientNet [28]和RegNet [25]。摘要：Freely available and easy-to-use audio editing tools make it straightforward to perform audio splicing. Convincing forgeries can be created by combining various speech samples from the same person. Detection of such splices is important both in the public sector when considering misinformation, and in a legal context to verify the integrity of evidence. Unfortunately, most existing detection algorithms for audio splicing use handcrafted features and make specific assumptions. However, criminal investigators are often faced with audio samples from unconstrained sources with unknown characteristics, which raises the need for more generally applicable methods. With this work, we aim to take a first step towards unconstrained audio splicing detection to address this need. We simulate various attack scenarios in the form of post-processing operations that may disguise splicing. We propose a Transformer sequence-to-sequence (seq2seq) network for splicing detection and localization. Our extensive evaluation shows that the proposed method outperforms existing dedicated approaches for splicing detection [3, 10] as well as the general-purpose networks EfficientNet [28] and RegNet [25].
【5】 Pronunciation-aware unique character encoding for RNN Transducer-based Mandarin speech recognition
标题：基于RNN变换器的汉语语音识别中语音感知的唯一字符编码
链接：https://arxiv.org/abs/2207.14578
* 与cs.SD语音【3】为同一篇
作者：Peng Shen,Xugang Lu,Hisashi Kawai摘要：适用于普通话端到端（E2E）自动语音识别（ASR）任务中，与基于字符的建模单元相比，基于发音的建模单元在模型训练中可以提高建模单元的共享性，但会遇到同音字问题，我们提出了一种新的语音感知的唯一字符编码方法来构建基于E2E RNN-T的普通话ASR系统，所提出的编码方法是语音-基本音节字索引通过引入CI，RNN-T模型克服了同音字问题，同时利用发音信息提取建模单元，通过一对一的映射，将模型输出转换为最终的识别结果.在Aishell和MagicData数据集上进行了实验，实验结果表明了该方法的有效性。摘要：For Mandarin end-to-end (E2E) automatic speech recognition (ASR) tasks, compared to character-based modeling units, pronunciation-based modeling units could improve the sharing of modeling units in model training but meet homophone problems. In this study, we propose to use a novel pronunciation-aware unique character encoding for building E2E RNN-T-based Mandarin ASR systems. The proposed encoding is a combination of pronunciation-base syllable and character index (CI). By introducing the CI, the RNN-T model can overcome the homophone problem while utilizing the pronunciation information for extracting modeling units. With the proposed encoding, the model outputs can be converted into the final recognition result through a one-to-one mapping. We conducted experiments on Aishell and MagicData datasets, and the experimental results showed the effectiveness of the proposed method.
【6】 Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network
标题：基于生成式对抗网络的非成对语音序列音素识别
链接：https://arxiv.org/abs/2207.14568
* 与cs.SD语音【4】为同一篇
作者：Da-rong Liu,Po-chun Hsu,Yi-chen Chen,Sung-feng Huang,Shun-po Chuang,Da-yi Wu,Hung-yi Lee摘要：近年来，ASR已被证明具有很好的性能，但大多数ASR都依赖于大量的成对数据，这对于低资源语言来说是不可行的.本文研究了如何直接从不成对的音素序列和语音中进行学习.我们设计了一个两阶段的迭代框架.第一阶段采用GAN训练来寻找不成对的语音和音素序列之间的映射关系.第二阶段采用GAN训练来寻找音素序列和语音之间的映射关系.在第二阶段，引入另一个HMM模型，从生成器的输出进行训练，这提高了性能，并为下一次迭代提供了更好的分割。在实验中，我们首先研究了不同的模型设计选择。然后我们将该框架与不同类型的基线进行了比较：（i）有监督的方法（ii）基于声学单元发现的方法（iii）从不成对数据学习的方法.我们的框架在TIMIT数据集上的性能一致地优于所有声学单元发现方法和从不成对数据学习的先前方法.摘要：ASR has been shown to achieve great performance recently. However, most of them rely on massive paired data, which is not feasible for low-resource languages worldwide. This paper investigates how to learn directly from unpaired phone sequences and speech utterances. We design a two-stage iterative framework. GAN training is adopted in the first stage to find the mapping relationship between unpaired speech and phone sequence. In the second stage, another HMM model is introduced to train from the generator's output, which boosts the performance and provides a better segmentation for the next iteration. In the experiment, we first investigate different choices of model designs. Then we compare the framework to different types of baselines: (i) supervised methods (ii) acoustic unit discovery based methods (iii) methods learning from unpaired data. Our framework performs consistently better than all acoustic unit discovery methods and previous methods learning from unpaired data based on the TIMIT dataset.
【7】 Domain Specific Wav2vec 2.0 Fine-tuning For The SE&R 2022 Challenge
标题：针对SE&R 2022挑战的特定领域Wav2vec 2.0微调
链接：https://arxiv.org/abs/2207.14418
* 与cs.SD语音【5】为同一篇
作者：Alef Iury Siqueira Ferreira,Gustavo dos Reis Oliveira备注：Proceedings of the First Workshop on Automatic Speech Recognition for Spontaneous and Prepared Speech & Speech Emotion Recognition in Portuguese (SE&R 2022), co-located with PROPOR 2022摘要：本文介绍了我们的努力，建立一个鲁棒的ASR模型，为共享任务自动语音识别自发和准备语音&语音情感识别在葡萄牙语（SE&R 2022）。挑战的目标是推进葡萄牙语的ASR研究，考虑不同方言中的准备语音和自发语音。我们的方法包括在特定领域的方法中微调ASR模型，该方法在4个可用的声道中有3个声道的测试集上提供了强基线，改进了强基线摘要：This paper presents our efforts to build a robust ASR model for the shared task Automatic Speech Recognition for spontaneous and prepared speech & Speech Emotion Recognition in Portuguese (SE&R 2022). The goal of the challenge is to advance the ASR research for the Portuguese language, considering prepared and spontaneous speech in different dialects. Our method consist on fine-tuning an ASR model in a domain-specific approach, applying gain normalization and selective noise insertion. The proposed method improved over the strong baseline provided on the test set in 3 of the 4 tracks available
机器翻译，仅供参考
每日语音速递