认知与发展心理学研究室知识库

语音与音乐的加工机制是听觉系统两大重要的主题。大量研究发现音乐经验影响了语音的加工过程，尤其促进噪声中的言语识别能力。音乐经验带来的改变可能会通过促进高级认知功能（比如听觉选择注意、听觉工作记忆）和跨通道整合能力实现；也有可能通过促进了语音的声学线索（比如基频、谐波等）的加工而实现。然而音乐经验在连续语音水平是否影响了语音声学时间线索的动态加工并没有得到充分的解答。

本研究将语音分解为不同的时间线索——包络与时间精细结构，并通过分频带的数量控制线索的完整程度。两种时间线索具有不同的时间动态特征与声学信息：包络是慢速变化的成分，代表了语音的能量波动，包含节奏、音节边界等信息；时间精细结构是快速变化的成分，代表语音的频谱成分，包含语调、共振峰等信息。本研究使用心理物理学方法和脑电技术，用三个实验考察音乐经验影响语音包络和时间精细结构加工的认知神经机制。

实验一使用心理物理学方法测试了平衡过认知功能的音乐家与非音乐家被试对包含两种不同时间线索的语音（包络语音、精细结构语音）在三种可懂度水平（三种频带数量）和三种噪音水平下的语句复述率。结果发现音乐经验降低了对包络语句的识别率，提高了精细结构语句的识别率。同时，两组被试对噪音中自然语句的识别率没有显著差异。这提示，音乐经验可能造成了对不同语音时间线索的资源分配差异和偏向性加工。

实验二使用脑干频率追随响应技术考察音乐经验对语音皮层下加工的影响机制。结果发现音乐家对语音音节的第一谐波的频率追随能力显著高于非音乐家被试，而且该追随能力与被试的音乐能力显著正相关。说明即使是声调语言母语者，对语音中与时间精细结构相关的频率信息的皮层下神经表征依然受到音乐经验的增强。

实验三使用脑电图技术考察音乐经验对连续的自然语音、包络语音和精细结构语音在安静和噪音下诱发的皮层包络跟踪响应的影响。结果发现，试次间一致性、神经时间响应函数与包络重构三种指标均显示，在所有条件下非音乐家比音乐家在 delta 频段（短语或语句时间尺度）与 theta 频段（音节与字词时间尺度）都具有更强的包络跟踪响应。此外，试次间一致性与时间响应函数指标与非音乐家在噪音中自然语音复述成绩相关，而包络重构精确度与音乐家在噪声中自然语音复述成绩相关。跨频耦合分析发现，delta 相位与 theta 幅度的耦合强度在音乐家中超过了非音乐家，尤其是在安静中的精细结构语音以及噪音中的自然语音中，并且安静中精细结构语音的跨频耦合强度与被试的音乐能力显著相关。这些结果说明音乐经验促进了大脑对语音包络分配更少的资源而更加利用语音精细结构的动态变化以一种自上而下的方式主动预测、构建和整合不同层级的语音单元，而非音乐家则更多依赖自下而上的语音包络跟踪进行言语知觉与理解。

本研究横向比较了音乐家与非音乐家对不同语音时间线索的加工策略差异，涉及了行为以及脑干到大脑皮层的神经机制，为研究音乐经验如何影响大脑对连续语音的层级式动态加工提供了证据。

Mechanisms of speech and music processing are two pillars in the study of auditory system. Numerous evidences have been provided that musical experience influences speech perception, especially in noisy environments. Such an experience induced facilitation might be achieved either via an enhancement of higher-level cognitive functions (e.g., auditory attention or auditory working memory) and the cross-modal integration ability, or strengthened processing of speech acoustic cues (e.g., pitch or harmonics). However, whether musical experience has an impact on the temporal processing of speech acoustic information at the sentence or paragraph level is still under debate.

Mechanisms of speech and music processing are two pillars in the study of auditory system. Numerous evidences have been provided that musical experience influences speech perception, especially in noisy environments. Such an experience induced facilitation might be achieved either via an enhancement of higher-level cognitive functions (e.g., auditory attention or auditory working memory) and the cross-modal integration ability, or strengthened processing of speech acoustic cues (e.g., pitch or harmonics). However, whether musical experience has an impact on the temporal processing of speech acoustic information at the sentence or paragraph level is still under debate.

In the first psychophysical experiment, cognitively matched musicians and nonmusicians were asked to hear and repeat speech sentences contain either TFS only or envelope only information at 3 difficulty levels (3 different numbers of frequency bands) and 3 signal-to-noise ratios (2 * 3 * 3 experiment design). Results showed that musical experience decreased the identification of envelope only sentences, and increased the identification of TFS only sentences. Besides, there was no significant difference in identifying natural speech-in-noise sentences between musicians and non-musicians. This indicates that musical experience might influence the allocation of cognitive resources and preferential processing of different temporal structures.

Experiment 2 investigated the influence of musical experience on the subcortical speech processing using brainstem frequency-following response (FFR). Results showed that compared to non-musicians, musicians exhibited enhanced FFR-H1 (the 1st harmonic) tracking of speech syllable. Furthermore, the FFR-H1 tracking strength had a positive correlation with subject’s musical ability. This provided evidence that the subcortical representation of spectral information within speech TFS was enhanced by musical experience even for native speakers of tonal language.

Experiment 3 used cortical EEG technique to explore the influence of musical experience on envelope tracking response induced by natural speech, TFS speech and envelope speech in both quiet and noisy conditions. Results of inter-trial correlation (ITC), neural temporal response function (TRF) and envelope reconstruction accuracy consistently found a stronger entrainment to speech envelope in non-musicians compared to musicians at delta and theta frequencies. Meanwhile, the ITC and TRF values were correlated with the repetition score of natural speech in noise in nonmusicians, while the reconstruction accuracy was correlated with the same score of musicians. In comparison, compared to non-musicians, musicians had a higher coupling between delta phase and theta amplitude when processing natural speech in noise and TFS speech in quiet. Besides, the cross-frequency coupling in processing TFS speech in quiet was significantly correlated with subject’s music ability. Our findings suggested that musicians may reduce resource allocation to speech envelope but have advantage for processing the dynamic feature of speech TFS to predict, chunk and integrate hierarchical speech units in a top-down manner, while non-musicians may rely more on the bottom-up tracking of speech envelope in comprehending speech.

To conclude, by exploring effects of musical experience on processing speech envelope and temporal fine structure from various aspects, including the behavior and EEG response from auditory brainstem to cortex, the present study shed light on the influence of musical experience on the hierarchical and dynamic processing of continuous speech in the brain.