认知与发展心理学研究室知识库

言语是人类交流的重要工具。最近数十年的研究表明，言语产出和知觉理解可能共享一套神经环路，即“说中有听，听中有说”，这被称为言语加工的感觉运动整合。在言语知觉和理解层面，已有研究发现位于左侧额叶和顶叶的背侧发音运动皮层通路参与其中，尤其在有挑战性的言语知觉中为进行声学解码和语义提取的颗叶腹侧通路提供辅助;另一方面，左侧运动皮层参与言语知觉时呈现出与其控制发音运动时相似的躯体拓扑特征。但是，言语知觉的感觉运动整合脑机制在较精细和较宏观的时空尺度上参与言语知觉的具体模式尚未被探明。在较精细的空间尺度上，尚未清楚喉部运动区是否以及如何参与由其控制发音的言语线索的知觉决策;在较精细的时间尺度上，运动皮层参与言语线索知觉决策的何种阶段尚存争议;在较宏观的空间尺度上，双侧听皮层与运动皮层如何协同参与到言语知觉中的机制尚未探明;在较宏观的时间尺度上，运动皮层参与连续多层级的自然语流知觉和理解的机制尚未厘清。本文通过三项研究(包括两个对前人研究的系统性综述和四个实验)，整合文献元分析、脑影像技术、无创脑刺激技术以及针对多模态行为和神经电生理数据的建模技术，针对汉语这一声调语言的知觉和理解，从多个角度对该问题进行了探讨。

研究一为系统综述研究，分为对言语线索知觉的脑影像研究的元分析(分析一)和对言语知觉和理解的无创脑刺激研究的系统性综述(分析二)。分析一发现，听皮层和运动皮层均参与到言语线索知觉判断任务中，且在不同的言语线索 (音段音位、声调和韵律)知觉中的激活位点的空间分布呈现出与线索的声学属性、语言学功能和发音运动特征相对应的梯度结构和功能偏侧化。分析二发现，更多证据表明左侧运动皮层因果性参与到决定语义的言语音位知觉，而右侧运动皮层则更多参与不传达意义的韵律知觉;另一方面，包含运动皮层在内的背侧通路通过多重功能连接与腹侧通路协同参与言语知觉。

研究二采用经颅磁刺激技术扰动左/右侧喉部运动区，并通过心理物理曲线拟合和对反应时分布进行漂移扩散模型拟合，测量被试普通话声调和声母知觉决策任务表现的变化，以研究短时程音位和音节知觉的感觉运动整合脑机制。其中，实验一采用重复性经颅磁刺激，实验二采用theta一爆发式刺激，普通话母语者被试在两个实验中进行相似的任务。两个实验的心理物理曲线拟合结果共同显示，双侧喉部运动区以效应器特异的方式参与言语线索的知觉决策。实验二的曲线拟合结果则进一步表明，左侧喉部运动区在言语声调和声母知觉决策中起到较为关键的作用，而右侧喉部运动区则在任务困难时有因果性的参与。同时，实验二的漂移扩散模型结果表明，喉部运动区参与言语知觉决策的各个阶段，但具体的参与阶段与所在脑半球和任务难度有关。

研究三采用经颅交流电刺激(实验三)、脑电图(实验三)和脑磁图(实验四)技术分别探究感觉运动环路在句子知觉中的因果性参与(实验三)和句子和语篇理解的感觉运动整合神经机制(实验四)。实验三同时在被试的听皮层和运动皮层施加同相位或反相位4-Hz经颅交流电刺激并让被试进行困难情景下的语句复述任务以观察听皮层和运动皮层低频神经振荡祸合被增强/破坏后行为成绩的变化，发现反相位交流电刺激降低被试在语谱噪声掩蔽句子复述任务中的成绩，且被试在无电刺激条件中相同任务时脑电信号的低频相位和alpha段幅值对言语信号的跟踪准确率显著预测电刺激调控效应。实验四让被试在脑磁图扫描中聆听普通话故事，并采用时间响应函数在通道水平上计算各频段神经振荡对言语时域包络结构、音高轮廓和嗓音起止线索的编码准确率，以探究运动皮层和听皮层对多维度言语声学线索的编码机制。实验四发现低频振荡(delta和theta段)同时编码三种线索，而alpha段振荡编码时域包络;在鸡尾酒会场景(有人声干扰)中，左侧运动皮层区域保留一定的编码能力，且在时域包络编码中体现在低频和alpha段，而在音高轮廓和嗓音起止线索的编码中则体现在低频信号。研究三的两个实验共同表明，左侧运动皮层在任务困难时辅助听皮层对言语声学线索进行编码，且这种辅助机制可能与额顶叶和颗叶的跨频神经振荡祸合有关。

综上所述，在精细空间尺度上，本文发现喉部运动区与唇、舌等运动皮层亚区共同构成效应器特异的“运动条带”参与言语知觉;在精细时间尺度上，运动皮层参与言语线索知觉决策的多个阶段;在宏观空间尺度上，双侧运动皮层参与言语线索知觉的功能偏侧化与其控制言语运动的方式和线索的语言学功能有关，并且左侧运动皮层起主导作用;在宏观时间尺度上，运动皮层参与多维度言语线索的编码且在任务困难时辅助听皮层编码，且运动皮层与听皮层低频(delta和theta段)神经振荡相位祸合和运动皮层中频(alpha段)神经振荡对此有所贡献。本文深化了对汉语言语知觉感觉运动整合脑机制及其理论模型的认识，对言语加工障碍人群的干预与类脑人工语言智能的发展具有启发意义。

Language is an essential tool for human communication. Research in the past decades has shown that the production and perception/comprehension of speech information may share the same neural circuits. This is the sensorimotor integration in speech processing, namely "the listening in speaking, and the speaking in listening". Speech perception and comprehension have been found to engage the dorsal articulatory motor cortical pathways located in the left frontal and parietal lobes. Particularly, this involvement offers compensation to the ventral pathways for auditory encoding and lexical retrieval located in the temporal cortices. On the other hand, the left motor cortex is engaged in speech perception with effector-specific somatotopy as in speech production. However, the sensorimotor integration mechanisms in speech perception at relatively micro and macro spatiotemporal scales remain unclear. At the micro-spatial scale, whether and how the laryngeal motor cortex (LMC) is recruited in the perceptual decision of the speech cues that it controls in articulation is unknown. At the micro-temporal scale, it remains debatable regarding the particular stages of motor involvement in the speech perceptual decision. At the macro一spatial scale, how bilateral auditory and motor cortices are cooperatively engaged in speech perception is incompletely characterized. At the macro-temporal scale, how the motor cortices subserve the perception and comprehension of hierarchically organized continuous speech is elusive. This dissertation reports three studies that explored these topics from multiple perspectives concentrating on the perception and comprehension of Chinese, a tonal language. Studies included two systematic reviews of previous research and four experiments and integrated meta-analysis, neuroimaging, and non-invasive brain stimulation techniques as well as computational modeling of multi-modal behavioral and neuro-electrophysiological data.

Study 1 includes two systematic reviews (Analysis 1 for a meta-analysis of neuroimaging studies on the perception of speech cues; Analysis 2 for a target-based systematic review of non-invasive brain stimulation studies on speech perception and comprehension. Analysis 1 found that the auditory and motor cortices are both engaged in the perceptual decision of speech features, and the spatial distributions of brain activations for the perception of different cues (segmental phoneme, lexical tone, and prosody) reflect the acoustic properties, linguistic functions, as well as articulatory motor gestures in terms of the gradient of representation and hemispheric functional asymmetry. Analysis 2 found more causal evidence for the recruitment of the left motor cortex in the perception of lexical phonemic cues, and that of the right motor cortex in the perception of non-lexical prosodic features. Moreover, in speech perception, the dorsal stream that contains the motor cortices cooperates with the ventral stream through multiple functional connectivity pathways.

Study 2 investigated the brain mechanisms subserving sensorimotor integration of the perception of short phonemes and syllables. Study 2 used transcranial magnetic stimulation (TMS) to interfere with the left or right LMC and measured the effects on the perceptual decision of Mandarin lexical tone and consonant, through psychometric curve fitting as well as drift-diffusion modeling (DDM) of reaction time distributions. Experiment 1 utilized repetitive TMS (rTMS), whereas Experiment 2 applied theta- burst stimulation (TBS), and Mandarin-speaking participants performed similar tasks in the two experiments. Psychometric curve fitting in the two experiments convergently showed that bilateral LMCs are involved in the perceptual decision of speech features in an effector-specific manner. Moreover, curve-fitting results in Experiment 2 showed that the left LMC is relatively essential in the perceptual decision of lexical tone and consonant, while its right counterpart is causally engaged in difficult tasks. Meanwhile, the DDM in Experiment 2 demonstrated that the activations of the LMC contribute to various stages of speech perceptual decision as a function of the hemisphere and task difficulty.

Study 3 used transcranial alternating current stimulation (tACS, Experiment 3), electroencephalography (EEG, Experiment 3), and magnetoencephalography (MEG, Experiment 4) techniques to investigate the causal engagement of the sensorimotor circuit in sentence perception (Experiment 3) and the sensorimotor integration neural mechanisms of sentence and discourse comprehension (Experiment 4). Experiment 3 applied in-phase or anti-phase 4-Hz dual-site tACS upon participants' auditory and motor cortices when they performed sentence repetition tasks in noise in order to detect changes in repetition scores after the auditory-motor low-frequency neural oscillation coupling being enhanced or hampered, correspondingly. Experiment 3 found that antiphase tACS impaired the performance in sentence repetition in speech spectrum noise. Meanwhile, the speech-tracking accuracies of the low-frequency EEG phase and alphaband EEG power in the same tasks without tACS significantly predicted stimulation effects. To investigate the neural tracking of multi-dimensional speech acoustic features in the auditory and motor cortices, Experiment 4 recorded MEG signals when participants listened to a story read in Mandarin and used the temporal response function (TRF) to quantify the encoding capacities of the temporal envelope, pitch contour, and voicing onset/offset at the sensor level. Experiment 4 found that lowfrequency (delta and alpha bands) neural oscillations subserve the encoding of all three features, and the alpha band oscillations also support envelope encoding. In the "cocktail party" situation (masked by speech), the left motor cortex keeps a certain encoding ability, and this occurs at the low frequency and alpha band for envelope encoding, and at the low-frequency bands for pitch contour and voicing onset/offset encoding. Two experiments from Study 3 convergently showed that the left motor cortex compensates for the encoding of speech acoustic features in challenging tasks, and the cross-frequency coupling of neural oscillations between the frontoparietal and the temporal areas may support this compensation.

In summary, this dissertation reports that, at the micro一spatial scale, the LMC is involved in speech perception with effector-specificity that constitutes a "motor strip" along with the lip and tongue motor areas. At the micro-temporal scale, the motor cortex is engaged in various phases of speech perceptual decision. At the macro-spatial scale, the hemispheric functional asymmetry of bilateral motor cortices is related to their roles in the articulation and the linguistic functions of the speech cues and is left-dominant. At the macro-temporal scale, the motor cortex encodes multi一dimensional speech cues and assists auditory encoding at adverse listening conditions. The low-frequency (delta and theta bands) auditory-motor neural oscillation coupling and the alpha band signals may subserve this integration. This dissertation deepens the knowledge of the brain mechanisms and theoretical models of sensorimotor integration in Chinese speech perception and is potentially constructive for clinical interventions on speech disabilities and the development of brain-inspired artificial intelligence.