认知与发展心理学研究室知识库

言语是人类社会中最为重要、也是最为常见的交流形式之一。为了理解言语信息，听者需要在极短的时间内处理具有极大可变性的语音信号，并迅速将模糊的声学信号转化为从音素到语义的表征层级结构。前人研究认为，人脑通过预测来解释即将输入的信息，提出了预测编码理论。该理论强调预测误差信息，即预测与实际输入之间的差异，被用于更新大脑内部的预测模型，从而塑造人类的感知体验和高级认知活动。本文以言语理解中的层级性预测编码框架为理论基础，从人脑言语信息加工的空间结构、时间结构以及脑结构基础三个方面，探究了言语理解中层级性动态时空预测编码的神经机制。研究一采用功能磁共振成像技术，以正常成语、关键字违反成语、关键字缺失成语以及乱序词组作为刺激材料，结合单变量分析和多变量分析，研究了预测编码的层级性空间表征模式。该研究细致地区分了目标单元违反造成语义和语音预测误差以及目标单元缺失诱发的预测误差的空间表征差异。

研究一 a 的单变量分析结果表明，违反和缺失条件相比于正常条件，在左侧额下回、颞上沟/回以及腹侧前运动皮层发现了由于关键字匹配失败引起的词汇表征增强效应。乱序条件由于缺少充足的语境信息，相比于正常条件在词汇语义选择和整合相关脑区表现出减弱效应，但在颞叶和脑岛等语音加工脑区表现出增强效应。 研究一 b 的多变量分析，通过比较不同实验条件中完全相同或部分相同成语与完全不同成语间的表征相似性差异，探究加工相同成语或相同关键字时的表征模式。结果发现，在违反条件下，即使语境信息完全不同，对相同关键字的词汇加工位于左侧喉部运动有关的运动皮层。完全相同的违反成语相比于完全不同的违反成语，以及相同关键字的缺失相比于不同关键字的缺失，在左侧颞上沟的前侧发现对关键字的表征和预测。该结果表明，发音运动区和左侧颞上沟前侧是表征关键字信息的重要区域。此外，缺失条件下，即使在不同语境信息下，相同关键字缺失也会导致额-顶叶一般性控制网络参与到关键字的预测表征过程中。这表明预测编码过程还受到实验任务调节，试图从缺失的关键字中提取语义信息需付出更多的认知努力。研究一 c 采用间接量化预测误差神经表征模式的方式，验证了核心额-颞叶语言网络在预测编码中的重要作用，并观测到语音和语义的预测误差信息分别位于额下回、颞上沟/回后侧以及顶叶语音相关区域和颞上沟前侧词汇语义加工区域的层级性表征结构中。研究一从信号增强和预测编码的不同神经表征机制上，证明了词汇违反和词汇缺失造成相似的神经表征结构，以及词汇违反时语音和语义单元预测误差的层级性表征结构。

研究二采用脑磁图成像技术，从多个神经特性的时间特征探究了听觉词汇短语预测编码的层级性时间结构特征。研究二 a 通过事件相关场分析，发现词汇违反诱发了与前人研究一致的 N400-P600 效应，但在缺失条件下并未观察到该效应。研究二 b 通过时频分析探究了实验条件间神经振荡的能量差异，结果发现在违反和缺失条件中，在关键字出现后 320 ms ~ 960 ms 间受到 beta 和 gamma 振荡的调节。研究二 c 采用与研究一中相同的量化预测误差神经表征模式的方法，结果发现在全脑传感器水平上，词汇违反和词汇缺失引发的预测误差在时间表征上的相似性。在全脑传感器的探照灯分析中，则发现违反条件下，左侧传感器特异性的语义信息预测误差表征早于语音信息预测误差表征。研究二从多个神经表征特征探讨了词汇违反和词汇缺失既相似又不同的预测误差表征机制，并验证了言语预测误差单元的层级性时间表征模式。

研究三采用高角度分辨率的扩散成像技术，进行全脑白质纤维束的概率性追踪，根据研究一中表现出层级性预测误差表征的节点，获得更精细化的白质纤维连接通路，并结合传统弥散张量指标和微观结构指标，更为多元化地评估了白质纤维束的连通性。结果表明，研究二中语音特征预测误差表征能力与连接额下回和颞上沟/回的背侧通路和腹侧通路的连通性相关。本研究首次观测到支持语音预测误差表征的时间特征的白质纤维通路。

综上，本文结合多模态脑影像技术，验证了言语理解中预测编码的层级性动态表征结构，及其相应神经环路的白质纤维束结构基础。结果发现，1）听觉短语理解过程中，语音和语义预测误差表征在左侧核心语言网络的额-颞叶皮层具有层级性动态时空表征特征；2）一般性执行控制网络受到任务调节以支持言语信息的预测编码加工；3）语音预测误差表征能力与左侧额下回-颞上沟/回间背侧通路和腹侧通路的白质纤维束连通性有关。本文的发现不仅有助于加深对人脑语言预测机制的理解，也对类脑语言模型的发展具有一定的启发。

Speech is a fundamental and ubiquitous form of communication in human society. To understand auditory speech, listeners need to rapidly process a wide range of variable speech signals and transform ambiguous auditory streams into a hierarchical structure, from phonemes to meanings. Prior research suggests that the human brain employs predictions to interpret upcoming speech information. Prediction coding theory highlights the role of prediction error information in updating the brain's predictive models by identifying discrepancies between anticipated and incoming perceptual information. This mechanism is crucial for shaping human perceptual experiences and higher cognitive functions. This dissertation delves into the neural underpinnings of hierarchical dynamic spatiotemporal prediction coding in language comprehension, within the framework of hierarchical predictive coding in language understanding. It examines the spatial and temporal structures, as well as the neural basis, of language processing in the human brain.

Study 1 utilized functional magnetic resonance imaging (fMRI) technology with stimuli comprising normal phrase, violated phrase, missing phrase, and disordered words. Through univariate and multivariate analyses, it explored the hierarchical spatial representation of predictive coding. This study dedicated to differentiate spatial representational differences in prediction errors caused by semantic and phonological violations of target units, as well as prediction errors induced by the missing targets. The univariate analysis results of study 1a revealed that compared to the normal phrase condition, the violation and missing conditions elicited higher activation in the left inferior frontal gyrus, superior temporal sulcus/gyrus, and ventral premotor cortex due to keyword matching failures, exhibiting the enhancement effect of lexical representation. Conversely, compared to the normal condition, the disordered words condition, due to insufficient contextual information, showed weaker activation in areas related to lexical and semantic selection and integration but stronger activation in phonological processing-related regions like the temporal lobe and insula. The multivariate analysis of study 1b，by comparing the representational similarity differences among completely the same or partially the same idioms and completely different idioms under different experimental conditions, explored the representation pattern when processing the same idioms or same key-words. The results found that under the violated condition, even with completely different contextual information, the lexical processing of the same key-words was located in the left part of the laryngealrelated motor cortex. Completely the same violated idioms compared to completely different violated idioms, and the same key-word missing compared to different keyword missing, found representation and prediction of key-words on the anterior part of the left superior temporal gyrus. This indicates that the articulatory motor area and the anterior part of the left superior temporal gyrus are important areas for representing key-word information. Moreover, under the missing condition, even with different contextual information, the same key-word missing led to the involvement of the frontal-parietal general control network in the prediction representation process of the key-word. This suggests that the predictive coding process is also subject to experimental task modulation, trying to extract semantic information from the missing key-word requires more cognitive effort. Study 1c adopted a indirect quantification of predictive error neural representational patterns, verifying the important role of the core frontal-temporal language network in predictive coding, and observed the hierarchical representation structure of speech and semantic predictive error information located in the inferior frontal gyrus, posterior superior temporal sulcus/gyrus, and parietal lobe speech-related areas and anterior superior temporal gyrus lexical semantic processing areas. Study 1 proved different neural representation mechanisms of sharpened signals and predictive coding, demonstrating similar neural representation structures caused by lexical violation and lexical missing, as well as the hierarchical representation structure of phonetic and semantic unit predictive errors during lexical violation.

Study 2 used magnetoencephalography imaging (MEG) to explore the hierarchical temporal structure features of auditory lexical phrase predictive coding，from the temporal characteristics of multiple neural attributes. Study 2a, through event-related field analysis, found that lexical violation induced an N400-P600 effect consistent with previous research, but this effect was not observed under the missing condition. Study 2b, through time-frequency analysis, explored the power differences in neural oscillations among experimental conditions, finding that beta and gamma oscillations regulated the violated and missing conditions between 320 ms ~ 960 ms after the appearance of the key-word. Study 2c, using the same indirect quantification of predictive error neural representation patterns method as in study 1, found similarities in the temporal representation of predictive errors induced by lexical violation and lexical missing at the whole-brain sensor level. In the whole-brain sensor beamforming analysis, it was found that under the violated condition, the representation of semantic information predictive error in the left sensors was earlier than the representation of phonetic information predictive error. Study 2 discussed the predictive error representation mechanisms that are both similar and different between lexical violation and lexical missing from multiple neural representation features, and verified the hierarchical temporal representation pattern of speech predictive error units.

Study 3, using high angular resolution diffusion imaging (HARDI) performed probabilistic tracking of whole-brain white matter fiber bundles, obtaining more refined white matter fiber connection pathways based on the nodes showing hierarchical predictive error representation in study 1, and combined with traditional diffusion tensor metrics and microstructural indices, more diversely assessed the connectivity of white matter fiber bundles. The results showed that the ability to represent phonetic feature predictive errors in study 2 was related to the connectivity of the dorsal and ventral pathways connecting the inferior frontal gyrus and superior temporal sulcus/gyrus. This study observed, for the initial time, white matter fiber pathways supporting the temporal characteristic of phonological prediction error representation.

In summary, this article, combining multimodal brain imaging technologies, validated the hierarchical dynamic representational structure of predictive coding in speech understanding, and its corresponding neural circuitry white matter fiber bundle structural basis. The findings revealed that 1) in the process of auditory phrase understanding, the representation of phonetic and semantic predictive errors in the left core language network's frontal-temporal cortex has hierarchical dynamic spatiotemporal representation features; 2) the general executive control network is modulated by tasks to support the predictive coding processing of speech information; 3) the ability to represent phonetic predictive errors is related to the connectivity of white matter fiber bundles between the left inferior frontal gyrus-superior temporal sulcus/gyrus dorsal and ventral pathways. The findings of this article not only enhance understanding of the human brain's language prediction mechanism but also have implications for the development of brain-like language models.