Computational protocol: “Neural overlap of L1 and L2 semantic representations across visual and auditory modalities: a decoding approach”

Similar protocols

Protocol publication

[…] Preprocessing and analysis of the fMRI data was performed using SPM8 software (Wellcome Department of Cognitive Neurology, London, UK). Reduction of T1 relaxation artefacts was pursued by exclusion of the first nine scans of all runs. The functional images were motion corrected with ArtRepair (Artifact Repair Toolbox v4), corrected for slice scan time differences and spatially realigned to their mean image by rigid body transformation. The anatomical image was normalized to the Montreal Neurological Institute (MNI) template brain image. The functional images were aligned with the high-resolution anatomical image to ensure an anatomically-based normalization. The low frequency artefacts in the time series data were removed using a high-pass filter with a cutoff at 128 s.For each modality and separately for the two language parts, statistical analyses were performed on individual subjects’ data using the general linear model (GLM) in SPM8. Trials with incorrect semantic categorization were excluded from the analysis. The fMRI time series data were modelled by 10 different vectors, one for each semantic concept. All these vectors were convolved with a hemodynamic response function (HRF), as well as with the temporal derivative and entered into the regression model (the design matrix). Additionally, six motion parameters were added to the design matrix as regressors of no interest to account for variance related to head motion. The statistical parameter estimates were computed separately for all columns in the design matrix. [...] To investigate the neural overlap between Dutch and French semantic representations, within and across the three tasks (naming, word reading and word listening), a multivariate decoding analysis was applied with the PyMVPA toolbox (). Multivariate decoding analyses were performed on the normalized but unsmoothed images to maximize the sensitivity to extract the full information in the spatial patterns, which might be reduced after smoothing (). Therefore smoothing was applied after multivariate decoding, prior to the second-level analyses with an 8 mm full-width half-maximum (FWHM) Gaussian kernel. A spherical whole brain searchlight with a radius of 3 voxels was applied to extract local spatial information from small brain spheres that carry information about the semantic concept (). The searchlight used the K Nearest Neighbours pattern classifier for this semantic classification (). Note that the use of other classifiers yielded similar results. More specifically, the classifier was trained to identify semantic activation of 10 concepts, associated with reading, listening to words or naming respective pictures, based on the neural pattern of brain activation elicited by reading/listening to /picture naming the same concepts in the other language. For instance, the classifier tried to predict neural activation triggered by the reading of the word cheval [horse] from the neural activation during reading (within-modalities) or listening/picture naming (across-modalities) of the translation equivalent paard, and vice versa.Because one aim of the present paper was to investigate cross-lingual overlap, within tasks, we primarily focused on the across-language decoding analysis. For within-language analyses, the exact same stimuli (identical pictures, written words and spoken words) are by definition included, making it difficult to disentangle semantic activation from other overlapping visual, auditory or lexical features when applying MVPA. Across languages, visual and phonetical/acoustical similarities between the stimulus pairs of a concept and lexical similarities between the translation equivalents were maximally reduced in all three tasks to assure that classifier performance only reflected access to the shared semantic representation needed for the task in the two languages. The classifier was trained on the task-specific activation pattern associated with each of the 10 concepts in one language in four of the five blocks (training data set). Subsequently, this pattern classifier was used to classify the task-specific activation pattern for each of the 10 concepts in the corresponding fifth block of the other language (test data set). This procedure was repeated 5 times, so that each block could function as a test block once, while the other blocks were used as training blocks. Mean decoding accuracy maps across all five classifications were achieved for each participant in two directions (Dutch as training blocks and French as test block and vice versa). These across-language decoding accuracies were then averaged across the two directions, resulting in one mean decoding accuracy map across languages for each participant.Additionally, in order to achieve our second aim, examining whether the semantic representations are shared across the three language modalities, MVPA was applied across modalities. Across modalities, we again only focused on the across-language decoding, because semantic overlap may by definition not be distinguished from lexical overlap in the within language decoding analysis, as this implies decoding activation after exposure to the same stimuli. For instance, a pattern classifier was trained on the activation pattern associated with the performance in L1 during the naming task, and then tested on how well it decoded the activation pattern associated with the performance in L2 during reading or listening. The underlying assumption was that the classifier would only be able to accurately predict which stimulus/concept was processed in the reading or listening task based on the activation in the naming task, if semantic representations overlap across these tasks. Across tasks there wasn’t any visual or auditory confound, because pictures, spoken words and written words of the same concepts relied on different sensory features. […]

Pipeline specifications