Computational protocol: Observer variability of absolute and relative thrombus density measurements in patients with acute ischemic stroke

Similar protocols

Protocol publication

[…] Thrombus measurements can be performed in any radiological workstation []. This task consists of detecting the thrombus in NCCT images and measuring its density by the placement standardized measurement tools (e.g., ROIs such as 2D ellipses or 3D spheres); see Fig. . As a support for the detection of low-density thrombi, CTA images can be displayed simultaneously with the NCCT images. For the simultaneous displaying of NCCT and CTA images, CTA images were automatically aligned with NCCT images using a rigid registration of the open source software Elastix® []. In this study, we performed the annotations and measurements with in Mevislab® developed software []. Thrombus density was measured in the proximal, middle, and distal part using three separate spherical volumes with a radius of 1 mm. In case of small thrombi, the ROIs often overlapped, which was accepted in our study. Our method assured that in case of overlapping ROIs, the attenuation values of every pixel were counted only once. In a similar fashion, contralateral density measures were performed at the corresponding site of occlusion []. The middle ROI density measurement of both the thrombus and contralateral artery was used for the single ROI measurement. Based on the CT attenuation values, the average absolute thrombus density in Hounsfield units (aHU) and average relative thrombus density in Hounsfield units (rHU) for three and one ROIs, abbreviated as aHU3, rHU3, aHU1, and rHU1, respectively, were determined.Fig. 1Three expert neuroradiologists each with more than 10 years of experience (AY, LB, CM) and two non-expert trained observers (MB and CW, one with a MSc in Biomedical Engineering and a fifth-year student in MSc Technical Medicine) placed the ROIs. The trained observers received approximately 3 h of training. The observers only had access to baseline NCCT and CTA images during the measurement sessions and were blinded from measured intensity, all clinical information, and each other’s measurements.Before the measurements, a calibration session was organized in which the observers performed the measurement in a training set of nine randomly selected datasets. This calibration session was used to assess any differences in interpretation and measurement strategy. Differences were discussed in a subsequent consensus meeting, after which the following additional instruction was provided; in case of thrombus at a bifurcation, only the longest branch should be used for the measurement.During the measurements, the observers could exclude a dataset if the image quality was considered insufficient to confidently place the ROIs, or if the occlusion was not distinguishable. If a dataset was excluded, the reason was recorded.After the measurements, a verification of the number of markers as well as a visual check of all ROIs was performed by a single external observer (ES) to detect missing, supernumerary, or incorrectly labeled ROI. These suspicious ROIs were discussed and corrected by the same observer if necessary. The number of incorrect ROI placements and performed corrections was recorded. [...] The interobserver agreement of the thrombus density measurements was evaluated by performing a paired comparison of the measurements of observers 1 and 2 and of observers 1 and 3. The accuracy of the trained non-expert observers was assessed by comparing measurements of observers 4 and 5 with that of observer 1 as the reference standard. The interobserver agreement of the trained non-expert observers was assessed by comparing measurements of observer 4 and observer 5. For all comparisons, Bland–Altman analysis was performed and the intraclass correlation coefficient (ICC) was calculated.To estimate the bias introduced by using only one instead of three ROIs, we evaluated the intermethod agreement by comparing the single ROI measurements with the three ROI measurements, using the latter as reference. Statistical significance of the differences between one and three ROI measurements was tested using paired t tests for normally distributed data or a related-sample Wilcoxon signed-rank test otherwise. Furthermore, a Bland–Altman analysis was performed and the ICC was determined.An ICC superior to 0.80 was considered as excellent, between 0.60 and 0.79 as good, between 0.40 and 0.59 as fair, and below 0.39 as poor. Normality of the distributions was tested using the Shapiro–Wilk test. Significance level was set to a p value of 0.05. To identify outliers prior to the ICC calculations, we used Tukey’s hinges method [], in which observations below Q1 − 1.5 (Q3 − Q1) and above Q3 + 1.5(Q3 − Q1) were excluded. All analyses were performed using IBM SPSS Statistics software, version 20.0 (IBM Corporation, Armonk, NY, USA). […]

Pipeline specifications

Software tools elastix, MeVisLab, SPSS
Applications Miscellaneous, Magnetic resonance imaging
Organisms Homo sapiens
Diseases Cerebral Infarction, Thrombosis