Phone: +31 88 75 56682
Nikolas studied Biomedical Engineering at the University of Lübeck, Germany. For his Bachelor’s thesis, he worked on X-ray based tracking to improve transbronchial biopsy of pulmonary nodules. He then spent one year at Philips Research in Hamburg, where he worked on automatic analysis of chest CT images for the detection of pulmonary embolism and on automatic segmentation of the pulmonary lobes. He is currently working on automatic calcium scoring and early detection of osteoporosis in lung cancer CT screening.
Papers in international journals
|1.||S.A.M. Gernaat, S.G.M. van Velzen, V. Koh, M.J. Emaus, I. Išgum, N. Lessmann, S. Moes, A. Jacobson, P.W. Tan, D.E. Grobbee, D.H.J. van den Bongard, J.I.Tang, H.M .Verkooijen. Automatic quantification of calcifications in the coronary arteries and thoracic aorta on radiotherapy planning CT scans of Western and Asian breast cancer patients. Radiotherapy and Oncology, in press, 2018 ( Abstract | pdf )
This study automatically quantified calcifications in coronary arteries (CAC) and thoracic aorta (TAC) on breast planning computed tomography (CT) scans and assessed its reproducibility compared to manual scoring.
Material and Methods
Dutch (n=1,199) and Singaporean (n=1,090) breast cancer patients with radiotherapy planning CT scan were included. CAC and TAC were automatically scored using deep learning algorithm. CVD risk categories were based on Agatson CAC: 0, 1-10, 11-100, 101-400 and >400. Reliability between automatic and manual scoring was assessed in 120 randomly selected CT scans from each population, with linearly weighted kappa for CAC categories and intraclass correlation coefficient for TAC.
Median age was higher in Dutch patients than Singaporean patients: 57 versus 52 years. CAC and TAC increased with age and were more present in Dutch patients than Singaporean patients: 24.2% versus 17.3% and 73.0% versus 62.2%, respectively. Reliability of CAC categories and TAC was excellent in the Netherlands (0.85 (95% confidence interval (CI)=0.77-0.93) and 0.98 (95% CI=0.96-0.98) respectively) and Singapore (0.90 (95% CI=0.84-0.96) and 0.99 (95% CI=0.98-0.99) respectively).
CAC and TAC prevalence was considerable and increased with age. Deep learning software is a reliable method to automatically measure CAC and TAC on radiotherapy breast CT scans.
|2.||N. Lessmann, B. van Ginneken, M. Zreik, P.A. de Jong, B.D. de Vos, M.A. Viergever, I. Išgum. Automatic calcium scoring in low-dose chest CT using deep neural networks with dilated convolutions. IEEE Transactions on Medical Imaging, 2018, vol. 37, nr. 2, pp. 615-625 ( Abstract | pdf )
Heavy smokers undergoing screening with low-dose chest CT are affected by cardiovascular disease as much as by lung cancer. Low-dose chest CT scans acquired in screening enable quantification of atherosclerotic calcifications and thus enable identification of subjects at increased cardiovascular risk. This paper presents a method for automatic detection of coronary artery, thoracic aorta and cardiac valve calcifications in low-dose chest CT using two consecutive convolutional neural networks. The first network identifies and labels potential calcifications according to their anatomical location and the second network identifies true calcifications among the detected candidates. This method was trained and evaluated on a set of 1744 CT scans from the National Lung Screening Trial. To determine whether any reconstruction or only images reconstructed with soft tissue filters can be used for calcification detection, we evaluated the method on soft and medium/sharp filter reconstructions separately. On soft filter reconstructions, the method achieved F1 scores of 0.89, 0.89, 0.67, and 0.55 for coronary artery, thoracic aorta, aortic valve and mitral valve calcifications, respectively. On sharp filter reconstructions, the F1 scores were 0.84, 0.81, 0.64, and 0.66, respectively. Linearly weighted kappa coefficients for risk category assignment based on per subject coronary artery calcium were 0.91 and 0.90 for soft and sharp filter reconstructions, respectively. These results demonstrate that the presented method enables reliable automatic cardiovascular risk assessment in all low-dose chest CT scans acquired for lung cancer screening.
|3.||M. Zreik, N. Lessmann, R. van Hamersvelt, J.M. Wolterink, M. Voskuil, M.A. Viergever, T. Leiner, I. Išgum. Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis. Medical Image Analysis, 2018, vol. 44, pp. 72-85 ( Abstract | pdf )
In patients with coronary artery stenoses of intermediate severity, the functional signicance needs to be determined. Fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA), is most often used in clinical practice. To reduce the number of ICA procedures, we present a method for automatic identication of patients with functionally signicant coronary artery stenoses, employing deep learning analysis of the left ventricle (LV) myocardium in rest coronary CT angiography (CCTA).
The study includes consecutively acquired CCTA scans of 166 patients who underwent invasive FFR measurements. To identify patients with a functionally signicant coronary artery stenosis, analysis is performed in several stages. First, the LV myocardium is segmented using a multiscale convolutional neural
network (CNN). To characterize the segmented LV myocardium, it is subsequently encoded using unsupervised convolutional autoencoder (CAE). As ischemic changes are expected to appear locally, the LV myocardium is divided into a number of spatially connected clusters, and statistics of the encodings
are computed as features. Thereafter, patients are classied according to the presence of functionally signicant stenosis using an SVM classier based on the
Quantitative evaluation of LV myocardium segmentation in 20 images resulted in an average Dice coecient of 0:91 and an average mean absolute distance
between the segmented and reference LV boundaries of 0:7 mm. Twenty CCTA images were used to train the LV myocardium encoder. Classication of
patients was evaluated in the remaining 126 CCTA scans in 50 10-fold cross-validation experiments and resulted in an area under the receiver operating
characteristic curve of 0.74 0.02. At sensitivity levels 0.60, 0.70 and 0.80, the corresponding specicity was 0.77, 0.71 and 0.59, respectively.
The results demonstrate that automatic analysis of the LV myocardium in a single CCTA scan acquired at rest, without assessment of the anatomy of the
coronary arteries, can be used to identify patients with functionally signicant coronary artery stenosis. This might reduce the number of patients undergoing unnecessary invasive FFR measurements.
|4.||E. Pompe, P.A. de Jong, D.A. Lynch, N. Lessmann, I. Isgum, B. van Ginneken, J.-W.J. Lammers, F.A.A. Mohamed Hoesein. Computed tomographic findings in subjects who died from respiratory disease in the National Lung Screening Trial. European Respiratory Journal, 2017, vol. 49, pp. 1601814 ( Abstract | pdf )
We evaluated the prevalence of significant lung abnormalities on computed tomography
(CT) in patients who died from a respiratory illness other than lung cancer in the National Lung
Screening Trial (NLST).
In this retrospective case–control study, NLST participants in the CT arm who died of respiratory illness
other than lung cancer were matched for age, sex, pack-years and smoking status to a surviving control. A
chest radiologist and a radiology resident blinded to the outcome independently scored baseline CT scans
visually and qualitatively for the presence of emphysema, airway wall thickening and fibrotic lung disease.
The prevalence of CT abnormalities was compared between cases and controls by using chi-squared tests.
In total, 167 participants died from a respiratory cause other than lung cancer. The prevalence of severe
emphysema, airway wall thickening and fibrotic lung disease were 28.7% versus 4.8%, 26.9% versus 13.2%
and 18.6% versus 0.5% in cases and controls, respectively. Radiological findings were significantly more
prevalent in deaths compared with controls (all p<0.001).
CT-diagnosed severe emphysema, airway wall thickening and fibrosis were much more common in
NLST participants who died from respiratory disease, and CT may provide an additional means of
identifying these diseases.
Papers in conference proceedings
|1.||N. Lessmann, B. van Ginneken, P. A. de Jong, I. Isgum. Iterative fully convolutional neural networks for automatic vertebra segmentation. In: Medical Imaging with Deep Learning (MIDL), 2018 ( Abstract | pdf )
Precise segmentation of the vertebrae is often required for automatic detection of vertebral abnormalities. This especially enables incidental detection of abnormalities such as compression fractures in images that were acquired for other diagnostic purposes. While many CT and MR scans of the chest and abdomen cover a section of the spine, they often do not cover the entire spine. Additionally, the first and last visible vertebrae are likely only partially included in such scans. In this paper, we therefore approach vertebra segmentation as an instance segmentation problem. A fully convolutional neural network is combined with an instance memory that retains information about already segmented vertebrae. This network iteratively analyzes image patches, using the instance memory to search for and segment the first not yet segmented vertebra. At the same time, each vertebra is classified as completely or partially visible, so that partially visible vertebrae can be excluded from further analyses. We evaluated this method on spine CT scans from a vertebra segmentation challenge and on low-dose chest CT scans. The method achieved an average Dice score of 95.8% and 92.1%, respectively, and a mean absolute surface distance of 0.194 mm and 0.344 mm.
|2.||N. Lessmann, B. van Ginneken, I. Isgum. Iterative convolutional neural networks for automatic vertebra identification and segmentation in CT images. In: SPIE Medical Imaging, 2018, pp. 1057408 ( Abstract | pdf )
Segmentation and identification of the vertebrae in CT images are important initial steps for automatic analysis of the spine. This paper presents an automatic method based on iteratively applied convolutional neural networks. This approach utilizes the inherent order of the vertebral column to simplify the detection problem, so that a deep neural network can be trained with a low number of manual reference segmentations.
Vertebrae are identified and segmented individually in sequential order relative to a reference vertebra. Additionally, a coarse-to-fine segmentation scheme is employed: The localization and identification of the vertebrae is first performed in low-resolution images that enable the analysis of context information. The fine segmentation is performed afterwards in the original high-resolution images. In contrast to most previous methods for vertebra segmentation, this approach is not focused on modeling shape information.
The method was trained and evaluated with 15 spine CT scans from the MICCAI CSI 2014 workshop challenge. These scans cover the whole thoracic and lumbar part of the spine of healthy young adults. In contrast to a non-iterative convolutional neural network, the proposed method correctly identified all vertebrae. The method achieved a mean Dice coefficient of 0.948 and a mean surface distance of 0.29 mm and thus outperforms the best method that participated in the original challenge.
|3.||N. Lessmann, I. Isgum, A.A.A. Setio, B.D. de Vos, F. Ciompi, P.A. de Jong, M. Oudkerk, W.P.Th.M. Mali, M.A. Viergever, B. van Ginneken. Deep convolutional neural networks for automatic coronary calcium scoring in a screening study with low-dose chest CT. In: SPIE Medical Imaging, 2016, vol. 9785, pp. 978511-1-978511-6 ( Abstract | pdf )
Coronary artery calcium (CAC) scoring can identify subjects at risk of cardiovascular events in screening programs with low-dose chest CT. We present an automatic method for CAC scoring based on deep convolutional neural networks. Candidates are extracted by intensity-based thresholding and subsequently classified by three concurrent networks that analyze three orthogonal 2D image patches per voxel. The networks consist of three convolutional steps and one fully-connected layer. In 231 subjects, this method detected on average 194.3 / 199.8mm3 CAC (sensitivity 97.2%), with 10.3mm3 false-positive volume per scan. Accuracy of cardiovascular risk category assignment was 84.4% (linearly weighted kappa 0.89).
|1.||R. van Hamersvelt, M. Zreik, N. Lessmann, J. Wolterink, M. Voskuil, M.A Viergever, T. Leiner, I. Isgum
. Improving Specificity of Coronary CT Angiography for the Detection of Functionally Significant Coronary Artery Disease: A Deep Learning Approach. Radiological Society of North America, 103rd Annual Meeting , 2017 ( Abstract )
Coronary computed tomography angiography (CCTA) is an increasingly important diagnostic tool for the detection of
coronary artery disease (CAD). However, due to calcium blooming and beam hardening, specificity for diagnosing
functionally significant CAD is limited. The purpose of this study was to evaluate to what extent the specificity of
CCTA for detection of functionally significant CAD could be improved by combining simple stenosis grading with
deep-learning based analysis of left ventricular myocardium (LVM).
METHOD AND MATERIALS
We retrospectively included 126 patients (77% male, 58.7±9.5 years) who underwent CCTA prior to invasive
fractional flow reserve (FFR). Functionally significant CAD was defined as an invasively measured FFR value below
0.78. First, the presence and degree of coronary artery stenosis was analyzed using the CAD-RADS system.
Patients without a significant stenosis reported on CCTA scans were scored as functionally non-significant. For the
remaining patients, fully automatic deep learning analysis of the LVM was used to identify presence of functionally
significant CAD. LVM was first segmented using a convolutional neural network and then characterized by a
convolutional auto-encoder (CAE). Based on the encodings generated by the CAE a support vector machine
classifier identified patients with functionally significant stenosis. Diagnostic performance of this combined analysis
was evaluated and compared with patient identification based only on ≥50% stenosis degree as measured in CCTA.
FFR was significant in 64 (51%) of the patients. Sensitivity and specificity of stenosis degree reported on CCTA
alone were 91% and 18%, respectively. Adding deep-learning based analysis of LVM to stenosis detection resulted
in improved specificity with a slight decline in sensitivity. The combined evaluation resulted in a sensitivity of 83%
and a specificity of 73%.
Our results show that, at the expense of only a mild sensitivity decrease, a combination of clinical stenosis evaluation
and automatic LVM analysis in CCTA led to substantial increase of the specificity.
Adding deep learning analysis of LVM to stenosis assessment holds the potential to substantially increase specificity
of CCTA and to decrease number of patients unnecessarily referred to invasive FFR.
|2.||N.Lessmann, B. van Ginneken, P.A. de Jong, W.B. Veldhuis, M.A. Viergever, I. Isgum. Deep learning analysis for automatic calcium scoring in routine chest CT. Radiological Society of North America, 103rd Annual Meeting, 2017 ( Abstract )
Coronary artery calcium (CAC) is a robust predictor of cardiovascular events (CVE) in asymptomatic individuals. Several guidelines recommend reporting of CAC scores in ungated chest CT exams. In addition, chest CT can be used to quantify thoracic aorta calcification (TAC) and cardiac valve calcification (CVC), which may further improve prediction of CVE. This study evaluates the performance of an automatic method for scoring of CAC, TAC and CVC on routine chest CT exams.
METHOD AND MATERIALS
The study includes 290 retrospectively collected chest CTs (16/64/256 slice scanners, 0.9/1.0mm slice thickness, 0.7mm increment, 100/120kvP, 60mAs, ungated non-contrast). For calcium scoring, the scans were resampled to 3mm thick slices with 1.5mm increment. Calcifications were manually identified and labeled to define a reference standard. A deep-learning-based method employing two convolutional neural networks was used to identify and label calcifications automatically. This automatic method was trained on 1012 low-dose chest CTs from the National Lung Screening Trial. Calcifications were quantified using volume and Agatston scores. Correlation of automatic and reference scores was assessed using two-way-mixed ICC. Additionally, patients were assigned to a cardiovascular risk category based on their total CAC Agatston score (0, 1-100, 101-1000, >1000). Risk category assignment was evaluated using proportion of agreement and linearly weighted к.
16 scans (5.5%) were excluded due to insufficient image quality for manual scoring. 38.7% of the remaining patients had no CAC, 18.6% had a score 1-100, 16.4% a score 101-1000, and 26.3% a score > 1000. There was excellent correlation between automatic and manual volume scores for CAC (ICC=0.925, 95% CI: 0.904-0.941), TAC (ICC=0.991, 95% CI: 0.987-0.994), aortic valve calcifications (ICC=0.791, 95% CI: 0.734-0.836) and mitral valve calcifications (ICC=0.935, 95% CI: 0.918-0.949). Automatic and manual risk categorization agreed in 81.8% and differed by one category in 15.0% of the patients with excellent reliability (к=0.82).
Fully automatic scoring of coronary, aortic and cardiac valve calcifications highly correlates with manual scoring, even in ungated routine chest CT.
|3.||B.D. de Vos, N. Lessmann, P.A. de Jong, M.A. Viergever, I. Isgum. Direct coronary artery calcium scoring in low-dose chest CT using deep learning analysis. Radiological Society of North America, 103rd Annual Meeting, 2017 ( Abstract )
Coronary artery calcium (CAC) score determined in screening with low-dose chest CT is a strong and independent predictor of cardiovascular events (CVE). However, manual CAC scoring in these images is cumbersome. Existing automatic methods detect CAC lesions and thereafter quantify them. However, precise localization of lesions may not be needed to facilitate identification of subjects at risk of CVE. Hence, we have developed a deep learning system for fully automatic, real-time and direct calcium scoring circumventing the need for intermediate detection of CAC lesions.
METHOD AND MATERIALS
The study included a set of 1,546 baseline CT scans from the National Lung Screening Trial. Three experts defined the reference standard by manually identifying CAC lesions that were subsequently quantified using the Agatston score. The designed convolutional neural network analyzed axial slices and predicted the corresponding Agatston score. Per-subject Agatston scores were determined as the sum of per-slice scores. Each subject was assigned to one of five cardiovascular risk categories (Agatston score: 0, 1-10, 10-100, 100-400, >400). The system was trained with 75% of the scans and tested with the remaining 25%. Correlation between manual and automatic CAC scores was determined using the intra class correlation coefficient (ICC). Agreement of CVD risk categorization was evaluated using accuracy and Cohen’s linearly weighted κ.
In the 386 test subjects, the median (Q1-Q3) reference Agatston score was 54 (1-321). By the reference, 95, 37, 86, 94 and 75 subjects were assigned to 0, 1-10, 10-100, 100-400, >400 risk categories, respectively. The ICC between the automatic and reference scores was 0.95. The method assigned 85% of subjects to the correct risk category with a κ of 0.90. The score was determined in <2 seconds per CT.
Unlike previous automatic CAC scoring methods, the proposed method allows for quantification of coronary calcium burden without the need for intermediate identification or segmentation of separate CAC lesions. The system is robust and performs analysis in real-time.
The proposed method may allow real-time identification of subjects at risk of a CVE undergoing CT-based lung cancer screening without the need for intermediate segmentation of coronary calcifications.
|4.||M. Zreik, N. Lessmann, R. van Hamersvelt, J. Wolterink, M. Voskuil, M.A Viergever, T. Leiner, I. Isgum
. Deep learning analysis of the left ventricular myocardium in cardiac CT images enables detection of functionally significant coronary artery stenosis regardless of coronary anatomy. Radiological Society of North America, 103rd Annual Meeting , 2017 ( Abstract )
Fractional flow reserve (FFR), performed during invasive coronary angiography (ICA), is the current reference standard to determine the functional significance of a coronary stenosis. Coronary Computed Tomography Angiography (CCTA) derived virtual FFR is a promising but time and computationally expensive non-invasive alternative that can reduce the number of unnecessary ICA procedures by modeling coronary artery flow dynamics. We propose a method for fully automatic identification of patients with significant coronary artery stenosis based on deep learning analysis of only the left ventricle (LV) myocardium in CCTA.
METHOD AND MATERIALS
The study included resting CCTA scans (Philips Brilliance iCT, 120kVp, 210-300mAs) of 166 consecutive patients (59.2 ± 9.5 years, 128 males) who underwent invasive FFR (0.79 ± 0.10). FFR provided the reference for presence of a functionally significant stenosis (cut-off 0.78) . Automatic analysis first segmented the LV myocardium using a multiscale convolutional neural network (CNN). Next, the segmented myocardium was represented with a number of encodings generated by a convolutional auto-encoder (CAE). To detect local ischemic changes, the LV myocardium was divided into a number of spatially connected clusters. Per-cluster statistics of the encodings were subsequently used by a support vector machine classifier to identify patients with functionally significant stenosis. CCTA scans of 20 patients were used to train the CNN, and an additional 20 scans were used to train the CAE. Accuracy of patient classification was evaluated using the remaining 126 CCTA scans in 50 ten-fold cross-validation experiments. In each experiment, patients were randomly assigned to training and test sets.
Classification of patients resulted in an area under the receiver operating characteristic curve of 0.74 ± 0.02. At sensitivity levels 0.60, 0.70 and 0.80, the corresponding specificity was 0.77, 0.71 and 0.59, respectively.
The results demonstrate that fully automatic analysis of only the LV myocardium in resting CCTA scans, without assessment of the anatomy of the coronary arteries, can be used to identify patients with functionally significant coronary artery stenosis.
Deep learning analysis of the LV myocardium could increase the specificity of the clinically used visual stenosis assessment in CCTA and reduce the number of patients undergoing unnecessary ICA.
|5.||F. Mohamed Hoesein, E. Pompe, D.A. Lynch, N. Lessmann, J.W.J. Lammers, I. Isgum, P.A. de Jong. Computed tomographic findings are associated with respiratory mortality in the National Lung Screening Trial. Radiological Society of North America, 102nd Annual Meeting, 2016 ( Abstract )
Almost 10% of all deaths in the computed tomography (CT) arm of the National Lung Cancer Screening Trial (NLST) were due to respiratory illnesses other than lung cancer. We evaluated the importance of lung abnormalities on screening CT for survival in NLST participants.
METHOD AND MATERIALS
Subjects were derived from the CT-arm of the NLST that died of a respiratory illness other than lung cancer, as defined on the death certificate, matched with an equal number of control subjects, based on age, sex, pack-years, and smoking status. A chest radiologist and senior radiology resident independently and blindly scored baseline CTs for the presence of emphysema, airway wall thickening, or fibrotic lung disease. Associations between CT abnormalities and death was evaluated with a logistic regression model.
172 died from a respiratory cause other than lung cancer. Radiologic diseases were significantly associated with higher mortality; severe emphysema OR (95%CI) 9.7 (4.6–20.4), airway wall disease OR (95%CI) 2.3 (1.3–3.9) or fibrotic lung disease OR (95%CI) 39.1 (5.1–289.6). 81 subjects were evaluated by the EVP and confirmed the diagnosis in 55 subjects. In this group, the presence of severe emphysema was significantly associated with mortality (OR=17.2, p<0.001), as well as airway remodeling (OR=3.2, p=0.01). In the 26 non-confirmed subjects no significant difference in CT lung abnormalities between participants who were alive and participants who died was found.
CT-diagnosis of fibrosis, emphysema, airway remodeling are important for survival. By screening CT-scans for these abnormalities, next to lung cancer, deaths related to respiratory causes other than lung cancer may be preventable.
|6.||N. Lessmann, I. Isgum, S. Lam, J. Mayo, P.A. de Jong, M.A. Viergever, B. van Ginneken. Automatic coronary calcium scoring and cardiovascular risk estimation in the Pan-Canadian lung cancer screening trial. Radiological Society of North America, 101th Annual Meeting, 2015 ( Abstract )
Coronary artery calcium (CAC) scores determined in low-dose ungated chest CT as acquired for lung cancer screening are a strong and independent predictor of cardiovascular events (CVE). Automatic CAC scoring can complement lung cancer screening by identifying subjects at risk of a CVE. We investigated agreement and reliability of an automatic CAC scoring method previously developed for CAC scoring in the Dutch-Belgian lung cancer screening trial (NELSON) in the Pan-Canadian Early Detection of Lung Cancer Study (PanCan).
METHOD AND MATERIALS
Our study included 149 low-dose chest CT scans from the PanCan (16x or 64x 1.0 or 1.25 mm, 120 kvP, 40-50 mAs, no IV contrast, no ECG synchronization). Prior to scoring, the scans were reconstructed to 3.1 mm slice thickness at 1.4 mm increment. In each scan, the reference standard was set by manual annotation of CAC by one observer. Only voxels with intensities above 130 HU and lesions with a minimum volume of 1.5 mm³ were considered. Subsequently, automatic CAC scoring was performed using a supervised pattern recognition method previously developed for CAC scoring in the NELSON trial. The algorithm was trained with 100 NELSON scans. Volume and Agatston scores were computed for manual and automatic scores. Subjects were assigned to a cardiovascular risk category based on the Agatston score (0-10, 11-100, 101-400, >400). Agreement was determined as proportion of agreement in risk category assignment. Reliability was determined using linearly weighted к for risk category assignment and two-way-mixed intraclass correlation coefficient (ICC) for volume scores.
Three (2.0%) scans were excluded due to metal artifacts. In the remaining scans, the reference median CAC volume was 52.3 mm3 (P25-P75: 0-287.3 mm3). 83.6% of these scans were automatically assigned to the correct risk category. Reliability of the automatic scoring was very good for both risk category assignment (к=0.83) and volume scores (ICC=0.80).
Automatic coronary calcium scoring in lung cancer screening CT scans is feasible. To achieve good agreement with manual scores representative training data was not necessary.