Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Segmentation of Arterial Walls in Intravascular Ultrasound Cross-Sectional Images Using Extremal Region Selection

Segmentation of Arterial Walls in Intravascular Ultrasound Cross-Sectional Images Using Extremal... Intravascular Ultrasound (IVUS) is an intra-operative imaging modality that facilitates observing and appraising the vessel wall structure of the human coronary arteries. Segmentation of arterial wall boundaries from the IVUS images is not only crucial for quantitative analysis of the vessel walls and plaque characteristics, but is also necessary for generating 3D reconstructed models of the artery. The aim of this study is twofold. Firstly, we investigate the feasibility of using a recently proposed region detector, namely Extremal Region of Extremum Level (EREL) to delineate the luminal and media-adventitia borders in IVUS frames acquired by 20 MHz probes. Secondly, we propose a region selection strategy to label two ERELs as lumen and media based on the stability of their textural information. We extensively evaluated our selection strategy on the test set of a standard publicly available dataset containing 326 IVUS B-mode images. We showed that in the best case, the average Hausdor Distances (HD) between the extracted ERELs and the actual lumen and media were 0:22 mm and 0:45 mm, respectively. The results of our experiments revealed that our selection strategy was able to segment the lumen with  0:3 mm HD to the gold standard even though the images contained major artifacts such as bifurcations, shadows, and side branches. Moreover, when there was no artifact, our proposed method was able to delineate media-adventitia boundaries with 0:31 mm HD to the gold standard. Furthermore, our proposed segmentation method runs in time that is linear in the number of pixels in each frame. Based on the results of this work, by using a 20 MHz IVUS probe with controlled pullback, not only can we now analyze the internal structure of human arteries more accurately, but also segment each frame during the pullback procedure because of the low run time of our proposed segmentation method. Keywords: Intravascular, Ultrasound, IVUS, Segmentation, Extremal Regions, Extremum Level, EREL 1. Introduction nose and treat cardiovascular diseases such as atheroscle- rosis (e.g., thin-cap broatheroma) that causes a heart attack and a brain stroke [1]. Aside from this, the IVUS Catheter-based Intravascular Ultrasound (IVUS) has technique can be helpful in visualizing some internal struc- captured considerable attention in the last two decades. tures of the human coronary such as the lumen, and thick- This worldwide attention is mostly due to the ability of the ness and distribution of the plaques [2]. Therefore, IVUS imaging method to picture the inside of the human coro- is regularly used to locate the atherosclerosis lesions in nary arteries and, hence, provide an opportunity to diag- Corresponding author Email addresses: faraji@ualberta.ca (Mehdi Faraji), locheng@ualberta.ca (Irene Cheng), iris_naudin@yahoo.fr (Iris Naudin), basu@ualberta.ca (Anup Basu ) Preprint submitted to Ultrasonics June 12, 2018 arXiv:1806.03695v1 [cs.CV] 10 Jun 2018 the coronary arteries to study the lumen and plaque di- have been modeled within a shape space in [5]. The lumen mensions, and to guide intervention and stent deployment segmentation is then performed by maximizing a nonpara- [3]. metric probability density energy. Also, the edge informa- tion has been used to segment the media-adventitia. A A typical IVUS imaging system consists of four parts: physics-based model of the IVUS signal scattered by the catheter, transducer, pullback device, and scanning con- structure of the vessel has been used in [6] to estimate the sole. The catheter is composed of a 150 cm long guidewire di erential backscattering cross-sections from the IVUS and a tip of 1.2-1.5 mm in size. It is usually inserted in the RF signal. The segmentation curve is obtained after train- femoral artery and proceeds toward the coronary arteries. ing a Support Vector Machine (SVM) model using the The catheter is responsible for carrying the ultrasound annotated data. Deformable models have been used in transducer, or other necessary devices, such as in atable [7, 8, 9] to detect the border of lumen/intima and me- balloons and stents [2]. The transducer is a miniatur- dia/adventitia. In [7] anisotropic di usion followed by an ized ultrasound probe that emits ultrasound pulses and edge detector are used to create an initial segmentation listens for the backscattered signal. After the catheter has which is then corrected using both geometric and para- reached the distal end of the coronary, it needs to be man- metric deformable models. In [8] an improved version of ually or automatically pulled back. The speed of the pull- gradient vector ow (iGVF) has been proposed which in- back varies between 0.5-1 mm/s [2]. The scanning console cludes a balloon force in the snake model that lets the is essentially a computer used to post-process the acquired contour pass over leaks and bifurcations. A probabilis- signals (using ampli cation, ltering, etc.) to provide a tic method that formulates the deformation of a lumen user-friendly environment for the surgeon to control the contour curve and can be minimized has been proposed device. in [9]. However, every rst frame of the sequence needs Segmentation of the acquired IVUS images is among user interactions to manually segment the lumen and me- the most challenging tasks in medical image analysis. In dia. Following this, a SVM is trained over the annotated particular, delineating the interior (lumen) and exterior data to compute the probability that each pixel is blood (media) vessel walls is problematic due to the presence or non-blood. A fourfold algorithm based on a determinis- of various artifacts such as motion of the catheter after a tic statistical strategy for segmenting the media has been heart contraction, guide wire e ects, bifurcation and side- proposed in [10]. Their method consists of preprocessing, branches or similar echogenicity between the vessel wall initial contour detection, active contour segmentation, and and some plaques. In some cases even the di erence in contour re nement. First, a sparse binary image is con- transducer frequencies a ect the segmentation results [2]. structed using the local appearance model and the initial The intrinsic diculty of IVUS segmentation has at- contour is elicited. To achieve this, a feature vector is built tracted many researchers to study and develop solutions for each pixel. It includes gray-level values of the pixel's using di erent methodologies, such as intensity-based, neighbours, the average intensity of neighbours and the statistics and probability-based, active contour and graph gray-level values of the pixel's neighbours in a contour- search-based approaches. In addition, several methods enhanced version of the image[10]. The K-SVD method is have been proposed to segment either the lumen or the utilized to classify the extracted feature vector. The initial media or both. A great number of approaches in the liter- contour is then re ned. Next, an active contour model is ature have utilized the 2D information provided as cross- used to delineate the media border in polar coordinates. sectional frames to segment the lumen and media. These The detected contours are then re ned using the informa- 2D cross-sectional gray-scale images are formed after dig- tion provided by identifying the calci cation and shadow itization of the backscattered RF signals and are called regions. Arti cial Neural Networks have been employed IVUS B-mode frames. To the best of our knowledge, re- in [11] to represent the spatial and neighbouring features cent approaches have mostly worked on the B-mode frames of the IVUS image data. As a result, two di erent vas- which will be reviewed in the following paragraph. For cular structures for lumen and media are extracted and more in-depth reviews of the methods published before optimized using two ANNs. The borders obtained are 2013, please refer to [2, 4]. then re ned and smoothed by an active contour model. The lumen and media-adventitia border variations 2 In [12] the lumen is segmented by a combination of image perts were not aware of other expert's manual annotations gradient and fuzzy connectedness model and the media- and two of them repeated the annotations after about one adventitia border is extracted by a fast marching model. week from their rst delineation [4]. A sequential forward selection process using SVMs and The test set contains several types of common artifacts. PR curve has been employed to conduct an in-depth anal- Speci cally, it includes 44 images containing bifurcation, ysis of several image features in [13]. It has been shown 93 images with a side vessel artifact, and 96 images that that the median ltered image and Haralick's texture fea- have been contaminated by a shadow artifact (some im- tures [14] provide stronger discrimination capabilities for ages contain more than one artifact). There are also 143 arterial structures. A limitation of their analysis is that it images that do not contain any serious artifacts except for only works for artifact-free IVUS sequences. plaque. As we can see, most approaches have employed either a type of energy minimization method or require annotated 2.2. Proposed Method data in order to train a classi er or an ANN. However, in In this paper we present a segmentation approach for this paper we propose a straightforward approach that not 20 MHz Intravascular Ultrasound images based on a region only does not require training but also does not use any detection strategy. Particularly, we investigate whether a variational method or deformable model. We show that recently proposed novel feature extraction method called by extracting EREL features [15, 16] the problem of the Extremal Regions of Extremum Levels (EREL) [15, 16] IVUS segmentation can be relaxed to a region selection. can segment the most essential regions of interest (lumen In particular, we illustrate that it is very likely to nd and media) from the IVUS images required to establish regions similar to lumen and media among the extracted the atherosclerotic plaque area [17]. The proposed method ERELs. Therefore, we propose a selection procedure that consists of four steps. We rst remove the typical artifacts eciently chooses ERELs that are most similar to lumen of IVUS frames, such as ring-down e ects and calibration and media. squares. Then ERELs are extracted and the obtained re- The rest of this paper is organized as follows. In Sec- gions are ltered based on their types. Next, we perform a tion 2 we present our proposed method and describe the region selection procedure to specify two regions as lumen sequence of Intravascular Ultrasound images that we use and media. Finally, the contour of the two selected regions throughout this paper. Section 3 illustrates the segmen- is traced and smoothed by an ellipse tting algorithm. tation results of our proposed method. In Section 4 we discuss the advantages and weaknesses of the proposed method. Finally, the concluding remarks are given in Sec- 2.3. Preprocessing tion 5. Most of the IVUS images have been contaminated by speckle noise [4]. Speckle is a multiplicative noise that 2. Materials and Method imposes diculties in processing the Ultrasound images [18]. Therefore, to decrease the sensibility of our method 2.1. Materials to speckle noise, we rst use a non-linear median operator Our proposed method has been evaluated on the test to lter the IVUS images. set of publicly available dataset consisting of 326 in-vivo One of the main identi able artifacts in IVUS images pullbacks of the human coronary artery frames that were is the ring-down e ects of the catheter that need to be acquired by the Si5 (Volcano Corporation), equipped with eliminated from the B-mode frame or from its polar im- a 20 MHz Eagle Eye monorail catheter [4]. The dataset age. Otherwise, there is a high risk of obtaining an erro- includes a multi-frame 3D context that has between 20 neous segmentation. To remove the ring-down e ects of to 50 gated frames acquired using a full pullback at the the catheter we employ the method proposed in [5] which end-diastolic cardiac phase from 10 patients. Manual an- is a very fast and straightforward procedure. Detecting notations for IVUS images are available in the dataset. the ring-down artifact can be done by processing the whole The annotations have been provided by four clinical ex- volume since the artifact is almost available in all of the perts who work regularly with IVUS echograph. The ex- IVUS frames. Therefore, taking the minimum over all the 3 (a) (b) (c) (d) (e) (f ) (g) (h) Figure 1: Artifact removal in B-mode and polar frames. a) A 40 MHz IVUS B-mode frame. b) Computed minimum image of (a). The yellow colour demonstrates higher values and the red colour represents lower values. c) Result of the IVUS frame shown in (a) after the artifact removal. d) A longitudinal cut of the whole volume. The horizontal lines are the e ects of the artifact revealed after cutting. e) Corresponding polar frame of (a). f ) Calculated minimum image of (e). The yellow colour demonstrates higher values and the red colour represents lower values. g) Corresponding polar frame of (e) after artifact removal. h) Result of artifact removal in all frames of the volume illustrated in a longitudinal view that is cut by the same plane as the one used to cut (d). frames generates an image where there is a signi cant con- the image with all possible integer thresholds and analyze trast between the artifact and the non-artifact pixels. the results obtained based on their global criterion and their local edge information. The regions belonging to the globally distinguished levels (Extremum Levels) are then I (x; y) = min I (x; y) (1) min i i2 extracted from the union- nd tree. where  is a set of available frames in a particular IVUS Generally, two types of regions can be extracted from sequence. We can then locate the artifacts' coordinates by a gray-level image. The rst type includes regions that subtracting the artifact zones from every frame, as in [5]. evolve from brighter surfaces to darker boundaries which Figure 1(b) and Figure 1(f ) show the resulted minimum are identi ed by Q . The superscript ' ' emphasizes the image in B-mode and polar frame respectively. fact that the intensity values are decreased from the sur- Another type of artifact that we can detect using Eq.1 face of the regions towards the boundaries. The second is the calibration square artifact. These small squares have type consists of regions that evolve from darker surfaces a very bright constant intensity in all frames that remains to brighter boundaries and are denoted by Q . This type bright in the minimum image. In several longitudinal cuts of the region is consistent with the inherent characteristics of the IVUS volume the e ects of these artifacts are re- of the lumen and media visualized by the backscattered vealed as horizontal lines. Figure 1 illustrates both the 20MHz IVUS signals. Therefore, we only need to extract artifacts and the resulting image after removing them. Q regions to obtain the lumen and media because both regions evolve from darker surfaces to brighter boundaries. 2.4. Extremal Regions of Extremum Levels To use EREL, we need to set several initialization EREL[15, 16] is a region detector that employs a union- parameters, namely A , A , , . These parame- min max nd structure [19] in conjunction with the edge informa- ters de ne the functionality of the detector and can be tion to detect a series of connected pixels from the im- tuned based on the application [16, 15]. In particular, age. The edge information of the image is included in we use A , A to set the minimum and maximum min max the method by using the Maxima of Gradient Magnitude area of the extracted regions. To better separate small (MGM) points. The idea underlying EREL is to binarize regions from bigger ones, we choose a value for A min 4 (a) (b) (c) (d) Figure 2: Extracted ERELs from a 20 MHz IVUS frame belonging to dataset [4]. The initial parameters of EREL are: = 0:5, = 1, A = (R  C)=100 = 1474 and A = (R  C)=3 = 49152. a) Q regions with small area. b) Q regions with small area. c) min max Q regions with large area. d) Q regions with large area. Contour colours have been randomly assigned and are only for visualization purposes. that correlates with image dimensions. Speci cally, we Local maxima searching is the approach that we em- set A = (R C )=100 and A = (R C )=3 where R ploy to select lumen and media from the nested set of min max and C represent the number of rows and columns of the ERELs. Assume that we have a vector (denoted by V ) IVUS image, respectively. The parameter is usually in representing the evolution of the Q regions. The index [0 2:5] and represents the strength of the resulting interest of each element of the vector V corresponds to a Q re- points [16]. In this study we set = 0:5. Finally, = 1 gion. Since the extracted ERELs start from the smallest denotes the width of the moving window over the global enclosing region and end with the largest enclosing region, criterion vector [16]. The extracted EREL regions using the actual region corresponding to the lumen should be the above-mentioned values are illustrated in Figure 2. found among the regions located at the early indices of However, not all four types of the regions depicted in the vector. Likewise, a region representing the media is Figure 2 encompass lumen and media regions. In fact, we expected to be found among the regions belonging to the only need to extract large area Q regions as illustrated in end section of the vector V . Figure 2(d). Since large area Q regions contain the ac- tual lumen and media segments, detectors need not track Our aim is to construct a vector that ultimately gives Q regions and, therefore, omit unnecessary computations us the stability of the regions in terms of the length of the which eventually helps to have a faster detector. boundary, average intensity and entropy variation. As we can see in Figure 2(d), the boundaries of the Q regions 2.5. EREL Selection are not smooth and are subject to large variations. There- The goal of this section is to address the problem of fore, lumen and media regions should be selected among nding the most appropriate ERELs to be designated as those Q regions that have more stable boundary length lumen and media. By the most appropriate we mean the variations. Also, the average intensity of the Q regions closest regions to the gold standard. As can be seen in should be stable enough (i.e., they should not change much Figure 2(d), although ERELs are nested regions, it is clear over several subsequent regions). Entropy measure can be that there is at least one EREL that is very close to the used in order to create a feature vector that is sensitive true lumen and similarly there is at least one EREL that to textural information of the regions. The calculation of corresponds to the true media. Therefore, we can relax these three features are straightforward. The boundary the problem of lumen and media segmentation to only a lengths of the regions are available as an output of the selection procedure, i.e., assigning two nested ERELs to EREL algorithm and are calculated based on a bottom- lumen and media. up tracking of boundary pixels along a parametric curve 5 (i) No Artifact (ii) Plaque and Shadow Figure 3: The evolution of the Q regions and their stability criteria in presence of no artifacts vs. plaque and shadow artifact. a) The best candidate region representing the lumen. b) The best region representing the media. The neglected regions are highlighted by the yellow colour and the selected regions for lumen and media are indicated by magenta and green colour. + + C (p). Q regions. It is strictly increasing because Q regions @C (p) are nested and non-repetitive. More stable sequences of L = j jdp (2) vector V are more likely to represent lumen and media @p since these stable sequences shows that the extremal re- Additionally, the calculation of the average intensity gions are subject to saturation and the subsequent regions of ERELs can be readily done. might contain a noticeable change. The best way to nd the stable regions is to create a vector describing the stabil- i=1 i E = (3) ity score of regions. Every element of the stability vector is calculated as follows. where N represents the area of the region and Q is the in- i i = (6) tensity value of pixel i. The entropy measure of a grayscale V V i+1 i1 region [14] is denoted as follows. where i speci es a speci c element of the vector V and varies from one to the number of the detected Q regions H = p log p (4) for an IVUS image. i i i The local maxima of the stability score points to re- gions with high stability because the ratio of their cur- where p is the value of the bin i of the normalized his- rent value to the change among their two neighbours is togram of the region and captures the probability of hav- larger than the other surrounding elements. So, we select ing a pixel with a certain gray-value. K is the number of lumen and media from the detected local maxima. Specif- available bins in the normalized histogram. ically, a Q region with higher prominence value among Afterwards, for every IVUS image, we create a vector the rst two peaks is considered as lumen. If the IVUS (V ) where each element is obtained from the product of image contains no artifacts, the media will be represented the above-mentioned measures for each region. by the last detected peak. Based on our observation, the stability score of the images that contain serious artifacts V = L E H L E H ::: L E H (5) 1 1 1 2 2 2 n n n have none or a small number of peaks since the presence where n is the number of Q regions extracted by the of the artifacts interferes with the natural extraction of EREL algorithm. Q regions (see Figure 3(ii)). Therefore, when a small Vector V illustrates the variation of textural informa- number of local maxima is detected, we consider the last tion of regions through the extraction and evolution of extracted region as media. This process for an IVUS with 6 no particular artifact is illustrated in Figure 3. The lo- where  ,  and  are the second order central mo- xx xy yy cal maxima of the stability score indicates the regions for ments and are calculated as follows [21]: which the variation of the textural characteristics is more = (x x ) (11) stable than their surrounding regions. As can be seen in xx Figure 3(i)(a), the second peak is selected as a suitable re- (x;y)2Q gion for the lumen since it has a higher prominence than the rst peak. Also, the region corresponding to the last  = (y y ) (12) yy peak of which is chosen as the media has been shown in (x;y)2Q Figure 3(i)(b). = (x x )(y y ) (13) It is important to note that before searching for the xy (x;y)2Q local maxima we need to remove outliers. To nd outliers we employ the modi ed Z-score normalization suggested where A represents the area of the region and (x;  y ) spec- in [20]. i es the coordinates of the region's centroid. 0:6745(A A) After having obtained matrix M (reported by EREL), M = (7) nding pixels belonging to ellipse border of the regions is MAD straightforward. We assume that M = VV where V where A represents the area of the region i, A is the me- denotes a matrix containing the eigenvectors and  indi- dian of a vector of all region areas, and MAD denotes cates a matrix of eigenvalues. The minimum ( ) and min the Median Absolute Deviation and is calculated by the maximum ( ) eigenvalues in  are used to calculate max following equation. the length of the minor(b) and major (a) axes of the el- lipse, respectively [22]. MAD = median(jA Aj) (8) a = p (14) Therefore, the regions that have a modi ed Z-score less max than Z = 3 and greater than Z = 3 are unlikely min max to represent lumen and media and hence can be removed b = (15) from the selection process. min The orientation of the ellipse () can be recovered by cal- culating the angle between the major axis and the x axis. 2.6. Contour Extraction T T Let V = [u v] where u = [u u ] and v = [v v ] are 1 2 1 2 Since the general shape of the lumen and media regions two column vectors corresponding to the minimum and of the vessel are very similar to conic sections, we propose maximum eigenvalues, respectively. The orientation of the to represent lumen and media by ellipses. To nd the pix- ellipse is then obtained as follows. els inside and on the ellipse border, it is sucient to nd the orientation of the ellipse, the major and minor axis = tan (16) length. Generally, EREL outputs all pixels belonging to 2 each extracted region in addition to its shape description 2.7. Computational Cost parameters which are three coecients of ellipse equation The run time of the proposed method depends only on [15, 16]. Speci cally, if a region Q is described by an ellipse the complexity of EREL which is O(N ) [16], where N is with three coecients, namely c,d and e then: the total number of pixels in the image. The subsequent 2 2 operation proposed in this paper in Section 2.5 works on 8x; y 2 Q : cx + 2dxy + ey  1 (9) a constant number of regions. The maximum number of nested regions that can be extracted from the same root There is a direct relationship between the parameters of pixel is at most 256 for an 8-bit image [16]. In our experi- the tted ellipse and the second central moments [21]: ments with IVUS images, the number of candidate ERELs 1 (Q ) rooted from the center of image are even less (lower c d yy xy M = = (10) than 75). Therefore, the region properties denoted in Eq.2, d e 4(   ) xy xx yy xy xx 7 Eq.3, Eq.4 and Eq.6 and the second moment matrix are To calculate d(C ; C ) rst the minimum of all Eu- auto man calculated on a constant number of regions containing far clidean distances from each point belonging to C to auto less than N pixels. Therefore, the overall run time of the all points in C is obtained. Then, d(C ; C ) is man auto man proposed method is O(N ) in the worst case. A compari- computed by taking the maximum of all the minimum son between the actual run time of the proposed method distances. Similarly, d(C ; C ) is computed by tak- man auto and the methods reported in [4] is shown in Table 1. ing the maximum of all minimum distances from C to man C [25]. auto The Percentage of Area Di erence calculates the seg- mentation area di erence between the automatic (A ) auto 3. Results and manual (A ) segmentation and is computed as fol- man lows. In this section, we present the segmentation results jA A j auto man of our method. Also, by showing the best case results, PAD = (19) we demonstrate that regions very close to the lumen and man media exist among the extracted ERELs and a proper se- lection strategy (the proposed method) can distinctively select the lumen and media regions from the extracted 3.2. Best Case Results ERELs. Furthermore, we present extensive evaluation re- sults of our method based on three standard evaluation In order to show that the extracted EREL regions have metrics on IVUS frames containing various artifacts. the potential to represent lumen and media regions, we evaluate all of the extracted ERELs by calculating the 3.1. Evaluation Measures evaluation metrics (Section 3.1) for the contours of each To assess the segmentation obtained by our method, EREL and the manually annotated contours. Then, the we employ three evaluation metrics, namely Jaccard Mea- ERELs that correspond to the maximum JM of lumen and sure (JM), Hausdor Distance (HD), and Percentage of media are selected as the best extracted ERELs for that Area Di erence (PAD). Using these metrics that have been frame. The quantitative results in comparison with the also used in [4] to evaluate the results of 8 state-of-the-art intra-observer and inter-observer variability are reported methods on the same dataset makes it possible to draw a in Table 2. fair comparison between our method and the results re- ported in [4]. The Jaccard Measure is calculated based on the com- 3.3. EREL Selection Results parison of the automatic segmentation result and the man- ual segmentation delineated by experts. It quanti es the overlap area between the automatic and manual segmen- Qualitative evaluations are illustrated in Figure 4 and tation as computed by the following equation. show the successful segmentation results of the proposed R \ R EREL selection strategy for 20 IVUS frames. The lumen auto man JM = (17) areas are highlighted by the magenta colour while the R [ R auto man media regions are green. Also, the manually annotated Where R is the vessel region segmented by the method auto contours for both lumen and media are drawn as yellow and R represents the region that has been segmented man dashed lines. As we can see, the chosen frames contain a manually by experts. variety of lumen and media morphologies. The Hausdor Distance between the automatic (C ) auto A detailed evaluation result and comparison with 9 re- and manual (C ) curves is the greatest distance of all man cently published IVUS segmentation methods are reported points belonging to C to the closest point in C and auto man in Table 3 where the performance of the proposed EREL is de ned as follows [25]. selection strategy in the presence of various artifacts is HD = maxfd(C ; C ); d(C ; C )g (18) shown as well. man auto auto man 8 Figure 4: Lumen and media segmentation results. Segmented lumen and media have been highlighted by magenta and green colours respectively. The yellow dashed lines illustrate the gold standard that have been delineated by four clinical experts [4]. 9 Table 1: Comparison of each method's run time (required for segmenting a frame) reported in [4] and the proposed method. Category Semi/Auto 2D/3D Time per frame Hardware used Proposed Method Lumen and media AUTO 2D 0.19 s Core i7-4700HQ, 2.4 GHz Unal et al. [5] Lumen and media SEMI 2D 3.25 s Pentium 6200 2.13 GHz Wang et al.[4] Lumen SEMI 2D 1 m 40 s Xeon 2.67 GHz Destrempes et al.[4] Lumen and media SEMI 2D 8.64 a Core i7 Q740 @ 1.73 GHz Downe et al. [23] Lumen and media AUTO 3D 0.16 s Core 2, 2.4 GHz Alberti et al.[4] Lumen AUTO 3D 13 s Core 2, Duo 2.13 GHz Ciompi et al. [24] Media AUTO 2D 20 s Core i7, 2.8 GHz Mendizabal et al. [9] Lumen SEMI 2D 4.96 s Core i7, 2 GHz Exarchos et al. [4] Lumen and media AUTO 2D 0.5 s Core 2, Duo 3.33 GHz Table 2: The best case performance results of the proposed method. Measures represent the mean and standard deviation (std) evaluated on 435 frames of dataset [4]. The measures are categorized based on the presence of a speci c artifact in each frame. The evaluation measures are Jaccard Measure (JM), Hausdor Distance (HD), and Percentage of Area Di erence (PAD). Lumen Media HD JM PAD HD JM Pad EREL 0.22 (0.12) 0.91 (0.04) 0.03 (0.03) 0.50 (0.45) 0.83 (0.15) 0.13 (0.15) General Intra-obs 0.28 (0.13) 0.88 (0.05) 0.11 (0.08) 0.24 (0.12) 0.92 (0.03) 0.06 (0.04) Performance Inter-obs 0.17 (0.13) 0.93 (0.05) 0.04 (0.06) 0.14 (0.12) 0.95 (0.03) 0.03 (0.03) EREL 0.20 (0.09) 0.92 (0.03) 0.02 (0.02) 0.23 (0.17) 0.92 (0.05) 0.03 (0.05) No Artifact Intra-obs 0.28 (0.13) 0.88 (0.05) 0.11 (0.08) 0.24 (0.12) 0.92 (0.03) 0.06 (0.04) Inter-obs 0.17 (0.13) 0.93 (0.05) 0.04 (0.06) 0.14 (0.12) 0.95 (0.03) 0.03 (0.03) EREL 0.36 (0.22) 0.85 (0.07) 0.07 (0.07) 0.95 (0.49) 0.67 (0.16) 0.27 (0.17) Bifurcation Intra-obs 0.30 (0.12) 0.88 (0.04) 0.09 (0.06) 0.24 (0.09) 0.92 (0.02) 0.06 (0.03) Inter-obs 0.18 (0.21) 0.92 (0.07) 0.05 (0.09) 0.15 (0.13) 0.95 (0.04) 0.03 (0.03) EREL 0.20 (0.09) 0.90 (0.04) 0.03 (0.03) 0.62 (0.53) 0.78 (0.16) 0.18 (0.16) Side Vessels Intra-obs 0.30 (0.13) 0.88 (0.05) 0.10 (0.08) 0.24 (0.11) 0.92 (0.04) 0.06 (0.04) Inter-obs 0.20 (0.11) 0.91 (0.05) 0.06 (0.05) 0.15 (0.10) 0.95 (0.03) 0.03 (0.04) EREL 0.22 (0.11) 0.89 (0.04) 0.03 (0.03) 1.10 (0.38) 0.64 (0.13) 0.31 (0.14) Shadow Intra-obs 0.31 (0.13) 0.88 (0.05) 0.11 (0.08) 0.27 (0.15) 0.92 (0.04) 0.06 (0.05) Inter-obs 0.18 (0.14) 0.93 (0.05) 0.04 (0.06) 0.14 (0.10) 0.96 (0.03) 0.02 (0.02) 4. Discussion based on creating a vector that represents several textu- ral characteristics of the regions and search for the re- gions which have higher textural stability scores. This The present study is twofold. We rst investigate method works very well when no artifact is present in the whether the extracted ERELs [15, 16] are proper regions IVUS image or more accurately, when no major artifact to be considered as lumen and media delineations of 20 is attached to the media. In this case, the average Haus- MHz Intravascular Ultrasound frames taken from the in- dor distance for both lumen and media is less than 0:3 side of the human coronary arteries. We quantitatively mm which shows that the proposed method works very (Table 2) validate that very close candidates can be found well for images without any strong artifact. Furthermore, from the small number (almost less than 50) of extracted the general performance of our proposed method for lu- ERELs for each IVUS frame. Speci cally, the best case men segmentation is superior to other available methods results demonstrate the average of the closest extracted in terms of the average Hausdor distance, which is 0:3 EREL to the lumen and media among all 435 images of mm. Likewise, our method is able to segment the lumen the dataset [4]. As the results of the best case study sug- when the image contains shadows (HD = 0:29 mm) or gest, if we design a proper selection strategy we are able side vessel (HD = 0:24 mm) artifacts, which is by far the to nd two regions out of the extracted ERELs that are lowest distance to the gold standard. very close to the actual lumen and media structure. Secondly, we propose an ERELs selection strategy If artifacts that are usually dark (low intensity value) 10 Table 3: Performance of the proposed EREL selection strategy. Measures represent the mean and standard deviation evaluated on 435 frames of dataset B [4] and categorized based on the presence of a speci c artifact in each frame. The evaluation measures are Jaccard Measure (JM), Hausdor Distance (HD), and Percentage of Area Di erence (PAD). Lumen Media HD JM PAD HD JM Pad Proposed Method 0.30 (0.20) 0.87 (0.06) 0.08 (0.09) 0.67 (0.54) 0.77 (0.17) 0.19 (0.18) Unal et al. [5] 0.47 (0.39) 0.81 (0.12) 0.14 (0.13) 0.64 (0.48) 0.76 (0.13) 0.21 (0.16) Wang et al.[4] 0.51 (0.25) 0.83 (0.08) 0.14 (0.12) { { { Destrempes et al.[4] 0.34 (0.14) 0.88 (0.05) 0.06 (0.05) 0.31 (0.12) 0.91 (0.04) 0.05 (0.04) General Downe et al. [23] 0.47 (0.22) 0.77 (0.09) 0.15 (0.12) 0.76 (0.48) 0.74 (0.17) 0.23 (0.19) Performance Alberti et al.[4] 0.46 (0.30) 0.79 (0.08) 0.16 (0.09) { { { Ciompi et al. [24] { { { 0.57 (0.39) 0.84 (0.10) 0.12 (0.12) Mendizabal et al. [9] 0.38 (0.26) 0.84 (0.08) 0.11 (0.12) { { { Exarchos et al. [4] 0.42 (0.22) 0.81 (0.09) 0.11 (0.11) 0.60 (0.28) 0.79 (0.11) 0.19 (0.19) Proposed Method 0.29 (0.17) 0.88 (0.05) 0.08 (0.07) 0.31 (0.23) 0.89 (0.07) 0.07 (0.08) No Artifact Lo Vercio et al. [13] { 0.83 (0.05) 0.18 (0.06) { { { Proposed Method 0.53 (0.34) 0.79 (0.10) 0.15 (0.17) 1.22 (0.45) 0.57 (0.13) 0.32 (0.19) Unal et al. [5] 0.65 (0.47) 0.76 (0.14) 0.18 (0.15) 0.57 (0.49) 0.78 (0.13) 0.19 (0.15) Wang et al.[4] 0.54 (0.27) 0.81 (0.11) 0.14 (0.13) { { { Destrempes et al.[4] 0.42 (0.18) 0.85 (0.06) 0.08 (0.06) 0.32 (0.13) 0.91 (0.03) 0.06 (0.04) Bifurcation Downe et al. [23] 0.64 (0.27) 0.70 (0.11) 0.21 (0.15) 0.79 (0.53) 0.71 (0.19) 0.24 (0.21) Alberti et al.[4] 0.61 (0.43) 0.75 (0.10) 0.20 (0.10) { { { Ciompi et al. [24] { { { 0.52 (0.29) 0.85 (0.07) 0.09 (0.07) Mendizabal et al. [9] 0.53 (0.36) 0.79 (0.12) 0.17 (0.18) { { { Exarchos et al. [4] 0.47 (0.23) 0.80 (0.09) 0.10 (0.09) 0.63 (0.25) 0.78 (0.11) 0.23 (0.23) Proposed Method 0.24 (0.11) 0.87 (0.05) 0.06 (0.05) 0.74 (0.18) 0.73 (0.60) 0.21 (0.18) Unal et al. [5] 0.51 (0.39) 0.79 (0.12) 0.17 (0.14) 0.57 (0.39) 0.78 (0.11) 0.18 (0.12) Wang et al.[4] 0.59 (0.23) 0.80 (0.10) 0.16 (0.13) { { { Destrempes et al.[4] 0.36 (0.15) 0.87 (0.04) 0.07 (0.04) 0.31 (0.12) 0.91 (0.04) 0.04 (0.04) Side Vessels Downe et al. [23] 0.46 (0.19) 0.77 (0.08) 0.15 (0.11) 0.76 (0.47) 0.74 (0.16) 0.22 (0.20) Alberti et al.[4] 0.47 (0.24) 0.79 (0.07) 0.17 (0.09) { { { Ciompi et al. [24] { { { 0.53 (0.37) 0.85 (0.09) 0.10 (0.13) Mendizabal et al. [9] 0.38 (0.19) 0.84 (0.07) 0.11 (0.11) { { { Exarchos et al. [4] 0.53 (0.24) 0.77(0.09) 0.16 (0.12) 0.63 (0.31) 0.78 (0.12) 0.18 (0.16) Proposed Method 0.29 (0.20) 0.86 (0.07) 0.08 (0.09) 1.24 (0.39) 0.58 (0.13) 0.37 (0.15) Unal et al. [5] 0.57 (0.39) 0.78 (0.12) 0.17 (0.12) 0.66 (0.50) 0.77 (0.13) 0.19 (0.15) Wang et al.[4] 0.59 (0.27) 0.81 (0.10) 0.18 (0.16) { { { Destrempes et al.[4] 0.39 (0.18) 0.87 (0.05) 0.06 (0.05) 0.33 (0.14) 0.92 (0.03) 0.05 (0.04) Shadow Downe et al. [23] 0.55 (0.26) 0.76 (0.11) 0.14 (0.13) 0.77 (0.48) 0.74 (0.16) 0.22 (0.19) Alberti et al.[4] 0.53 (0.29) 0.78 (0.08) 0.18 (0.09) { { { Ciompi et al. [24] { { { 0.58 (0.36) 0.84 (0.09) 0.11 (0.11) Mendizabal et al. [9] 0.43 (0.27) 0.83 (0.09) 0.12 (0.11) { { { Exarchos et al. [4] 0.46 (0.19) 0.80 (0.10) 0.12 (0.12) 0.57 (0.28) 0.82 (0.11) 0.14 (0.17) 11 Figure 5: Several inaccurate lumen and media segmentation results in the presence of various artifacts. Segmented lumen and media have been highlighted by magenta and green colours respectively. The yellow dashed lines illustrate the gold standard that have been delineated by four clinical experts [4]. regions, join the lumen and media regions, a type of leak The success of the proposed method depends strictly will be created. This leak signi cantly increases the size on the correct extraction of ERELs. In cases when contin- and the mean intensity of the regions and generates irreg- uous and clear boundaries do not exist between extremal ular patterns that are very dicult to distinguish by the regions, EREL cannot extract correct candidate regions EREL extraction algorithm. Some of the inaccurate seg- for lumen and media. For example, in IVUS images taken mentations that are caused by the presence of the artifacts with a 40 MHz catheter probe, some lumen pixels located are shown in Figure 5. As can be seen in Figure 5, al- on the boundaries adjacent to intima do not have clear though the presence of the artifacts disrupts the detection connections to each other. A similar situation occurs for of the media regions, most of the lumen regions can still the boundaries between media and adventitia. This cre- be segmented accurately. This is also supported by the ates leakages that attach media region to the adventitia. quantitative results presented in Table 3. Evidently, even So, the sequential process of EREL's enumeration con- in the presence of artifacts such as bifurcations, side ves- nects pixels from two semantically di erent regions and sels, and shadows, the lumen segmentation performance constructs a big connected component that includes both remains high, though the accuracy of the media segmen- regions. Eventually, the leakage obstructs EREL's region tation drops. In fact, even in the worst case condition of extraction capability for detecting Q regions. This leak- lumen segmentation that happens when the image con- age can also be created in IVUS 20 MHz images by various tains bifurcation artifacts, the average Hausdor distance artifacts that somehow attach the media regions to the ad- to the gold standard is only 0:53 mm which is still lower ventitia as illustrated in Figure 5. than most of the other methods. On the other hand, the Apart from the above, our proposed method o ers sev- media segmentation is more sensitive to the presence of eral bene ts. Not only is our method automatic, but works artifacts. For instance, the Hausdor metric for media also solely on the B-mode images (in contrast to some in some cases increases to 1:22 mm for bifurcation and methods available in the literature that work on the po- to 1:24 mm for shadow artifact. Accordingly, it can be lar space [5, 7, 9, 10]) and, thus, it eliminates the need concluded that the lumen segmentation is robust to the for transformation to the polar space. In addition, some common artifacts of the IVUS images. methods [26, 27] require the whole volume to be presented 12 in order to segment the arterial walls. This makes it im- at the beginning section of the texture vector is consid- possible to segment the lumen and media in online seg- ered as the lumen. Similarly, the last region with a stable mentation applications (during the in-vivo pullback pro- textural variation is labeled as media. cedures). On the other hand, our method delineates lumen We evaluated our segmentation method on the test set and media contours using only the information available of a standard publicly available IVUS dataset using three in the current frame and, hence, performs no redundant common evaluation metrics; namely, JM, HD, and PAD. processing. Furthermore, due to the low computational Extensive quantitative comparisons show the high accu- complexity of the EREL [15, 16], every frame can be seg- racy of our proposed method. In general, the average HD mented in linear time based on the number of pixels in the for our lumen segmentation results is 0:3 mm which is the frame. The average run time per frame in the proposed closest distance to the gold standard among all existing method over the test set of [4] is 0:19 seconds. methods. In addition, when no artifact was present, our method segments both lumen and media achieving average HD lower than 0:29 mm and 0:31 mm, respectively. 5. Conclusion As reported in the best case results, among the ex- tracted EREL regions, there exist regions very close to the actual structure of the arterial walls. In future work, In this study, we investigated the suitability of a re- we would like to devise a better selection strategy that cently proposed region detector called Extremal Regions chooses ERELs more accurately. of Extremum Levels for detecting the lumen and media Since IVUS is completely radiation free, unlike X-ray regions. The results of our experiments reveal that among and CT, in the future we would also like to investigate the ERELs extracted from a single IVUS frame there exist the feasibility of combining the segmentation results from regions with very low di erence from the manually anno- individual slices to obtain 3D views of collateral arteries. tated data. Therefore, we presented a region selection ap- proach to automatically segment the arterial walls from Intravascular Ultrasound images taken with a 20 MHz 6. Acknowledgment catheter probe. We embedded the mean intensity value, the entropy, and the boundary length of the regions into a We gratefully acknowledge the assistance of Prof. Si- single feature vector representing the texture information mone Balocco at the University of Barcelona for providing of the region. Then, our selection method looks for regions the labeling information on the existing artifacts in the with higher stability score. The region that is most stable dataset. References References [1] J. Frosteg ard, Sle, atherosclerosis and cardiovascular disease, Journal of internal medicine 257 (6) (2005) 485{495. [2] A. Katouzian, E. D. Angelini, S. G. Carlier, J. S. Suri, N. Navab, A. F. Laine, A state-of-the-art review on seg- mentation algorithms in intravascular ultrasound (IVUS) images, IEEE Transactions on Information Technology in Biomedicine 16 (5) (2012) 823{834. [3] T. Ma, B. Zhou, T. K. Hsiai, K. K. Shung, A review of intravascular ultrasound-based multimodal intravascular imaging: the synergistic approach to characterizing vulnerable plaques, Ultrasonic imaging 38 (5) (2016) 314{331. [4] S. Balocco, C. Gatta, F. Ciompi, A. Wahle, P. Radeva, S. Carlier, G. Unal, E. Sanidas, J. Mauri, X. Carillo, et al., Standardized evaluation methodology and reference database for evaluating ivus image segmentation, Computerized Medical Imaging and Graphics 38 (2) (2014) 70{90. 13 [5] G. Unal, S. Bucher, S. Carlier, G. Slabaugh, T. Fang, K. Tanaka, Shape-driven segmentation of the arterial wall in intravascular ultrasound images, IEEE Transactions on Information Technology in Biomedicine 12 (3) (2008) 335{347. [6] G. Mendizabal-Ruiz, I. A. Kakadiaris, A physics-based intravascular ultrasound image reconstruction method for lumen segmentation, Computers in biology and medicine 75 (2016) 19{29. [7] A. Taki, Z. Naja , A. Roodaki, S. K. Setarehdan, R. A. Zoroo , A. Konig, N. Navab, Automatic segmentation of calci ed plaques and vessel borders in ivus images, International Journal of Computer Assisted Radiology and Surgery 3 (3-4) (2008) 347{354. [8] X. Zhu, P. Zhang, J. Shao, Y. Cheng, Y. Zhang, J. Bai, A snake-based method for segmentation of intravascular ultrasound images and its in vivo validation, Ultrasonics 51 (2) (2011) 181{189. [9] E. G. Mendizabal-Ruiz, M. Rivera, I. A. Kakadiaris, Segmentation of the luminal border in intravascular ultra- sound b-mode images using a probabilistic approach, Medical image analysis 17 (6) (2013) 649{670. [10] F. S. Zakeri, S. K. Setarehdan, S. Norouzi, Automatic media-adventitia ivus image segmentation based on sparse representation framework and dynamic directional active contour model, Computers in Biology and Medicine. [11] S. Su, Z. Hu, Q. Lin, W. K. Hau, Z. Gao, H. Zhang, An arti cial neural network method for lumen and media- adventitia border detection in IVUS, Computerized Medical Imaging and Graphics 57 (2017) 29{39. [12] J. Yan, D. Lv, Y. Cui, A novel segmentation approach for intravascular ultrasound images, Journal of Medical and Biological Engineering 37 (3) (2017) 386{394. [13] L. Lo Vercio, J. I. Orlando, M. del Fresno, I. Larrabide, Assessment of image features for vessel wall segmentation in intravascular ultrasound images, International journal of computer assisted radiology and surgery 11 (8) (2016) 1397{1407. [14] R. M. Haralick, Statistical and structural approaches to texture, Proceedings of the IEEE 67 (5) (1979) 786{804. [15] M. Faraji, J. Shanbehzadeh, K. Nasrollahi, T. B. Moeslund, Erel: Extremal regions of extremum levels, in: Image Processing (ICIP), 2015 IEEE International Conference on, IEEE, 2015, pp. 681{685. [16] M. Faraji, J. Shanbehzadeh, K. Nasrollahi, T. Moeslund, Extremal regions detection guided by maxima of gradient magnitude, Image Processing, IEEE Transactions on. [17] F. Destrempes, M.-H. R. Cardinal, L. Allard, J.-C. Tardif, G. Cloutier, Segmentation method of intravascular ultrasound images of human coronary arteries, Computerized Medical Imaging and Graphics 38 (2) (2014) 91{103. [18] C. P. Loizou, C. S. Pattichis, Despeckle ltering algorithms and software for ultrasound imaging, Synthesis lectures on algorithms and software in engineering 1 (1) (2008) 1{166. [19] R. Sedgewick, K. Wayne, Algorithms, 4th Edition., Addison-Wesley, 2011. [20] B. Iglewicz, D. C. Hoaglin, How to detect and handle outliers, Vol. 16, Asq Press, 1993. [21] L. Shapiro, G. C. Stockman, Computer vision. 2001, ed: Prentice Hall. [22] R. Szeliski, Computer vision: algorithms and applications, Springer Science & Business Media, 2010. 14 [23] R. Downe, A. Wahle, T. Kovarnik, H. Skalicka, J. Lopez, J. Horak, M. Sonka, Segmentation of intravascular ultrasound images using graph search and a novel cost function, in: Proc. 2nd MICCAI workshop on computer vision for intravascular and intracardiac imaging, Citeseer, 2008, pp. 71{79. [24] F. Ciompi, O. Pujol, C. Gatta, M. Alberti, S. Balocco, X. Carrillo, J. Mauri-Ferre, P. Radeva, Holimab: A holistic approach for media{adventitia border detection in intravascular ultrasound, Medical image analysis 16 (6) (2012) 1085{1100. [25] F. Molinari, K. M. Meiburger, G. Zeng, A. Nicolaides, J. S. Suri, Caudles-ef: carotid automated ultrasound double line extraction system using edge ow, Journal of digital imaging 24 (6) (2011) 1059{1077. [26] Z. Sun, C. Liu, A parallel method for segmenting intravascular ultrasound image sequence, in: Mechanical and Electronics Engineering III, Vol. 130 of Applied Mechanics and Materials, Trans Tech Publications, 2012, pp. 2051{2055. doi:10.4028/www.scientific.net/AMM.130-134.2051. [27] M.-H. Cardinal, J. Meunier, G. Soulez, R. L. Maurice, E. Therasse, G. Cloutier, Intravascular ultrasound image segmentation: a three-dimensional fast-marching method based on gray level distributions, IEEE transactions on medical imaging 25 (5) (2006) 590{601. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Electrical Engineering and Systems Science arXiv (Cornell University)

Segmentation of Arterial Walls in Intravascular Ultrasound Cross-Sectional Images Using Extremal Region Selection

Loading next page...
 
/lp/arxiv-cornell-university/segmentation-of-arterial-walls-in-intravascular-ultrasound-cross-dbnwU1EUnq

References (26)

ISSN
0041-624X
eISSN
ARCH-3348
DOI
10.1016/j.ultras.2017.11.020
Publisher site
See Article on Publisher Site

Abstract

Intravascular Ultrasound (IVUS) is an intra-operative imaging modality that facilitates observing and appraising the vessel wall structure of the human coronary arteries. Segmentation of arterial wall boundaries from the IVUS images is not only crucial for quantitative analysis of the vessel walls and plaque characteristics, but is also necessary for generating 3D reconstructed models of the artery. The aim of this study is twofold. Firstly, we investigate the feasibility of using a recently proposed region detector, namely Extremal Region of Extremum Level (EREL) to delineate the luminal and media-adventitia borders in IVUS frames acquired by 20 MHz probes. Secondly, we propose a region selection strategy to label two ERELs as lumen and media based on the stability of their textural information. We extensively evaluated our selection strategy on the test set of a standard publicly available dataset containing 326 IVUS B-mode images. We showed that in the best case, the average Hausdor Distances (HD) between the extracted ERELs and the actual lumen and media were 0:22 mm and 0:45 mm, respectively. The results of our experiments revealed that our selection strategy was able to segment the lumen with  0:3 mm HD to the gold standard even though the images contained major artifacts such as bifurcations, shadows, and side branches. Moreover, when there was no artifact, our proposed method was able to delineate media-adventitia boundaries with 0:31 mm HD to the gold standard. Furthermore, our proposed segmentation method runs in time that is linear in the number of pixels in each frame. Based on the results of this work, by using a 20 MHz IVUS probe with controlled pullback, not only can we now analyze the internal structure of human arteries more accurately, but also segment each frame during the pullback procedure because of the low run time of our proposed segmentation method. Keywords: Intravascular, Ultrasound, IVUS, Segmentation, Extremal Regions, Extremum Level, EREL 1. Introduction nose and treat cardiovascular diseases such as atheroscle- rosis (e.g., thin-cap broatheroma) that causes a heart attack and a brain stroke [1]. Aside from this, the IVUS Catheter-based Intravascular Ultrasound (IVUS) has technique can be helpful in visualizing some internal struc- captured considerable attention in the last two decades. tures of the human coronary such as the lumen, and thick- This worldwide attention is mostly due to the ability of the ness and distribution of the plaques [2]. Therefore, IVUS imaging method to picture the inside of the human coro- is regularly used to locate the atherosclerosis lesions in nary arteries and, hence, provide an opportunity to diag- Corresponding author Email addresses: faraji@ualberta.ca (Mehdi Faraji), locheng@ualberta.ca (Irene Cheng), iris_naudin@yahoo.fr (Iris Naudin), basu@ualberta.ca (Anup Basu ) Preprint submitted to Ultrasonics June 12, 2018 arXiv:1806.03695v1 [cs.CV] 10 Jun 2018 the coronary arteries to study the lumen and plaque di- have been modeled within a shape space in [5]. The lumen mensions, and to guide intervention and stent deployment segmentation is then performed by maximizing a nonpara- [3]. metric probability density energy. Also, the edge informa- tion has been used to segment the media-adventitia. A A typical IVUS imaging system consists of four parts: physics-based model of the IVUS signal scattered by the catheter, transducer, pullback device, and scanning con- structure of the vessel has been used in [6] to estimate the sole. The catheter is composed of a 150 cm long guidewire di erential backscattering cross-sections from the IVUS and a tip of 1.2-1.5 mm in size. It is usually inserted in the RF signal. The segmentation curve is obtained after train- femoral artery and proceeds toward the coronary arteries. ing a Support Vector Machine (SVM) model using the The catheter is responsible for carrying the ultrasound annotated data. Deformable models have been used in transducer, or other necessary devices, such as in atable [7, 8, 9] to detect the border of lumen/intima and me- balloons and stents [2]. The transducer is a miniatur- dia/adventitia. In [7] anisotropic di usion followed by an ized ultrasound probe that emits ultrasound pulses and edge detector are used to create an initial segmentation listens for the backscattered signal. After the catheter has which is then corrected using both geometric and para- reached the distal end of the coronary, it needs to be man- metric deformable models. In [8] an improved version of ually or automatically pulled back. The speed of the pull- gradient vector ow (iGVF) has been proposed which in- back varies between 0.5-1 mm/s [2]. The scanning console cludes a balloon force in the snake model that lets the is essentially a computer used to post-process the acquired contour pass over leaks and bifurcations. A probabilis- signals (using ampli cation, ltering, etc.) to provide a tic method that formulates the deformation of a lumen user-friendly environment for the surgeon to control the contour curve and can be minimized has been proposed device. in [9]. However, every rst frame of the sequence needs Segmentation of the acquired IVUS images is among user interactions to manually segment the lumen and me- the most challenging tasks in medical image analysis. In dia. Following this, a SVM is trained over the annotated particular, delineating the interior (lumen) and exterior data to compute the probability that each pixel is blood (media) vessel walls is problematic due to the presence or non-blood. A fourfold algorithm based on a determinis- of various artifacts such as motion of the catheter after a tic statistical strategy for segmenting the media has been heart contraction, guide wire e ects, bifurcation and side- proposed in [10]. Their method consists of preprocessing, branches or similar echogenicity between the vessel wall initial contour detection, active contour segmentation, and and some plaques. In some cases even the di erence in contour re nement. First, a sparse binary image is con- transducer frequencies a ect the segmentation results [2]. structed using the local appearance model and the initial The intrinsic diculty of IVUS segmentation has at- contour is elicited. To achieve this, a feature vector is built tracted many researchers to study and develop solutions for each pixel. It includes gray-level values of the pixel's using di erent methodologies, such as intensity-based, neighbours, the average intensity of neighbours and the statistics and probability-based, active contour and graph gray-level values of the pixel's neighbours in a contour- search-based approaches. In addition, several methods enhanced version of the image[10]. The K-SVD method is have been proposed to segment either the lumen or the utilized to classify the extracted feature vector. The initial media or both. A great number of approaches in the liter- contour is then re ned. Next, an active contour model is ature have utilized the 2D information provided as cross- used to delineate the media border in polar coordinates. sectional frames to segment the lumen and media. These The detected contours are then re ned using the informa- 2D cross-sectional gray-scale images are formed after dig- tion provided by identifying the calci cation and shadow itization of the backscattered RF signals and are called regions. Arti cial Neural Networks have been employed IVUS B-mode frames. To the best of our knowledge, re- in [11] to represent the spatial and neighbouring features cent approaches have mostly worked on the B-mode frames of the IVUS image data. As a result, two di erent vas- which will be reviewed in the following paragraph. For cular structures for lumen and media are extracted and more in-depth reviews of the methods published before optimized using two ANNs. The borders obtained are 2013, please refer to [2, 4]. then re ned and smoothed by an active contour model. The lumen and media-adventitia border variations 2 In [12] the lumen is segmented by a combination of image perts were not aware of other expert's manual annotations gradient and fuzzy connectedness model and the media- and two of them repeated the annotations after about one adventitia border is extracted by a fast marching model. week from their rst delineation [4]. A sequential forward selection process using SVMs and The test set contains several types of common artifacts. PR curve has been employed to conduct an in-depth anal- Speci cally, it includes 44 images containing bifurcation, ysis of several image features in [13]. It has been shown 93 images with a side vessel artifact, and 96 images that that the median ltered image and Haralick's texture fea- have been contaminated by a shadow artifact (some im- tures [14] provide stronger discrimination capabilities for ages contain more than one artifact). There are also 143 arterial structures. A limitation of their analysis is that it images that do not contain any serious artifacts except for only works for artifact-free IVUS sequences. plaque. As we can see, most approaches have employed either a type of energy minimization method or require annotated 2.2. Proposed Method data in order to train a classi er or an ANN. However, in In this paper we present a segmentation approach for this paper we propose a straightforward approach that not 20 MHz Intravascular Ultrasound images based on a region only does not require training but also does not use any detection strategy. Particularly, we investigate whether a variational method or deformable model. We show that recently proposed novel feature extraction method called by extracting EREL features [15, 16] the problem of the Extremal Regions of Extremum Levels (EREL) [15, 16] IVUS segmentation can be relaxed to a region selection. can segment the most essential regions of interest (lumen In particular, we illustrate that it is very likely to nd and media) from the IVUS images required to establish regions similar to lumen and media among the extracted the atherosclerotic plaque area [17]. The proposed method ERELs. Therefore, we propose a selection procedure that consists of four steps. We rst remove the typical artifacts eciently chooses ERELs that are most similar to lumen of IVUS frames, such as ring-down e ects and calibration and media. squares. Then ERELs are extracted and the obtained re- The rest of this paper is organized as follows. In Sec- gions are ltered based on their types. Next, we perform a tion 2 we present our proposed method and describe the region selection procedure to specify two regions as lumen sequence of Intravascular Ultrasound images that we use and media. Finally, the contour of the two selected regions throughout this paper. Section 3 illustrates the segmen- is traced and smoothed by an ellipse tting algorithm. tation results of our proposed method. In Section 4 we discuss the advantages and weaknesses of the proposed method. Finally, the concluding remarks are given in Sec- 2.3. Preprocessing tion 5. Most of the IVUS images have been contaminated by speckle noise [4]. Speckle is a multiplicative noise that 2. Materials and Method imposes diculties in processing the Ultrasound images [18]. Therefore, to decrease the sensibility of our method 2.1. Materials to speckle noise, we rst use a non-linear median operator Our proposed method has been evaluated on the test to lter the IVUS images. set of publicly available dataset consisting of 326 in-vivo One of the main identi able artifacts in IVUS images pullbacks of the human coronary artery frames that were is the ring-down e ects of the catheter that need to be acquired by the Si5 (Volcano Corporation), equipped with eliminated from the B-mode frame or from its polar im- a 20 MHz Eagle Eye monorail catheter [4]. The dataset age. Otherwise, there is a high risk of obtaining an erro- includes a multi-frame 3D context that has between 20 neous segmentation. To remove the ring-down e ects of to 50 gated frames acquired using a full pullback at the the catheter we employ the method proposed in [5] which end-diastolic cardiac phase from 10 patients. Manual an- is a very fast and straightforward procedure. Detecting notations for IVUS images are available in the dataset. the ring-down artifact can be done by processing the whole The annotations have been provided by four clinical ex- volume since the artifact is almost available in all of the perts who work regularly with IVUS echograph. The ex- IVUS frames. Therefore, taking the minimum over all the 3 (a) (b) (c) (d) (e) (f ) (g) (h) Figure 1: Artifact removal in B-mode and polar frames. a) A 40 MHz IVUS B-mode frame. b) Computed minimum image of (a). The yellow colour demonstrates higher values and the red colour represents lower values. c) Result of the IVUS frame shown in (a) after the artifact removal. d) A longitudinal cut of the whole volume. The horizontal lines are the e ects of the artifact revealed after cutting. e) Corresponding polar frame of (a). f ) Calculated minimum image of (e). The yellow colour demonstrates higher values and the red colour represents lower values. g) Corresponding polar frame of (e) after artifact removal. h) Result of artifact removal in all frames of the volume illustrated in a longitudinal view that is cut by the same plane as the one used to cut (d). frames generates an image where there is a signi cant con- the image with all possible integer thresholds and analyze trast between the artifact and the non-artifact pixels. the results obtained based on their global criterion and their local edge information. The regions belonging to the globally distinguished levels (Extremum Levels) are then I (x; y) = min I (x; y) (1) min i i2 extracted from the union- nd tree. where  is a set of available frames in a particular IVUS Generally, two types of regions can be extracted from sequence. We can then locate the artifacts' coordinates by a gray-level image. The rst type includes regions that subtracting the artifact zones from every frame, as in [5]. evolve from brighter surfaces to darker boundaries which Figure 1(b) and Figure 1(f ) show the resulted minimum are identi ed by Q . The superscript ' ' emphasizes the image in B-mode and polar frame respectively. fact that the intensity values are decreased from the sur- Another type of artifact that we can detect using Eq.1 face of the regions towards the boundaries. The second is the calibration square artifact. These small squares have type consists of regions that evolve from darker surfaces a very bright constant intensity in all frames that remains to brighter boundaries and are denoted by Q . This type bright in the minimum image. In several longitudinal cuts of the region is consistent with the inherent characteristics of the IVUS volume the e ects of these artifacts are re- of the lumen and media visualized by the backscattered vealed as horizontal lines. Figure 1 illustrates both the 20MHz IVUS signals. Therefore, we only need to extract artifacts and the resulting image after removing them. Q regions to obtain the lumen and media because both regions evolve from darker surfaces to brighter boundaries. 2.4. Extremal Regions of Extremum Levels To use EREL, we need to set several initialization EREL[15, 16] is a region detector that employs a union- parameters, namely A , A , , . These parame- min max nd structure [19] in conjunction with the edge informa- ters de ne the functionality of the detector and can be tion to detect a series of connected pixels from the im- tuned based on the application [16, 15]. In particular, age. The edge information of the image is included in we use A , A to set the minimum and maximum min max the method by using the Maxima of Gradient Magnitude area of the extracted regions. To better separate small (MGM) points. The idea underlying EREL is to binarize regions from bigger ones, we choose a value for A min 4 (a) (b) (c) (d) Figure 2: Extracted ERELs from a 20 MHz IVUS frame belonging to dataset [4]. The initial parameters of EREL are: = 0:5, = 1, A = (R  C)=100 = 1474 and A = (R  C)=3 = 49152. a) Q regions with small area. b) Q regions with small area. c) min max Q regions with large area. d) Q regions with large area. Contour colours have been randomly assigned and are only for visualization purposes. that correlates with image dimensions. Speci cally, we Local maxima searching is the approach that we em- set A = (R C )=100 and A = (R C )=3 where R ploy to select lumen and media from the nested set of min max and C represent the number of rows and columns of the ERELs. Assume that we have a vector (denoted by V ) IVUS image, respectively. The parameter is usually in representing the evolution of the Q regions. The index [0 2:5] and represents the strength of the resulting interest of each element of the vector V corresponds to a Q re- points [16]. In this study we set = 0:5. Finally, = 1 gion. Since the extracted ERELs start from the smallest denotes the width of the moving window over the global enclosing region and end with the largest enclosing region, criterion vector [16]. The extracted EREL regions using the actual region corresponding to the lumen should be the above-mentioned values are illustrated in Figure 2. found among the regions located at the early indices of However, not all four types of the regions depicted in the vector. Likewise, a region representing the media is Figure 2 encompass lumen and media regions. In fact, we expected to be found among the regions belonging to the only need to extract large area Q regions as illustrated in end section of the vector V . Figure 2(d). Since large area Q regions contain the ac- tual lumen and media segments, detectors need not track Our aim is to construct a vector that ultimately gives Q regions and, therefore, omit unnecessary computations us the stability of the regions in terms of the length of the which eventually helps to have a faster detector. boundary, average intensity and entropy variation. As we can see in Figure 2(d), the boundaries of the Q regions 2.5. EREL Selection are not smooth and are subject to large variations. There- The goal of this section is to address the problem of fore, lumen and media regions should be selected among nding the most appropriate ERELs to be designated as those Q regions that have more stable boundary length lumen and media. By the most appropriate we mean the variations. Also, the average intensity of the Q regions closest regions to the gold standard. As can be seen in should be stable enough (i.e., they should not change much Figure 2(d), although ERELs are nested regions, it is clear over several subsequent regions). Entropy measure can be that there is at least one EREL that is very close to the used in order to create a feature vector that is sensitive true lumen and similarly there is at least one EREL that to textural information of the regions. The calculation of corresponds to the true media. Therefore, we can relax these three features are straightforward. The boundary the problem of lumen and media segmentation to only a lengths of the regions are available as an output of the selection procedure, i.e., assigning two nested ERELs to EREL algorithm and are calculated based on a bottom- lumen and media. up tracking of boundary pixels along a parametric curve 5 (i) No Artifact (ii) Plaque and Shadow Figure 3: The evolution of the Q regions and their stability criteria in presence of no artifacts vs. plaque and shadow artifact. a) The best candidate region representing the lumen. b) The best region representing the media. The neglected regions are highlighted by the yellow colour and the selected regions for lumen and media are indicated by magenta and green colour. + + C (p). Q regions. It is strictly increasing because Q regions @C (p) are nested and non-repetitive. More stable sequences of L = j jdp (2) vector V are more likely to represent lumen and media @p since these stable sequences shows that the extremal re- Additionally, the calculation of the average intensity gions are subject to saturation and the subsequent regions of ERELs can be readily done. might contain a noticeable change. The best way to nd the stable regions is to create a vector describing the stabil- i=1 i E = (3) ity score of regions. Every element of the stability vector is calculated as follows. where N represents the area of the region and Q is the in- i i = (6) tensity value of pixel i. The entropy measure of a grayscale V V i+1 i1 region [14] is denoted as follows. where i speci es a speci c element of the vector V and varies from one to the number of the detected Q regions H = p log p (4) for an IVUS image. i i i The local maxima of the stability score points to re- gions with high stability because the ratio of their cur- where p is the value of the bin i of the normalized his- rent value to the change among their two neighbours is togram of the region and captures the probability of hav- larger than the other surrounding elements. So, we select ing a pixel with a certain gray-value. K is the number of lumen and media from the detected local maxima. Specif- available bins in the normalized histogram. ically, a Q region with higher prominence value among Afterwards, for every IVUS image, we create a vector the rst two peaks is considered as lumen. If the IVUS (V ) where each element is obtained from the product of image contains no artifacts, the media will be represented the above-mentioned measures for each region. by the last detected peak. Based on our observation, the stability score of the images that contain serious artifacts V = L E H L E H ::: L E H (5) 1 1 1 2 2 2 n n n have none or a small number of peaks since the presence where n is the number of Q regions extracted by the of the artifacts interferes with the natural extraction of EREL algorithm. Q regions (see Figure 3(ii)). Therefore, when a small Vector V illustrates the variation of textural informa- number of local maxima is detected, we consider the last tion of regions through the extraction and evolution of extracted region as media. This process for an IVUS with 6 no particular artifact is illustrated in Figure 3. The lo- where  ,  and  are the second order central mo- xx xy yy cal maxima of the stability score indicates the regions for ments and are calculated as follows [21]: which the variation of the textural characteristics is more = (x x ) (11) stable than their surrounding regions. As can be seen in xx Figure 3(i)(a), the second peak is selected as a suitable re- (x;y)2Q gion for the lumen since it has a higher prominence than the rst peak. Also, the region corresponding to the last  = (y y ) (12) yy peak of which is chosen as the media has been shown in (x;y)2Q Figure 3(i)(b). = (x x )(y y ) (13) It is important to note that before searching for the xy (x;y)2Q local maxima we need to remove outliers. To nd outliers we employ the modi ed Z-score normalization suggested where A represents the area of the region and (x;  y ) spec- in [20]. i es the coordinates of the region's centroid. 0:6745(A A) After having obtained matrix M (reported by EREL), M = (7) nding pixels belonging to ellipse border of the regions is MAD straightforward. We assume that M = VV where V where A represents the area of the region i, A is the me- denotes a matrix containing the eigenvectors and  indi- dian of a vector of all region areas, and MAD denotes cates a matrix of eigenvalues. The minimum ( ) and min the Median Absolute Deviation and is calculated by the maximum ( ) eigenvalues in  are used to calculate max following equation. the length of the minor(b) and major (a) axes of the el- lipse, respectively [22]. MAD = median(jA Aj) (8) a = p (14) Therefore, the regions that have a modi ed Z-score less max than Z = 3 and greater than Z = 3 are unlikely min max to represent lumen and media and hence can be removed b = (15) from the selection process. min The orientation of the ellipse () can be recovered by cal- culating the angle between the major axis and the x axis. 2.6. Contour Extraction T T Let V = [u v] where u = [u u ] and v = [v v ] are 1 2 1 2 Since the general shape of the lumen and media regions two column vectors corresponding to the minimum and of the vessel are very similar to conic sections, we propose maximum eigenvalues, respectively. The orientation of the to represent lumen and media by ellipses. To nd the pix- ellipse is then obtained as follows. els inside and on the ellipse border, it is sucient to nd the orientation of the ellipse, the major and minor axis = tan (16) length. Generally, EREL outputs all pixels belonging to 2 each extracted region in addition to its shape description 2.7. Computational Cost parameters which are three coecients of ellipse equation The run time of the proposed method depends only on [15, 16]. Speci cally, if a region Q is described by an ellipse the complexity of EREL which is O(N ) [16], where N is with three coecients, namely c,d and e then: the total number of pixels in the image. The subsequent 2 2 operation proposed in this paper in Section 2.5 works on 8x; y 2 Q : cx + 2dxy + ey  1 (9) a constant number of regions. The maximum number of nested regions that can be extracted from the same root There is a direct relationship between the parameters of pixel is at most 256 for an 8-bit image [16]. In our experi- the tted ellipse and the second central moments [21]: ments with IVUS images, the number of candidate ERELs 1 (Q ) rooted from the center of image are even less (lower c d yy xy M = = (10) than 75). Therefore, the region properties denoted in Eq.2, d e 4(   ) xy xx yy xy xx 7 Eq.3, Eq.4 and Eq.6 and the second moment matrix are To calculate d(C ; C ) rst the minimum of all Eu- auto man calculated on a constant number of regions containing far clidean distances from each point belonging to C to auto less than N pixels. Therefore, the overall run time of the all points in C is obtained. Then, d(C ; C ) is man auto man proposed method is O(N ) in the worst case. A compari- computed by taking the maximum of all the minimum son between the actual run time of the proposed method distances. Similarly, d(C ; C ) is computed by tak- man auto and the methods reported in [4] is shown in Table 1. ing the maximum of all minimum distances from C to man C [25]. auto The Percentage of Area Di erence calculates the seg- mentation area di erence between the automatic (A ) auto 3. Results and manual (A ) segmentation and is computed as fol- man lows. In this section, we present the segmentation results jA A j auto man of our method. Also, by showing the best case results, PAD = (19) we demonstrate that regions very close to the lumen and man media exist among the extracted ERELs and a proper se- lection strategy (the proposed method) can distinctively select the lumen and media regions from the extracted 3.2. Best Case Results ERELs. Furthermore, we present extensive evaluation re- sults of our method based on three standard evaluation In order to show that the extracted EREL regions have metrics on IVUS frames containing various artifacts. the potential to represent lumen and media regions, we evaluate all of the extracted ERELs by calculating the 3.1. Evaluation Measures evaluation metrics (Section 3.1) for the contours of each To assess the segmentation obtained by our method, EREL and the manually annotated contours. Then, the we employ three evaluation metrics, namely Jaccard Mea- ERELs that correspond to the maximum JM of lumen and sure (JM), Hausdor Distance (HD), and Percentage of media are selected as the best extracted ERELs for that Area Di erence (PAD). Using these metrics that have been frame. The quantitative results in comparison with the also used in [4] to evaluate the results of 8 state-of-the-art intra-observer and inter-observer variability are reported methods on the same dataset makes it possible to draw a in Table 2. fair comparison between our method and the results re- ported in [4]. The Jaccard Measure is calculated based on the com- 3.3. EREL Selection Results parison of the automatic segmentation result and the man- ual segmentation delineated by experts. It quanti es the overlap area between the automatic and manual segmen- Qualitative evaluations are illustrated in Figure 4 and tation as computed by the following equation. show the successful segmentation results of the proposed R \ R EREL selection strategy for 20 IVUS frames. The lumen auto man JM = (17) areas are highlighted by the magenta colour while the R [ R auto man media regions are green. Also, the manually annotated Where R is the vessel region segmented by the method auto contours for both lumen and media are drawn as yellow and R represents the region that has been segmented man dashed lines. As we can see, the chosen frames contain a manually by experts. variety of lumen and media morphologies. The Hausdor Distance between the automatic (C ) auto A detailed evaluation result and comparison with 9 re- and manual (C ) curves is the greatest distance of all man cently published IVUS segmentation methods are reported points belonging to C to the closest point in C and auto man in Table 3 where the performance of the proposed EREL is de ned as follows [25]. selection strategy in the presence of various artifacts is HD = maxfd(C ; C ); d(C ; C )g (18) shown as well. man auto auto man 8 Figure 4: Lumen and media segmentation results. Segmented lumen and media have been highlighted by magenta and green colours respectively. The yellow dashed lines illustrate the gold standard that have been delineated by four clinical experts [4]. 9 Table 1: Comparison of each method's run time (required for segmenting a frame) reported in [4] and the proposed method. Category Semi/Auto 2D/3D Time per frame Hardware used Proposed Method Lumen and media AUTO 2D 0.19 s Core i7-4700HQ, 2.4 GHz Unal et al. [5] Lumen and media SEMI 2D 3.25 s Pentium 6200 2.13 GHz Wang et al.[4] Lumen SEMI 2D 1 m 40 s Xeon 2.67 GHz Destrempes et al.[4] Lumen and media SEMI 2D 8.64 a Core i7 Q740 @ 1.73 GHz Downe et al. [23] Lumen and media AUTO 3D 0.16 s Core 2, 2.4 GHz Alberti et al.[4] Lumen AUTO 3D 13 s Core 2, Duo 2.13 GHz Ciompi et al. [24] Media AUTO 2D 20 s Core i7, 2.8 GHz Mendizabal et al. [9] Lumen SEMI 2D 4.96 s Core i7, 2 GHz Exarchos et al. [4] Lumen and media AUTO 2D 0.5 s Core 2, Duo 3.33 GHz Table 2: The best case performance results of the proposed method. Measures represent the mean and standard deviation (std) evaluated on 435 frames of dataset [4]. The measures are categorized based on the presence of a speci c artifact in each frame. The evaluation measures are Jaccard Measure (JM), Hausdor Distance (HD), and Percentage of Area Di erence (PAD). Lumen Media HD JM PAD HD JM Pad EREL 0.22 (0.12) 0.91 (0.04) 0.03 (0.03) 0.50 (0.45) 0.83 (0.15) 0.13 (0.15) General Intra-obs 0.28 (0.13) 0.88 (0.05) 0.11 (0.08) 0.24 (0.12) 0.92 (0.03) 0.06 (0.04) Performance Inter-obs 0.17 (0.13) 0.93 (0.05) 0.04 (0.06) 0.14 (0.12) 0.95 (0.03) 0.03 (0.03) EREL 0.20 (0.09) 0.92 (0.03) 0.02 (0.02) 0.23 (0.17) 0.92 (0.05) 0.03 (0.05) No Artifact Intra-obs 0.28 (0.13) 0.88 (0.05) 0.11 (0.08) 0.24 (0.12) 0.92 (0.03) 0.06 (0.04) Inter-obs 0.17 (0.13) 0.93 (0.05) 0.04 (0.06) 0.14 (0.12) 0.95 (0.03) 0.03 (0.03) EREL 0.36 (0.22) 0.85 (0.07) 0.07 (0.07) 0.95 (0.49) 0.67 (0.16) 0.27 (0.17) Bifurcation Intra-obs 0.30 (0.12) 0.88 (0.04) 0.09 (0.06) 0.24 (0.09) 0.92 (0.02) 0.06 (0.03) Inter-obs 0.18 (0.21) 0.92 (0.07) 0.05 (0.09) 0.15 (0.13) 0.95 (0.04) 0.03 (0.03) EREL 0.20 (0.09) 0.90 (0.04) 0.03 (0.03) 0.62 (0.53) 0.78 (0.16) 0.18 (0.16) Side Vessels Intra-obs 0.30 (0.13) 0.88 (0.05) 0.10 (0.08) 0.24 (0.11) 0.92 (0.04) 0.06 (0.04) Inter-obs 0.20 (0.11) 0.91 (0.05) 0.06 (0.05) 0.15 (0.10) 0.95 (0.03) 0.03 (0.04) EREL 0.22 (0.11) 0.89 (0.04) 0.03 (0.03) 1.10 (0.38) 0.64 (0.13) 0.31 (0.14) Shadow Intra-obs 0.31 (0.13) 0.88 (0.05) 0.11 (0.08) 0.27 (0.15) 0.92 (0.04) 0.06 (0.05) Inter-obs 0.18 (0.14) 0.93 (0.05) 0.04 (0.06) 0.14 (0.10) 0.96 (0.03) 0.02 (0.02) 4. Discussion based on creating a vector that represents several textu- ral characteristics of the regions and search for the re- gions which have higher textural stability scores. This The present study is twofold. We rst investigate method works very well when no artifact is present in the whether the extracted ERELs [15, 16] are proper regions IVUS image or more accurately, when no major artifact to be considered as lumen and media delineations of 20 is attached to the media. In this case, the average Haus- MHz Intravascular Ultrasound frames taken from the in- dor distance for both lumen and media is less than 0:3 side of the human coronary arteries. We quantitatively mm which shows that the proposed method works very (Table 2) validate that very close candidates can be found well for images without any strong artifact. Furthermore, from the small number (almost less than 50) of extracted the general performance of our proposed method for lu- ERELs for each IVUS frame. Speci cally, the best case men segmentation is superior to other available methods results demonstrate the average of the closest extracted in terms of the average Hausdor distance, which is 0:3 EREL to the lumen and media among all 435 images of mm. Likewise, our method is able to segment the lumen the dataset [4]. As the results of the best case study sug- when the image contains shadows (HD = 0:29 mm) or gest, if we design a proper selection strategy we are able side vessel (HD = 0:24 mm) artifacts, which is by far the to nd two regions out of the extracted ERELs that are lowest distance to the gold standard. very close to the actual lumen and media structure. Secondly, we propose an ERELs selection strategy If artifacts that are usually dark (low intensity value) 10 Table 3: Performance of the proposed EREL selection strategy. Measures represent the mean and standard deviation evaluated on 435 frames of dataset B [4] and categorized based on the presence of a speci c artifact in each frame. The evaluation measures are Jaccard Measure (JM), Hausdor Distance (HD), and Percentage of Area Di erence (PAD). Lumen Media HD JM PAD HD JM Pad Proposed Method 0.30 (0.20) 0.87 (0.06) 0.08 (0.09) 0.67 (0.54) 0.77 (0.17) 0.19 (0.18) Unal et al. [5] 0.47 (0.39) 0.81 (0.12) 0.14 (0.13) 0.64 (0.48) 0.76 (0.13) 0.21 (0.16) Wang et al.[4] 0.51 (0.25) 0.83 (0.08) 0.14 (0.12) { { { Destrempes et al.[4] 0.34 (0.14) 0.88 (0.05) 0.06 (0.05) 0.31 (0.12) 0.91 (0.04) 0.05 (0.04) General Downe et al. [23] 0.47 (0.22) 0.77 (0.09) 0.15 (0.12) 0.76 (0.48) 0.74 (0.17) 0.23 (0.19) Performance Alberti et al.[4] 0.46 (0.30) 0.79 (0.08) 0.16 (0.09) { { { Ciompi et al. [24] { { { 0.57 (0.39) 0.84 (0.10) 0.12 (0.12) Mendizabal et al. [9] 0.38 (0.26) 0.84 (0.08) 0.11 (0.12) { { { Exarchos et al. [4] 0.42 (0.22) 0.81 (0.09) 0.11 (0.11) 0.60 (0.28) 0.79 (0.11) 0.19 (0.19) Proposed Method 0.29 (0.17) 0.88 (0.05) 0.08 (0.07) 0.31 (0.23) 0.89 (0.07) 0.07 (0.08) No Artifact Lo Vercio et al. [13] { 0.83 (0.05) 0.18 (0.06) { { { Proposed Method 0.53 (0.34) 0.79 (0.10) 0.15 (0.17) 1.22 (0.45) 0.57 (0.13) 0.32 (0.19) Unal et al. [5] 0.65 (0.47) 0.76 (0.14) 0.18 (0.15) 0.57 (0.49) 0.78 (0.13) 0.19 (0.15) Wang et al.[4] 0.54 (0.27) 0.81 (0.11) 0.14 (0.13) { { { Destrempes et al.[4] 0.42 (0.18) 0.85 (0.06) 0.08 (0.06) 0.32 (0.13) 0.91 (0.03) 0.06 (0.04) Bifurcation Downe et al. [23] 0.64 (0.27) 0.70 (0.11) 0.21 (0.15) 0.79 (0.53) 0.71 (0.19) 0.24 (0.21) Alberti et al.[4] 0.61 (0.43) 0.75 (0.10) 0.20 (0.10) { { { Ciompi et al. [24] { { { 0.52 (0.29) 0.85 (0.07) 0.09 (0.07) Mendizabal et al. [9] 0.53 (0.36) 0.79 (0.12) 0.17 (0.18) { { { Exarchos et al. [4] 0.47 (0.23) 0.80 (0.09) 0.10 (0.09) 0.63 (0.25) 0.78 (0.11) 0.23 (0.23) Proposed Method 0.24 (0.11) 0.87 (0.05) 0.06 (0.05) 0.74 (0.18) 0.73 (0.60) 0.21 (0.18) Unal et al. [5] 0.51 (0.39) 0.79 (0.12) 0.17 (0.14) 0.57 (0.39) 0.78 (0.11) 0.18 (0.12) Wang et al.[4] 0.59 (0.23) 0.80 (0.10) 0.16 (0.13) { { { Destrempes et al.[4] 0.36 (0.15) 0.87 (0.04) 0.07 (0.04) 0.31 (0.12) 0.91 (0.04) 0.04 (0.04) Side Vessels Downe et al. [23] 0.46 (0.19) 0.77 (0.08) 0.15 (0.11) 0.76 (0.47) 0.74 (0.16) 0.22 (0.20) Alberti et al.[4] 0.47 (0.24) 0.79 (0.07) 0.17 (0.09) { { { Ciompi et al. [24] { { { 0.53 (0.37) 0.85 (0.09) 0.10 (0.13) Mendizabal et al. [9] 0.38 (0.19) 0.84 (0.07) 0.11 (0.11) { { { Exarchos et al. [4] 0.53 (0.24) 0.77(0.09) 0.16 (0.12) 0.63 (0.31) 0.78 (0.12) 0.18 (0.16) Proposed Method 0.29 (0.20) 0.86 (0.07) 0.08 (0.09) 1.24 (0.39) 0.58 (0.13) 0.37 (0.15) Unal et al. [5] 0.57 (0.39) 0.78 (0.12) 0.17 (0.12) 0.66 (0.50) 0.77 (0.13) 0.19 (0.15) Wang et al.[4] 0.59 (0.27) 0.81 (0.10) 0.18 (0.16) { { { Destrempes et al.[4] 0.39 (0.18) 0.87 (0.05) 0.06 (0.05) 0.33 (0.14) 0.92 (0.03) 0.05 (0.04) Shadow Downe et al. [23] 0.55 (0.26) 0.76 (0.11) 0.14 (0.13) 0.77 (0.48) 0.74 (0.16) 0.22 (0.19) Alberti et al.[4] 0.53 (0.29) 0.78 (0.08) 0.18 (0.09) { { { Ciompi et al. [24] { { { 0.58 (0.36) 0.84 (0.09) 0.11 (0.11) Mendizabal et al. [9] 0.43 (0.27) 0.83 (0.09) 0.12 (0.11) { { { Exarchos et al. [4] 0.46 (0.19) 0.80 (0.10) 0.12 (0.12) 0.57 (0.28) 0.82 (0.11) 0.14 (0.17) 11 Figure 5: Several inaccurate lumen and media segmentation results in the presence of various artifacts. Segmented lumen and media have been highlighted by magenta and green colours respectively. The yellow dashed lines illustrate the gold standard that have been delineated by four clinical experts [4]. regions, join the lumen and media regions, a type of leak The success of the proposed method depends strictly will be created. This leak signi cantly increases the size on the correct extraction of ERELs. In cases when contin- and the mean intensity of the regions and generates irreg- uous and clear boundaries do not exist between extremal ular patterns that are very dicult to distinguish by the regions, EREL cannot extract correct candidate regions EREL extraction algorithm. Some of the inaccurate seg- for lumen and media. For example, in IVUS images taken mentations that are caused by the presence of the artifacts with a 40 MHz catheter probe, some lumen pixels located are shown in Figure 5. As can be seen in Figure 5, al- on the boundaries adjacent to intima do not have clear though the presence of the artifacts disrupts the detection connections to each other. A similar situation occurs for of the media regions, most of the lumen regions can still the boundaries between media and adventitia. This cre- be segmented accurately. This is also supported by the ates leakages that attach media region to the adventitia. quantitative results presented in Table 3. Evidently, even So, the sequential process of EREL's enumeration con- in the presence of artifacts such as bifurcations, side ves- nects pixels from two semantically di erent regions and sels, and shadows, the lumen segmentation performance constructs a big connected component that includes both remains high, though the accuracy of the media segmen- regions. Eventually, the leakage obstructs EREL's region tation drops. In fact, even in the worst case condition of extraction capability for detecting Q regions. This leak- lumen segmentation that happens when the image con- age can also be created in IVUS 20 MHz images by various tains bifurcation artifacts, the average Hausdor distance artifacts that somehow attach the media regions to the ad- to the gold standard is only 0:53 mm which is still lower ventitia as illustrated in Figure 5. than most of the other methods. On the other hand, the Apart from the above, our proposed method o ers sev- media segmentation is more sensitive to the presence of eral bene ts. Not only is our method automatic, but works artifacts. For instance, the Hausdor metric for media also solely on the B-mode images (in contrast to some in some cases increases to 1:22 mm for bifurcation and methods available in the literature that work on the po- to 1:24 mm for shadow artifact. Accordingly, it can be lar space [5, 7, 9, 10]) and, thus, it eliminates the need concluded that the lumen segmentation is robust to the for transformation to the polar space. In addition, some common artifacts of the IVUS images. methods [26, 27] require the whole volume to be presented 12 in order to segment the arterial walls. This makes it im- at the beginning section of the texture vector is consid- possible to segment the lumen and media in online seg- ered as the lumen. Similarly, the last region with a stable mentation applications (during the in-vivo pullback pro- textural variation is labeled as media. cedures). On the other hand, our method delineates lumen We evaluated our segmentation method on the test set and media contours using only the information available of a standard publicly available IVUS dataset using three in the current frame and, hence, performs no redundant common evaluation metrics; namely, JM, HD, and PAD. processing. Furthermore, due to the low computational Extensive quantitative comparisons show the high accu- complexity of the EREL [15, 16], every frame can be seg- racy of our proposed method. In general, the average HD mented in linear time based on the number of pixels in the for our lumen segmentation results is 0:3 mm which is the frame. The average run time per frame in the proposed closest distance to the gold standard among all existing method over the test set of [4] is 0:19 seconds. methods. In addition, when no artifact was present, our method segments both lumen and media achieving average HD lower than 0:29 mm and 0:31 mm, respectively. 5. Conclusion As reported in the best case results, among the ex- tracted EREL regions, there exist regions very close to the actual structure of the arterial walls. In future work, In this study, we investigated the suitability of a re- we would like to devise a better selection strategy that cently proposed region detector called Extremal Regions chooses ERELs more accurately. of Extremum Levels for detecting the lumen and media Since IVUS is completely radiation free, unlike X-ray regions. The results of our experiments reveal that among and CT, in the future we would also like to investigate the ERELs extracted from a single IVUS frame there exist the feasibility of combining the segmentation results from regions with very low di erence from the manually anno- individual slices to obtain 3D views of collateral arteries. tated data. Therefore, we presented a region selection ap- proach to automatically segment the arterial walls from Intravascular Ultrasound images taken with a 20 MHz 6. Acknowledgment catheter probe. We embedded the mean intensity value, the entropy, and the boundary length of the regions into a We gratefully acknowledge the assistance of Prof. Si- single feature vector representing the texture information mone Balocco at the University of Barcelona for providing of the region. Then, our selection method looks for regions the labeling information on the existing artifacts in the with higher stability score. The region that is most stable dataset. References References [1] J. Frosteg ard, Sle, atherosclerosis and cardiovascular disease, Journal of internal medicine 257 (6) (2005) 485{495. [2] A. Katouzian, E. D. Angelini, S. G. Carlier, J. S. Suri, N. Navab, A. F. Laine, A state-of-the-art review on seg- mentation algorithms in intravascular ultrasound (IVUS) images, IEEE Transactions on Information Technology in Biomedicine 16 (5) (2012) 823{834. [3] T. Ma, B. Zhou, T. K. Hsiai, K. K. Shung, A review of intravascular ultrasound-based multimodal intravascular imaging: the synergistic approach to characterizing vulnerable plaques, Ultrasonic imaging 38 (5) (2016) 314{331. [4] S. Balocco, C. Gatta, F. Ciompi, A. Wahle, P. Radeva, S. Carlier, G. Unal, E. Sanidas, J. Mauri, X. Carillo, et al., Standardized evaluation methodology and reference database for evaluating ivus image segmentation, Computerized Medical Imaging and Graphics 38 (2) (2014) 70{90. 13 [5] G. Unal, S. Bucher, S. Carlier, G. Slabaugh, T. Fang, K. Tanaka, Shape-driven segmentation of the arterial wall in intravascular ultrasound images, IEEE Transactions on Information Technology in Biomedicine 12 (3) (2008) 335{347. [6] G. Mendizabal-Ruiz, I. A. Kakadiaris, A physics-based intravascular ultrasound image reconstruction method for lumen segmentation, Computers in biology and medicine 75 (2016) 19{29. [7] A. Taki, Z. Naja , A. Roodaki, S. K. Setarehdan, R. A. Zoroo , A. Konig, N. Navab, Automatic segmentation of calci ed plaques and vessel borders in ivus images, International Journal of Computer Assisted Radiology and Surgery 3 (3-4) (2008) 347{354. [8] X. Zhu, P. Zhang, J. Shao, Y. Cheng, Y. Zhang, J. Bai, A snake-based method for segmentation of intravascular ultrasound images and its in vivo validation, Ultrasonics 51 (2) (2011) 181{189. [9] E. G. Mendizabal-Ruiz, M. Rivera, I. A. Kakadiaris, Segmentation of the luminal border in intravascular ultra- sound b-mode images using a probabilistic approach, Medical image analysis 17 (6) (2013) 649{670. [10] F. S. Zakeri, S. K. Setarehdan, S. Norouzi, Automatic media-adventitia ivus image segmentation based on sparse representation framework and dynamic directional active contour model, Computers in Biology and Medicine. [11] S. Su, Z. Hu, Q. Lin, W. K. Hau, Z. Gao, H. Zhang, An arti cial neural network method for lumen and media- adventitia border detection in IVUS, Computerized Medical Imaging and Graphics 57 (2017) 29{39. [12] J. Yan, D. Lv, Y. Cui, A novel segmentation approach for intravascular ultrasound images, Journal of Medical and Biological Engineering 37 (3) (2017) 386{394. [13] L. Lo Vercio, J. I. Orlando, M. del Fresno, I. Larrabide, Assessment of image features for vessel wall segmentation in intravascular ultrasound images, International journal of computer assisted radiology and surgery 11 (8) (2016) 1397{1407. [14] R. M. Haralick, Statistical and structural approaches to texture, Proceedings of the IEEE 67 (5) (1979) 786{804. [15] M. Faraji, J. Shanbehzadeh, K. Nasrollahi, T. B. Moeslund, Erel: Extremal regions of extremum levels, in: Image Processing (ICIP), 2015 IEEE International Conference on, IEEE, 2015, pp. 681{685. [16] M. Faraji, J. Shanbehzadeh, K. Nasrollahi, T. Moeslund, Extremal regions detection guided by maxima of gradient magnitude, Image Processing, IEEE Transactions on. [17] F. Destrempes, M.-H. R. Cardinal, L. Allard, J.-C. Tardif, G. Cloutier, Segmentation method of intravascular ultrasound images of human coronary arteries, Computerized Medical Imaging and Graphics 38 (2) (2014) 91{103. [18] C. P. Loizou, C. S. Pattichis, Despeckle ltering algorithms and software for ultrasound imaging, Synthesis lectures on algorithms and software in engineering 1 (1) (2008) 1{166. [19] R. Sedgewick, K. Wayne, Algorithms, 4th Edition., Addison-Wesley, 2011. [20] B. Iglewicz, D. C. Hoaglin, How to detect and handle outliers, Vol. 16, Asq Press, 1993. [21] L. Shapiro, G. C. Stockman, Computer vision. 2001, ed: Prentice Hall. [22] R. Szeliski, Computer vision: algorithms and applications, Springer Science & Business Media, 2010. 14 [23] R. Downe, A. Wahle, T. Kovarnik, H. Skalicka, J. Lopez, J. Horak, M. Sonka, Segmentation of intravascular ultrasound images using graph search and a novel cost function, in: Proc. 2nd MICCAI workshop on computer vision for intravascular and intracardiac imaging, Citeseer, 2008, pp. 71{79. [24] F. Ciompi, O. Pujol, C. Gatta, M. Alberti, S. Balocco, X. Carrillo, J. Mauri-Ferre, P. Radeva, Holimab: A holistic approach for media{adventitia border detection in intravascular ultrasound, Medical image analysis 16 (6) (2012) 1085{1100. [25] F. Molinari, K. M. Meiburger, G. Zeng, A. Nicolaides, J. S. Suri, Caudles-ef: carotid automated ultrasound double line extraction system using edge ow, Journal of digital imaging 24 (6) (2011) 1059{1077. [26] Z. Sun, C. Liu, A parallel method for segmenting intravascular ultrasound image sequence, in: Mechanical and Electronics Engineering III, Vol. 130 of Applied Mechanics and Materials, Trans Tech Publications, 2012, pp. 2051{2055. doi:10.4028/www.scientific.net/AMM.130-134.2051. [27] M.-H. Cardinal, J. Meunier, G. Soulez, R. L. Maurice, E. Therasse, G. Cloutier, Intravascular ultrasound image segmentation: a three-dimensional fast-marching method based on gray level distributions, IEEE transactions on medical imaging 25 (5) (2006) 590{601.

Journal

Electrical Engineering and Systems SciencearXiv (Cornell University)

Published: Jun 10, 2018

There are no references for this article.