Image Analysis & Image Acquisition


The image analysis was carried out on two levels: 1) macroscopic analysis of CECT images to build a 3-D model of anatomical structures, and 2) microscopic analysis to segment and to register images of histological slices.

The following methods have been developed for CECT image processing: a liver / tumour / vessel segmentation method, a registration method and a surface meshing method. Together they form all necessary tools for creating subject-specific 3-D models. All developed tools have been extensively evaluated and their performance has been carefully benchmarked against human experts and existing methods and their robustness over different image acquisition protocols. Evaluation results have shown that the developed methods are suitable for planned application and that improvements over existing methods have been made. The developed tools have been used to create subject-specific liver models. Figure 1 shows an example of constructed 3-D models: liver is shown grey, lesions in yellow, the hepatic artery in red, the portal vein in blue and the hepatic vein in green. During the project, we have reconstructed seven 3-D models from patients and ten 3-D models from pigs.

Figure 1. Example 3D model of a pig. Liver is shown grey, lesions in yellow, the hepatic artery in red, the portal vein in blue and the hepatic vein in green.

Probabilistic classification followed by the level-set segmentation has been employed for robust identification of those structures from CAB- and MB-stained histology slices. Two additional steps have been included into a modified processing line comprising micro-CT: (1) histology - Micro-CT registration and (2) Micro-CT - CECT registration. The development of algorithms for histology segmentation and fusion to micro-CT is completed as a whole. All necessary algorithms are implemented and are integrated within an interactive segmentation tool. Nevertheless, the large variability in histology appearance makes it necessary to adapt the algorithms to each data set. Micro-CT and CT registration is the most challenging and erroneous aspect. Software has been developed to interactively register 3D models by selecting corresponding landmarks. Preliminary results on the 1-Month-survivor show promising results but the achieved accuracy needs to be improved. Additional pigs are being processed which will allow us to refine the registration process and improve on the registration accuracy.

Details for CT data:

Dedicated image processing algorithms have been designed and corresponding tools have been implemented to construct the subject-specific 3-D liver models from CT-data. These models provide input for simulation and visualization, and therefore should include accurate presentation of the liver and its sub-structures (vessels, tumours, and ablation zones). To generate such a 3-D model from CT scans, specific methods have been designed to 1) segment each anatomical structure; 2) register all the segmented data in a common coordinate system, and 3) generate a surface-mesh converting the volumetric data into a form that is more suitable for visualization and simulation purposes.

Vessel segmentation:

A semiautomatic method for the segmentation of bright, connected tubular structures in CT images has been applied to build an accurate 3-D vessel model. The segmentation method employs several pre-processing steps including image enhancement using Hessian filters, linear normalization and isotropy correction. Vessel segmentation is based on an adaptive region-growing algorithm with manual initialization of seed points. The segmented binary image is converted into a graph-skeleton in line with distance transform methods. The skeleton graph is presented in the form of a directed piecewise linear tree. Using the developed refinement tools it is possible to prune the vessel trees. After pruning, the vessel skeleton is inflated back to its original size.

Liver segmentation:

Three different method were implemented and tested for liver segmentation Their results are all qualitatively similar to manual segmentations but differ in the algorithmic complexity and the expected run-time.

Liver segmentation with multi-classifier approach: For liver segmentation a multi-classifier approach was developed. All CECT images were first registered with the non-contrast CT image of same patient. In a pre-processing step, the diaphragm was segmented to separate liver and heart to obtain a rough ROI mask. Two different segmentation methods were used to created independent segmentations to (1) refine the ROI mask in a low resolution processing step and (2) finally to obtain the final segmentation by consensus decison. The first segmentation method was based on thresholding air and fat from tissue (-60 in Hounsfield units) followed by morphological opening and largest connected component analysis. That segmentation result was used in the segmentation method 2 to calculate the median liver intensity which is used in turn to softly classify liver voxels based on their intensity. The final map was obtained by smooting the classification result.

Liver segmentation with multi-atlas approach: An atlas consists of intensity data and corresponding binary segmentations. In the segmentation process a new data and atlas (intensity data) were registered and a binary segmentation of the corresponding atlas was transferred to the coordinate system of the new image. This provided a segmentation for the new image. In multi-atlas segmentation a bank of 20 atlases was used, where each atlas was independently registered with the new data. The final segmentation was obtained by majority voting of the independent segmentations.

Liver segmentation with histogram peak thresholding approach: This method is based on detecting peaks corresponding to different tissue types from the intensity histogram of the denoised maximum of the co-registered CECT images, and performing classification to different tissues types by thresholding (Figure 2). The peaks of the histogram are first detected as local minima of the absolute value of the histogram’s derivative and classified according to their neighboring histogram values as local maxima, local minima, and ascending or descending blended peaks. The main peak is determined as the highest local maximum in a pre-determined Hounsfield unit range and a suitable amount of neighboring peaks are then chosen according to predetermined criteria to facilitate the classification by thresholding. After the initial segmentation the segmentation result is refined by detaching the largest connected region consisting of image voxels falling within the class corresponding to the main peak.

Figure 2. Example of classification result using the histogram thresholding approach. The picture on the left shows all of the peaks detected from the intensity histogram (dots), the detected main peak (square), and its neighboring peaks (circle). The picture on the right shows the resulting classification after thresholding with the HU values corresponding to the final detected peaks.

Tumour segmentation:

A novel semi-automatic liver tumor segmentation method was developed with special focus on tumors that are typical for RFA treatment. The method is based on the Hidden Markov Random Field (HMMF) model and non-parametric intensity distribution estimation. In addition to segmentation of tumors from single CECT images, a multiphase method was developed, taking simultaneously advantage of all the available preoperative image data of the patient. The method requires minimal initial input from the user, after which the segmentation is computed automatically.

Registration of CT data:

The registration module is intended to register all data to common co-ordinate system so that a) single geometry can used in pre-operative planning and b) pre- and post-operative data can be compared in meaningful way. We implemented linear and non-rigid registration frameworks. The non-rigid registration was based on uniformly distributed control points, whose positions were iteratively optimized. The cubic B-spline interpolation was used between the control points. The cost function was based on normalized mutual information and additional regularization terms. In optimization, we used the conjugate gradient method. To speed up the convergence hierarchical refinement of both image resolutions and control point spacing were adopted.

Surface meshing and mesh refinement

As a final step in 3-D model creation, the isosurface was extracted from segmented volumes with Marching Cubes algorithm. In addition, a mesh refinement tool IMPPACT-SNAP has been developed by using the widely-used open source software package ITK-SNAP ( as a starting point. A screen shot for example use of the segmentation refinement tool is shown in Figure 3. The main modification was the development and incorporation of an elastic surface mesh deformation tool, as well as the I/O functionality for standard mesh file formats (VTK, STL, BYU). The developed software tool enables the user to inspect and manually adjust a triangle surface mesh.

Figure 3. An example of deforming a surface mesh representing a porcine liver (enlarged axial view). The deformation tool is illustrated as the circle around the navigation crosshair representing the standard deviation (half-axes) of the applied Gaussian kernel.

Details for histological data

At the beginning of year 2 it became clear that registration of histological segments into a coherent lesion volume and its further registration into the CT- built 3D model is not feasible because of 1) about 1:100 scale difference between the CT and histology slices, and 2) significant physical deformations of histology slices introduced in the process of their preparation. To resolve this problem, micro-CT image of lesion blocks has been introduced into the segmentation & registration loop. Micro-CT image 1) introduce an intermediate reference scale between CT and histology images, thus filling the scaling gap; and 2) geometry preserving as compared to arbitrary deformed histology slices.

Histology lesion reconstruction:

We have developed a framework for interactive reconstruction of the histological lesion and registration into in vivo CT data (Figure 4). All registration steps are based solely on non-lesion features and external landmarks introduced into the liver tissue. This allowed a well-defined correlation to be established between lesion features visible in CT and the “true” histological lesion. The framework consists of a segmentation stage, where foreground, vessels, landmarks and lesion are identified, and three subsequent registration stages:
i) the histology slides are registered into a high-resolution ex vivo CT (embedded MicroCT) scan of the uncut histology block;
ii) the obtained histology/MicroCT block is registered into the native MicroCT using manually selected corresponding points on segmented vessels;
iii) final registration of the native MicroCT block into the in vivo CT data is done based on the artificial landmarks and the segmented vessels.
In all these steps, only segmented foreground masks, vessels and landmarks are used to establish correspondence to avoid introducing the correlation of lesion features we want to analyze.

Figure 4. Data preparation and the histology processing pipeline with introduced deformations and associated registration steps for lesion reconstruction. Image data shows (left) example slices and (right) the approximate scale of (a) in-vivo CT (portal venous phase), (b) native MicroCT, (c) embedded MicroCT and (d) histology. (All data is enhanced for visualisation)

The introduction of the micro-CT data of the uncut histology block allowed us to recover the original configuration of the histology by registering each histology slide onto the corresponding micro-CT slice. The correspondence is defined by manually selected landmarks to allow the reconstruction of heavily distorted slices and, mainly, to ensure that lesion features are not used to register the slices. To facilitate that registration task and also the segmentation refinment of the histology data we have developed an interactive segmentation and registration tool for two-dimensional histology segmentation as well as slice based micro-CT segmentation (Figure 5) implemented in ITK, VTK & Qt.

Figure 5. Interactive segmentation and registration tool used for histology and micro-CT fusion.

A software tool for interactive fusion of micro-CT and CECT has been developed which facilitates the detection of corresponding landmark point, corresponding vessels and interactive testing of different registration methods (rigid, similar, affine, TPS) (Figure 6). Standard algorithms implemented in VTK are used to calculate the optimal transformation between corresponding points.

Figure 6. Interactive registration tool for micro-CT and CECT model fusion. Micro-CT segmentation is shown in the left window, the CECT model in the middle window and the combined(registered) in the right window.

Histology segmentation:

Final goal of the histology image segmentation is to build 3-D regions with dead/ partly dead/ live cells. Knowledge about these regions will play a role of "ground truth" when identifying actual results of the RFA on a particular liver. For this, segmented regions of ablated (dead) tissue should be registered to the 3D model of the liver as generated from the contrast enhanced CT images. Therefore, large vessels which are visible on CT need to be segmented from the histology data as well. The above image acquisition procedure provides all necessary data to accomplish this goal.

To accommodate for those requirements a multi-resolution, semi-automatic, probabilistic algorithm for combined slice-by-slice segmentation and registration is being developed. The algorithm combines variations of three basic algorithms which are explained in turn:
1.Probabilistic classification:
The classification step uses a probabilistic mixture of Gaussians clustering algorithm to label the pixels in the histology image. Furthermore, it outputs the probability of each pixel of belonging to a tissue class, which can be interpreted as a probabilistic measure of each class' membership. The probability model used is a multivariate Gaussian mixture (GMM) which is trained using the standard (unsupervised) expectation maximization (EM) algorithm.
2.Slice-by-slice registration:
Slice-by-slice registration is done using standard multi-resolution registration framework implemented in ITK using rigid transformations, normalized mutual information and a genetic optimization algorithm. Registration features are the red channel for two CAB images, the blue channel for two MB images and the value channel for a CAB and a MB image. For robustness, multiple resolution levels are typically used. Furthermore, the registration algorithm is applied not only to consecutive slices but to several neighbours to allow for a post-processing step to detect and to eliminate poorly aligned slices.
The segmentation of tissue structures is based on the classification in that current slice and of neighbouring slices. The probabilistic classification results of those slices are combined into one feature image weighted by the distance to the current slice and the quality of registration. The resulting feature image represents the probability of each pixel of belonging to a tissue type given that probability for neighbouring slices. A level-set algorithm is used to segment that feature image into smooth regions of similar probability values. The level-set parameters are set by hand to result in multiple smooth regions for the vessel segmentation and only a few regions for region of ablation. The histology data is processed on two resolution levels with almost identical workflow. The main difference is that on the low resolution level the whole slices are processed and, subsequently, regions of interest (ROI) are extracted. The full resolution slices are processed only in those ROIs to refine the results of the low resolution level.

Image data aquisition from pre-clinical experiments:

Parameters of the CT images during the RFA in animals have been mostly defined by the type of the CT device and RFA protocol in use. We examined three samples of three-phase high resolution contrast enhanced CT-images from porcine liver. The 512x512x320 size images have planar resolution of 0.454 mm and slice thickness of 0.5 mm. All three phases, namely arterial, portal venous and hepatic venous are separately scanned.
We studied the image quality of contrast enhanced CT images during the manual segmentation and during the development for 3D segmentation algorithms. The current image acquisition protocol is accurate enough for our methods if the bolus timing is correct. Few images in early acquisition with in-accurate bolus timing have been observed, but this has been corrected in later acquisitions.

To sucessfully reconstruct the lesion from strongly deformed histology slides, the original configuration of the histology block has to be known. Therefore a Micro-CT scan is taken before cutting.

Acquisition of histological image data has the purpose of displaying the status of cell tissue after the RFA treatment. Treated cells usually fall into one of three categories. 1) dead cells, 2) live cells, and 3) intermediate tissue with cells of both previous types. The task of histological image acquisition is to enhance best differentiation between these three types and facilitate automatic segmentation procedures of the cell types.
Extensive experiments have been conducted to establish optimal procedure for histological image acquisition. These experiments evaluated two types of scanning devices and different parameter settings and compositions of stainings applied in neighbouring slices. Three staining substances were tested:

1.CAB — Chromotrope-Anilin Blue.
The CAB staining shows reliably good differentiation between the connecting liver tissue (blue), normal liver tissue (red-violet) dead liver tissue (light red) and vessels (background colour) within the liver.
2.MB — Methylene Blue.
The MB staining exhibits different intensities of blue colour for dead and living cells. The disadvantage is that this difference varies from sample to sample, which makes it difficult to use in any automatic segmentation procedure.
3.TCR — Trichrom.
TCR exhibits differentiation between the connecting tissue (green) and ablated region of (light red), but requires exhaustive manual colouring.
Final procedure for histological image acquisition has been developed in a close collaboration of partners. Special attention was paid to provide features which can be used in automatic segmentation and registration methods as well as sensible work load in terms of technical preparation of histology slices and their further segmentation.
This final procedure is as follows:

Copyright 2006 - Design by LS