Model development and numerical simulation


Visualization A completely new rendering pipeline was developed to meet the demands of the project data. A software-renderer was implemented on the graphics hardware which is able to combine multiple volumetric datasets with polyhedral surface meshes of data sources in different data formats, different resolutions and different scale. . The raw data which has to be considered for visualization is summarized in the following bullet list.

Medical Scans:
The raw data are contrast enhanced volumetric CT scans stored in DICOM format. Furthermore histologies are stored as stack of images.
Segmentation Results:
The liver surface, all vessel trees and RFA lesions or tumours are available as high-polygonal surface meshes. These datasets are rendered in the same scene with the volumetric representation of a patient.
Simulation Results:
Simulation is done on volumetric unstructured grids, which allow a denser simulation around interesting regions like the tumour or vessels and computationally more economic calculations in homogenous regions. These datasets cannot be integrated directly into a rendering system and are post-processed — namely resampling in a volumetric dataset — or visualized separately.

A virtual model of the RITA Starburst electrode in all extension states was created. The model is suitable for visualization as well as computation. The needle is represented as polygones. From the work of the simulation of RFA ablation using finite element techniques, we receive a number of interesting 3D datasets, which must be inspected through visualization. One key property here is that the data has uncertainty (of the simulation) as an attribute. Uncertainty visualization is an emerging topic in scientific visualization, and we have started work on novel uncertainty visualization techniques for this problem domain. In TUG we are developing a novel method for displaying isosurfaces of uncertain 3D datasets by augmenting the isosurface rendition with generalized error bars that have all the strengths of classic error bars used in 2D visualization of uncertain datasets, such as unambiguousness and intuitiveness. Several other simulation visualization procedures have been implemented. A color coded 2D overlay together with an iso-surface which shows a certain temperature or estimated necrosis have revealed to be best suited for the clinical practice.


Visualization of simulation results fused with treatment images:

While techniques for visualization of anatomical 3D reconstructions have been used in clinical practice for years, the future of medical applications is no longer oriented on showing images as acquired by medical technology. Added value arises from combining information from different sources as well as additional knowledge into one single presentation. From this fact arise today's challenges in medical virtual reality (VR) research. The fundamental tool for modern medical VR is volume rendering, the process of visualizing data stored aligned to a grid. The volume data can either be transformed into a set of geometries followed by the conventional rendering, or else, the visualization is directly derived from the volume data set, which leads to Direct Volume Rendering (DVR). DVR offers higher quality images and a larger degree of freedom, since no data is lost during a transformation.
Our direct volume rendering tools must be able to deal with multiple huge volumes and multi-variant, high-dimensional data sets. This can hardly be achieved by straight forward extensions of the conventional CPU vertex/fragment shader model used in today's implementations. While the fundamental technique for producing the visualizations will still be based on raycasting, the use of the emerging GPU compute languages (specifically, CUDA) allow the use of a new class of optimized algorithms.
According to the needs described above, we developed a new GPU-based rendering system for ray casting of multiple volumes (Figure 1). Our approach supports a large number of volumes, complex translucent and concave polyhedral objects as well as Constructive Solid Geometry (CSG) intersections of volumes and geometry in any combination. The system (including the rasterization stage) is implemented entirely in CUDA, which allows full control of the memory hierarchy, in particular access to high bandwidth and low latency shared memory. High depth complexity, which is problematic for conventional approaches based on depth peeling, can be handled successfully. As far as we know, our approach is the first framework for multi-volume rendering which provides interactive frame rates when concurrently rendering more than 50 arbitrarily overlapping volumes on current graphics hardware.

Figure 1. The flow-chart shows an overview of our Direct Volume Rendering (DVR) system. The main steps - triangle processing and pixel processing - are executed in separate compute kernels. The left part is necessary to rasterize the available geometry into screen pixels and the left part is responsible for the actual ray casting and combination with polyhedral geometry.

Accuracy of the ablation prediction achieved in the experimental loop

Overall accuracy of the ablation prediction is defined by the accuracy of single steps in the experimental loop e.g. segmentation, registration, and numerical modelling. The overall accuracy is not cumulative over these steps but a step with the largest source of error defines the overall accuracy of the ablation prediction. We therefore performed detailed evaluation of possible inaccuracies based on clinically required accuracy.

Clinically required accuracy: The simulation has to meet certain requirements with respect to the spatial accuracy. The clinically relevant accuracy depends on tumour location and tumour type. For critical locations such as the liver hilus near large vessels, for sub-capsular lesions close to the lung or the stomach and for metastases of a colorectal carcinoma the required accuracy is below 5 mm. For other more intraparenchymal tumour locations and for HCCs, an accuracy of 10 mm would be sufficient. For the present project, the required accuracy limit for the prediction of the ablation size was set to 5 mm.

Evaluation metric(s): Evaluation of segmentation and registration tasks without a well-defined ground-truth is a typical problem in (medical) computer vision. As usual, we use expert segmentation(s) and registration to define the ground-truth. Segmentation of liver, tumour and histology data is evaluated by the average symmetric surface distance (SD) and the maximum symmetric surface distance (MD) which are most informative for the validation of the RFA model.
These error metric is only applicable for mass-like objects such as liver and tumours. For vessels, manually extracted landmarks were compared to the semi-automatically segmented vessels. The root mean square error (RMSE) of centre point location and vessel diameter and hit/success rate were used in evaluation.
Registration is evaluated by performing two independent registration steps and comparing the created deformed segmentations with each other using the MD metric.

Liver segmentation error: Liver segmentation errors for three developed methods (multi-classifier, peak detection and multi-atlas) are calculated for evaluation data from partner MUL including 8 RFA patients and data from liver segmentation competition (SLIVER07) respectively. The typical SD is between 2.3 and 1.1 mm and MD between 29.0 and 19.0 mm which is comparable to the accuracy of expert annotations. Although the maximum error is well above the aim of 5-10 mm, it has only a significant impact on the simulation if the error occurs near the tumor. But maximum errors are spatially concentrated on connected organ borders, where no visible borders between the liver and neighbouring organ e.g. the stomach exists. However, if the tumor is located near those borders, percutaneous RFA intervention is typically not used.

Tumor segmentation error: The evaluation data for tumor segmentation came from liver tumour segmentation challenge (LTS). For our algorithm the SD is about 1.9 mm (MD 8.1 mm) and as such outperforms the currently best method [Smeets et al., 2010] (SD 2.0 mm, MD 10.1 mm) but refinement by a human expert (SD 0.4 mm, MD 4.0mm) is necessary to achive maximum error below our target of 5mm.

Vessel segmentation error: Vessel segmentation evaluation is based on the root mean square error (RMSE) of centre point location and vessel diameter and hit/success rates of corresponding vessels. Results indicate that we can sucessfully segment 97% of thermally significant vessel i.e. vessels with diameter 3.0 mm or larger. To have any impact on simulation the missing vessels should be located in the close neighbourhood of the ablation centre. This probability is small (<< 3%). The errors in vessel location ranges from 0.46 to 0.85 mm and the error in vessel size (diameter) ranges from 0.61 to 1.90 mm. Both errors are well below the 5 mm target error.

CECT lesion segmentation error: In standard clinical practice, the RFA ablation zone is assessed from CECT data. The CECT lesion is characterised by lower brightness as compared to the surrounding non-damaged tissue, which appears lighter in CECT images. The border between the irreversibly ablated (dead) tissue and the remaining healthy tissue is however blurred and about 5-8mm thick. That value is also the smallest possible accuracy for CECT lesion segmentation and subsequently for lesion prediction when using the CECT lesion as validation model.

Histology lesion segmentation error: For the 1-month-survivor data set , six typical MB and CAB slides were manually segmented by two medical experts. The differences between their segmentations as well as comparison to automatic and refined segmentation are reported in the following table (Median and worst values for segmentation error of histology of 1-Month-survivor. Worst case values are given in brackets (1pixel = 0.017mm)):

CAB slides (MD in mm) MB Slides (MD in mm)
Inter-observer 1.14(1.49) 0.94(1.44)
Automatic segmentation 1.18(1.40) 2.71(19.6)
Manual refined segmentation 1.09(1.67) 0.87(1.44)
It shows that errors for the manually refined segmentation are well below the clinically significant error of 5mm.

Histology, micro-CT, CECT model registration error: Evaluation of registration without a given ground-truth is an unsolved problem. Therefore, no objective quantitative evaluation can be given here. Instead we employ the standard test-point-error metric. Here two disjoint sets of corresponding points are interactively segmented. Each set is used to create the registration transform. The source points of the other set are transformed as well and the Euclidean distance to the corresponding target points measured. These distances give a qualitative overview of expected errors in the registration process. The clinically relevant variation of the reconstructed lesion is evaluated by transforming the segmented ablation zone for both point sets and calculating the Maximum symmetric surface distance (MD) between them. The registration of histology and micro-CT was evaluated on three typical slides of the 1-month-survivor data set. Registration of the 3D micro-CT/histology volume and 3D CECT model was evaluated on the 1-month-survivor data set as well. Results summarised in the following table (Median and worst values for registration error for 1-Month-survivor data set. Worst case values are given in brackets).

test-point-error (in mm) MD (in mm)
Histology micro-CT 0.17(4.5) 0.11(0.28)
Micro-CT CECT 3.2(9.4) 1.4(3.8)
These results indicate that the highest degradation of accuracy occurs during the micro-CT and CECT registration.

Summary on the error of registration: So far the registration error has been assessed as max deviation between registrations carried out by different experts. This accuracy assessment, though below the clinically defined requirement of 5mm, does not fully reflect the true accuracy, which can only be obtained via comparison with the benchmark. However IMPPACT consortium could not establish any benchmark registration between CECT and micro-CT /histology images. It is therefore that the consortium discussed an option to introduce an additional reference frame into the RFA protocol and the acquisition of micro-CT images in pigs. The aim of the additional reference frame is to provide landmarks points and by that a reliable benchmark data for the registration of micro-CT images in the CECT 3D model.
As off today, the accuracy of the established histology lesion is principally restricted by the accuracy of the micro-CT-CT registration step, for which no benchmark assessment exist. The accuracy of all other steps is well below the clinically defined 5mm.

Accuracy of the numerical modelling: The RFA simulation accuracy is dependent on the animal specific material properties, needle locations (boundary conditions) and heat transfer modelling assumptions for the blood and tissue. These will affect the size of the predicted lesion both qualitatively and quantitatively, dependent upon the sensitivity of the predictions to each of these effects. We will be able to assess the "overall" accuracy level for the RFA prediction in terms of the lesion volume size and shape. By superimposing the segmented and predicted lesions we can calculate an error band variation over the outer surface.

Further clinically relevant information is the correspondence between the ablation lesion as it is seen and reconstructed from the CECT images and the histologically defined lesion. Currently, there is no reliable clinical data available that would relate these two different measures. One main objective of the image analysis is therefore to draw the histologically justified border over the blurred region seen in the CECT images.
Benchmark for such a definition, however, is a tissue section in which alive and dead cells have been appropriately stained. The key question for the radiologist is to which extent the CT border corresponds to that defined by histology. This is necessary for an early detection of areas that are under treated and possibly require another RF ablation. The most beneficial scenario would be to avoid a second RFA and the associated risk of complications by using simulation software to predict the ablated volume and to control the RF ablation procedure in order to provide the required security margins. One aim of IMPPACT is therefore to assess the true ablation zone from the histology data and establish the currently unknown correspondence between histological findings and the CECT lesion. The accuracy of that correspondence is defined by the maximum error of histology segmentation, histology and micro-CT as well as micro-CT and CECT model registration.

Visual Error Estimation:

The goal of visual validation was to get an overview over the plausibility of the simulation and the following alignment steps. We used the multi-volume renderer and applied it to both, the simulated lesion volume and the segmented lesion from the histology slices. Visualization of such 3D datasets with uncertainty is an emerging topic in scientific visualization, which receives more and more attention from the visualization researchers. Development and employment of such techniques may have a significant impact on the decision-making process of a radiologist using the RFA planning application. We are developing a novel method for displaying isosurfaces of uncertain 3D datasets by augmenting the isosurface rendition with generalized error bars that have all the strengths of classic error bars used in 2D visualization of uncertain datasets, such as unambiguousness and intuitiveness. The open problems that are being solved are the optimal placement of error bars in a 3D environment, the choice of their direction, combining error bars for multiple datasets in a single visualization as well as the rendering technique for error bars themselves.


[Deng, X. and Du, G. 2008] Editorial: 3-D segmentation in the clinic: A grand challenge II-Liver tumour segmentation. International Conference on Medical Image Computing and Computer Assisted Intervention, 2008.

[Smeets, D., and et al. 2010] Semi-automatic level set segmentation of liver tumours combining a spiral-scanning technique with supervised fuzzy pixel classification, Medical Image Analysis, Vol. 14 (1), pp. 13-20, 2010

Copyright 2006 - Design by LS