TRANS-FUSIMO Treatment System (TTS)

The TRANS-FUSIMO treatment software (TTS) is one of the integral and essential developments during the TRANS-FUSIMO project. Only with the TTS, a focused ultrasound treatment in moving abdominal organs becomes feasible. To our knowledge, this system is unique in the research community. No other group of researchers has been able to integrate the necessary components into a real-time steering system that is extensively validated. The system is implemented as a generic treatment system that abstracts from actual vendor-specific hardware detail. MRgFUS is an image guided therapy (see above figure), thus, the system continuously requires images as input in order to extract information for adjustments of the therapy in real time. In the TRANS-FUSIMO case, images are acquired with a GE MRI device. This data is then processed and the resulting information leads to a therapy action which is the update of the FUS device.

We aimed at developing a generic motion-compensated FUS system that is independent of any algorithmic implementations needed for the real-time processing pipelines or hardware specific details. This generic system takes care of all the incoming information and data and handles the core functionality. The grey box in the left middle image visualizes the so-called core part of the software. The implementations of the algorithms can be added like plug-ins that, however, are mandatory for the full functionality of the software. The green boxes in the same figure depict the algorithms needed for the motion compensation and monitoring whereas the blue boxes represent the connection to hardware.

In a next step, we implemented the FUS control of the physical transducer CBS 2100 of INSIGHTEC as well as the imaging receiver for the MRI device of GE. The lower figure on the left shows the specific hardware implementations of the generic FUS system. This specific implementation of the generic motion-compensated FUS system was used for all pre-clinical experiments performed during TRANS-FUSIMO.

There are two concepts that are in principle applicable for any real-time control system such as developed in the TRANS-FUSIMO context.

The left part of the figure on the left shows a first design approach for the processing of real-time information: A forward control loop that receives images at a certain image update rate. On these images the motion observation and also the temporal motion prediction is performed to compute the target location for the next couple of milliseconds. This new information is then sent to the FUS control which adapts the phases of the elements for a certain sonication location in real-time. However, this forward control loop has the following drawback: The image update frequency of the MR limits the update frequency of all downstream calculations. In other words, the FUS update frequency is bound to the MR update frequency.

To tackle this problem, we decompose the simple forward pipeline into a motion observation loop and a FUS control loop that can run in parallel and at independent frequencies, see the right part of the figure on the left. The splitting into multiple loops, however, requires an interface between the loops: The so-called motion model serves as such. It stores all information gathered from the motion observations, computes the temporal motion predictions and can provide this information at any time point. Thus, the FUS control loop is able to fetch new position information at its own speed and completely decoupled from any other existing system.

To provide a flexible design of the generic treatment system which is not bound to any specifics of used hardware systems, we implemented the decoupled control loop approach. In addition to the previously described loops, there is also a third loop calculating the actual temperature difference based on a set of echo planar imaging phase data, as depicted in the full system overview in the lower left figure. This loop has the same update rate as the motion monitoring loop since it also depends on the imaging rate of the MRI device.

A central part of the motion monitoring component is the extraction of motion information from MR or US imaging data. For this, a new initialization routine and the related tracking routine have been successfully applied in the MICCAI challenge for liver ultrasound tracking in 2014, even though the choice of important points for the observation model had not been optimized. This optimization has been worked on later and found to deliver decent results which allow for a more elaborate selection of important candidate points in the landmark model. 

An automatic routine for placing tracking landmarks inside a region of interest has been implemented. Once landmarks are placed their performance is observed and if found to be insufficient landmarks are replaced by new ones.

For the optimization of the tracking component, a human volunteer dataset of MR images previously acquired by UNIVDUN during FUSIMO was used to explore the influence of the parametrization on the tracking component. The dataset features data of ten volunteers performing different breathing maneuvers, imaged via two parallel sagittal slices, in six sessions of 300 frames each. For this dataset manual ground truth annotations are maintained by ETH. The tracking algorithm was compared against the test set and for the implemented version a mean error of 1.72mm (~1 voxel, in-plane) was found. Neglecting landmarks with high uncertainty, the algorithm reaches an average error of 1.49mm. This version of the algorithm is currently integrated into the TTS. The tracking algorithm has been further optimized to lower the error.

The left figure shows the automatic selection and tracking of landmarks inside sagittal liver MR slices. The green arrows indicate the motion between the reference and current image. The red circles indicate the uncertainty. The smaller the circle, the more certain is the motion detection.

The data from the motion tracking is used in the motion compensation pipeline to predict a motion state in the near-term future. This prediction can be of temporal and/or of spatial nature.

The upper left image shows an example of the use of the motion model concept in TRANS-FUSIMO: from an acquired observation image, surrogates are automatically extracted. New observations are then performed. The combined information is then used by a motion model that is able to predict the position of the surrogates to a time point in the future. The lower the time to predict, the higher the accuracy.

In summary, three motion model approaches have been implemented in TRANS-FUSMO. A summary of the features of all included improved motion methods methods are shown inthe figure on the left.

The first motion prediction method incorporated into the TTS was a simple linear prediction based method that is based on the most recent observation samples. The linear prediction method does not need any manual setup and thus was a suitable first implementation of the motion prediction model. However, it has problems with the turning points of motion and for large temporal prediction horizons. To remedy this, a history-based motion prediction method was implemented as a second method. The history-based prediction stores all motion observations and performs a full search for the most similar respiratory state in the database. A graphical presentation of the performance of the linear vs. the history based motion prediction model is given in the lower left figure.

The history-based prediction performs better for greater prediction times. Still the method is solely a temporal prediction while the spatial prediction is done using a rigid translational model in both approaches.

As a third method, the ETH motion model was incorporated into the TTS to further improve the motion compensation. The mapping of the motion model to a new patient requires a liver segmentation, thus, the TTS was extended to allow for loading of a liver segmentation file and to map the model to the patient.

The figure shows the workflow applied when using the ETH motion model: Prior to the therapy a static dataset is acquired where the target organ is delineated. This segmentation is then used to individualize the generic motion model which was extracted from multiple 4D MRI datasets. From the partial motion observations from ultrasound or MR tracking, it is now possible to perform a temporal and spatial prediction.

The average motion and prediction errors observed in this study are very similar to that achieved for the whole liver for humans with a population model, where mean (95%) motion of 4.4 (11.9) mm was reduced to 2.0 (4.7) mm. As previously observed, prediction errors increased with distance to the tracked locations. General motion of the pig increased mean prediction errors by about 5%.

Real-time thermometry is one of the most important models in the TTS because it gives feedback about the treatment effect to the physician.

We employ a multi-baseline PRFS approach to compute temperature in the moving scenario: prior to the treatment a baseline library of monitoring images covering several respiratory cycles is acquired. For each monitoring image during treatment, the corresponding baseline library image is selected to calculate phase differences with respect to the unheated situation. The best-fitting library image is determined by computing an image similarity measure on the magnitude image part of a given monitoring image and all library images. The phase part of the library image with the highest similarity value is then subtracted from the current phase image to give the phase differences for temperature calculation. With high image update rates and baseline libraries over several breathing cycles this process becomes computationally expensive. As a remedy, a fast library search algorithm is implemented in TRANS-FUSIMO to improve the speed of the computations.

A comprehensive test suite for focusing methods was developed. Focusing methods from literature and newly developed methods were implemented. All methods were tested and compared to assess whether they might be applicable in our TRANS-FUSIMO treatment system. Both the StraightRay and MultipleRefractedRays method seem to be good candidates for an actual implementation in the treatment system. To assess the performance of the algorithms the numerical simulation was enhanced and used for the verification.

This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 611889