The user can select any number of medical images in DICOM format from a group of thumbnails to process. These thumbnails will appear in a resizable area at the bottom of the screen as shown in Fig. (1).

Fig. (1). DoctorEye: Snapshot of the main window. The user is able to easily navigate through the available slides and select the one to process.

At this point, the user can also adjust the level and width values by using a system of axes. Two options are also available here. The first allows the user to zoom in or out and the second to shift the position of the selected image on the screen in order to have a better view of the regions of interest.

The user can activate the “Annotations” panel to delineate either areas or contours of potential tumors in the selected medical image. These areas can be annotated, categorized, deleted, added and redefined as depicted in Fig. (2).

Fig. (2). A single slide is selected for further processing. A working area has been set (green rectangle) and the segmentation algorithm has been applied. The selected area is labeled in pink.

To start the processing, the user must first press the “add” button in the annotations panel to create a new label. This label can be added to a single slide or to a group of slides automatically by choosing those from a window with the thumbnailed medical images.

The segmentation process starts with the first image of the selected group of images and at this point the user has to choose a segmentation algorithm. The user can then define a “working area” where the segmentation method will be applied (Fig. 2). At the end of this process the above mentioned label will correspond to the area which will be identified by the algorithm. The user may also specify the color of the selected region (red being the default color) in order to define two or more areas in the same region of the image.

Two segmentation methods are currently available in the platform as discussed in section II-B.

Finally, contour refinement is possible by using the “eraser” and “pencil” tools, where any mistakenly selected or ignored pixel can be removed or added, respectively, in order to end up with the best possible delineation of the identified tumor.

The label that had been given at the beginning of the process is now available in the “annotations” panel corresponding to this specified tumor. It can be selected and the specified tumor will be displayed.

The platform also gives the ability to select and delineate multiple tumors (i.e. in the case of metastasis) with different labels and colors, either in the same or any other slide as shown in Fig. (3).

Fig. (3). Different areas of interest may be annotated accordingly using respective labels in different colors. This can be done in the same slide in the case of metastasis. (Blue and Gold depict areas which are easy to segment while Red and Green depict more complex areas).

The “save” option in the main menu saves the selected areas in the corresponding medical images the user has been working on. The stored information is displayed automatically and can be accessed when pressing “open” a new image or a sequence of images.

The “Save as’ option in the main menu provides the following options to save your selections (annotations) alternatively as:

1. “Annotation” (store the selected areas of your annotations in a .txt file)
2. “Annotation’s Outline” (store the contour of your annotations in a .txt file)
3. “Xml file” (presents the area of your annotations in a .xml file)
4. The 3^rd party application” option produces 3 files:
   1. The “Size.dat” which includes the size (number of rows x number of columns) of the MRI Images and the total number of the slices contained in the specific MRI dataset (ex. 512,512,22 means that the dimension of the images are 512 rows x 512 columns x 22 slices).
   2. The “TumorPoints.dat” which contains data in the form of “x,y,z,concentration”. Here x,y are the coordinates for a specific point in the tumor in a slice while z indicates the number of the slice where this point is located. The concentration value is currently set to 200 until a new method of getting the correct concentration value is developed.
   3. The “Volume.dat” which includes all the points of the tumor, starting from the first point in the first slice and finishing in the last point in the last slice in a sequence (as a volume), showing the gray-level value of every point (takes values from 0 to 255)

The function of the Magic Wand Algorithm is based on finding and selecting all the pixels around a pre-specified user-selected initial point that are similar in gray intensity. A tolerance value can also be set by the user to determine how closely to match colors (higher tolerance ends up in a larger selection) [9].

All the points selected by the algorithm are automatically stored in an image mask of the same size as the original image. Each pixel in the image mask represents the position of the selected pixels in the original image and is used to label the delineated tumors.

To enhance its effectiveness, we have implemented a faster version of this algorithm which excludes all the pixels that have already been examined ensuring that the algorithm does not check them again.

Apart from being able to work with the original image(s), another integrated option allows the user to work with the image displayed on the screen after the level and the width values have been adjusted by the user, in order to facilitate the image processing task.

The Spatially Adaptive Active Contour technique developed by our group [10], is an extension on the traditional active contours, or snakes, which have been widely used in image processing for segmenting image entities and delineate object boundaries. The snake algorithm, first introduced by Kass et al. [11], is a semi-automatic method, based on the deformation of an initial contour towards the boundary of the desired object, which is accomplished by trying to minimize an energy functional, designed so that its minimum is obtained at the desired boundary. The snake evolution can be driven by adjusting four constant and global parameters, which control the curve’s smoothness and continuity, and the force that pushes the snake to expand.

In the platform described in this paper we integrated an improved version of the traditional active contours, which we call the Spatially Adaptive Active Contours¹. This algorithm is based on the discrimination of image regions according to underlying characteristics, such as gradient magnitude and corner strength, and the assignation to each region with a different “localized” set of parameters, one corresponding to a very flexible snake, and the other corresponding to a very rigid one, according to the local image features. Therefore, the snake exhibits different behavior within image regions with diverse characteristics, thus, providing more accurate boundary delineation.

This segmentation technique is semi-automatic, which means that the user only needs to place an initial contour inside the desired object (tumor area or organ) and let it evolve towards its boundaries. It has been extensively tested on real data of nephroblastoma cases and seems to follow very satisfyingly the clinical expert’s intuition, concerning the true tumor boundaries. A theoretical background on the traditional snakes follows, as well as the details of the approach used.

A traditional snake is a curve that evolves in time, in order to adapt to nearby edges, through the minimization of a suitably formulated energy function. In continuous space, a snake is represented as:

(1)

where S ∈ [0, 1] is the normalized length around the snake. In practice, a snake is a set of x and y coordinates of its N points and the parameter S ∈ [0, N] is the actual index of the contour points.

The total energy functional of the traditional snake, which will be minimized, has the following form:

(2)

where E_internal, E_image and E_internal denote the internal energy of the curve, the image energy and the energy produced by external constraints, respectively.

The internal contour energy is defined as:

(3)

where and denote the first and second derivative of v (s) with respect to s. The parameters α and β are weighting parameters that control snake’s elasticity and curvature, respectively: low values of α allow the snake points to be unevenly distributed along the contour, whereas higher values force the snake to retain evenly spaced contour points. Correspondingly, low values of β allow the snake to become second-order discontinuous and develop corners along its perimeter, whereas high values predispose the snake to smooth contours.

The image energy term, E_image depends on the gradient of the image and is associated with the forces that pull the snake towards the desired image boundaries. Given a gray-level image I(x,y), it is formed as:

(4)

where G_σ is generally a pre-processing image filter that is convolved with image I in order to improve image characteristics, and ∇ denotes the gradient operator. The parameter k controls the extent of the influence of that term to the total energy.

The balloon energy G_baloon is produced by a pressure force that pushes the snake to specific direction, either to expand, or shrink. This force is defined as:

(5)

where is the unit vector, normal to the curve, at point v(s), and f is the amplitude of this force. The balloon force is used in order to overcome non-important edges that would cause the snake to stop in local minima.

According to variational calculus [12], a snake that minimizes its total energy E_smoke, as defined in (2), must satisfy the Euler equation:

(6)

In order to solve the above equation, we treat v as a function of both time t and space s and by introducing a gradient descent scheme for the partial derivative of v with respect to t, we obtain the resulting equation:

(7)

At equilibrium v_t = 0 and we reach a solution of (6). Equation (7) can be solved iteratively by using the finite difference method for the discretization of the function derivatives, as described in [11].

In discrete space, the snake is actually a set of x and y coordinates of its N points and the parameter s ∈ [1, N]is the actual index of the contour points. The number of the snake points, N, depends directly on the resolution selected for the snake contour, that is the distance between two consecutive snake points. Normally, a distance of two pixels is quite satisfying (this resolution used in this implementation). However, since the snake is constantly expanding, if the number of snake points was kept constant, then the distance between them would grow larger during each iteration. In order to keep the resolution steady, we re-parameterise the snake every five iterations and add new points so that the distance between them is always the same. This way, the number of snake points is growing larger, while its resolution remains constant.

As was explained in the previous section, the snake’s behavior during its evolution is controlled by four parameters.

By adjusting those parameters, traditional snakes can yield satisfying results in cases where the boundary of the desired object is quite distinct and its inner region homogeneous.

However, in medical images, the regions of interest are often inhomogeneous and include several small areas with different intensity levels than the original organ or tumor, thus segmentation becomes a difficult task.

In such cases, if the snake parameters are adjusted so that extreme curve bending is allowed, the snake will be ‘caught’ up in local edge maxima, produced by small internal high-contrast regions that lie far away from the correct border.

On the contrary, if the parameters are adjusted so that we obtain a very rigid snake, local inhomogeneities of the outline would not be detected, leading to poor boundary definition.

In order to overcome these difficulties, we integrated on the platform the spatially adaptive active contours, which are based on dividing the image pixels into two different groups and assigning a different parameter set to each one, thus allowing the snake to topologically adapt its behavior according to the characteristics of each pixel region.

For the efficient determination of those image regions, a binary mask is produced, where the white pixels correspond to the regions where we wish the snake to be flexible, and the black pixels indicate the regions where we want the snake to be rigid. The extraction of this binary mask is based on gradient and corner features.

The overall approach involves the following major steps, which are shown in Fig. (4):

Fig. (4). A flowchart, illustrating the steps of the Spatially Adaptive Active Contours.

1. Anisotropic diffusion filtering: First, a pre-processing step of the input image is incorporated, using an edge-preserving anisotropic diffusion filter. This filter was first formulated by Perona and Malik [13] as a diffusion process that encourages intraregional smoothing, while inhibiting interregional smoothing. This way, the image appearance is simplified and small image artifacts are smoothed, while structures of interest, such as edges, are enhanced.
2. Employment of gradient information. We obtain gradient information, by applying the Sobel operator [14] to the image. Fig. (5b) shows the resulting image representing the strength of the existing edges on the original image. Considering that strong edges are more dominant along the boundary of the object to be segmented, compared to the region that this boundary encompasses, a threshold is imposed on the gradient image, in order for the boundary to be coarsely detected and discriminated from the internal region. Therefore, an initial binary mask is constructed (Fig. 5c) where white pixels correspond to sharp edges, whereas black ones correspond to homogeneous regions.
3. Employment of corner information. In this step, the Harris corner detector [15] is used to detect high curvature points in the image (Fig. 5d). Morphological dilation is performed on the output of Harris detector (Fig. 5e), thus small circular areas are obtained, which represent the neighboring regions around the detected corners. The resulting binary mask is shown in Fig. (5f).
4. Construction of the final mask. In order to combine those two masks produced so far, a logical OR operator is applied on them, as shown in Fig. (6). Then, the 8-connected components of the binary mask are extracted, and the components with small area are discarded and a final dilation of the resulting mask leads to the final mask, shown in the bottom image of Fig. (6).
5. Extraction of the parameters matrices. Pixels of the white region will be assigned with low α and β, and high k (which results in a very flexible snake), as well as very low f (which lets the snake be attracted by the boundary rather than be pushed to it). The pixels of the black region will be assigned with high α and β compared to k, and a very high value for the force f. This will push the snake to evolve towards the object boundary and when approaching it, it will slow down and fit correctly to the border details.
6. Iterative snake deformation. The snake deformation takes place, as in the case of traditional snakes, but instead of using constant parameter values, each point of the snake has its own value, which can be attained by a look-up table process. This way, the user only needs to define the initial contour, by dragging an ellipse inside the desired object, whereas no parameter adjustment is necessary, since the parameters are predefined by the algorithm. Thus, minimum user interaction is accomplished. The stopping criterion for the iterative procedure in this implementation is the length of the contour: if the snake has the same length (that is, the same number of points) for more than 10 iterations, it is assumed that a steady solution has been reached and the procedure ends.

Fig. (7) shows an example of the application of both traditional active contours and the spatially adaptive active contour on an MR brain image.

Fig. (5). Extraction of the initial binary masks, using gradient and corner information: (a) Original image, (b) Gradient image using the Sobel operator, (c) Thresholded gradient image, resulting into a binary mask, (d) Detected corners using Harris corner detector, represented as green points, (e) Dilated corner neighboring regions, represented as green disks, (f) The binary corner regions.

Fig. (6). The two binary masks that have been constructed using gradient and corner information, are combined by applying a logical OR operator and then the small connected components are discarded.

Fig. (7). (a) Result of the traditional active contours on an MR brain image, (b) Result of the application of the application of the spatially adaptive active contours on the same image.

It is obvious that the improved version of the snakes can reach the true tumor boundary and at the same time, attain its detailed outline, while the traditional snake is caught up on the interior high-contrast inhomogeneities of the tumor.