myProjects

Java volumetric rendering (ray casting) using VTK

Some years ago I worked in medical setting. Job started with my graduate thesis when I won a research grant in the biggest hospital in Turin (1998)… as result of work I wrote (using VTK 4.2.2 and Java 1.4/C) VolumStudio3D a radiology software that :

  • Load DICOM volume
  • Volumetric Rendering
  • Virtual endoscopy
  • Segmentation (extract/isolate parts from volume)
  • Pattern Search (intestinal polyps search)

Sources code of this implementation can be downloaded here:  https://sourceforge.net/projects/volumstudio3d/

About VTK The Visualization Toolkit (click-here) and below few notes…

#1 Images Elaboration

Most code uses Java+Visualization Toolkit (VTK)

#1.a Volumetric rendering

Basically this kind of rendering is based on a technique named “Ray Casting“. Using this technique,  for each 2D pixel on display screen, the rendering engine shoots a beam in order to cross the whole densitometric volume (volume is composed by voxels == 3D pixels).

Volumetric Rendering (ray casting)
Volumetric Rendering (ray casting)

So a ray crosses the whole volume and strikes, in its trajectory, some voxels. The impact of the ray with a voxel changes the value of  the “densityaccumulated on the ray … and… working on this “density” (accumulated on the ray) and a “transfer function” it is possible to show the 2D pixel with the wanted color. The map between the couple (RGBA, “density”)(*) is the “transfer function” and is useful in order to make transparent some tissue and to show solid other (for example bones).
 (*) (RGBA, “density”)   RGBA is the color (RGB) and Alpha of 2D pixel / density is the end value of accumulated density on the ray

 Rendering (using VTK and Java):

Sources code of this implementation can be downloaded here:  https://sourceforge.net/projects/volumstudio3d/

#1.b Navigation and virtual endoscopy

Using the same engine of volumetric rendering  I wrote an extension for virtual endoscopy (still using Java+VTK). Using the “path control panel” (image below) the doctor can trace the camera path and start the navigation. Mouse and keyboard allow the radiologist to adjust the trajectory of navigation. A video can be recorded during the navigation (video below).

EndVirtVirtual Endoscopy
Virtual Endoscopy “path control panel”
Virtual Endoscopy :

Sources code of this implementation can be downloaded here:  https://sourceforge.net/projects/volumstudio3d/

#1.c Segmentation

VTK offers several algorithms of segmentation.  Collaborating  with University of Turin (Italy) I wrote a filling 3D algorithm : the radiologist starts selecting (on the 2D slice of the volume) a point of germination. So initially (t0) the volume is composed just by 1-voxel. The algorithm starts to grow this 1-voxel volume in the region with density ~similar to the starting point (point of germination). The algorithm ends when the volume can not further grow.

#1.d Object recognition

My work in this case was focused on intestinal polyps recognition. Starting from a 3D volume the algorithm was designed in order to search and to mark (with a red box)  intestinal polyps. Algorithm consists in two phases :

Step 1 : Fast descent using intersection of histogram :
goal is the fast exclusion of regions that don’t contain polyps
 algorithm looks for an area with histogram compatible with the histogram of a polyp (i.e. the area could contain polyps)
 then algorithm divides this area into 4 subareas and continue recursively into each subarea
 algorithm stops when a subarea isn’t compatible: in this case it jumps to step 2 on border of subarea

Step 2 : Fine search and tag polyps using feature form
goal is accurate search of polyps
 algorithm looks for an area with shape compatible with a polyp.

Polyps recognition algorithm
Polyps recognition algorithm

Histogram (in case of volume obtained by a DICOM data set) is the distribution of density values: how much is frequent a value of density. The Step 1 has the goal to descent into a subarea if the related histogram of the area shows that there is compatibility with the polyp. The check of compatibility is done using intersection between the histogram of area and the histogram of a polyp. If (intersection) is > of a threshold then this mean that in subarea it’s present a density region that is compatible with a polyp. If (intersection) is < threshold then this mean that we have cut out the polyp and algorithm goes in Step 2. The Step 2 is more accurate search using feature forms.

Below the video related to this algorithm in action (search of intestinal polyps on a 2D radiological image). The moving red rectangle is the area which could contain a polyp. The unmoving red rectangle is what is a polyp for the algorithm. Note the presence of several false positives. The gray rectangle on right is the polyp.

Algorithm : intestinal polyps search 
Powered by WP Review