August 30, 2012
The quest for providing tissue characterization and functional mapping during minimally invasive surgery (MIS) has motivated the development of new surgical tools that extend the current functional capabilities of MIS. Miniaturized biophotonic probes can be inserted into the instrument channel of standard endoscopes to reveal cellular and subcellular microstructures of the tissue, allowing excision-free optical biopsy. One of the limitations of such a point based imaging and tissue characterization technique is the difficulty of tracking biopsied sites in vivo. Since the probe needs to be placed in contact with the tissue when the optical biopsy takes place, tracking the tip of the probe enables the localization of the biopsy site. An image-based approach will be presented for localizing optical probes in endoscopic images without the use of fiducial markers. The colour attributes of the shaft of the probe in the HSV (Hue Saturation Value) space are used to segment the probe in the scene. A novel probabilistic approach has been employed to estimate the orientation of the tangential lines to the shaft of the probe. Perspective image analysis is used to detect the distal tip of the probe. In order to enable large area surveillance and integrated functional mapping, an image-based tracking framework has been proposed based on SLAM (Simultaneous Localisation and Mapping) for optical probes. This will allow for subsequent localization and contextual analysis of microstructures or guiding real tissue biopsy. The SLAM system is combined with probe tracking to create a 3D model of the tissue surface and spatiotemporally tracked biopsy sites. These biopsy sites are subsequently re-projected back onto the image plane to provide a live augmented view in vivo, thus facilitating retargeting and serial examination. The proposed method has been validated on ex-vivo and phantom data with known ground truth and the accuracy derived demonstrates the strength and practical clinical value of the technique. The method facilitates a move from the current point based optical biopsy towards large area multi-scale image integration in a routine clinical environment.