Abstract

Background: There are approximately 5.4M Basal Cell (BCC) and Squamous Cell Carcinomas (SCC) diagnosed in the US with the numbers expanding each year. Currently, the gold standard for skin cancer diagnosis is histopathology, which requires the surgical excision of the tumor followed by pathological evaluation of a tissue biopsy. The Three-Dimensional (3D) nature of human tissue suggests that Two-Dimensional (2D) cross-sections may be insufficient in some cases to represent the complex structure due to sampling bias. There is a need for new techniques that can be used to classify skin lesion types and margins noninvasively.

Methods: We use optical coherence tomography volume scan images and Artificial Intelligence (AI) to noninvasively create 3D images of basal cell and squamous cell carcinomas.

Results: 3D optical coherence tomography images can be broken down into a series of cross sections that can be classified as benign or cancerous using convolutional neural network models developed in this study. These models can identify cancerous regions as well as clear edges. Cancerous regions can also be verified based on visual review of the color-coded images and the loss of the green and blue subchannel pixel intensities.

Conclusion: 3D optical coherence tomography cross-sections of cancerous lesions can be collected noninvasively and using AI the skin lesions can be classified as well the clear edges. These images may provide a means to speed up treatment and promote better screening of cancer patients, especially for older patients who will likely develop more than one cancerous lesion as they age.

Keywords: Skin Cancer; Basal Cell Carcinoma; Squamous Cell Carcinoma; Melanoma; Optical Coherence Tomography; 3D Image; Fibrotic Tissue; Cancer Associated Fibroblasts; Thin Blood Vessels; Intermediate Filaments; AI

Introduction

Melanomas are pigmented skin cancers that are increasingly being diagnosed each year throughout the world. The American Cancer Society estimates that nearly 105,000 new cases of invasive melanoma of the skin and over 107,000 new cases of melanoma in-situ will be diagnosed in the United States in 2025 [1]. Most melanomas are asymmetrical brown spots or nodular bumps greater than 6 mm in diameter with irregular pigmentation. Lesions with distinctive colors such as white, red, slate gray, pink, black or blue are commonly seen [2]. A variety of dermoscopic features are associated with melanomas including irregular pigment distribution, pigmented finger-like projections arranged asymmetrically, white streaks, blue veils and atypical linear or highly convoluted blood vessels [3]. There are many different histologic variants of melanoma and pigmented lesions that are similar when viewed with a dermoscope. No specific stain or special technique alone can be used to distinguish between a benign nevus and melanoma [4]. Increased Breslow’s thickness and the depth of lesion penetration into the dermis remains the most important factors in gaging the severity of melanomas [4].

One study reported that use of noninvasive devices provides high sensitivity when used to detect melanoma [5]. The FotoFinder Moleanalyzer Pro is reported to have a sensitivity and specificity like that of expert dermatologists [5]. Classification of dermoscopic skin lesion images based on Convolutional Neural Network models (CNNs) is reported to work well in predicting melanoma [6]. In comparison, in another study, diagnostic performance of artificial intelligence for histologic melanoma recognition was compared to the diagnosis by expert pathologists. It was concluded that the diagnosis was confounded by the disagreement among pathologists as to the presence of melanoma [7]. In that study only half (47%) of the melanoma diagnoses were unanimously confirmed by all 18 pathologists showing the difficulty in routine diagnosis of this cancer [7].

The diagnosis, interpretation and classification of melanocytic tumors have become more complicated considering that several other melanocyte lesions can be confused with melanoma. In addition, different names have been given to associated melanocytic lesions described in the literature [8]. One of these lesions is known as a Dysplastic Nevus and is abbreviated as DN. This is a nevus with architectural disorder and atypical melanocytes [8]. Despite its name, DN is not considered as a true dysplastic precursor or premalignant lesion [8]. DN cytological atypia is characterized by enlargement of nuclei (with varying degrees of irregularity), chromatin clumping and hyperchromatism and variably prominent nucleoli [8]. A lack of standardized diagnostic terminology makes it challenging to gauge clinical behavior and guide treatment recommendations [9]. It is thought that most lesions currently reported as atypical melanocytic proliferations may be benign and that their significance lies in the potential pitfall for overdiagnosis of malignant melanoma [10]. The literature regarding atypical melanocytic proliferations highlights the potential pitfalls, such as diagnostic disagreement, institutional variations in nomenclature, uncertain biological potential, lack of histologic criteria and unclear management recommendations [11]. A lack of standardized diagnostic terminology makes it challenging to gauge clinical behavior and guide treatment [12].

Benign melanocytic nevi are the proliferation of uniform melanocytes which occur on the skin after birth. They are initially located at the intradermal-epidermal junction and over time tend to migrate into the dermis [13]. Compound nevi identification by dermoscopy do not always correlate well with the histopathological diagnosis. Most clinically diagnosed compound nevi are found intradermally on histopathology [13].

The purpose of this paper is to conduct a pilot study to evaluate the use of Optical Coherence Tomography (OCT) along with AI to classify the differences between melanomas and other melanocytic skin lesions. The results of this pilot study suggest that further studies are warranted to evaluate the use of OCT and AI to differentiate melanoma from benign melanocytic lesions.

Methodology

An OptoScope was used in this study to collect OCT images. It consists of a modified Lumedica OQ 2.0 OCT (Lumedica Inc, Durham, NC) as described previously operating at a wavelength of 840 nm collecting 13,000 frames per second. OCT gray-scale images were used to classify both normal skin and melanocytic lesions. The images were collected on intact normal skin and biopsied pigmented lesions as described previously [14,15]. The pigmented lesions were studied immediately after a biopsy was taken and then processed and stained with H&E for histopathology. All images were collected as part of an IRB approved clinical study on skin cancers at Summit Health (Berkeley Heights, NJ) after obtaining informed consent from the patients. Clinical diagnoses were made by more than one board-certified dermatopathologist after H&E staining and review of the tissue sections as part of routine clinical skin excisional protocols. Measurements were made on OCT images of control skin (N=200), melanomas (N=106) and 2 atypical melanocytic lesions, a dysplastic nevus and a typical benign nevus. The OptoScope includes a handpiece that collects reflected red light from the skin that is connected to an OCT that operates using an I5 computer. All OCT images were created by horizontally scanning the cancerous lesion cross sections. The gray scale-scans were color-coded using image J as reported previously [14,15]. The OCT gray-scale pixel images were broken into green, blue and red subchannel images using a lookup table [14,15]. By breaking up the pixel intensity distribution at each point into low (green), medium (blue) and high (red) intensities, it is possible to examine differences in reflection of the different layers of skin and skin lesions. The clinical evaluation of the skin required only 2 minutes for analysis of each lesion. Images collected on skin lesions were compared to images of normal skin using a Convolutional Neural Network (CNN). 80% of the data was used for model training and 20% for testing. The sensitivity, specificity and Area Under the Operating Curve (AUOC) were calculated for CNN models developed for each cancer type studied and are listed in Table 1. The sensitivity and specificity of these models ranged between 92 and 100% as reported previously [14,15]. All samples evaluated in this study were not used in training the CNN model.

In this study, a transfer learning-based Convolutional Neural Network (CNN) using the ResNet18 architecture was implemented to classify OCT images of normal skin versus melanoma. The gray-scale images were converted into three-channel inputs to match the ResNet18 requirements and data augmentation techniques such as random horizontal flipping, rotation and resizing were applied to enhance generalization, while normalization followed ImageNet standards. Using a 5-fold cross-validation strategy, to ensure robust performance evaluation, the final classification layer of ResNet18 was modified to output a single node for binary classification. The network was trained using the Adam optimizer with Binary Cross-Entropy loss over multiple epochs, with the best-performing model saved based on validation accuracy. To provide interpretability, Gradient-weighted Class Activation Mapping (Grad-CAM) was employed to highlight the discriminative regions used by the model when distinguishing skin lesions from normal skin. Finally, model performance was assessed through test accuracy, confusion matrices, sensitivity, specificity and ROC-AUOC analysis, offering both predictive power and clinical relevance. Each unknown lesion was compared to normal skin to calculate a probability that the sample was cancerous or similar to normal skin.

Results

Fig. 1 shows a typical color-coded OCT image of normal skin. Note the stratum corneum in yellow marked by the yellow arrow in A, the granulating layers in pink and red (see arrow) and the papillary and basal cell layers shown in blue area. In the green subchannel layer shown in B the stratum corneum is seen while in the blue subchannel image the hyporeflective layer is shown by a yellow arrow.

Figure 1: Color-coded OCT image of normal skin (A) and the green (B), blue (C) and red (D) sub-channels images. The low (B), medium (C) and high (D) pixel intensities were determined from a look-up table. Note the stratum corneum in yellow marked by the meddle yellow arrow in A, the granulating layers in pink  (middle arrow) and red and the papillary and basal cell layers shown in blue (lowest arrow). In the green subchannel layer shown in B the stratum corneum is seen while in the blue subchannel image the hyporeflective layer is shown by an arrow.

Fig. 2 shows a typical color-coded OCT image (A) as well as green (B), blue (C) and red subchannel images (D) of a lesion diagnosed by a board-certified dermatopathologist as a melanoma. Note the rough surface of the melanoma due to the presence of lesions marked with an arrow in A. The green subchannel disappears at the location of the lesion (see B) which is marked by the arrow. The hyporeflective layer is absent possibly due to mutation of the keratin of the intermediate filaments (see arow in C).

Figure 2: Typical color-coded (A) and subchannel green (B), blue (C) and red (D) images of a melanoma diagnosed by board-certified dermatopathologists based on histopathology. Arrows indicate the location of the lesion in the different images.

Fig. 3 shows the results of a Convolutional Neural Network (CNN) model comparing gray scale OCT images of melanomas to normal skin. The number of samples tested are shown in the boxes. The sensitivity and specificity of the CNN model is highly effective in distinguishing melanoma from normal skin as shown in Table 1.

Figure 3: Convolutional neural network results distinguishing melanoma (1) from normal skin (0). Note the sensitivity of distinguishing melanoma from normal is 100% while the specificity is 100%. Table 1 lists the results from the CNN model distinguishing melanoma from normal skin.

Figure 4: Typical color-coded and subchannel images of an atypical melanocytic lesion diagnosed by a board-certified dermatopathologist. The arrows point to areas where there are groups of cells possibly melanocytes.

Fig. 5 is a color-coded subchannel image of a dysplastic nevus. Note the large proliferation of cells at the lesion surface in A and in B where the green cellular layer is quite thick there is thick hyporeflective region in C.

Figure 5: Typical color-coded and subchannel images of a dysplastic melanocytic lesion diagnosed by a board-certified dermatopathologist. Note the cellular proliferation in B demonstrated by the thick green band and the large hyporeflective region in C (black region between top layer of epidermis and the papillary dermis) of a benign lesion. The probability of this being a melanoma is given in Table 2.

Fig. 5 is a typical color-coded and subchannel images of a dysplastic melanocytic lesion diagnosed by a board-certified dermatopathologist. Note the cellular proliferation in B and the large hyporeflective region in C representative (black region) of a benign lesion. The probability of this being a melanoma is only 14.3% as given in Table 2.

Fig. 6 is a typical color-coded image of a nevus (A) and subchannel images including green (B), blue (C) and red (D). Note the rete ridges standout at the base of the epidermis compared to normal skin (Fig. 1) (see arrow in B and C) compared to normal nevus. Areas with enhanced melanin particles tend to scatter light deeper into the sample due to Mie scattering.

Figure 6: Typical color-coded image of a nevus (A) and including green (B), blue (C) and red (D) subchannel images. Note the rete ridges stand out compared to normal skin (Fig. 1) in the nevus. Areas with enhanced melanin particles tend to scatter light deeper into the sample due to Mie scattering therefore preventing back scattering of the infrared light.  This is why the green channel is very faint.

Figure 7: A melanoma with areas of normal skin and a cancerous lesion. This is the lesion shown in Fig. 6. A split screen with normal skin on the left and melanoma on the right.

Table 3 shows the data on the probability that the lesion is a melanoma is increased by dividing it into two parts based on the OCT image.