3.1 First Step: Analysis by Hue Property
To estimate the extent of spilled oil at sea, it is necessary to separate the fraction of oil from seawater. In this study, histograms of 20 images, as shown in
Fig. 4, of distinct oil-spilled patterns were collected and analyzed for a clear separation. The images used in the analysis were color-expressed scenes in a digital format and were regarded as a population of various oil patterns appearing at the accident site.
To obtain the desired characteristics from an image, it is necessary to extract the distinct feature included in the image. It is clear that most viewers can intuitively identify an oil spill from seawater using their eyes. Because human intuition is influenced by the color difference shown in the image, the first step is to select a method that distinguishes the areas using this color difference.
A digital image is a medium that expresses an entire image as a mathematical model, called a color model, by combining the specific colors of each pixel of an image. The typical color models used in image processing include red, green, blue (RGB); hue, saturation, value (HSV); and hue, saturation, lightness (HSL). The RGB model is a numerical representation of the changes in the three major colors, which allows it to be easily understood. However, when the three colors are combined, the results are difficult to predict. Meanwhile, the HSV and HSL models have structures that make it easy for humans to perceive color; thus, they are widely used in image processing. When only the desired color information from the RGB model must be extracted, RGB is converted to either HSV or HSL.
The OpenCV library is used for the computational treatment of image processing techniques. OpenCV is an open-source library for developing computer vision applications (
OpenCV, 2019). It provides numerous functions necessary for image processing; furthermore, it can be seamlessly executed on multiple platforms.
An OpenCV function is utilized to convert the color space read in RGB into the HSV color space. General equations for changing the color space are easily found in the literature or online, as in
Eq. (1).
After the color space conversion, when processing for a single color is required, a three-channel (HSV) image is separated into single channels (H/S/V). Subsequently, the histogram of the separated channel image is obtained, and binarization is performed.
An example of changing the RGB color space to HSV is shown in
Fig. 5, followed by three images after separation into individual channels in
Fig. 6. The histogram for the left hue image of
Fig. 6 is depicted in
Fig. 7. After the minimum and maximum values are obtained from the histogram, the peaks and valleys are determined using the slopes of the neighboring values. To find the peaks or valleys, the histogram is first blurred and curved. To distinguish neighboring peaks, certain parameters are used as criteria. The parameters are determined in the image analysis step according to the image size, that is, the total number of pixels of the image, and the determined parameters are applied in the image processing step.
Table 1 lists the number of peaks in the hue histograms of the 20 sample images.
Even though multiple peaks appear in the color range of seawater, not all of these peaks represent meaningful values. Thus, it is plausible to discard peaks other than the peak with the lowest value within the range. The number of valid peaks adjusted from the number of peaks in
Table 1 is rewritten in
Table 2.
The spilled crude oil initially appears black (#9 in
Fig. 4; the image index is denoted by a mark (#) followed by its number, hereinafter) and becomes emulsified to a reddish color over time. By analysis, the color histogram of the black or red image shows two peaks near both ends and one peak in between. With three peaks, the two valleys between the peaks can be used as the thresholds. Images with two peaks, e.g., #2 and #11, do not show red or black colors, but the seawater and oil can easily be distinguished by different colors when confirmed visually. Histograms with silver or iridescent oil films such as #5, #6, #12, #13, and #18 have either one, four, or more than four peaks, which make the analysis more complicated. The histograms for the selected sample images are illustrated in
Fig. 8.
When the images were binarized using the threshold value obtained by the histogram analysis, the separation between water and oil was possible in the 13 three-peak images (#1, #3, #4, #7, #8, #9, #10, #14, #15, #16, #17, #19, and #20) through two binarizations. However, in the images with other peak numbers, the separation was not successful in whole or in part. The separation failure could be attributed to several reasons, but it can be deduced that an important factor was the fact that the hue characteristic considered in this step was limited to chromatic colors. This implies that other separation methods are needed.
3.2 Second Step: Analysis by Saturation Property
Saturation is used for images whose hue histogram has either a single threshold value or when a valid threshold value cannot be obtained. This is a strategy to separate different regions by utilizing the level of sharpness, which the hue itself cannot express. In other words, through saturation, some achromatic information can be utilized, which can provide a clue for additionally processing regions not separated by the hue.
Saturation histograms were created and analyzed for the seven sample images that did not have valid threshold values in the hue analysis. The results are summarized in
Table 3.
If the saturation histogram is bimodal, then one effective threshold is obtained. In this case, two binary images obtained by the hue and saturation analyses, separately, are synthesized to generate a combined image that attempts to complement the two results. Consequently, valid threshold values could be obtained for images #2, #5, #6, and #11.
Fig. 9, for instance, illustrates the synthesis process for sample #2, which combines the hue image (left) and saturation image (middle) to yield the synthesized result shown on the right.
If a binarized result is obtained from the hue images such as #2 or #11, the oil spill area can be obtained by additionally considering the result of the saturation image. However, when three or more valleys are obtained from the hue images, such as #5 or #6, some regions are still not detected, as shown in
Fig. 10.
In some cases (#5, #6, #12, #13, and #18), hue and saturation analyses were not applicable. Hence, the regions had to be separated using a feature other than the hue or saturation.
3.3 Third Step: Analysis by Lightness Property
A smaller spill or already decomposed oil appears as a thin film that shows silvery or iridescent colors. The silvery film shows little difference from the sea in terms of color, while the rainbow colors produce many different colors simultaneously, rendering it difficult to calculate a valid threshold value with hue and saturation analyses.
In this case, it will be effective to use the lightness information as a new characteristic by focusing on the fact that the oil film reflects more light than the seawater. This is due to the nature of the oil film floating on the water surface. In this study, the separation was performed using the grayscale lightness after the original image was converted to grayscale.
Table 4 lists the number of peaks obtained from the lightness histogram. Because the lightness histogram indicates the increase or decrease in lightness, even if two or more peaks occur, only one binarization is performed using the value located at the lowest valley as a threshold. For example, sample image #6 shows three peaks, that is, two valleys, after the lightness histogram analysis. In this case, one binarization is performed using the lowest threshold value.
Fig. 11 shows the results of binarization using the lightness valley as a threshold. The three-step binarization reveals a reasonable separation. Nevertheless, all of the results are far from ideal. In some regions, water and oil are reversed. The fact that the oil and water areas in #13 are difficult to distinguish even with the naked eye is also a cause of incomplete results.
Because the #12 and #18 images with one peak could not be determined because no valley existed, the binarization was performed using the
Otsu (1979) method. The results are shown in
Fig. 12, and it is difficult to infer the oil spill area from the binarization result. In the #12 image, the binarization result becomes meaningless because the spilled oil and highlight of the reflected light are mixed for the binarization. In this study, it was considered impossible to automatically separate images such as #12 and #18. It is more logical for the user to perform the separation semi-automatically or manually.