Underwater Acoustic Research Trends with Machine Learning: Active SONAR Applications

2.1 Active Target Detection and Classification

Traditional active SONAR signal processing can be largely classified into the processes of (1) detecting the signal of interest, (2) identifying (or classifying) the signal of interest, and (3) tracking the signal of interest (Abraham, 2019). In particular, because active SONAR transmits a signal and detects a scattered signal from the target, noise that differs from the transmitted signal is removed before signal detection is performed, by using a matched filter after beamforming. However, because the signal scattered by the rough interface or underwater scatterer is also similar to the sound source signal, it cannot be conveniently removed even using a matched filter. In addition, the clutter signal scattered by the local domain, such as a sea floor outcrop, exhibits a behavior similar to that of the target signal. In this manner, when an active SONAR is in operation, a false alarm can be triggered by a scattered/clutter signal other than the target signal. Consequently, the target detection performance deteriorates.

In the typical target detection process, the target is identified based on a threshold. In particular, the cell-averaging constant false alarm rate (CFAR) is a tool for normalizing the signal considering the noise effect around the target signal of interest and for applying a threshold to increase the target detection performance (Fig. 1) (Abraham, 2019; Abraham and Willett, 2002). In addition, in another technique for improving the performance, a variable that incorporates the variation of the signal-to-noise ratio over time is designed to explore a section in which a scattered signal by the target exists (Abraham and Willett, 2002).

Fig. 1.

Flow diagram of the normalization scheme depicting how the reverberation and background noise power is estimated from leading and lagging windows of auxiliary data. A conventional matched filter detector would compare the normalized data to a threshold to declare detection (Abraham and Willett, 2002).

In the aforementioned active SONAR signal processing, after the detection of the signal of interest, a post-processing technique such as clustering is applied to remove the clutter signals that exhibit a behavior similar to that of the target signal as much as possible. Finally, a SONAR operator manually determines whether the signal is the target or clutter by using the visual (e.g., spectrogram) or auditory (e.g., timbre) information of the signal. In an environment where clutter signals are likely to occur, such as shallow waters, the number of tasks for the SONAR operator increases. To reduce this workload, studies on the application of machine learning to active target signal detection have begun. In particular, the following techniques have been proposed to reduce the amount of computation. First, a target signal candidate group is searched by a conventional target detection method. In addition, a detector that has been trained with the target signal characteristics of active SONAR data is applied to determine whether the detected signal is the target signal (Young and Hines, 2007). However, if the machine learning-based target signal classifier is applied directly to the entire acquired signal rather than being limited to the first detected target signal, the detection and classification steps of the conventional active SONAR signal processing can be integrated into one process (Shin et al., 1997). In this regard, machine learning-based target detection and machine learning-based target classification are used without distinction in this review article.

Gorman and Sejnowski (1988b) conducted the first classification of target signals by applying machine learning in underwater acoustics. In their studies, a metal cylinder and a cylindrical rock were selected as the target and the clutter, respectively. The target signals and clutter-scattered signals were measured according to the aspect angles. In particular, the linear frequency modulation signal was used as the sound source signal in their study. Based on the concept that the frequency variation of a scattered signal over time can be observed with a short-time Fourier transform, a spectral envelop displaying the frequency energy according to the observation time was used as the input value of the neural network. In addition, they experimentally demonstrated that the targets can be detected with a high and stable classification performance (that is, the effect of the initial value of an irregularly generated neural network is marginal) if the scattered signal of various aspect angles is included in the training data and the training is performed using a hidden layer and a neural network with a sufficient number of neurons. Furthermore, Gorman and Sejnowski (1988a) attempted to describe the neural network's classification process for target signals (using weights of a hidden layer and neurons activated thereby) in association with the scattered signal’s characteristics (bandwidth and onset/decay characteristics).

As described above, the machine learning-based target detection technique that applies the classify-before-detect strategy must utilize comprehensive information of the observation signal similar to how a SONAR operator uses various information to determine the target. In particular, to reduce false alarms caused by clutter signals, raw data must be converted to a domain that makes it convenient to distinguish target signals from clutter signals (Fig. 2). In this regard, as a representative example, Shin et al. (1997) observed and analyzed different target signals with respect to the shape or internal structure of the target. Furthermore, features effective for target/clutter signal identification were extracted to improve the target detection performance. These included time variations, frequency spectra, frequency variation over time, and auditory characteristics of the observed signals. Then, features optimized for target signal classification were selected and combined with several machine learning algorithms, including neural networks, to detect the target signals. This method displayed performance superior to that of the conventional method by producing fewer false alarms and higher target signal classification accuracy.

Fig. 2.

Selection of appropriate projection spaces can improve clutter-reduction performance. Stochastic impulsive sources generally enlarge the target and clutter spaces spanned by raw time-domain data. The main goal of projection-space investigation is to represent both classes (i.e., target and clutter) compactly in the transformation space for improved class separation and lower false alarm rate (Shin et al., 1997).

It was mentioned earlier that target detection becomes challenging owing to signals scattered by various underwater clutter. Nevertheless, the SONAR operator is trained to recognize the differences between the scattered target signal and the scattered clutter signal. Based on this, Young and Hines (2007) proposed a target signal detector reflecting the SONAR operator's auditory perception. In particular, acoustic cognitive features (timbre characteristics), which are the main aspects of interest in musical acoustics, were extracted from the signal duration containing the target or clutter signal. That is, the equivalent rectangular bandwidth (ERB) scale is defined based on the human tendency of analyzing the low frequency band in more detail while detecting a sound. In this nonlinear frequency scale, when a gammatone filter is applied to the original signal, the filtered time series signal can be obtained in each frequency band. The time delay between the start (or end) of the echo and the peak of the temporal envelop and the slope of the line joining the start (or end) of the echo and the peak of the temporal envelop obtained through the above process were used as features (Fig. 3(a)). In addition, the energy in each frequency band of the signal can be calculated through the above process, and a perceptual loudness function can be derived through the calculation (Fig. 3(b)). The feature vectors reflecting the auditory perception characteristics of humans were defined by combining the peak value of this function and the corresponding frequency with the previously designed features. Then, to avoid having an excessive number of dimensions, elements effective for target/clutter classification were selected from among the feature vector elements. Furthermore, the feature vector dimension was reduced through principal component analysis, and the resulting vectors were used as final input values. In this manner, the characteristics reflecting human auditory perception characteristics were combined with a simple machine learning algorithm such as a Gaussian classifier to classify target signals such as marine structures. Thereby, the target signals differing from clutter signals were classified with high accuracy.

Fig. 3.

(a) Extraction of sub-band attack time feature values from several channels of the auditory filter bank; the lowest trace exhibits the maximum sub-band attack time (maxSBAT) and the frequency of this filter bank channel corresponds to maxSBAT-F. (b) Extraction of peak loudness frequency (PLF), peak loudness value (PLV), and loudness centroid (LC) feature values from a perceptual loudness function (Young and Hines, 2007).

The performance of the target signal classifier reflecting the timbre characteristics is affected by the wave propagation phenomenon depending on the ocean environment, noise, and time-varying ocean environment. In this regard, Murphy and Hines (2014) tested the proposed target signal classifier for robustness against variations in the ocean environment, using target/clutter signals (Clutter07/Clutter09) measured in different environments. In this case, apart from characteristics of the ocean environment (such as underwater sound speed or noise), the experimental design factors including the transmitted signal for measuring the target/cutter signal, ship route, and object used as the target/clutter were set identically in both of experiments.

In their study, the Gaussian classifier was used as a classifier similarly as in the previous study (Young and Hines, 2007). Furthermore, the data obtained from Clutter07 were used for training. The performance of the classifier can be evaluated through a receiver-operating-characteristic (ROC) curve displaying the relationship between the probability of detection and the probability of false alarm. The target classifier performance is determined by calculating the area under the ROC curve (AUC). The AUC has a value between zero and one. The closer the value is to one, the higher is the target classification performance. A value of 0.91 was obtained when the AUC was calculated by applying the classifier trained with Clutter07 data to data from among Clutter07 not used for training. This demonstrates that the timbre-based target detector displays remarkable classification performance. Furthermore, a classifier trained with Clutter07 data was applied to Clutter09 data to obtain an AUC value of 0.86 (Fig. 4). Although this value is low compared to the result of using test data obtained in an environment identical to that of the training data, it evidently exhibits remarkable classification performance.

Fig. 4.

(a) Scatter plot of testing echoes in the reduced (two-dimensional) feature space. The circular gray target region contains 90% of the target echoes (black dots) and the surrounding white clutter region contains 64% of the clutter echoes (outlined white squares). (b) ROC curves corresponding to the training dataset (solid line) and testing dataset (dashed line). The training AUC value is 0.91 (outstanding discrimination), and the testing AUC value is 0.86 (excellent discrimination). The Bayes-rate operating point (black circle) is shown at (0.23, 0.89) on the training curve and at (0.36, 0.90) on the testing curve (Murphy and Hines, 2014).

Another related study tested the auditory perception capability of humans to classify target/clutter signals and compared this capability with the performance of a timbre-based target signal classifier (Allen et al., 2011). The subjects participating in the target/clutter signal classification experiment were able to distinguish the target signal from the clutter signal while using acoustic signals in all the frequency bands. However, when the experiment was performed with scattered signals that removed the signal component of the low frequency band, the target classification rate was substantially reduced. However, the timbre-based target signal classifier showed similar performance to those of most subjects. In particular, it was observed that unlike humans, the performance of the classifier slightly decreased when the signal bandwidth was limited.

A study that analyzed the probability distribution characteristics of clutter signals that limit the operation of active SONAR and applied them to classification is described as follows. Gelb et al. (2010) distinguished between target and clutter using the distribution characteristics of the matched clutter signal according to space–time. The clutter signals can be divided into bottom-like, compact stationary, and compact nonstationary classes. In this study, the probability distribution of the observed signal according to the above classes was derived by applying K-distribution and generalized Pareto distribution to unthresholded data and thresholded data, respectively. At this time, if the parameters of each distribution are adjusted and fitted to the observed signal, the distribution of parameters depends on the clutter class. Meanwhile, it is possible to directly calculate cumulants from the locally observed signal according to the class. These cumulants are also distributed differently according to the clutter signal class. By combining this with a classifier that estimates a class based on the probability distribution of input features, such as a Gaussian mixture model, it was demonstrated that the clutter signal could be classified according to its class.

2.2 Seafloor Target Detection and Classification

Active SONAR can be used to detect targets on the seafloor or buried in sediment, as well as underwater targets. Early statistical studies on the classification of buried targets using the monostatic SONAR system were performed to develop a target classification technique using signals received from different aspect angles of the target by utilizing one sensor. Runkle et al. (1999a; 1999b) extracted features from the signals scattered from the target using a matching pursuit technique, one of the sparse representation algorithms, and classified five types of objects using a hidden Markov model. In addition, at approximately the same time, the results of applying pattern recognition techniques different from those used in the above study to detect buried targets were also published (Trucco and Pescetto, 2000; Trucco, 2001). They segmented the beamformed signals into partially overlapping frames, projected them into the time-frequency space, and classified the extracted features using a multivariate Gaussian classifier. In particular, they demonstrated that this method performed well even with the data obtained by an active SONAR system operating remotely with a low grazing angle.

Azimi-Sadjadi, et al. (2000) applied wavelet packet decomposition and linear predictive coding to backscattered target/clutter signals to extract features that can conveniently classify target/clutter, and they combined them with a neural network to obtain a classifier that can classify a target signal with high performance. At this time, the target signals comprised a signal scattered from a bullet-shaped metal object and one from a plastic material object with a truncated cone shape. The clutter signals were scattered from four types of clutters: a water-filled drum, an irregularly shaped limestone rock, soft granite, and water-logged logs. These targets and clutters were placed in a water tank environment with a limited size. The target/clutter backscattered signals were measured according to the aspect angle. Then, to generate a realistic target/clutter backscattered signal, preprocessing was applied to remove the interference of signals from multiple paths in a limited tank environment. Subsequently, a simulated reverberation signal was added. Furthermore, in their study, pre-processed observation signals and transmission signals were divided into multiple bands using wavelet packet decomposition. Moreover, cross-correlation of observation signals and transmission signals in each segmented band was performed, and the results were expressed using an autoregressive (AR) model. Here, when high-dimensional feature vectors are generated by combining AR coefficients derived according to each segmented band, features effective for convenient identification of target/clutter signals are extracted using the Fisher discriminant function. In addition, the dimension of the input feature vectors is reduced. Training and test data were sorted according to aspect angle, and performance was evaluated by training a two-layer neural network using the aforementioned feature vectors as inputs. The classifier proposed in their study exhibited performance superior to those of other existing classifiers. In particular, the results obtained by combining the results of backscattered signal classification at adjacent aspect angles showed higher performance improvements over those using the backscattered signals of a single aspect angle.

Azimi-Sadjadi et al. (2002) improved the previously developed target classifier further and proposed an adaptive target classifier that can achieve high performance even when using test data obtained in a different environment from the ocean environment of the training data. Here, the basic classifier used as a non-adaptive classifier is identical to the aforementioned two-layer neural network. The adaptive target classifier proposed in this study transforms the input vector into a vector insensitive to variations in the environment by adding a linear mapping between the input and the hidden layer of the non-adaptive classifier. In this process, the non-adaptive classifier is fixed, and the weight for linear mapping is learned. In particular, the difference between the output values of the adaptive and non-adaptive classifiers is increased using K-nearest neighbors (K-NN) and a 2-D sigmoid cost function. In their study, the effectiveness of the proposed method was verified by using test data having reverberation levels different from those of the training data. The proposed adaptive classifier exhibited higher performance in an environment with a reverberation level higher than that of the existing classifier.

Meanwhile, while developing a target signal classifier, even when the same input vector is used, classification performance may vary according to the machine learning algorithm used and according to the characteristic of the input data. Li et al. (2004) combined the input vectors defined by Azimi-Sadjadi et al. (2000) with multivariate Gaussian, K-NN, probabilistic neural networks, and support vector machines (SVMs) in addition to neural networks. They also compared and analyzed their performances. Among these, the SVM-based target signal classifier exhibited the most stable and remarkable performance. In addition, Yao et al. (2002) compared the results of classification by using the classifier developed by Azimi-Sadjadi et al. (2000) for different data having bandwidths of 40 kHz and 80 kHz. The result was presented in terms of ROC, error locations, and generalization and robustness for the noise. In their study, the result obtained using data with the 80 kHz bandwidth showed higher classification performance. Accordingly, a sub-band fusion technique based on the contribution of each band with subdivision of the broadband data was also proposed.

2.3 AUV and Automatic Target Recognition System

Underwater image information obtained via sensors such as side scan SONAR or synthetic aperture SONAR is generally used when classifying targets on the seafloor, such as mines, using an autonomous underwater vehicle (AUV). Recently, several studies have been presented that produced high-performance, high-resolution image processing and classification results through the application of machine learning for automatic target recognition (Isaacs, 2015; Kriminger et al., 2015; Myers and Fawcett, 2010). However, these methods have a limitation in that real-time processing is challenging. In particular, targets buried in the sediment cannot be classified following these approaches. In this regard, Fischell and Schmidt (2015) proposed a machine learning-based signal processing technique capable of real-time target classification in a system installed on an AUV. In general, when a signal emitted from a fixed sound source is scattered by a target, the scattering intensity depends on the angle between the line joining the sound source and target and that joining the target and receiver (azimuthal angle). Here, the intensity of scattered radiation for a specific target varies only marginally according to the distance between the sound source and receiver or the depth of the receiver. However, it varies substantially according to the azimuthal angle. In addition, the scattering radiation pattern displayed for all the azimuthal angles is highly dependent on the type of target (Fig. 5). Therefore, in their study, an input vector for machine learning was defined by using a bistatic scattering radiation pattern. Furthermore, it was combined with SVM to perform classification into spherical and cylindrical targets. The scattering radiation pattern used for training was normalized using the mean and standard deviation of the scattering intensity and discretized at regular azimuthal angle intervals. In this case, if multiple scattering intensities exist in the same discrete azimuthal angle, the feature value of the corresponding azimuthal angle is calculated as the median value. In this manner, scattering radiation patterns for all the azimuthal angles discretized at specific intervals can be represented using vectors, which can be used as inputs to the SVM. This target classifier was tested according to the discrete interval of the azimuthal angle and the number of data. It was verified that real-time target shape classification is possible using data (BayEx'14) obtained by an AUV in a real underwater environment.

Fig. 5.

Simulated scattering amplitude dependence on angle for spherical and cylindrical targets. is calculated by setting the target at (0, 0) and the source at (−60, 0) such that the source is at 180°. Amplitudes shown here are for multiple receiver depths and ranges to the target. (a) Spherical target. (b) Cylindrical target (Fischell and Schmidt, 2015).

Meanwhile, the target scattering amplitude according to the azimuthal angle depends on the properties of the seafloor, depth of the surface layer of the marine sediments, and depth of burial of the target. Therefore, the performance of the target classifier may deteriorate when the ocean environment and the experimental conditions for obtaining the training data differ from those of the test data. To address this problem, Fischell and Schmidt (2017a) analyzed the performance of the sphere/cylinder target classifier according to the inconsistencies between the ocean environment and the experimental conditions and proposed a method to reduce the performance degradation. In particular, their study reduced the dependence on similar environmental variations by adding a process whereby the mean and variance of the vectorized discrete scattering radiation pattern became zero and one, respectively. In addition, in this study, bistatic scattering tendency according to the seafloor properties, surface depth of marine sediments, and depth of buried targets were calculated using a simulator and utilized as training data for learning. The actual measurement data were used as test data. Then, it was verified that in the training environment with seafloor properties identical to those of the operating environment of the AUV, good classification performance was obtained even under conditions where the depth of the surficial sediment or the degree of buried targets were inconsistent. However, a rapid performance degradation of the classifier was observed when the seafloor properties differed from those in the environment for actual data acquisition. In this case, simulation was performed by varying the depth of the surface layer and that of the buried target, using these as training data to improve the classifier performance. It is considered that these improved results were achieved because various scattering tendencies and conditions were provided as observation information compared to the use of only specific conditions.

In the case of a cylindrical-shaped target, the scattering radiation pattern varies depending on the aspect angle between the axis of the target and the line joining the sound source and target. Fischell and Schmidt (2017b) proposed a method for estimating the aspect of a target from the scattered signal of the target measured using an AUV by utilizing a scattering radiation pattern that depends on the aspect angle. In this technique utilizing SVM regression, the simulator generated normalized scattering radiation pattern according to the target's aspect angle and trained it. Even for the cases when the ocean environment of the simulation did not match the environment of actual test data acquisition, it was verified that the target aspect classifier trained with simulation data derived a specific aspect angle with high accuracy when feature vectors were constructed using a sufficient number of measurement signals.