In this article, a single-spectrum-driven binary-class sparse representation target detector (SSBSTD) via target and background dictionary construction (BDC) is proposed. The SSBSTD leans upon the binary-class sparse representation (BSR) model. Due to the fact that a background spectrum usually consists in background samples composed low-dimensional subspace and a target spectrum also consists in target samples composed low-dimensional subspace, only background samples should be used for sparsely representing the test pixel under the target absent hypothesis and the samples from target-only dictionary for target present hypothesis. To alleviate the problem that there are insufficient available target samples in the sparse representation model, this article proposed a predetection method to construct the target dictionary utilizing the given target spectrum. With regard to the BDC, we proposed an approach based on the classification to generate a global over-complete background dictionary. The detection output is composed of the residual difference between the BSR. Extensive experiments were made on four benchmark hyperspectral images and the experimental results indicate that our SSBSTD algorithm demonstrates superior detection performances.
The presence of mixed pixels in the hyperspectral data makes unmixing to be a key step for many applications. Unsupervised unmixing needs to estimate the number of endmembers, their spectral signatures, and their abundances at each pixel. Since both endmember and abundance matrices are unknown, unsupervised unmixing can be considered as a blind source separation problem and can be solved by nonnegative matrix factorization (NMF). However, most of the existing NMF unmixing methods use a least-squares objective function that is sensitive to the noise and outliers. To deal with different types of noises in hyperspectral data, such as the noise in different bands (band noise), the noise in different pixels (pixel noise), and the noise in different elements of hyperspectral data matrix (element noise), we propose three self-paced learning based NMF (SpNMF) unmixing models in this article. The SpNMF models replace the least-squares loss in the standard NMF model with weighted least-squares losses and adopt a self-paced learning (SPL) strategy to learn the weights adaptively. In each iteration of SPL, atoms (bands or pixels or elements) with weight zero are considered as complex atoms and are excluded, while atoms with nonzero weights are considered as easy atoms and are included in the current unmixing model. By gradually enlarging the size of the current model set, SpNMF can select atoms from easy to complex. Usually, noisy or outlying atoms are complex atoms that are excluded from the unmixing model. Thus, SpNMF models are robust to noise and outliers. Experimental results on the simulated and two real hyperspectral data sets demonstrate that our proposed SpNMF methods are more accurate and robust than the existing NMF methods, especially in the case of heavy noise.
Random Gaussian noise and striping artifacts are common phenomena in hyperspectral images (HSI). In this article, an effective restoration method is proposed to simultaneously remove Gaussian noise and stripes by merging a denoising and a destriping submodel. A denoising submodel performs a multiband denoising, i.e., Gaussian noise removal, considering Gaussian noise variations between different bands, to restore the striped HSI from the corrupted image, in which the striped HSI is constrained by a weighted nuclear norm. For the destriping submodel, we propose an adaptive anisotropy total variation method to adaptively smoothen the striped HSI, and we apply, for the first time, the truncated nuclear norm to constrain the rank of the stripes to 1. After merging the above two submodels, an ultimate image restoration model is obtained for both denoising and destriping. To solve the obtained optimization problem, the alternating direction method of multipliers (ADMM) is carefully schemed to perform an alternative and mutually constrained execution of denoising and destriping. Experiments on both synthetic and real data demonstrate the effectiveness and superiority of the proposed approach.
Oblique photogrammetry with multiple cameras onboard unmanned aerial vehicle (UAV) has been widely applied in the construction of photorealistic three-dimensional (3-D) urban models, but how to obtain the optimal building facade texture images (BFTIs) from the abundant oblique images has been a challenging problem. This article presents an optimization method for selection of BFTIs from the image flows acquired by five oblique cameras onboard UAV. The proposed method uses multiobjective functions, which consists of the smallest occlusion of the BFTI and the largest façade texture area, to select the optimal BFTIs. Geometric correction, color equalization, and texture repairment are also considered for correction of BFTI’s distortions, uneven color, and occlusion by other objects such as trees. Visual C++ and OpenGL under the Windows Operating System are used to implement the proposed methods and algorithms. The proposed method is verified using 49 800 oblique images collected by five cameras onboard the Matrice 600 Pro (M600 Pro) UAV system over Dongguan Street, in the City of Ji’nan, Shandong, China. To restore the partially occluded textures, different thresholds and different sizes of windows are experimented, and a template window of $200times200$ pixels2 is recommended. With the proposed method, 2740 BFTIs are extracted from 49 800 oblique images. As compared with the Pix4Dmapper and Smart 3-D method, it can be concluded that the optimal texture can be selected from the image flow acquired by multiple cameras onboard UAV and the approximately 95% memory occupied by the original BFTIs is reduced.
Ship detection in remote sensing plays an important role in civil and military fields. Owing to the complex background and uncertain direction, ship detection is full of challenge by using the commonly used object-detection methods. In this article, a new framework for detecting the arbitrary direction ships is proposed based on the improvement in the Faster region-based convolutional network (R-CNN), in which the shape of the bounding box is described by three sides, namely, vertical side, horizontal side, and short side, respectively. The inclination of the ship is obtained by calculating the arc-tangent value of the vertical side to the horizontal side. First, the better performing ResNet-101 is adopted to extract features over an entire image, which are shared by the region proposal network (RPN) and the head network. Then, the multidirection proposal regions that may contain ships are generated by the RPN. Next, the global and local features of the proposal regions are combined as the whole features of the regions by a multiregion feature-fusion (MFF) module, which can provide more detailed information of the regions. Finally, the head network uses the whole features of the proposal regions for bounding-box recognition through multitask learning, including classification, regression, and incline direction prediction (left or right). The proposed method is tested and compared with other state-of-the-art ship-detection methods on two open remote-sensing data sets and some large-scale and real images. The experimental results validate that the proposed approach has achieved better performance.
Image stitching aims to generate a natural seamless high-resolution panoramic image free of distortions or artifacts as fast as possible. In this article, we propose a new seam cutting strategy based on superpixels for unmanned aerial vehicle (UAV) image stitching. Explicitly, we decompose the issue into three steps: image registration, seam cutting, and image blending. First, we employ adaptive as-natural-as-possible (AANAP) warps for registration, obtaining two aligned images in the same coordinate system. Then, we propose a novel superpixel-based energy function that integrates color difference, gradient difference, and texture complexity information to search a perceptually optimal seam located in continuous areas with high similarity. We apply the graph cut algorithm to solve the problem and thereby conceal artifacts in the overlapping area. Finally, we utilize a superpixel-based color blending approach to eliminate visible seams and achieve natural color transitions. Experimental results demonstrate that our method can effectively and efficiently realize seamless stitching, and is superior to several state-of-the-art methods in UAV image stitching.
As a fundamental and critical task in feature-based remote sensing image registration, feature matching refers to establishing reliable point correspondences from two images of the same scene. In this article, we propose a simple yet efficient method termed linear adaptive filtering (LAF) for both rigid and nonrigid feature matching of remote sensing images and apply it to the image registration task. Our algorithm starts with establishing putative feature correspondences based on local descriptors and then focuses on removing outliers using geometrical consistency priori together with filtering and denoising theory. Specifically, we first grid the correspondence space into several nonoverlapping cells and calculate a typical motion vector for each one. Subsequently, we remove false matches by checking the consistency between each putative match and the typical motion vector in the corresponding cell, which is achieved by a Gaussian kernel convolution operation. By refining the typical motion vector in an iterative manner, we further introduce a progressive strategy based on the coarse-to-fine theory to promote the matching accuracy gradually. In addition, an adaptive parameter setting strategy and posterior probability estimation based on the expectation–maximization algorithm enhance the robustness of our method to different data. Most importantly, our method is quite efficient where the gridding strategy enables it to achieve linear time complexity. Consequently, some sparse point-based tasks may inspire from our method when they are achieved by deep learning techniques. Extensive feature matching and image registration experiments on several remote sensing data sets demonstrate the superiority of our approach over the state of the art.
By studying the spectral reflectance features of different land cover types and leveraging information of primarily “BLUE” band along with “RED” and “NIR” bands, this article seeks to introduce a new built-up index such as powered B1 built-up index (PB1BI). The proposed index, while being conceptually simple and computationally inexpensive, can extract the built-up areas from Landsat7 satellite images efficiently. For Landsat7 satellite imagery, classification performances of the proposed index along with support vector machine (SVM), artificial neural network (ANN), and three existing built-up indices have been examined for three study sites of 1° Latitude $times 1^circ $ Longitude ( $approx 12,100~{mathrm {sq}}cdot {mathrm {km}}$ ) area from three diverse geographical regions in India. The computed value of the M-Statistics for PB1BI is consistently greater than 1.80, indicating a better spectral separability between built-up and nonbuilt-up classes by the index. In order to improve the performance of the built-up indices, this article has suggested a bootstrapping method for threshold estimation in addition to the existing Otsu’s method for the same. It has been found that using bootstrapping method instead of Otsu’s method for threshold estimation has helped to improve the classification performance of built-up indices up to 17.75% and 40.49% in terms of overall accuracy and kappa ( $kappa $ ) coefficient, respectively. It has been observed that for the validation set, average overall accuracy (97.45%) and kappa ( $kappa $ ) coefficient (0.907) of PB1BI for considered study sites are not only significantly higher than existing indices but also comparable with the same of SVM (99.10% and 0.942) and ANN (87.24% and 0.450). This article has also shown that the proposed index provides a stable performance for multitemporal analysis of the study sites and is able to capture growth in built-up region in time horizon. The classification performance of PB1BI has also been verified for Landsat8 imagery across 11 study sites from different continents around the globe, and the results show overall accuracy and $kappa $ to be consistently more than 90% and 0.75, respectively. For considered study sites, the reported values of average accuracy and $kappa $ of PB1BI for built-up classification using Landsat8 satellite data are 95.7151% and 0.8843, respectively.
Deep neural networks, which can learn the representative and discriminative features from data in a hierarchical manner, have achieved state-of-the-art performance in the remote sensing scene classification task. Despite the great success that deep learning algorithms have obtained, their vulnerability toward adversarial examples deserves our special attention. In this article, we systematically analyze the threat of adversarial examples on deep neural networks for remote sensing scene classification. Both targeted and untargeted attacks are performed to generate subtle adversarial perturbations, which are imperceptible to a human observer but may easily fool the deep learning models. Simply adding these perturbations to the original high-resolution remote sensing (HRRS) images, adversarial examples can be generated, and there are only slight differences between the adversarial examples and the original ones. An intriguing discovery in our study shows that most of these adversarial examples may be misclassified into the wrong category by the state-of-the-art deep neural networks with very high confidence. This phenomenon, undoubtedly, may limit the practical deployment of these deep learning models in the safety-critical remote sensing field. To address this problem, the adversarial training strategy is further investigated in this article, which significantly increases the resistibility of deep models toward adversarial examples. Extensive experiments on three benchmark HRRS image data sets demonstrate that while most of the well-known deep neural networks are sensitive to adversarial perturbations, the adversarial training strategy helps to alleviate their vulnerability toward adversarial examples.
Super-resolution (SR) techniques play a crucial role in increasing the spatial resolution of remote sensing data and overcoming the physical limitations of the spaceborne imaging systems. Though the convolutional neural network (CNN)-based methods have obtained good performance, they show limited capacity when coping with large-scale super-resolving tasks. The more complicated spatial distribution of remote sensing data further increases the difficulty in reconstruction. This article develops a dense-sampling super-resolution network (DSSR) to explore the large-scale SR reconstruction of the remote sensing imageries. Specifically, a dense-sampling mechanism, which reuses an upscaler to upsample multiple low-dimension features, is presented to make the network jointly consider multilevel priors when performing reconstruction. A wide feature attention block (WAB), which incorporates the wide activation and attention mechanism, is introduced to enhance the representation ability of the network. In addition, a chain training strategy is proposed to optimize further the performance of the large-scale models by borrowing knowledge from the pretrained small-scale models. Extensive experiments demonstrate the effectiveness of the proposed methods and show that the DSSR outperforms the state-of-the-art models in both quantitative evaluation and visual quality.
Ocean color satellite sensors have become an indispensable component in the Earth Observing System, in which the use of multiple ocean color satellite sensors not only improves the spatiotemporal coverage of the global oceans but also maintains the continuity of the data products for long-term monitoring. In this research, the performance of a new ocean color satellite sensor Chinese Ocean Color and Temperature Scanner (COCTS) from HY1C launched in September 2018 is thoroughly evaluated with two important aspects: the signal-to-noise ratio (SNR) at the top of the atmosphere and the uncertainty in the remote-sensing reflectance Rrs) products. The results showed that the SNR of the COCTS can satisfy the requirements of the ocean color applications, and the uncertainty in the Rrs at the blue bands in the ocean waters meets the demand of less than 5%. A further comparison with other well-known ocean color sensors indicates that not only the COCTS can provide reliable ocean color data but also the processing system is robust and reliable. These results provide a solid base for merging the COCTS products with other ocean color sensors for the studies of ocean biogeochemistry.
Motivated by the development of deep convolution neural networks (DCNNs), aircraft detection has gained tremendous progress. State-of-the-art DCNN-based detectors mainly belong to top-down approaches, which enumerate massive potential locations of aircraft with the form of rectangular regions, and then identify whether they are objects or not. Compared with these top-down detectors, this article shows that aircraft detection via a type of bottom-up method can have better performances in the era of deep learning. In this article, we propose a novel bottom-up detector named X-LineNet. It formulates the aircraft detection task as prediction and clustering of paired intersecting line segments inside each target. Aircraft detection is then a purely appearance-based line segments estimation problem, without any rectangular regions classification or implicit features learning. With simple postprocessing, X-LineNet can simultaneously provide multiple representation forms of the detection result: the horizontal bounding box, the oriented bounding box, and the pentagonal mask. The pentagonal mask is a more accurate representation form of aircraft which has less redundancy than that of a rectangular box. Experiments show that X-LineNet outperforms prevalent top-down and region-based detectors on UCAS-AOD, NWPU VHR-10, and DIOR public data sets in the field of aircraft detection.
Airborne light detection and ranging (LiDAR) data are widely applied in building reconstruction, with studies reporting success in typical buildings. However, the reconstruction of curved buildings remains an open research problem. To this end, we propose a new framework for curved building reconstruction via assembling and deforming geometric primitives. The input LiDAR point clouds are first converted into contours where individual buildings are identified. After recognizing geometric units (primitives) from building contours, we get initial models by matching the basic geometric primitives to these primitives. To polish assembly models, we employ a warping field for model refinements. Specifically, an embedded deformation (ED) graph is constructed via downsampling the initial model. Then, the point to model displacements is minimized by adjusting node parameters in the ED graph based on our objective function. The presented framework is validated on several highly curved buildings collected by various LiDAR in different cities. The experimental results, as well as accuracy comparison, demonstrate the advantage and effectiveness of our method. The new insight attributes to an efficient reconstruction manner. Moreover, we prove that the primitive-based framework significantly reduces the data storage to 10%–20% of classical mesh models.
The 3-D information collected from sample plots is significant for forest inventories. Terrestrial laser scanning (TLS) has been demonstrated to be an effective device in data acquisition of forest plots. Although TLS is able to achieve precise measurements, multiple scans are usually necessary to collect more detailed data, which generally requires more time in scan preparation and field data acquisition. In contrast, mobile laser scanning (MLS) is being increasingly utilized in mapping due to its mobility. However, the geometrical peculiarity of forests introduces challenges. In this article, a test backpack-based MLS system, i.e., backpack laser scanning (BLS), is designed for forest plot mapping without a global navigation satellite system/inertial measurement unit (GNSS-IMU) system. To achieve accurate matching, this article proposes to combine the line and point features for calculating transformation, in which the line feature is derived from trunk skeletons. Then, a scan-to-map matching strategy is proposed for correcting positional drift. Finally, this article evaluates the effectiveness and the mapping accuracy of the proposed method in forest sample plots. The experimental results indicate that the proposed method achieves accurate forest plot mapping using the BLS; meanwhile, compared to the existing methods, the proposed method utilizes the geometric attributes of the trees and reaches a lower mapping error, in which the mean errors and the root square mean errors for the horizontal/vertical direction in plots are less than 3 cm.
Classification of airborne laser scanning (ALS) point clouds is needed in digital cities and 3-D modeling. To efficiently recognize objects in ALS point clouds, we propose a novel hierarchical aggregated deep feature representation method, which can adequately employ spatial association of multilevel structures and deep feature discrimination. In our method, a 3-D deep learning model is constructed to represent the discriminative feature of each point cluster in a hierarchical structure by decreasing the within-class distance and increasing the between-class distance. Our method aggregates the discriminative deep features in different levels into a hierarchical aggregated deep feature that considers the spatial hierarchy and feature distinctiveness. Lastly, we build a multichannel 1-D convolutional neural network to classify the unknown points. Our tests demonstrate that the proposed hierarchical aggregated deep feature method can enhance point cloud classification results. Comparing with seven state-of-the-art methods, those results also verified the superior performance of our method.
In order to improve the accuracy of the International Reference Ionosphere (IRI)-2016 model for application in the Antarctic region, total electron content (TEC) data from the Global Navigation Satellite Systems (GNSS) observation data in 2018 are assimilated into the IRI-2016 model by updating the effective ionospheric parameter, IG12 index on a daily basis. The functional relationship between the IG12 index and the longitude, latitude, and the day of year (DOY) is fitted by using the spherical crown harmonic function and the polynomial, and finally establish an updated experiential model of IG12 indices over the Antarctic region. Conclusions that were reached were: 1) the updated IG12 index varies greatly over different geographical locations and 2) it is also apparent that the accuracy of the IRI-2016 model is worse in the perpetual night than that in the perpetual day. In order to verify our method, the TEC calculated by the IRI-2016 model driven by the updated IG12 index and that calculated by the original IRI-2016 model are compared with the GNSS-TEC, and the results show that the updated IRI-2016 model has improved the accuracy of the BIAS and root mean square (RMS) of the TEC calculation by 97% and 87%, respectively, on the fitting moments, while 75% and 54% on the predicting moments. In addition, compared with the original IRI-2016 model, it is found that the updated IRI-2016 model improves the accuracy of the NmF2 calculation by approximately 23% on average for the fitting time and 8% for the predicting time.
Seismic inversion problems often involve strong nonlinear relationships between model and data so that their misfit functions usually have many local minima. Global optimization methods are well known to be able to find the global minimum without requiring an accurate initial model. However, when the dimensionality of model space becomes large, global optimization methods will converge slow, which seriously hinders their applications in large-dimensional seismic inversion problems. In this article, we propose a new method for large-dimensional seismic inversion based on global optimization and a machine learning technique called autoencoder. Benefiting from the dimensionality reduction characteristics of autoencoder, the proposed method converts the original large-dimensional seismic inversion problem into a low-dimensional one that can be effectively and efficiently solved by global optimization. We apply the proposed method to seismic impedance inversion problems to test its performance. We use a trace-by-trace inversion strategy, and regularization is used to guarantee the lateral continuity of the inverted model. Well-log data with accurate velocity and density are the prerequisite of the inversion strategy to work effectively. Numerical results of both synthetic and field data examples clearly demonstrate that the proposed method can converge faster and yield better inversion results compared with common methods.
With the dramatic growth and complexity of seismic data, manual seismic facies analysis has become a significant challenge. Machine learning and deep learning (DL) models have been widely adopted to assist geophysical interpretations in recent years. Although acceptable results can be obtained, the uninterpretable nature of DL (which also has a nickname “alchemy”) does not improve the geological or geophysical understandings on the relationships between the observations and background sciences. This article proposes a noble interpretable DL model based on 3-D (spatial–spectral) attention maps of seismic facies features. Besides regular data-augmentation techniques, the high-resolution spectral analysis technique is employed to generate multispectral seismic inputs. We propose a trainable soft attention mechanism-based deep dilated convolutional neural network (ADDCNN) to improve the automatic seismic facies analysis. Furthermore, the dilated convolution operation in the ADDCNN generates accurate and high-resolution results in an efficient way. With the attention mechanism, not only the facies-segmentation accuracy is improved but also the subtle relations between the geological depositions and the seismic spectral responses are revealed by the spatial–spectral attention maps. Experiments are conducted, where all major metrics, such as classification accuracy, computational efficiency, and optimization performance, are improved while the model complexity is reduced.
Seismic data are usually contaminated by various noises. Noise suppression plays an important role in seismic processing. In this article, we propose a new denoising method based on the nonlocal weighted robust principal component analysis (RPCA). First, seismic data are divided into many patches and grouped based on the nonlocal similarity. For each group, then, we establish a similar block matrix and set up the objective function of the RPCA. Next, we introduce the iterative log-thresholding algorithm into the augmented Lagrangian method to solve the problem. Furthermore, varying weights are specified to different singular values when minimizing the objective function. Finally, aggregating all recovered matrices can obtain the denoised seismic data. The proposed method considers the nonlocal similarity and adaptively sets weights with local noise variance. It performs well also owing to the superiority of the iterative log-thresholding method. The presented method is assessed using a synthetic seismic section with several crossover events. We also apply this novel approach to a real seismic data, which shows good results. Comparison with other approaches reveals the effectiveness of the proposed approach.
Noise attenuation is a very important step in seismic data processing, which facilitates accurate geologic interpretation. Random noise is one of the main factors that lead to reductions in the signal-to-noise ratio (SNR) of seismic data. It is necessary for seismic data, including complex geological structures, to develop a number of new noise attenuation technologies. In this article, we concern with a new variational regularization method for random noise attenuation of seismic data. Considering that seismic reflection events often have spatially varying directions, we first employ the gradient structure tensor (GST) to estimate the spatially varying dips point by point and propose the structure-oriented directional total generalized variation (DTGV) (SODTGV) functional. Then, we employ the SODTGV as a regularizer to establish an $ell _{2}$ -SODTGV model and develop the primal-dual algorithm for solving this model. Next, the choice of the model parameters is discussed. Finally, the proposed model is applied to restore noisy synthetic and field data to verify the effectiveness of the proposed workflow. For contrastive methods, we select the structure adaptive median filtering (SAMF), anisotropic total variation (ATV), total generalized variation (TGV), DTGV, median filtering, KL transform, SVD transform, and curvelet transform. The synthetic and real seismic data examples indicate that our proposed method can preferably improve the vertical resolution of seismic profiles, enhance the lateral continuity of reflection events, and preserve local geologic features while improving the SNR. Moreover, the proposed regularization method can also be applied to other inverse problems, such as image processing, medical imaging, and remote sensing.
