IEEE Transactions on Geoscience and Remote Sensing

The fusion of hyperspectral (HS) and multispectral (MS) images designed to obtain high-resolution HS (HRHS) images is a very challenging work. A series of solutions has been proposed in recent years. However, the similarity in the structure of the HS image has not been fully used. In this article, we present a novel HS and MS image-fusion method based on nonlocal low-rank tensor approximation and sparse representation. Specifically, the HS image and the MS image are considered the spatially and spectrally degraded versions of the HRHS image, respectively. Then, the nonlocal low-rank constraint term is adopted in order to form the nonlocal similarity and the spatial–spectral correlation. Meanwhile, we add the sparse constraint term to describe the sparsity of abundance. Thus, the proposed fusion model is established and its optimization is solved by alternative direction method of multipliers (ADMM). The experimental results on three synthetic data sets and one real data set show the advantages of the proposed method over several state-of-the-art competitors.

Blind hyperspectral unmixing (BHU) is an important technology to decompose the mixed hyperspectral image (HSI), which is actually an ill-posed problem. The ill-posedness of the BHU is deteriorated by nonlinearity, endmember variability (EV) and abnormal points, which are considered as three challenging intractable interferences currently. To sidestep the challenges, we present a novel unmixing model, where a latent multidiscriminative subspace is explored and the inherent self-expressiveness property is employed. The most existing unmixing approaches directly decompose the HSI utilizing original features in an interference corrupted single subspace, unlike them, our model seeks the underlying intrinsic representation and simultaneously reconstructs HSI based on the learned latent subspace. With the help of both clustering homogeneity and intrinsic features selection, structural differences in the HSI and the spectral property of a certain material are exploited perfectly, and an ideal multiheterogeneous subspace is recovered from the heavily contaminated original HSI. Based on the multiheterogeneous subspace, the reconstructed differentiated transition matrix is split into two matrices to avoid the emergence of the artificial endmember. Experiments are conducted on synthetic and four representative real HSI sets, and all the experimental results demonstrate the validity and superiority of our proposed method.

The use of hyperspectral (HS) data is growing over the years, thanks to the very high spectral resolution. However, HS data are still characterized by a spatial resolution that is too low for several applications, thus motivating the design of fusion techniques aimed to sharpen HS images with high spatial resolution data. To reach a significant resolution enhancement, high-resolution images should be acquired by different satellite platforms. In this article, we highlight the pros and cons of employing real multiplatform data, using the EO-1 satellite as an exemplary case. The spatial resolution of the HS data collected by the Hyperion sensor is improved by exploiting both the ALI panchromatic image collected from the same platform and acquisitions from the WorldView-3 and the QuickBird satellites. Furthermore, we tackle the problem of assessing the final quality of the fused product at the nominal resolution, which presents further difficulties in this general environment. Useful indications for the design of an effective sharpening method in this case are finally outlined.

In hyperspectral image (HSI) classification, spatial context has demonstrated its significance in achieving promising performance. However, conventional spatial context-based methods simply assume that spatially neighboring pixels should correspond to the same land-cover class, so they often fail to correctly discover the contextual relations among pixels in complex situations, and thus leading to imperfect classification results on some irregular or inhomogeneous regions such as class boundaries. To address this deficiency, we develop a new HSI classification method based on the recently proposed graph convolutional network (GCN), as it can flexibly encode the relations among arbitrarily structured non-Euclidean data. Different from traditional GCN, there are two novel strategies adopted by our method to further exploit the contextual relations for accurate HSI classification. First, since the receptive field of traditional GCN is often limited to fairly small neighborhood, we proposed to capture long-range contextual relations in HSI by performing successive graph convolutions on a learned region-induced graph which is transformed from the original 2-D image grids. Second, we refine the graph edge weight and the connective relationships among image regions simultaneously by learning the improved similarity measurement and the “edge filter,” so that the graph can be gradually refined to adapt to the representations generated by each graph convolutional layer. Such updated graph will in turn result in faithful region representations, and vice versa. The experiments carried out on four real-world benchmark data sets demonstrate the effectiveness of the proposed method.

Sparse representation-based graph embedding methods have been successfully applied to dimensionality reduction (DR) in recent years. However, these approaches usually become problematic in the presence of the hyperspectral image (HSI) that contains complex nonlinear manifold structure. Inspired by recent progress in manifold learning and hypergraph framework, a novel DR method named local constraint-based sparse manifold hypergraph learning (LC-SMHL) algorithm is proposed to discover the manifold-based sparse structure and the multivariate discriminant sparse relationship of HSI, simultaneously. The proposed method first designs a new sparse representation (SR) model named local constrained sparse manifold coding (LCSMC) by fusing local constraint and manifold reconstruction. Then, two manifold-based sparse hypergraphs are constructed with sparse coefficients and label information. Based on these hypergraphs, LC-SMHL learns an optimal projection for mapping data into low-dimensional space in which embedding features not only discover the manifold structure and sparse relationship of original data but also possess strong discriminant power for HSI classification. Experimental results on three real HSI data sets demonstrate that the proposed LC-SMHL method achieves better performance in comparison with some state-of-the-art DR methods.

Sparse unmixing, as a semisupervised unmixing method, has attracted extensive attention. The process of sparse unmixing involves treating the mixed pixels of hyperspectral imagery as a linear combination of a small number of spectral signatures (endmembers) in a standard spectral library, associated with fractional abundances. Over the past ten years, to achieve a better performance, sparse unmixing algorithms have begun to focus on the spatial information of hyperspectral images. However, less accurate spatial information greatly limits the performance of the spatial-regularization-based sparse unmixing algorithms. In this article, to overcome this limitation and obtain more reliable spatial information, a novel sparse unmixing algorithm named superpixel-based reweighted low-rank and total variation (SUSRLR-TV) is proposed to enhance the performance of the traditional spatial-regularization-based sparse unmixing approaches. In the proposed approach, superpixel segmentation is adopted to consider both the spatial proximity and the spectral similarity. In addition, a low-rank constraint is enforced on the objective function as pixels within each superpixel have the same endmembers and similar abundance values, and they naturally satisfy the low-rank constraint. Differing from the traditional nuclear norm, a reweighted nuclear norm is used to achieve a more efficient and accurate low-rank constraint. Meanwhile, low-rank consideration is also used to enhance the spatial continuity and suppress the effects of random noise. Furthermore, TV regularization is introduced to promote the smoothness of the abundance maps. Experiments on three simulated data sets, as well as a well-known real hyperspectral imagery data set, confirm the superior performance of the proposed method in both the qualitative assessment and the quantitative evaluation, compared with the state-of-the-art sparse unmixing methods.

We propose a novel graph Laplacian-guided coupled tensor decomposition (gLGCTD) model for fusion of hyperspectral image (HSI) and multispectral image (MSI) for spatial and spectral resolution enhancements. The coupled Tucker decomposition is employed to capture the global interdependencies across the different modes to fully exploit the intrinsic global spatial–spectral information. To preserve local characteristics, the complementary submanifold structures embedded in high-resolution (HR)-HSI are encoded by the graph Laplacian regularizations. The global spatial–spectral information captured by the coupled Tucker decomposition and the local submanifold structures are incorporated into a unified framework. The gLGCTD fusion framework is solved by a hybrid framework between the proximal alternating optimization (PAO) and the alternating direction method of multipliers (ADMM). Experimental results on both synthetic and real data sets demonstrate that the gLGCTD fusion method is superior to state-of-the-art fusion methods with a more accurate reconstruction of the HR-HSI.

Remote sensing opens opportunities to assess spatial patterns on ecological data for a wide range of ecosystems. This information can be used to more effectively design sampling strategies for fieldwork, either to capture the maximum spatial dependence related to the ecological data or to completely avoid it. The sampling design and the autocorrelation observed in the field will determine whether there is a need to use a spatial model to predict ecological data accurately. In this article, we show the effects of different sampling designs on predictions of a plant trait, as an example of an ecological variable, using a set of simulated hyperspectral data with an increasing range of spatial autocorrelation. Our findings show that when the sample is designed to estimate population parameters such as mean and variance, a random design is appropriate even where there is strong spatial autocorrelation. However, in remote sensing applications, the aim is usually to predict characteristics of unsampled locations using spectral information. In this case, regular sampling is a more appropriated design. Sampling based on close pairs of points and clustered over a regular design may improve the accuracy of the training model, but this design generalizes poorly. The use of spatially explicit models improves the prediction accuracy significantly in landscapes with strong spatial dependence. However, such models have low generalization capacities to extrapolate to other landscapes with different spatial patterns. When the combination of design and size results in sample distances similar to the range of the spatial dependence in the field, it increases predictions uncertainty.

In this article, we focus on tackling the problem of weakly supervised object detection from high spatial resolution remote sensing images, which aims to learn detectors with only image-level annotations, i.e., without object location information during the training stage. Although promising results have been achieved, most approaches often fail to provide high-quality initial samples and thus are difficult to obtain optimal object detectors. To address this challenge, a dynamic curriculum learning strategy is proposed to progressively learn the object detectors by feeding training images with increasing difficulty that matches current detection ability. To this end, an entropy-based criterion is firstly designed to evaluate the difficulty for localizing objects in images. Then, an initial curriculum that ranks training images in ascending order of difficulty is generated, in which easy images are selected to provide reliable instances for learning object detectors. With the gained stronger detection ability, the subsequent order in the curriculum for retraining detectors is accordingly adjusted by promoting difficult images as easy ones. In such way, the detectors can be well prepared by training on easy images for learning from more difficult ones and thus gradually improve their detection ability more effectively. Moreover, an effective instance-aware focal loss function for detector learning is developed to alleviate the influence of positive instances of bad quality and meanwhile enhance the discriminative information of class-specific hard negative instances. Comprehensive experiments and comparisons with state-of-the-art methods on two publicly available data sets demonstrate the superiority of our proposed method.

Currently, reliable and accurate ship detection in optical remote sensing images is still challenging. Even the state-of-the-art convolutional neural network (CNN)-based methods cannot obtain very satisfactory results. To more accurately locate the ships in diverse orientations, some recent methods conduct the detection via the rotated bounding box. However, it further increases the difficulty of detection because an additional variable of ship orientation must be accurately predicted in the algorithm. In this article, a novel CNN-based ship-detection method is proposed by overcoming some common deficiencies of current CNN-based methods in ship detection. Specifically, to generate rotated region proposals, current methods have to predefine multioriented anchors and predict all unknown variables together in one regression process, limiting the quality of overall prediction. By contrast, we are able to predict the orientation and other variables independently, and yet more effectively, with a novel dual-branch regression network, based on the observation that the ship targets are nearly rotation-invariant in remote sensing images. Next, a shape-adaptive pooling method is proposed to overcome the limitation of a typical regular region of interest (ROI) pooling in extracting the features of the ships with various aspect ratios. Furthermore, we propose to incorporate multilevel features via the spatially variant adaptive pooling. This novel approach, called multilevel adaptive pooling, leads to a compact feature representation more qualified for the simultaneous ship classification and localization. Finally, a detailed ablation study performed on the proposed approaches is provided, along with some useful insights. Experimental results demonstrate the great superiority of the proposed method in ship detection.

Cloud detection is a crucial preprocessing step for optical satellite remote sensing (RS) images. This article focuses on the cloud detection for RS imagery with cloud-snow coexistence and the utilization of the satellite thumbnails that lose considerable amount of high resolution and spectrum information of original RS images to extract cloud mask efficiently. To tackle this problem, we propose a novel cloud detection neural network with an encoder–decoder structure, named CDnetV2, as a series work on cloud detection. Compared with our previous CDnetV1, CDnetV2 contains two novel modules, that is, adaptive feature fusing model (AFFM) and high-level semantic information guidance flows (HSIGFs). AFFM is used to fuse multilevel feature maps by three submodules: channel attention fusion model (CAFM), spatial attention fusion model (SAFM), and channel attention refinement model (CARM). HSIGFs are designed to make feature layers at decoder of CDnetV2 be aware of the locations of the cloud objects. The high-level semantic information of HSIGFs is extracted by a proposed high-level feature fusing model (HFFM). By being equipped with these two proposed key modules, AFFM and HSIGFs, CDnetV2 is able to fully utilize features extracted from encoder layers and yield accurate cloud detection results. Experimental results on the ZY-3 satellite thumbnail data set demonstrate that the proposed CDnetV2 achieves accurate detection accuracy and outperforms several state-of-the-art methods.

Estimation of downward shortwave radiation (DSR) is of great importance in global energy budget and climatic modeling. Although various algorithms have been proposed, effective validation methods are absent for rugged terrains due to the lack of rigorous methodology and reliable field measurements. We propose a two-step validation method for rugged terrains based on computer simulations. The first step is to perform point-to-point validation at local scale. Time-series measurements were applied to evaluate a three-dimensional (3-D) radiative transfer model. The second step is to validate the DSR at pixel-scale. A semiempirical model was built up to interpolate and upscale the DSR. Key terrain parameters were weighted by empirical coefficients retrieved from ground-based observations. The optimum number and locations of ground stations were designed by the 3-D radiative transfer model and Monte Carlo method. Four ground stations were selected to upscale the ground-based observations. Additional three ground stations were set up to validate the interpolated results. The upscaled DSR was finally applied to validate the satellite products provided by MODIS and Himawari-8. The results showed that the modeled and observed DSR exhibited good consistency at point scale with correlation coefficients exceeding 0.995. The average error was around 20 W/m2 for the interpolated DSR and 10 W/m2 for the upscaled DSR in theory. The accuracies of the satellite products were acceptable at most times, with correlation coefficients exceeding 0.94. From an operational point of view, our method has an advantage of using small amount of ground stations to upscale DSR with relatively high accuracy over rugged terrains.

Clouds and cloud shadows heavily affect the quality of the remote sensing images and their application potential. Algorithms have been developed for detecting, removing, and reconstructing the shaded regions with the information from the neighboring pixels or multisource data. In this article, we propose an integrated cloud detection and removal framework using cascade convolutional neural networks, which provides accurate cloud and shadow masks and repaired images. First, a novel fully convolutional network (FCN), embedded with multiscale aggregation and the channel-attention mechanism, is developed for detecting clouds and shadows from a cloudy image. Second, another FCN, with the masks of the detected cloud and shadow, the cloudy image, and a temporal image as the input, is used for the cloud removal and missing-information reconstruction. The reconstruction is realized through a self-training strategy that is designed to learn the mapping between the clean-pixel pairs of the bitemporal images, which bypasses the high demand of manual labels. Experiments showed that our proposed framework can simultaneously detect and remove the clouds and shadows from the images and the detection accuracy surpassed several recent cloud-detection methods; the effects of image restoring outperform the mainstream methods in every indicator by a large margin. The data set used for cloud detection and removal is made open.

Pole-like objects provide important street infra- structure for road inventory and road mapping. In this article, we proposed a novel pole-like object extraction algorithm based on plane filtering from mobile Light Detection and Ranging (LiDAR) data. The proposed approach is composed of two parts. In the first part, a novel octree-based split scheme was proposed to fit initial planes from off-ground points. The results of the plane fitting contribute to the extraction of pole-like objects. In the second part, we proposed a novel method of pole-like object extraction by plane filtering based on local geometric feature restriction and isolation detection. The proposed approach is a new solution for detecting pole-like objects from mobile LiDAR data. The innovation in this article is that we assumed that each of the pole-like objects can be represented by a plane. Thus, the essence of extracting pole-like objects will be converted to plane selecting problem. The proposed method has been tested on three data sets captured from different scenes. The average completeness, correctness, and quality of our approach can reach up to 87.66%, 88.81%, and 79.03%, which is superior to state-of-the-art approaches. The experimental results indicate that our approach can extract pole-like objects robustly and efficiently.

This article is focused on a challenging topic emerging from the registration of point clouds, specifically the registration of dynamic objects with low overlapping ratio. This problem is especially difficult when the static scanner is installed on a floating platform, and the objects it scans are also floating. These issues make most of the automatic registration methods and software solutions invalid. To solve this problem, explicit exploration of the static region is necessary for both the coarse and fine registration steps. Fortunately, determining the corresponding regions can be eased by the intuitive realization that in urban environments, natural objects neither present straight boundaries nor stack vertically. This intuition has guided the authors to develop a robust approach for the detection of static regions using planar structures. Then, silhouettes of the objects are extracted from the planar structures, which assist in the determination of an SE(2) transformation in the horizontal direction by a novel line matching method. The silhouettes also enable identification of the correspondences of planes in the step of fine registration using a variant of the iterative closest point method. Experimental evaluations using point clouds of cargo ships with different sizes and shapes reveal the robustness and efficiency of the proposed method, which gives 100% success and reasonable accuracy in rapid time, suitable for an online system. In addition, the proposed method is evaluated systematically with regard to several practical situations caused by the floating platform, and it demonstrates good robustness to limited scanning time and noise.

Accurate and efficient extraction of road marking plays an important role in road transportation engineering, automotive vision, and automatic driving. In this article, we proposed a dense feature pyramid network (DFPN)-based deep learning model, by considering the particularity and complexity of road marking. The DFPN concatenated its shallow feature channels with deep feature channels so that the shallow feature maps with high resolution and abundant image details can utilize the deep features. Thus, the DFPN can learn hierarchical deep detailed features. The designed deep learning model was trained end to end for road marking instance extraction with mobile laser scanning (MLS) point clouds. Then, we introduced the focal loss function into the optimization of deep learning model in road marking segmentation part, to pay more attention to the hard-classified samples with a large extent of background. In the experiments, our method can achieve better results than state-of-the-art methods on instance segmentation of road markings, which illustrated the advantage of the proposed method.

In the last decades, 3-D object recognition has received significant attention. Particularly, in the presence of clutter and occlusion, 3-D object recognition is a challenging task. In this article, we present an object recognition pipeline to identify the objects from cluttered scenes. A highly descriptive, robust, and computationally efficient local shape descriptor (LSD) is first designed to establish the correspondences between a model point cloud and a scene point cloud. Then, a clustering method, which utilizes the local reference frames (LRFs) of the keypoints, is proposed to select the correct correspondences. Finally, an index is developed to verify the transformation hypotheses. The experiments are conducted to validate the proposed object recognition method. The experimental results demonstrate that the proposed LSD holds high descriptor matching performance and the clustering method can well group the correct correspondences. The index is also very effective to filter the false transformation hypotheses. All these enhance the recognition performance of our method.

Precise extraction of ionospheric total electron content (TEC) observations with high precision is the precondition for establishing high-precision ionospheric TEC models. Nowadays, there are several ways to extract TEC observations, e.g., raw-code method (Raw-C), phase-leveled code method (PL-C), and undifferenced and uncombined precise point positioning method (UD-PPP); however, their accuracy is affected by multipath and noise. Considering the limitations of the three traditional methods, we try directly to use the phase observation based on zero-difference integer ambiguity to extract ionospheric observations, namely, PPP-Fixed method. The main goal of this work is to: 1) deduce the expression of ionospheric observables of these four extraction methods in a mathematical formula, especially the satellite and receiver hardware delays; 2) investigate the performance and precision of ionospheric observables extracted from different algorithms using two validation methods, i.e., the co-location experiment by calculating the single difference for each satellite, and the single-frequency PPP (SF-PPP) test by two co-location stations; and 3) use the short arc experiment to demonstrate the advantages of the PPP-Fixed method. The results show that single-difference mean errors of TEC extracted by PL-C, UD-PPP, and PPP-Fixed are 1.81, 0.59, and 0.15 TEC unit (TECU), respectively, and their corresponding maximum single-difference values are 5.12, 1.68, and 0.43 TECU, respectively. Compared with PL-C, the precision of the TEC observations extracted by the PPP-Fixed method is improved by 91.7%, while it is 67.3% for UD-PPP. The SF-PPP experiment shows that PPP-Fixed is the best among these methods in terms of convergence speed, correction accuracy, and reliability of positioning performance. Moreover, the PPP-Fixed method can achieve high accuracy even when the observed arc is short, e.g., within 40 mi- .

This article presents an adaptive hybrid-tracking (AHT) algorithm designed to process GNSS-R signals with a sufficient coherent component. Coherent GNSS-R signals have the potential to enable high-precision and high-resolution carrier-phase measurements for altimetry, sea-level monitoring, soil-moisture monitoring, flood mapping, snow–water equivalent measurements, and so on. The AHT algorithm incorporates the model inputs typically used in the master–slave open-loop (MS-OL) architecture into a closed-phase lock loop. Raw IF data recorded by the CYGNSS satellites over in-land water, land, and open-ocean surface are used to demonstrate the performance of the AHT. The results show that the AHT algorithm achieves comparable robustness with the MS-OL implementation while maintaining centimeter-level accuracy and excellent carrier-phase continuity that can be achieved with a fine-tuned Kalman filter (KF)-based adaptive closed-loop (ACL) system. Moreover, the AHT is suitable for real-time implementation and is applicable to other radio signals-of-opportunity.

In tunnel construction, to prevent the occurrence of water inrush, the geological conditions of faults and underground rivers must be determined in advance. As a direct detection method of groundwater, magnetic resonance sounding (MRS) has been applied for the advanced detection of water-related hazards in tunnels and mines recently. However, the results of conventional 1-D MRS cannot correctly reflect the spatial distribution characteristics of complicated water-bearing structures. In this article, we propose a measurement scheme using a rotating coil with separated transmitter and receiver loop configuration (SEP) for the 2-D imaging of water-bearing structures, such as faults and conduits. In this scheme, the receiver coil rotates several times, while the transmitter coil, which is separated from the receiver coil by a certain distance, remains stationary. Moreover, all the observed data participate in the inversion to achieve the 2-D magnetic resonance tomography (MRT). Numerical simulations of water-bearing faults are performed, and we compare and analyze the 2-D sensitivity and inversion results of two measurement schemes for a rotating coil, i.e., an overlapping transmitter and receiver loop configuration (OVE) and SEP. The results showed that the imaging results of the SEP are better than those of the OVE because the OVE imaging has a symmetric artifact. Finally, we discuss the influence of the transceiver distance, the resistivity, and the environmental noise on the imaging results. Moreover, the imaging results of the water-bearing conduit at different locations were obtained to validate the effectiveness of the SEP rotating coil measurement scheme.