Modern spaceborne synthetic aperture radar (SAR) features a capacity of multiple imaging modes. It comes with Earth-observation data archives consisting of SAR images acquired in various modes with different resolutions and coverage. In this context, in addition to using single-mode images for SAR interferometry (InSAR), exploiting images acquired in different imaging modes for InSAR can provide extra interferograms and, thus, favors the retrieval of interferometric information. The interferometric processing of multimode image pairs requires special considerations due to significant variations in the Doppler spectra. Conventionally, the InSAR technique only uses the spectral band common in both master and slave images, and the remaining band is discarded before interferogram formation. Therefore, conventional processing cannot make full use of the observed data, and the interferogram quality is limited by the common band spectra. In this article, by exploiting the conventionally discarded spectrum, we present a new interferometric phase retrieval method for multimode InSAR data to improve interferogram quality. To this end, first, we propose a linear model to characterize the interferometric phase of a multimode image pair based on image spectral relation. Second, we adopt a sparse recovery method to inverse the linear model for the retrieval of the interferometric phase. Finally, we present real-data experiments on TerraSAR-X staring spotlight to sliding spotlight interferometry and Sentinel-1 strip map to Terrain Observation by Progressive Scan (TOPS) interferometry to test the proposed method. The experiment results show that the proposed method can provide interferograms with reduced phase noise and defocusing effect for multimode InSAR.
Low-earth-orbit (LEO) synthetic aperture radar (SAR) can achieve advanced remote sensing applications benefiting from the large beam coverage and long duration time of interested area provided by a geosynchronous (GEO) SAR illuminator. In addition, the receiving LEO SAR system is also cost-effective because the transmitting module can be omitted. In this article, an imaging method for GEO-LEO bistatic SAR (BiSAR) is proposed. First, the propagation delay characteristics of GEO-LEO BiSAR are studied. It is found that the traditional “stop-and-go” propagation delay assumption is not appropriate due to the long transmitting path and high speed of the LEO SAR receiver. Then, an improved propagation delay model and the corresponding range model for GEO-LEO BiSAR are established to lay the foundation of accurate imaging. After analyzing the sampling characteristics of GEO-LEO BiSAR, it is found that only 12.5% sampling data can be acquired in the azimuth direction. To handle the serious sub-Nyquist sampling problem and achieve good focusing results, an imaging method combined with multireceiving technique and compressed sensing is proposed. The multireceiving observation model is first obtained based on the inverse process of a nonlinear chirp-scaling imaging method, which can handle 2-D space-variant echo. Following that, the imaging problem of GEO-LEO BiSAR is converted to an $L_{1}$ regularization problem. Finally, an effective recovery method named complex approximate message passing (CAMP) is applied to obtain the final nonambiguous image. Simulation results show that the proposed method can suppress eight times Doppler ambiguity and obtain the well-focused image with three receiving channels. With the proposed method, the number of required receiving channels can be greatly reduced.
A new method based on the Looking-Around-and-Into (LAaI) mode is proposed for the task of change detection in large-scope Polarimetric Synthetic Aperture Radar (PolSAR) image. Specifically, the LAaI mode consists of two processes named Look-Around and Look-Into, which are accomplished by attention proposal network (APN) and recurrent convolutional neural network (CNN) (Recurrent CNN), respectively. The former provides certain subregions efficiently, and the latter detects changes in subregions accurately. In Look-Around, difference image (DI) of whole PolSAR images is calculated first to get global information; then, APN is established to locate the position of interested subregions intentionally by paying special attention to; next interested subregions that contain changed area in high probability are picked out as candidate-regions. Moreover, candidate-regions are sorted in importance descending order so that highly interested regions have priority to be detected. In Look-Into, candidate-regions of different scales are selected at first; then, Recurrent CNN is constructed and employed to deal with multiscale PolSAR subimages so that clearer and finer change detection results are generated. The process is repeated until all candidate-regions are detected. As a whole, the proposed algorithm based on the LAaI mode looks around whole images first to find out the possible position of changes (candidate-regions generation in Look-Around) and then reveal the exact shape of changes in different scales (multiscale change detection in Look-Into). The effect of APN and Recurrent CNN is verified in experiments, and it shows that the proposed method performs well in the task of change detection in the large-scope PolSAR image.
Ship target detection using large-scale synthetic aperture radar (SAR) images has important application in military and civilian fields. However, ship targets are difficult to distinguish from the surrounding background and many false alarms can occur due to the influence of land area. False alarms always occur with ship target detection because most of the area in large-scale SAR images are treated as background and clutter, and the ship targets are considered unevenly distributing small targets. To address these issues, a ship detection method in large-scale SAR images via CenterNet is proposed in this article. As an anchor-free method, CenterNet defines the target as a point, and the center point of the target is located through key point estimation, which can effectively avoid the missing detection of small targets. At the same time, the spatial shuffle-group enhance (SSE) attention module is introduced into CenterNet. Through SSE, the stronger semantic features are extracted while suppressing some noise to reduce false positives caused by inshore and inland interferences. The experiments on the public SAR-ship-data set show that the proposed method can detect all targets without missed detection with dense-docking targets. For the ship targets in large-scale SAR images from Sentinel 1, the proposed method can also detect targets near the shore and in the sea with high accuracy, which outperforms the methods like faster R-convolutional neural network (CNN), single-shot multibox detector (SSD), you only look once (YOLO), feature pyramid network (FPN), and their variations.
The lack of in situ data has always posed challenges to remote-sensing-data product validation. Herein, the products of sea-ice concentration (SIC) data derived using the arctic radiation and turbulence interaction study (ARTIST) sea ice (ASI) algorithm were evaluated by comparing them with SICs from a high-resolution sea-ice aerial image obtained during the 27th China Antarctic expedition in January 2011. Results suggest that data obtained from the advanced microwave scanning radiometer for the earth-observing system (AMSR-E) underestimate SICs by 19%. We performed step-by-step comparisons among the aerial image, moderate-resolution-imaging spectroradiometer (MODIS), and AMSR-E SIC. These types of comparisons have not been made in previous validation studies. First, SICs acquired from MODIS-Terra imagery were acquired using a tie-point method and corrected by SICs derived from the aerial photography. Second, SICs of MODIS-Aqua images were trained based on the consistency of SIC results between MODIS-Terra and MODIS-Aqua over the selected region on the same day. Finally, the MODIS-Aqua SICs were employed to validate synchronous AMSR-E swath SIC products. The results show that the AMSR-E products underestimate SICs by 8.5% in the marginal ice zone in comparison with MODIS SICs. According to our further analysis between sea-ice types and AMSR-E biases, the higher the proportion of first-year ice, the smaller the AMSR-E SIC bias. In other words, results suggest that the higher the thin ice proportion, the more the AMSR-E underestimates the SIC.
In this article, we first present a generalized methodology for deriving sensor brightness temperature sensor data records (SDR) from antenna temperature data records (TDR) applicable for Advanced Microwave Sounding Unit-A (AMSU-A) instruments. It includes corrections for antenna sidelobe contributions, antenna emission, and radiation perturbation due to the difference of Earth radiance in the main beam and that in the sidelobes that lie outside the main beam but within the Earth disk. For practical purposes, we simplify the methodology by neglecting the components other than the antenna sidelobe contributions to establish a consistent formulation with the legacy AMSU-A antenna pattern correction (APC) formula. The simplified formulation is then applied to the final AMSU-A instrument onboard the Metop-C satellite that was launched in November 2018, in order to compute APC coefficients for deriving SDR from TDR data. Furthermore, the performance of the calculated correction coefficients is validated by calculating the differences between the daily averaged AMSU-A (TDR and SDR) observations against radiative transfer model (O–B) simulations under clear sky conditions, and over open oceans. The validation results show that the derived temperature corrections are channel and scan position dependent, and can add 0.2–1.6 K to the antenna temperatures. In addition, the derived SDR O-B results exhibit a reduced and more symmetric scan angle-dependent bias when compared with corresponding TDR antenna temperatures.
Compressive spectral imaging (SI) (CSI) acquires few random projections of an SI reducing acquisition, storage, and, in some cases, processing costs. Then, this acquisition framework has been widely used in various tasks, such as target detection, video processing, and fusion. Particularly, compressive spectral image fusion (CSIF) aims at obtaining a high spatial–spectral resolution SI from two sets of compressed measurements: one from a hyperspectral image with a high-spectral low-spatial resolution, and one from a multispectral image with high-spatial low-spectral resolution. Most of the literature approaches include prior information, such as global low rank, smoothness, and sparsity, to solve the resulting ill-posed CSIF inverse problem. More recently, the high self-similarities exhibited by SIs have been successfully used to improve the performance of CSI inverse problems, including a nonlocal low-rank (NLLR) prior. However, to the best of our knowledge, this NLLR prior has not been implemented in the solution of the CSIF inverse problem. Therefore, this article formulates an approach that jointly includes the global low rank, the smoothness, and the NLLR priors to solve the CSIF inverse problem. The global low-rank prior is introduced with the linear mixture model that describes the SI as a linear combination of a set of few end-members to specific abundances. In this article, either the end-members are accurately estimated from the compressed measurements or initialized from a fast reconstruction of the hyperspectral image. Also, it assumes that the abundances preserve the smoothness and NLLR priors of the SI so that the fused image is obtained from the end-members and abundances that result when minimizing a cost function including the sum of two data fidelity terms and two regularizations: the smoothness and the NLLR. Simulations over three data sets show that the proposed approach increases the CSIF performance compared with literature -
pproaches.
The trade-off between feature representation power and spatial localization accuracy is crucial for the dense classification/semantic segmentation of remote sensing images (RSIs). High-level features extracted from the late layers of a neural network are rich in semantic information, yet have blurred spatial details; low-level features extracted from the early layers of a network contain more pixel-level information but are isolated and noisy. It is therefore difficult to bridge the gap between high- and low-level features due to their difference in terms of physical information content and spatial distribution. In this article, we contribute to solve this problem by enhancing the feature representation in two ways. On the one hand, a patch attention module (PAM) is proposed to enhance the embedding of context information based on a patchwise calculation of local attention. On the other hand, an attention embedding module (AEM) is proposed to enrich the semantic information of low-level features by embedding local focus from high-level features. Both proposed modules are lightweight and can be applied to process the extracted features of convolutional neural networks (CNNs). Experiments show that, by integrating the proposed modules into a baseline fully convolutional network (FCN), the resulting local attention network (LANet) greatly improves the performance over the baseline and outperforms other attention-based methods on two RSI data sets.
Developed here is an algorithm for determining the infrared (IR) cloud-top phase for advanced Himawari imager (AHI) measurements from the Japanese geostationary satellite Himawari-8. The tests and decision tree used in the AHI algorithm are different from those in the Moderate Resolution Imaging Spectroradiometer (MODIS) Level-2 cloud product algorithm. Verification of AHI cloud-top phase results with the Cloud–Aerosol Lidar with orthogonal polarization product over a four-month period from March to June of 2017 over the North Pacific gives hit rates of 80.20% (66.33%) and 86.51% (80.61%) for liquid-water and randomly oriented-ice cloud tops, respectively, if clear-sky pixels are excluded (included) from the statistics. Also made are intercomparisons between AHI and MODIS IR cloud-top phase products over the North Pacific in June 2017. AHI liquid-water-phase determinations agree with MODIS liquid-water-phase determinations at an agreement rate of 83.68%, showing a dependence on MODIS zenith angles. The agreement rate of ice phase classifications between AHI and MODIS is 93.54%. The MODIS IR product contains some unreasonable ice-phase pixels over oceans, as well as uncertain-phase pixels over land, and has limitations for daytime liquid-water-phase identifications over the Indo-China Peninsula. Limitations of the AHI cloud-top phase algorithm are mainly caused by difficulties in identifying liquid-water-phase clouds over sun-glint regions and during twilight.
In the last five years, deep learning has been introduced to tackle the hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, due to the high dimensionality of HSI and equal treatment of all bands, the performance of these methods is hampered by learning features from useless bands for classification. Moreover, for patchwise-based CNN models, equal treatment of spatial information from the pixel-centered neighborhood also hinders the performance of these methods. In this article, we propose an end-to-end residual spectral–spatial attention network (RSSAN) for HSI classification. The RSSAN takes raw 3-D cubes as input data without additional feature engineering. First, a spectral attention module is designed for spectral band selection from raw input data by emphasizing useful bands for classification and suppressing useless bands. Then, a spatial attention module is designed for the adaptive selection of spatial information by emphasizing pixels from the same class as the center pixel or those are useful for classification in the pixel-centered neighborhood and suppressing those from a different class or useless. Second, two attention modules are also used in the following CNN for adaptive feature refinement in spectral–spatial feature learning. Third, a sequential spectral–spatial attention module is embedded into a residual block to avoid overfitting and accelerate the training of the proposed model. Experimental studies demonstrate that the RSSAN achieved superior classification accuracy compared with the state of the art on three HSI data sets: Indian Pines (IN), University of Pavia (UP), and Kennedy Space Center (KSC).
Hyperspectral (HS) pansharpening, as a special case of the superresolution (SR) problem, is to obtain a high-resolution (HR) image from the fusion of an HR panchromatic (PAN) image and a low-resolution (LR) HS image. Though HS pansharpening based on deep learning has gained rapid development in recent years, it is still a challenging task because of the following requirements: 1) a unique model with the goal of fusing two images with different dimensions should enhance spatial resolution while preserving spectral information; 2) all the parameters should be adaptively trained without manual adjustment; and 3) a model with good generalization should overcome the sensitivity to different sensor data in reasonable computational complexity. To meet such requirements, we propose a unique HS pansharpening framework based on a 3-D generative adversarial network (HPGAN) in this article. The HPGAN induces the 3-D spectral–spatial generator network to reconstruct the HR HS image from the newly constructed 3-D PAN cube and the LR HS image. It searches for an optimal HR HS image by successive adversarial learning to fool the introduced PAN discriminator network. The loss function is specifically designed to comprehensively consider global constraint, spectral constraint, and spatial constraint. Besides, the proposed 3-D training in the high-frequency domain reduces the sensitivity to different sensor data and extends the generalization of HPGAN. Experimental results on data sets captured by different sensors illustrate that the proposed method can successfully enhance spatial resolution and preserve spectral information.
As one of the most important algorithms in target detection, constrained energy minimization (CEM) has been widely used and developed in recent years. However, it is easy to verify that the target detection result of CEM varies with the data origin, which is apparently unreasonable since the distribution of the target of interest is objective and, therefore, unrelated to the selection of data origin. The clever eye (CE) algorithm tries to solve this problem by adding the data origin as a new variable from the perspective of the filter output energy. However, due to the nonconvexity of the objective function, CE can only obtain locally optimal solutions by using the gradient ascent method. In this article, we find a striking conclusion that there exists an analytical solution for CE that corresponds to the solution of a linear equation and further prove that all the solutions of the linear equation are globally optimal.
Conventional low-rank (LR)-based hyperspectral image (HSI) denoising models generally convert high-dimensional data into 2-D matrices or just treat this type of data as 3-D tensors. However, these pure LR or tensor low-rank (TLR)-based methods lack flexibility for considering different correlation information from different HSI directions, which leads to the loss of comprehensive structure information and inherent spatial–spectral relationship. To overcome these shortcomings, we propose a novel multidirectional LR modeling and spatial–spectral total variation (MLR-SSTV) model for removing HSI mixed noise. By incorporating the weighted nuclear norm, we obtain the weighted sum of weighted nuclear norm minimization (WSWNNM) and the weighted sum of weighted tensor nuclear norm minimization (WSWTNNM) to estimate the more accurate LR tensor, especially, to remove the dead-line noise better. Gaussian noise is further denoised and the local spatial–spectral smoothness is preserved effectively by SSTV regularization. We develop an efficient algorithm for solving the derived optimization based on the alternating direction method of multipliers (ADMM). Extensive experiments on both synthetic data and real data demonstrate the superior performance of the proposed MLR-SSTV model for HSI mixed noise removal.
Class-wise adversarial adaptation networks are investigated for the classification of hyperspectral remote sensing images in this article. By adversarial learning between the feature extractor and the multiple domain discriminators, domain-invariant features are generated. Moreover, a probability-prediction-based maximum mean discrepancy (MMD) method is introduced to the adversarial adaptation network to achieve a superior feature-alignment performance. The class-wise adversarial adaptation in conjunction with the class-wise probability MMD is denoted as the class-wise distribution adaptation (CDA) network. The proposed CDA does not require labeled information in the target domain and can achieve an unsupervised classification of the target image. The experimental results using the Hyperion and Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) hyperspectral data demonstrated its efficiency.
Recently, convolution neural network (CNN)-based hyperspectral image (HSI) classification has enjoyed high popularity due to its appealing performance. However, using 2-D or 3-D convolution in a standalone mode may be suboptimal in real applications. On the one hand, the 2-D convolution overlooks the spectral information in extracting feature maps. On the other hand, the 3-D convolution suffers from heavy computation in practice and seems to perform poorly in scenarios having analogous textures along with consecutive spectral bands. To solve these problems, we propose a mixed CNN with covariance pooling for HSI classification. Specifically, our network architecture starts with spectral–spatial 3-D convolutions that followed by a spatial 2-D convolution. Through this mixture operation, we fuse the feature maps generated by 3-D convolutions along the spectral bands for providing complementary information and reducing the dimension of channels. In addition, the covariance pooling technique is adopted to fully extract the second-order information from spectral–spatial feature maps. Motivated by the channel-wise attention mechanism, we further propose two principal component analysis (PCA)-involved strategies, channel-wise shift and channel-wise weighting, to highlight the importance of different spectral bands and recalibrate channel-wise feature response, which can effectively improve the classification accuracy and stability, especially in the case of limited sample size. To verify the effectiveness of the proposed model, we conduct classification experiments on three well-known HSI data sets, Indian Pines, University of Pavia, and Salinas Scene. The experimental results show that our proposal, although with less parameters, achieves better accuracy than other state-of-the-art methods.
Blind hyperspectral unmixing is the process of expressing the measured spectrum of a pixel as a combination of a set of spectral signatures called endmembers and simultaneously determining their fractional abundances in the pixel. Most unmixing methods are strictly spectral and do not exploit the spatial structure of hyperspectral images (HSIs). In this article, we present a new spectral–spatial linear mixture model and an associated estimation method based on a convolutional neural network autoencoder unmixing (CNNAEU). The CNNAEU technique exploits the spatial and the spectral structure of HSIs both for endmember and abundance map estimation. As it works directly with patches of HSIs and does not use any pooling or upsampling layers, the spatial structure is preserved throughout and abundance maps are obtained as feature maps of a hidden convolutional layer. We compared the CNNAEU method to four conventional and three deep learning state-of-the-art unmixing methods using four real HSIs. Experimental results show that the proposed CNNAEU technique performs particularly well and consistently when it comes to endmembers’ extraction and outperforms all the comparison methods.
The fusion of hyperspectral (HS) and multispectral (MS) images designed to obtain high-resolution HS (HRHS) images is a very challenging work. A series of solutions has been proposed in recent years. However, the similarity in the structure of the HS image has not been fully used. In this article, we present a novel HS and MS image-fusion method based on nonlocal low-rank tensor approximation and sparse representation. Specifically, the HS image and the MS image are considered the spatially and spectrally degraded versions of the HRHS image, respectively. Then, the nonlocal low-rank constraint term is adopted in order to form the nonlocal similarity and the spatial–spectral correlation. Meanwhile, we add the sparse constraint term to describe the sparsity of abundance. Thus, the proposed fusion model is established and its optimization is solved by alternative direction method of multipliers (ADMM). The experimental results on three synthetic data sets and one real data set show the advantages of the proposed method over several state-of-the-art competitors.
Blind hyperspectral unmixing (BHU) is an important technology to decompose the mixed hyperspectral image (HSI), which is actually an ill-posed problem. The ill-posedness of the BHU is deteriorated by nonlinearity, endmember variability (EV) and abnormal points, which are considered as three challenging intractable interferences currently. To sidestep the challenges, we present a novel unmixing model, where a latent multidiscriminative subspace is explored and the inherent self-expressiveness property is employed. The most existing unmixing approaches directly decompose the HSI utilizing original features in an interference corrupted single subspace, unlike them, our model seeks the underlying intrinsic representation and simultaneously reconstructs HSI based on the learned latent subspace. With the help of both clustering homogeneity and intrinsic features selection, structural differences in the HSI and the spectral property of a certain material are exploited perfectly, and an ideal multiheterogeneous subspace is recovered from the heavily contaminated original HSI. Based on the multiheterogeneous subspace, the reconstructed differentiated transition matrix is split into two matrices to avoid the emergence of the artificial endmember. Experiments are conducted on synthetic and four representative real HSI sets, and all the experimental results demonstrate the validity and superiority of our proposed method.
The use of hyperspectral (HS) data is growing over the years, thanks to the very high spectral resolution. However, HS data are still characterized by a spatial resolution that is too low for several applications, thus motivating the design of fusion techniques aimed to sharpen HS images with high spatial resolution data. To reach a significant resolution enhancement, high-resolution images should be acquired by different satellite platforms. In this article, we highlight the pros and cons of employing real multiplatform data, using the EO-1 satellite as an exemplary case. The spatial resolution of the HS data collected by the Hyperion sensor is improved by exploiting both the ALI panchromatic image collected from the same platform and acquisitions from the WorldView-3 and the QuickBird satellites. Furthermore, we tackle the problem of assessing the final quality of the fused product at the nominal resolution, which presents further difficulties in this general environment. Useful indications for the design of an effective sharpening method in this case are finally outlined.
In hyperspectral image (HSI) classification, spatial context has demonstrated its significance in achieving promising performance. However, conventional spatial context-based methods simply assume that spatially neighboring pixels should correspond to the same land-cover class, so they often fail to correctly discover the contextual relations among pixels in complex situations, and thus leading to imperfect classification results on some irregular or inhomogeneous regions such as class boundaries. To address this deficiency, we develop a new HSI classification method based on the recently proposed graph convolutional network (GCN), as it can flexibly encode the relations among arbitrarily structured non-Euclidean data. Different from traditional GCN, there are two novel strategies adopted by our method to further exploit the contextual relations for accurate HSI classification. First, since the receptive field of traditional GCN is often limited to fairly small neighborhood, we proposed to capture long-range contextual relations in HSI by performing successive graph convolutions on a learned region-induced graph which is transformed from the original 2-D image grids. Second, we refine the graph edge weight and the connective relationships among image regions simultaneously by learning the improved similarity measurement and the “edge filter,” so that the graph can be gradually refined to adapt to the representations generated by each graph convolutional layer. Such updated graph will in turn result in faithful region representations, and vice versa. The experiments carried out on four real-world benchmark data sets demonstrate the effectiveness of the proposed method.
