A Conditional Generative Adversarial Network for urban area classification using multi-source data

Abstract

Classifying urban land use/cover types poses significant challenges due to the complex and heterogeneous nature of urban landscapes. Recent years have witnessed notable advancements in land use/cover classification, driven by improvements in classification methods and the utilization of data from multiple sources. Deep learning networks, especially, have gained prominence in various image analysis tasks, including land use/cover classification. However, when it comes to urban areas, the classification of urban land use/cover encounters additional obstacles, including the complexity of classes, limited training data, and the presence of numerous urban categories. To overcome the limitations arising from similar classes and insufficient training data, we propose a novel approach that integrates hyperspectral and LiDAR data through a Conditional Generative Adversarial Network (CGAN) for semantic segmentation. Our methodology leverages the UNet + + generator and PatchGAN discriminator to achieve accurate segmentation. The CGAN-generated segmented images are then processed by a fully connected neural network (FCN) to classify 20 land use/cover classes. By validating our approach on the 2018 GRSS Data Fusion dataset, our study demonstrates its exceptional operational performance. The CGAN architecture outperforms previous algorithms in terms of class diversity and training data volume. By generating synthetic data that closely resembles the ground truth, the CGAN enhances the classification performance. Clear visual distinctions are observed among various urban features, such as vegetation, trees, buildings, roads, and cars. Classes associated with healthy grass, stressed grass, bare earth, and stadium seats achieve high accuracy. However, road and railway classes exhibit poorer performance due to their similarity with sidewalk, highway, major thoroughfare, and crosswalk classes. Overall, our study showcases a significant improvement in classification accuracy, achieving an approximate accuracy of 96.98% compared to the winning articles presented in the 2018 competition, which achieved accuracies of 64.95% and 76.54%, respectively. This improvement in accuracy can be attributed to the effective extraction and combination of high and low-level urban land cover/land use features within our proposed architecture. © 2023, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.