Published on July 25, 2022
Amid explosive growth in availability of multimodal remotely sensed imagery (RSI) data from a constellation of overhead sensors, a lack of understanding persists concerning the actual content of these data sources, in particular the nature of spatial variation in the visual and contextual features in the landscape being imaged. Whether described as spatial domain shift, geographic feature variance or simply geodiversity, this gap of knowledge about RSI dataset content comes with important implications.
On one hand, there is a lack of tools to evaluate heterogeneity and representativeness of objects classes found in labeled RSI training datasets, in particular methods for regional comparison of object class appearance and their geographic context. On the other hand, the classic elements of overhead imagery interpretation (e.g., color, size, texture, association) remain unexplored computationally over the spatial and temporal extent of massive unlabeled RSI databases, such that spatial models of visual change across the global landscape are unavailable to the scientific community.
The proposed research is positioned near the beginning of the academic conversation regarding diversity-related analysis of ‘big RSI data’. It aims to provide deep learning-based methods to address the above gaps, concentrating on the most basic type of geodiversity analysis: detection of regional changes of appearance (i.e., high-dimensional image features) and context (i.e., object & landscape co-occurrence) in large labeled and unlabeled RSI databases.
A set of deep-feature spatial analysis methods for RSI is developed to enable researchers to detect and explore a variety of geodiversity-related characteristics in visible RSI at arbitrary spatial scales. Specifically, Geospatial Fréchet Inception Distance (GeoFID) is proposed as a measure of visual variation in geographic objects and landscapes using a feature distance-based computation of dissimilarity of learned image features among regions. A complementary Geospatial Inception Score (GeoIS) is proposed as measure of scene-level contextual geodiversity of objects and land cover classes detected in a region using an entropy-based computation.
The proposed methods are applied to simulated RSI as well as benchmark RSI training datasets and unlabeled overhead imagery databases to demonstrate their broad practical utility for a range of spatial modeling and analysis problems such as geographic change detection, dataset quality control, landscape heterogeneity analysis, and characterization of dataset bias / diversity issues.