Publication Date
2024
Document Type
Thesis
Committee Members
Wen Zhang, Ph.D. (Advisor); Lingwei Chen, Ph.D. (Committee Member); Krishnaprasad Thirunarayan, Ph.D. (Committee Member)
Degree Name
Master of Science (MS)
Abstract
Aerial imagery provides crucial insights for various fields, including remote monitoring, environmental assessment, and autonomous navigation. However, the availability of aerial image datasets is limited due to privacy concerns and imbalanced data distribution, impeding the development of robust deep learning models. While recent text-guided generative models have shown promise in synthesizing high-quality images, they fall short in handling the unique challenges of aerial imagery, including densely packed objects, intricate spatial relationships, and the absence of paired text-aerial image datasets. To tackle these limitations, we propose STARS, a groundbreaking framework for Semantic-aware Text-guided Aerial image Refinement and Synthesis. STARS introduces a three-pronged approach: context-aware text generation using chain-of-thought prompting for precise and diverse text annotations, feature-augmented image representation with multi-head attention to preserve small-object details and spatial coherence, and a latent diffusion mechanism conditioned on multi-modal embedding fusion for high-fidelity image synthesis. STARS's is the first to extend deep generative models for high-resolution, text-guided aerial image generation, including the synthesis of images from novel viewpoints. To this end, we contribute paired text-aerial image datasets and rigorously validate the performance of our proposed model. Extensive evaluations across five benchmarks, using the VisDrone-DET and UAV-VisLoc datasets, achieve FID scores of 88.28 and 94.22, respectively. These results demonstrate a significant advancement over state-of-the-art models, including DDPM (217.95 and 232.34), Stable Diffusion (119.13 and 135.80), ARLDM (111.59 and 87.72), Versatile Diffusion (124.12 and 122.42), and Make-a-Scene (114.75 and 122.42).
Page Count
61
Department or Program
Department of Computer Science and Engineering
Year Degree Awarded
2024
Copyright
Copyright 2024, all rights reserved. My ETD will be available under the "Fair Use" terms of copyright law.
