Publication Date
2025
Document Type
Dissertation
Committee Members
Soon M. Chung, Ph.D. (Advisor); Thomas Wischgoll, Ph.D. (Committee Member); Vincent A. Schmidt, Ph.D. (Committee Member); Wen Zhang, Ph.D. (Committee Member)
Degree Name
Doctor of Philosophy (PhD)
Abstract
Social media, AI systems, IoT sensors, and other platforms generate vast amounts of streaming data. Given this vast volume of information, techniques that can reduce and aggregate data into meaningful topics are essential. One such technique is the two-phase stream clustering approach. In the first, online micro-clustering phase, the system forms micro-clusters from the incoming data stream, incrementally merges new items into related existing micro-clusters, and prunes or fades micro-clusters as they become inactive, producing a constantly updating yet compact set of micro-clusters representing potential topics and subtopics of the stream. In the second, offline macro-clustering phase, these micro-clusters are grouped into higher-level topics over user-defined intervals. The dissertation begins with the development of a sequential stream clustering system, proceeds to accelerate the online micro-clustering phase on a Graphics Processing Unit (GPU), and finishes by scaling this phase across multiple nodes using a Message Passing Interface (MPI) framework. The sequential TextClust implementation on a single CPU provides a functional baseline but it cannot keep pace with the high-volume, unbounded streams. GPU acceleration of the micro-clustering stage achieves a 2.9 speedup over this baseline through batching and parallel similarity computation; however, host-to-device transfers, a static device-resident model, and single-node memory limited end-to-end gains. To address these limits, the design transitions to a MPI architecture that distributes ingestion and micro-cluster maintenance across many nodes in a high-performance computing (HPC) environment. The framework supports two streaming modes: a hybrid mode with communication-free local epochs and brief synchronizations to realign global state across ranks, and a strict global mode that periodically consolidates micro-clusters for exact cross-rank alignment. Depending on fixed hyperparameters, the MPI hybrid configuration increases speed by a factor of 9.2–43.0 when compared to the GPU baseline, while the strict global mode is approximately 4.0 times faster. Both MPI modes are 160–1740 times faster than the sequential baseline. Together, the sequential clustering foundation, GPU acceleration study, and multi-node MPI framework demonstrate that through careful design of algorithms and systems near-real-time micro-clustering can be achieved at scale while preserving a flexible micro-cluster representation for downstream analytics.
Page Count
134
Department or Program
Department of Computer Science and Engineering
Year Degree Awarded
2025
Copyright
Copyright 2025, all rights reserved. My ETD will be available under the "Fair Use" terms of copyright law.
ORCID ID
0000-0001-6465-5303
