Publication Date

2023

Document Type

Thesis

Committee Members

Lingwei Chen, Ph.D. (Advisor); Meilin Liu, Ph.D. (Committee Member); Junjie Zhang, Ph.D. (Committee Member)

Degree Name

Master of Science in Cyber Security (M.S.C.S.)

Abstract

Malware detection is a critical task in ensuring the security of computer systems. Due to a surge in malware and the malware program sophistication, machine learning methods have been developed to perform such a task with great success. To further learn structural semantics, Graph Neural Networks abbreviated as GNNs have emerged as a recent practice for malware detection by modeling the relationships between various components of a program as a graph, which deliver promising detection performance improvement. However, this line of research attends to individual programs while overlooking program interactions; also, these GNNs tend to perform feature aggregation from neighbors without considering any label information and significantly suffer from over-smoothing on node presentations. To address these issues, this thesis constructs a graph over program collection to capture the program relations and designs two enhanced graph convolutional network (GCN)architectures for malware detection. More specifically, the first proposed GCN model in-corporates label propagation into GCN to take advantage of label information for facilitating neighborhood aggregation, which is used to propagate labels from the labeled nodes to the unlabeled nodes; the second proposed GCN model introduces residual connections between the original node features and the node representations produced by GCN layer to enhance the flow of information through the network and address over-smoothing is-sue. The experimental results after substantial experiments show that the proposed models outperform the baseline GCN and classic machine learning methods for malware detection, which confirm their effectiveness in program representation learning using either label propagation or residual connections and malware detection using program graph. Furthermore, these models can be used for other graph-based tasks other than malware detection, demonstrating their versatility and promise.

Page Count

71

Department or Program

Department of Computer Science and Engineering

Year Degree Awarded

2023

Download


Share

COinS