Publication Date
2025
Document Type
Thesis
Committee Members
Fathi Amsaad, Ph.D. (Advisor); Paul M. Simon, Ph.D. (Committee Member); Ashutosh Shivakumar, Ph.D. (Committee Member); Sean Banerjee, Ph.D. (Committee Member)
Degree Name
Master of Science in Computer Engineering (MSCE)
Abstract
Image sensors are at the heart of machine vision systems in robotics, industrial automation, and surveillance systems which ideally operate with minimal human supervision and only occasional maintenance. The image sensors convert visible light into electrical signals which are locally decoded to image on the printed circuit board (PCB) by an ordinary embedded processor System on Chip (SoC). This thesis investigates a critical vulnerability in such systems, targeting the communication protocol at the signal level during runtime. Specifically, it focuses on a novel attack in the Digital Video Port (DVP) protocol, possible to exploit with PCB-based hardware Trojans, to craft precise pixel injection during runtime. Through a series of experimental hardware and attacker firmware approaches, this work manipulates the signal composition, exploiting the decoding mechanisms of the DVP protocol, by which an adversary can inject malicious pixels without disrupting the system. In this study, the Proof-of-Concept (PoC) for this attack is demonstrated and a hardwarebased PCB Trojan layout was designed and fabricated in the lab as a part of the PoC, which can compromise the quality of images, alter pixel colors, and cause mispredictions in AI-based algorithms. The designed Trojan is only 6.85x4.35mm in size and requires 0.6mA of current in stealth mode, making it diffcult to detect. Different variants of poisoned images were generated to analyze the downstream impact of such an attack. The impacts were comprehensively studied by injecting runtime pixels into images tied to critical vision applications. Experiments included testing the impact on checkerboard-based camera calibration algorithms, AI-based image classification models (MobileNet, ResNet50, EffcientNet, DenseNet, VGG16, and more), and object detection frameworks (SSD and all YOLO models). The results show that the camera calibration algorithm fails to detect the checkerboard in almost 50% of the samples, while processed poisoned images led the algorithm to an RMS error of 2.99, resulting in dysfunctional state. All popular AI-based methods showed considerable confidence degradation and misclassifications. Poisoned images bypass 90% of the tested, highly popular object detection models, raising significant concerns about the trust between camera systems and processing units.
Page Count
84
Department or Program
Department of Computer Science and Engineering
Year Degree Awarded
2025
Copyright
Copyright 2025, all rights reserved. My ETD will be available under the "Fair Use" terms of copyright law.
ORCID ID
0009-0005-2022-2940
