Publication Date
2022
Document Type
Dissertation
Committee Members
Nikolaos G. Bourbakis, Ph.D. (Advisor); Soon M. Chung, Ph.D. (Committee Member); Bin Wang, Ph.D. (Committee Member); Maria Virvou, Ph.D. (Committee Member); Ioannis Hatzilygeroudis, Ph.D. (Committee Member)
Degree Name
Doctor of Philosophy (PhD)
Abstract
The technical document is an entity that consists of several essential and interconnected parts, often referred to as modalities. Despite the extensive attention that certain parts have already received, per say the textual information, there are several aspects that severely under researched. Two such modalities are the utility of diagram images and the deep automated understanding of mathematical formulas. Inspired by existing holistic approaches to the deep understanding of technical documents, we develop a novel formal scheme for the modelling of digital diagram images. This extends to a generative framework that allows for the creation of artificial images and their annotation. We contribute on the field with the creation of a novel synthetic dataset and its generation mechanism. We propose the conversion of the pseudocode generation problem to an image captioning task and provide a family of techniques based on adaptive image partitioning. We address the mathematical formulas’ semantic understanding by conducting an evaluating survey on the field, published in May 2021. We then propose a formal synthesis framework that utilized formula graphs as metadata, reaching for novel valuable formulas. The synthesis framework is validated by a deep geometric learning mechanism, that outsources formula data to simulate the missing a priori knowledge. We close with the proof of concept, the description of the overall pipeline and our future aims.
Page Count
196
Department or Program
Department of Computer Science and Engineering
Year Degree Awarded
2022
Copyright
Copyright 2022, some rights reserved. My ETD may be copied and distributed only for non-commercial purposes and may not be modified. All use must give me credit as the original author.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.
