Publication Date
2025
Document Type
Thesis
Committee Members
Lingwei Chen, Ph.D. (Advisor); Wen Zhang, Ph.D. (Committee Member); Michael Raymer, Ph.D. (Committee Member)
Degree Name
Master of Science (MS)
Abstract
Natural-language inference (NLI) asks whether a hypothesis is entailed by, contradicts, or is neutral with respect to a premise. Modern transformers reach high raw accuracy on benchmarks such as SNLI, MNLI, and ANLI, yet they often rely on brittle lexical shortcuts and provide little insight into their decision process. This thesis shows that counterfactual-augmented knowledge distillation can simultaneously boost robustness and supply faithful, token-level explanations—without scaling model size. Four T5-v1_1 students (60M, 220M, 770M, 3B parameters) are trained under four curricula: (1) standard fine-tuning, (2) fine-tuning with free-text rationales, (3) multi-task distillation with naive counterfactuals, and (4) multi-task distillation with guided keyword counterfactuals. On ANLI Round 1, naive counterfactual training lifts T5-small from 0.408 → 0.671 accuracy (+44% error-rate reduction) and T5-base from 0.492 → 0.707, while guided counterfactuals push those scores to 0.678 and 0.729, respectively, and provide automatically recoverable “edit scripts” that flip the label in 82% of cases. Smaller gains persist for T5-xl (0.736 → 0.820). On a 15k / 900 eSNLI split, counterfactual supervision still reduces relative error by 2–6% despite a stronger baseline. Training cost rises sub-linearly: a T5-base+guided-CF run requires 5.8 GPU-hours versus 15 GPU-hours for a vanilla T5-large that attains lower accuracy. Qualitative analysis confirms that single-token edits correct habitual negation and attribute errors better than attention heat-maps. Overall, the results argue that smart data beats sheer scale: counterfactual-enhanced distillation yields lean NLI models that are both more accurate and transparently explainable, charting a path toward deployable, trustworthy language understanding systems.
Page Count
56
Department or Program
Department of Computer Science and Engineering
Year Degree Awarded
2025
Copyright
Copyright 2025, all rights reserved. My ETD will be available under the "Fair Use" terms of copyright law.
ORCID ID
0009-0003-1723-8816
