Interpretability of few-shot intent detection | |
| Author | Sitiporn Sae Lim |
| Call Number | AIT Thesis no.DSAI-22-09 |
| Subject(s) | Natural language processing (Computer science) Machine learning |
| Note | A thesis submitted in partial fulfillment of the requirements for the degree of Master of Engineering in Data Science and Artificial Intelligence |
| Publisher | Asian Institute of Technology |
| Abstract | Few shot intent detection system is a key task in dialogue system. However, a few works have focused on the interpretability of few shot intent detection system which is the first stepping stone towards better failure detection in intent detection. By leveraging supervised and unsupervised on Simple Contrastive Learning of Sentence Embeddings models (SimCSE), we attempted to interpret the models by using attention head as an interpretable testbed to obtain the dependency syntax at three different levels which are: 1) global dependency anal ysis; 2) class level analysis; and 3) error level analysis and to compare them to the gold standard provided by stanza recent dependency parser. In particular, our experiments in global analysis revealed that there some attention heads that perform consistency on syntax across the combination of models and intent detection test set. Furthermore, in class level analysis , the difference between classes can be explained by looking the combination of attention head and dependency may not share across classes. In Error level analysis, each syntax, the attention head of predicted correct and incorrect groups are mostly different po sitions. Then, we quantify this properties for every intent to differentiate the ability to track predicted incorrect samples. However, further work can find how to manipulate incorrect attention heads. |
| Year | 2022 |
| Type | Thesis |
| School | School of Engineering and Technology |
| Department | Department of Information and Communications Technologies (DICT) |
| Academic Program/FoS | Data Science and Artificial Intelligence (DSAI) |
| Chairperson(s) | Chaklam Silpasuwanchai |
| Examination Committee(s) | Dailey, Matthew N.;Mongkol Ekpanyapong |
| Scholarship Donor(s) | Royal Thai Government Fellowship |
| Degree | Thesis (M. Eng.) - Asian Institute of Technology, 2022 |