publications

2023

1. Meta-Learning a Cross-lingual Manifold for Semantic Parsing Sherborne, Tom, and Lapata, Mirella Transactions of the Association for Computational Linguistics 2023 [Abs]

   Localizing a semantic parser to support new languages requires effective cross-lingual generalization. Recent work has found success with machine-translation or zero-shot methods, although these approaches can struggle to model how native speakers ask questions. We consider how to effectively leverage minimal annotated examples in new languages for few-shot cross-lingual semantic parsing. We introduce a first-order meta-learning algorithm to train a semantic parser with maximal sample efficiency during cross-lingual transfer. Our algorithm uses high-resource languages to train the parser and simultaneously optimizes for cross-lingual generalization to lower-resource languages. Results across six languages on ATIS demonstrate that our combination of generalization steps yields accurate semantic parsers sampling ≤10\% of source training data in each new language. Our approach also trains a competitive model on Spider using English with generalization to Chinese similarly sampling ≤10\% of training data.1

2022

1. Extrinsic Evaluation of Machine Translation Metrics Moghe, Nikita, Sherborne, Tom, Steedman, Mark, and Birch, Alexandra 2022
2. Zero-Shot Cross-lingual Semantic Parsing Sherborne, Tom, and Lapata, Mirella In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) 2022 [Abs]

   Recent work in cross-lingual semantic parsing has successfully applied machine translation to localize parsers to new languages. However, these advances assume access to high-quality machine translation systems and word alignment tools. We remove these assumptions and study cross-lingual semantic parsing as a zero-shot problem, without parallel data (i.e., utterance-logical form pairs) for new languages. We propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-logical form paired data and in-domain natural language corpora in each new language. Our model encourages language-agnostic encodings by jointly optimizing for logical-form generation with auxiliary objectives designed for cross-lingual latent representation alignment. Our parser performs significantly above translation-based baselines and, in some cases, competes with the supervised upper-bound.

2020

1. Bootstrapping a Crosslingual Semantic Parser Sherborne, Tom, Xu, Yumo, and Lapata, Mirella In Findings of the Association for Computational Linguistics: EMNLP 2020 2020 [Abs]

   Recent progress in semantic parsing scarcely considers languages other than English but professional translation can be prohibitively expensive. We adapt a semantic parser trained on a single language, such as English, to new languages and multiple domains with minimal annotation. We query if machine translation is an adequate substitute for training data, and extend this to investigate bootstrapping using joint training with English, paraphrasing, and multilingual pre-trained models. We develop a Transformer-based parser combining paraphrases by ensembling attention over multiple encoders and present new versions of ATIS and Overnight in German and Chinese for evaluation. Experimental results indicate that MT can approximate training data in a new language for accurate parsing when augmented with paraphrasing through multiple MT engines. Considering when MT is inadequate, we also find that using our approach achieves parsing accuracy within 2% of complete translation using only 50% of training data.

2. Going Beneath the Surface: Evaluating Image Captioning for Grammaticality, Truthfulness and Diversity Xie, Huiyuan, Sherborne, Tom, Kuhnle, Alexander, and Copestake, Ann In Evaluating Evaluation of AI Systems 2020 [Abs]

   Image captioning as a multimodal task has drawn much interest in recent years. However, evaluation for this task remains a challenging problem. Existing evaluation metrics focus on surface similarity between a candidate caption and a set of reference captions, and do not check the actual relation between a caption and the underlying visual content. We introduce a new diagnostic evaluation framework for the task of image captioning, with the goal of directly assessing models for grammaticality, truthfulness and diversity (GTD) of generated captions. We demonstrate the potential of our evaluation framework by evaluating existing image captioning models on a wide ranging set of synthetic datasets that we construct for diagnostic evaluation. We empirically show how the GTD evaluation framework, in combination with diagnostic datasets, can provide insights into model capabilities and limitations to supplement standard evaluations.

2018

1. On the Impact of Fixed Point Hardware for Optical Fiber Nonlinearity Compensation Algorithms Sherborne, T., Banks, B., Semrau, D., Killey, R. I., Bayvel, P., and Lavery, D. Journal of Lightwave Technology 2018

2017

1. CacheMAsT: Cache Management Analysis and Visualization Tool Tuncer, D., Sherborne, T., Charalambides, M., and Pavlou, G. In 2017 IFIP/IEEE Symposium on Integrated Network and Service Management (IM) 2017