The BLA benchmark: Investigating basic language abilities of pre-trained multimodal models
Published in EMNLP, 2023
BLA is a novel, automatically constructed benchmark to evaluate vision-and-language models on Basic Language Abilities. In the BLA tasks, we explore to what extent the models handle basic linguistic constructions—active-passive voice, coordination, and relative clauses—that even preschool children can typically master.
Recommended citation: Xinyi Chen, Raquel Fernández, and Sandro Pezzelle. 2023. The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5817–5830, Singapore. Association for Computational Linguistics.
Download Paper