ADAPT Academic and Assistant Professor at Dublin City University, Dr Sheila Castilho, recently had a paper accepted into the 24th Annual Conference of the European Association for Machine Translation. The paper, titled “Do Online Machine Translation Systems Care for Context? What about a GPT Model?”, addresses the challenge of evaluating document level machine translation (MT) under the conditions of recent advances in context-aware neural machine translation (NMT). Dr Castilho’s students Clodagh Mallon, Rahel Meister, and Shengya Yue assisted with the Irish, German, and Mandarin translations within the paper.
As a result of the study, Dr Castilho and team discovered that a change of input punctuation displayed a great variability in the output while the context position did not have a similar level of impact. Dr Castilho noted that the ChatGPT model currently outperformed the NMT systems but performed poorly for Irish Low Resource as well as carrying gender bias.
The 24th Annual Conference of the European Association for Machine Translation (EAMT 2023) will take place in Finland 12th – 15th June 2023. The conference is jointly organised by the European Association for Machine Translation, Tampere University and the University of Eastern Finland.
Read the paper here.
