ADAPT researcher Dr. Sheila Castilho delivered an online lecture titled Context-aware MT evaluation: what have we learned? For the University of Surrey on Wednesday March 1st.
The challenge of evaluating translations in context has been raising interest in the machine translation (MT) field. However, the definition of what constitutes a document-level (doc-level) MT evaluation, in terms of how much of the text needs to be shown, is still unclear. Few works have taken into account doc-level human evaluation, and one common practice is the usage of test suites with context-aware markers.
But why do we need context in MT evaluation? Document-level evaluation of MT allows for a more thorough examination of the output quality with context, avoiding common cases of misevaluations. The main objective of the DELA Project is to define best practices for doc-level MT evaluation, and test the existing human and automatic sentence-level evaluation metrics to the doc-level.
In this talk, Dr. Castilho looked into the results of the project regarding methodologies for adding context in MT evaluation tasks, a document-level corpus annotated with context-related issues, and how much context span is enough to solve those issues.
Read more information>
