On Wednesday 29th November, representatives from two research and innovation projects—EASIER (Intelligent Automatic Sign Language Translation), SignON (Sign Language Translation Mobile Application and Open Communications Framework)—and the LEAD-ME (COST) Action met in Brussels to discuss the progress and results of the initiatives, and sign language research more broadly. This was part of the concertation event, “Outcomes and Outlooks of Research on Sign Language in Europe”, organised with the support of the European Commission and moderated by the European Union of the Deaf, a partner in both projects.
The event brought together researchers and end user community representatives who discussed the future of sign language technology in Europe.
In an opening address Member of European Parliament, Brando Benifei, called for accessibility for all citizens to be entrenched in the way we do things. He also stressed that we need to do more. This was echoed in a keynote presentation delivered by the President of the European Union of the Deaf (EUD), Sofia Isari, who noted that we must work together to ensure that the digital world is an accessible world for all.
The projects were introduced by Dimitar Shterionov (Scientific Coordinator for SignON), Eleni Efthimiou (Scientific Coordinator for EASIER), and Krishna Chandramouli (WG3 Leader of the ‘LEAD-ME’ European Cooperation in Science and Technology (COST) Action). Eleni Efthimiou highlighted that minority languages remain disconnected from technology, and that this has a major impact on inclusion.
A panel discussion moderated by Humberto Insolera (Executive Committee Member of the European Disability Forum) explored the way forward for sign language research in Europe, and included Frankie Picron (SignON, EASIER, EUD), Dimitar Shterionov (SignON), Eleni Efthimiou (EASIER), and Krishna Chandramouli (LEAD-ME). Discussing the co-creation process, Dimitar Shterionov highlighted that communication and trust with the community are the most important things to establish. Expanding on this, Eleni Efthimiou discussed the importance of providing features that the community has asked for, including elements of grammar and expressions of emotion. Krishna Chandramouli described the normalisation of captioning in media, and how we might use this momentum to include synthetically generated sign language. Frankie Picron commented on avatars, stressing that they will not replace humans, but rather offer an alternative option—and as he pointed out, “we need more options”. The panel discussed the different possible use cases for sign language machine translation, and reflected on SignON and EASIER—both projects have demonstrated the possibility of sign language machine translation, and have highlighted the challenges posed by lack of data.
A lunchtime session of poster presentations and demonstrations showcased a range of perspectives on this area of research and, in the afternoon, a series of technical presentations focused on specific aspects of the projects, and shared what the teams have learnt throughout the research process. Eleni Efthimiou discussed sign language processing technologies, emphasising that we need to join forces to ensure sign language technologies keep up with the overall technology landscape.
Dimitar Shterionov explained the sign language machine translation pipeline, and how important it is that we standardise the way we collect and analyse data. Expanding on this point, after the event, Shterionov explained:
“Since the inception of machine translation for spoken languages, language technology has evolved significantly, reaching unprecedented levels with end-to-end neural models trained on huge amounts of data. However, when it comes to signed languages, we need to turn to a more analytical approach, decomposing the translation into smaller, more focused tasks, i.e., recognition, translation, synthesis. But even more so, we need to involve the right people. At this stage, we need researchers, linguists, and developers composed of and/or working with deaf communities.”
Caro Brosens (SignON, VGTC) presented on the challenges with sign language data, and emphasised the importance of investing in high quality data and better technological environments to process this data so that we can use it. Brosens expanded on this by saying,
“The data bottleneck in sign language research is often reduced to ‘there is not enough data’ but the situation is a lot more nuanced than that. Not only is there not enough data, only a small amount of said data is of high quality and/or computer readable. Money should therefore not only be put towards creating new (high quality) data, but into developing the necessary technical environment to properly and efficiently process the data as well, otherwise the data remains unusable.”
Davy Van Landuyt (SignON, EASIER, EUD) presented on user engagement within the research process, and shared recommendations for future work—that sign language research projects must have deaf professionals at the steering wheel; that transparent and trustworthy co-creation and science communication processes, led and carried out by deaf professionals are essential to building a relationship with deaf communities; and that hearing researchers must be aware of their social responsibility.
Carlos Duarte (LEAD-ME) discussed web accessibility, noting that sign language must keep up with upcoming developments resulting from the strong progress being made on Artificial Intelligence (AI) models. Krishna Chandramouli explored the role of AI in digital accessibility as digital content becomes more pervasive—”AI technology offers new opportunities to be leveraged through content transformation methodologies”.
Closing the event, Frankie Picron (SignON, EASIER, and EUD) summarised the presentations and emphasised the importance of diversity and inclusivity in the consortium. The red thread or the central theme of the event was focused on advancing research on sign language technologies in Europe. Key elements included collaboration. Much like the European Union, where progress is contingent upon the collective efforts of all Member States, research consortia also need to involve all stakeholders to foster a way forward in sign language technology research, with the ultimate goal of enhancing the lives of the deaf communities within the European Union.
Watch the video in International Sign:
