Posted: 15/11/18
The ADAPT Centre received the best-paper award at the 27th International Conference on Artificial Neural Networks (ICANN 2018) for Dr Joeran Beel and Mark Collier’s paper ‘Implementing Neural Turing Machines’. Trinity College Dublin (TCD) student, and co-author of the paper, Mark Collier was at ICANN to present the work about implementing a Neural Turing Machine. Mark Collier was an undergraduate student at TCD and the paper resulted from his thesis.
Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. Neural Turing Machines have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implementations of Neural Turing Machines exist but are unstable during training and/or fail to replicate the reported performance of NTMs. This paper presents the details of the team’s successful implementation of a Neural Turing Machine. The implementation learns to solve three sequential learning tasks from the original NTM paper. The team found that the choice of memory contents initialisation scheme is crucial in successfully implementing a Neural Turing Machine. Networks with memory contents initialised to small constant values converge on average twice as fast than the next best memory contents initialisation scheme.
Read more about the paper here.
Share this article:
