“Be well-prepared & be insistent on your choice of topic”
PhD Candidate (2017-1019), completion time: 2.01 years.
Using modern machine learning and memory, Hung Le’s research is working towards A²I²’s ethos of advancing knowledge and building systems that can benefit society for the good.
Hung Le earned a scholarship for his PhD position at A²I² in August 2017 with A/Prof. Truyen Tran and Alfred Deakin Professor Svetha Venkatesh.
Hung’s research interest is biological and machinery memory mechanisms that support long-term episodic retrieval, inferential reasoning, and hybrid temporal adaptions in artificial neural networks. He aims to build a cognitive architecture using deep learning. Hung believes memory is the core of intelligence and his architecture will revolve around it.
During his candidature, Hung was also involved in teaching at Deakin University, as a teaching assistant for Unit SIT-112 in 2018, and reviewing for a number of journals (TKDE, KAIS, Journal of Machine Learning) and conferences (ACML 2017 – 2020, NeurIPS 2020).
Hung’s PhD thesis is about memory and attention in deep learning. Under the supervision of Associate Professor Truyen Tran and Alfred Deakin Professor Svetha Venkatesh, Hung has five papers accepted at top-tier peer-reviewed conferences: PAKDD 2018, KDD 2018, NeurlPS 018, ICLR 2019 (Oral) and ICLR 2020 (see Hung’s publication below)
Hung said: “A PhD is an interesting journey. I was lucky as before starting my PhD in deep learning, I had spent years building a background in the field. This is important as a PhD in Australia takes around 3 years, thus normally, PhD students here do not have time to learn the basics during the course. Moreover, choosing the right topic is not as important as consistently sticking with it. Diving deeper into the memory network, I gained more interests and new ideas. Papers are a by-product of the process of studying something long enough”.
Hung’s advice for prospective PhD candidates:
- Be well-prepared before starting a PhD
- Be insistent on your choice of topic
- Do not aim too high for the first stage
Hung is currently working as an Associate Research Fellow at A2I2, on several projects. The main one is opening a new research direction on neural memory—the neural stored-program concept. His other side projects are focusing on temporal relational reasoning, episodic reinforcement learning, memorisation optimisation and generative memory.
His ultimate research goal is a multi-stage plan. The first stage will be maintaining his track record and reputation in the field by regularly publishing quality papers. The second stage will be creating his own research space and to attract other researchers in the field to join. Hung is interested in a research direction on neural memory, which hopefully can capture the interest of a new generation of young researchers who are tired of traditional concepts.
Hung’s publications during his candidature are:
- Neural Stored-program Memory. Hung Le, Truyen Tran, Svetha Venkatesh. Published in ICLR’20
- Learning to Remember More with Less Memorization. Hung Le, Truyen Tran, Svetha Venkatesh. Published in ICLR’19 (Oral)
- Variational Memory Encoder-Decoder. Hung Le, Truyen Tran, Thin Nguyen, Svetha Venkatesh. Published in NeurIPS’18
- Dual Memory Neural Computer for Asynchronous Two-view Sequential Learning. Hung Le, Truyen Tran, Svetha Venkatesh. Published in KDD’18
- Dual Control Memory Augmented Neural Networks for Treatment Recommendations. Hung Le, Truyen Tran, Svetha Venkatesh. Published in PAKDD’18.