Deep learning is a vast field that is evolving fast. The following references and blogs are merly the tip of the iceberg but might give a good starting point to delve deeper into the subject.
- The Unreasonable Effectiveness of Recurrent Neural Networks
- Deep Learning, NLP, and Representations
- Understanding LSTM Networks
- Conv Nets: A Modular Perspective
- Recurrent Neural Networks Tutorial, Part 1 - Introduction to RNNs
- Recurrent Neural Networks Tutorial, Part 2 – Implementing a RNN with Python, Numpy and Theano
- Recurrent Neural Networks Tutorial, Part 3 – Backpropagation Through Time and Vanishing Gradients
- Recurrent Neural Network Tutorial, Part 4 – Implementing a GRU/LSTM RNN with Python and Theano
- Understanding Convolutional Neural Networks for NLP
- Implementing a CNN for Text Classification in TensorFlow
- Deep Learning for Chatbots, Part 1 – Introduction
- Deep Learning for Chatbots, Part 2 – Implementing a Retrieval-Based Model in Tensorflow
- Attention and Memory in Deep Learning and NLP
- An Explanation of Xavier Initialization
- An overview of gradient descent optimization algorithms
- A curated list of resources dedicated to recurrent neural networks
Convolutional Neural Networks
- Cireşan et al. (2011)
Recurrent Neural Networks
- Greff et al. (2015)
- Zaremba, Sutskever, and Vinyals (2014)
- Appleyard, Kociský, and Blunsom (2016)
Image Recognition and Object Detection
- Krizhevsky, Sutskever, and Hinton (2012)
- Szegedy et al. (2014)
Neural Machine Translation
- Bahdanau, Cho, and Bengio (2014)
- Vinyals et al. (2014)
- Jean et al. (2014)
- S. Bengio et al. (2015)
- Cho et al. (2014)
- Chiu and Nichols (2015)
- Tomas Mikolov (2013)
Appleyard, Jeremy, Tomás Kociský, and Phil Blunsom. 2016. “Optimizing Performance of Recurrent Neural Networks on GPUs.” CoRR abs/1604.01946. http://arxiv.org/abs/1604.01946.
Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2014. “Neural Machine Translation by Jointly Learning to Align and Translate.” CoRR abs/1409.0473. http://arxiv.org/abs/1409.0473.
Bengio, Samy, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. “Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks.” CoRR abs/1506.03099. http://arxiv.org/abs/1506.03099.
Chiu, Jason P. C., and Eric Nichols. 2015. “Named Entity Recognition with Bidirectional LSTM-CNNs.” CoRR abs/1511.08308. http://arxiv.org/abs/1511.08308.
Cho, Kyunghyun, Bart van Merrienboer, Çaglar Gülçehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. “Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation.” CoRR abs/1406.1078. http://arxiv.org/abs/1406.1078.
Cireşan, Dan C., Ueli Meier, Jonathan Masci, Luca M. Gambardella, and Jürgen Schmidhuber. 2011. “Flexible, High Performance Convolutional Neural Networks for Image Classification.” In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence - Volume Volume Two, 1237–42. IJCAI’11. Barcelona, Catalonia, Spain: AAAI Press. doi:10.5591/978-1-57735-516-8/IJCAI11-210.
Greff, Klaus, Rupesh Kumar Srivastava, Jan Koutník, Bas R. Steunebrink, and Jürgen Schmidhuber. 2015. “LSTM: A Search Space Odyssey.” CoRR abs/1503.04069. http://arxiv.org/abs/1503.04069.
Jean, Sébastien, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2014. “On Using Very Large Target Vocabulary for Neural Machine Translation.” CoRR abs/1412.2007. http://arxiv.org/abs/1412.2007.
Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. 2012. “ImageNet Classification with Deep Convolutional Neural Networks.” In NIPS, edited by Peter L. Bartlett, Fernando C. N. Pereira, Christopher J. C. Burges, Léon Bottou, and Kilian Q. Weinberger, 1106–14. https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.
Szegedy, Christian, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2014. “Going Deeper with Convolutions.” CoRR abs/1409.4842. http://arxiv.org/abs/1409.4842.
Tomas Mikolov, Geoffrey Zweig, Scott Wen-tau Yih. 2013. “Linguistic Regularities in Continuous Space Word Representations.” In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT-2013). Association for Computational Linguistics. https://www.microsoft.com/en-us/research/publication/linguistic-regularities-in-continuous-space-word-representations/.
Vinyals, Oriol, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey E. Hinton. 2014. “Grammar as a Foreign Language.” CoRR abs/1412.7449. http://arxiv.org/abs/1412.7449.
Zaremba, Wojciech, Ilya Sutskever, and Oriol Vinyals. 2014. “Recurrent Neural Network Regularization.” CoRR abs/1409.2329. http://arxiv.org/abs/1409.2329.