Deep Learning

some general deep learning techniques

Survey Papers / Books
Deep Learning Book [deeplearningbook.org]​
Deep learning [Nature]​
Resources
Courses
​Coursera Deep Learning Specialization by Andrew Ng
Frameworks
​TensorFlow / Keras​
​PyTorch / fastai​
​mxnet / GLUON​
hardly used now (2019.11)
​Caffe​
​Caffe2 (now part of PyTorch)
​Torch (Lua)
​Matlab / Octave​
Models
Optimizer
Stochastic Gradient Descent (SGD)
RMSProp
Momentum
AdaGrad
Adam [1412.6980]​
AdamW [1711.05101]​
AdaBound [1902.09843]​
AMSGrad [1904.09237]​
RAdam [LiyuanLucasLiu/RAdam] [1908.03265]​
Normalization
Batch Normalization [1502.03167]​
Weight Normalization [1602.07868]​
Layer Normalization [1607.06450]​
Instance Normalization [1607.08022]​
Spectral Normalization [1802.05957]​
Group Normalization [1803.08494]​
Switchable Normalization [1806.10779]​
Weight Initialization
Xavier Initialization [JMLR'10]​
He Initialization [1502.01852]​
Activation Function
Sigmoid
Tanh
ReLU
Leaky ReLU
Swish [1710.05941]​
Mish [digantamisra98/Mish] [1908.08681]​

Machine Learning - Previous

Traditionals

Next - Machine Learning

Image Classification (CNN)

Last modified 1yr ago