📝
Tzu-Heng's wiki
  • Tzu-Heng's wiki
  • Machine Learning
    • Traditionals
    • Deep Learning
    • Image Classification (CNN)
    • Detection
    • Semantic Segmentation
    • Generative Adversarial Networks
    • Style Transfer
    • Recommender Systems
    • Meta Learning
  • Notes
    • Differientiable Sampling and Argmax
    • GAN theory
    • Multi-task Learning (MTL)
    • Disentanglement in GANs
    • CNN practical notes
    • 3D Clothes
    • OpenGL
    • Generative Art
    • nginx usage
    • Deploy Deep Learning Models
    • Character Motion Synthesis
  • Data Structure & Algorithms
    • Sorting Algorithms
Powered by GitBook
On this page
  • Survey Papers / Books
  • Resources
  • Courses
  • Frameworks
  • Models
  • Optimizer
  • Normalization
  • Weight Initialization
  • Activation Function

Was this helpful?

  1. Machine Learning

Deep Learning

some general deep learning techniques

PreviousTraditionalsNextImage Classification (CNN)

Last updated 3 years ago

Was this helpful?

Survey Papers / Books

  • Deep Learning Book

  • Deep learning

Resources

Courses

  • by Andrew Ng

Frameworks

  • /

  • /

  • /

  • hardly used now (2019.11)

    • (now part of PyTorch)

    • (Lua)

    • /

Models

Optimizer

  • Stochastic Gradient Descent (SGD)

  • RMSProp

  • Momentum

  • AdaGrad

Normalization

Weight Initialization

Activation Function

  • Sigmoid

  • Tanh

  • ReLU

  • Leaky ReLU

Adam

AdamW

AdaBound

AMSGrad

RAdam

Batch Normalization

Weight Normalization

Layer Normalization

Instance Normalization

Spectral Normalization

Group Normalization

Switchable Normalization

Xavier Initialization

He Initialization

Swish

Mish

[deeplearningbook.org]
[Nature]
Coursera Deep Learning Specialization
TensorFlow
Keras
PyTorch
fastai
mxnet
GLUON
Caffe
Caffe2
Torch
Matlab
Octave
[1412.6980]
[1711.05101]
[1902.09843]
[1904.09237]
[LiyuanLucasLiu/RAdam]
[1908.03265]
[1502.03167]
[1602.07868]
[1607.06450]
[1607.08022]
[1802.05957]
[1803.08494]
[1806.10779]
[JMLR'10]
[1502.01852]
[1710.05941]
[digantamisra98/Mish]
[1908.08681]