📝
Tzu-Heng's wiki
  • Tzu-Heng's wiki
  • Machine Learning
    • Traditionals
    • Deep Learning
    • Image Classification (CNN)
    • Detection
    • Semantic Segmentation
    • Generative Adversarial Networks
    • Style Transfer
    • Recommender Systems
    • Meta Learning
  • Notes
    • Differientiable Sampling and Argmax
    • GAN theory
    • Multi-task Learning (MTL)
    • Disentanglement in GANs
    • CNN practical notes
    • 3D Clothes
    • OpenGL
    • Generative Art
    • nginx usage
    • Deploy Deep Learning Models
    • Character Motion Synthesis
  • Data Structure & Algorithms
    • Sorting Algorithms
Powered by GitBook
On this page
  • Train CIFAR10
  • Pretrained models

Was this helpful?

  1. Notes

CNN practical notes

some notes on my training with CNN

Train CIFAR10

  • On 2019.12.6, I tried to use torchvision.models.resnet18 to train CIFAR10

  • Some findings in time sequential orders

    • without weight decay, model can only reach 70% something.

    • pretrained weights from ImageNet does not actually helps.

    • using torchvision.models.resnet18 can only achieve 85% something accuracy. This is because torchvision.models is perfectly tuned for ImageNet, and when training on other datasets, the results usually won't went well.

      • See also

        • How to train resnet18 to the best accuracy? #1166

        • Adding "narrow" ResNet models for CIFAR-10 #1570

        • https://github.com/akamaster/pytorch_resnet_cifar10 (I use this implementation and achieve comparable claimed test set accuracy 8.27%)

        • https://github.com/kuangliu/pytorch-cifar

  • Takeaway

    • weight decay is important, yet torch.optim disable it by default. set it to 1e-4 or 5e-4.

    • It's not preferrable to directly use torchvision.models or other pretrained model architectures on datasets other than ImageNet. That's what so called 'Hyperparameter tuning is important'.

Pretrained models

  • https://github.com/rwightman/pytorch-image-models

  • https://github.com/Cadene/pretrained-models.pytorch

PreviousDisentanglement in GANsNext3D Clothes

Last updated 3 years ago

Was this helpful?