 有三的新書《深度學(xué)習(xí)之模型優(yōu)化：核心算法與案例實踐》已經(jīng)正式上市，本次書籍為我寫作并出版的第7本書籍，大家可以在我們的知識平臺或者京東/當(dāng)當(dāng)?shù)鹊赇佭M(jìn)行購買，本書配套的代碼與數(shù)據(jù)資源也已經(jīng)正式上傳我們的官方GitHub項目，請大家知悉。 開源代碼項目 有三AI社區(qū)維護(hù)了一個匯總了我們的免費實踐案例，紙質(zhì)版書籍資源、電子版開源學(xué)習(xí)手冊的綜合性GitHub項目，地址為： https://github.com/longpeng2008/yousan.ai  新書《深度學(xué)習(xí)之模型優(yōu)化：核心算法與案例實踐》的相關(guān)資料已經(jīng)上傳，資源預(yù)覽界面如下，請購買了書籍的朋友及時獲??！  本書內(nèi)容 本書是深度學(xué)習(xí)模型使用系列書籍中的第二本，內(nèi)容上承前啟后。本書是在該系列第一本書《深度學(xué)習(xí)之模型設(shè)計》的基礎(chǔ)上講解更深入的模型設(shè)計與壓縮方法。 本書的第4章講解了輕量級模型設(shè)計方法，本書的第8章講解了自動化模型設(shè)計方法，它們都可以看作是對《深度學(xué)習(xí)之模型設(shè)計》書籍內(nèi)容的補(bǔ)充。而剩下的模型剪枝，模型量化，模型蒸餾，則是模型壓縮與優(yōu)化最核心的技術(shù)。除此之外，我們也補(bǔ)充了模型可視化的內(nèi)容，以便讀者增加對模型的理解。 而在本書的第9章中，我們簡單介紹了一些常用的開源模型優(yōu)化和部署工具，這也是為下一本書，即模型部署書籍進(jìn)行了提前的鋪墊。
 全書正文約230頁，共計9章，目錄如下：
第1章 引言 本章對人工智能技術(shù)發(fā)展的重要要素，數(shù)據(jù)、模型、框架、硬件進(jìn)行了介紹，充足的數(shù)據(jù)配合優(yōu)秀的模型才能學(xué)習(xí)到復(fù)雜的知識，框架和硬件則是完成模型學(xué)習(xí)不可或缺的軟硬件設(shè)施，希望讀者能夠在閱讀本章內(nèi)容后，充分認(rèn)識到人工智能本質(zhì)上是一門綜合性的工程技術(shù)。
第2章 模型性能評估 本章介紹了常用的模型性能評估指標(biāo)，包括參數(shù)量、計算量、內(nèi)存訪問、計算速度等，最后介紹了工業(yè)界的一個模型壓縮相關(guān)競賽。
第3章 模型可視化 本章系統(tǒng)性地介紹了模型可視化的內(nèi)容，包括模型結(jié)構(gòu)可視化、參數(shù)與特征可視化、輸入?yún)^(qū)域可視化以及激活模式可視化，通過掌握相關(guān)原理和3個典型的實踐案例，我們可以更深入地理解模型的性能表現(xiàn)以及參數(shù)細(xì)節(jié)，從而為設(shè)計和改進(jìn)模型結(jié)構(gòu)提供指導(dǎo)思想。
第4章 輕量級模型設(shè)計 本章系統(tǒng)性地介紹了當(dāng)下輕量級模型設(shè)計的方法，包括卷積核的使用和設(shè)計、卷積拆分與分組設(shè)計、參數(shù)與特征重用設(shè)計、動態(tài)自適應(yīng)模型設(shè)計、卷積乘法操作的優(yōu)化和設(shè)計、重參數(shù)化技術(shù)、新穎算子設(shè)計、低秩稀疏化技術(shù)。通過在一開始就使用輕量級的基礎(chǔ)模型架構(gòu)，可以大大減少后續(xù)對其進(jìn)一步進(jìn)行模型壓縮與優(yōu)化的工作量，因此這也是本書中非常核心的內(nèi)容。
第5章 模型剪枝 本章介紹了模型剪枝的主要算法理論與實踐，主要包括模型稀疏正則化技術(shù)，非結(jié)構(gòu)化模型剪枝與結(jié)構(gòu)化模型剪枝等算法，最后通過案例實踐讓讀者掌握結(jié)構(gòu)化模型剪枝中原始模型的訓(xùn)練與訓(xùn)練后的稀疏裁剪。
第6章 模型量化 本章介紹了模型量化的主要算法理論與實踐，主要包括1bit量化，對稱與非對稱的8bit量化，混合量化等算法，最后通過案例實踐讓讀者掌握對稱的8bit量化方法代碼實現(xiàn)以及基于TensorRT框架的模型量化與推理流程。
第7章 遷移學(xué)習(xí)與知識蒸餾 本章介紹了模型蒸餾的主要算法理論與實踐，主要包括基于優(yōu)化目標(biāo)與結(jié)構(gòu)匹配的模型蒸餾算法，最后通過案例實踐讓讀者掌握經(jīng)典的知識蒸餾框架的模型訓(xùn)練，比較學(xué)生模型在蒸餾前后的性能變化。
第8章 自動化模型設(shè)計 本章介紹了自動化模型設(shè)計中神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)搜索技術(shù)，主要包括基于柵格搜索的神經(jīng)網(wǎng)絡(luò)搜索方法，基于強(qiáng)化學(xué)習(xí)的神經(jīng)網(wǎng)絡(luò)搜索方法，基于進(jìn)化算法的神經(jīng)網(wǎng)絡(luò)搜索方法，可微分神經(jīng)網(wǎng)絡(luò)搜索方法。自動化模型設(shè)計是難度較高的工程技術(shù)，也是模型設(shè)計與壓縮的最終發(fā)展形態(tài)。
第9章 模型優(yōu)化與部署工具 本章介紹了當(dāng)下工業(yè)界常用的開源模型優(yōu)化和部署工具，主要包括Tensorflow、PaddlePaddle、Pytorch生態(tài)相關(guān)的模型優(yōu)化工具，各類通用的移動端模型推理框架以及ONNX標(biāo)準(zhǔn)與NVIDIA的模型優(yōu)化與部署工具TensorRT，并基于NCNN框架在嵌入式硬件上進(jìn)行了部署實戰(zhàn)。熟練掌握好模型優(yōu)化與部署工具，是深度學(xué)習(xí)算法工程師的必修課，本章內(nèi)容可供大家作為入門參考，更加系統(tǒng)的模型部署內(nèi)容，將在本系列書籍的下一本中進(jìn)行講解。
詳細(xì)內(nèi)容請大家直接閱讀書籍。本書內(nèi)容由淺入深，講解圖文并茂，緊隨工業(yè)界和學(xué)術(shù)界的最新發(fā)展，理論和實踐緊密結(jié)合，給出了大量的圖表與案例分析。本書拋開了過多的數(shù)學(xué)理論，完整地剖析了模型壓縮與優(yōu)化的主流技術(shù)，不是只停留于理論的闡述和簡單的結(jié)果展示，更是從夯實理論到完成實戰(zhàn)一氣呵成。相信讀者跟隨著本書進(jìn)行學(xué)習(xí)，將會對深度學(xué)習(xí)領(lǐng)域的模型壓縮技術(shù)有更深的理解。本書是一本專門講解深度學(xué)習(xí)模型壓縮與優(yōu)化，尤其是針對深度卷積神經(jīng)網(wǎng)絡(luò)的書籍，本書內(nèi)容屬于深度學(xué)習(xí)領(lǐng)域中高級內(nèi)容，對讀者的基礎(chǔ)有一定的要求，建議預(yù)先學(xué)習(xí)CNN模型設(shè)計基礎(chǔ)知識。 【本書所有實戰(zhàn)算法代碼統(tǒng)一使用Pytorch框架，TensorRT+Jetson開發(fā)版推理代碼使用Python語言，NCNN+EAIDK-610開發(fā)版部署代碼使用C++語言】。 本書參考資料 由于出版社管控原因，本書所有的參考資料出處都被刪除，因此我們在這里列出所有的參考資料列表供大家索引。 第三章，模型可視化參考資料如下： [1] Erhan D, Bengio Y, Courville A, et al. Visualizing higher-layer features of a deep network[J]. University of Montreal, 2009, 1341(3): 1. [2] Simonyan K, Vedaldi A, Zisserman A. Deep inside convolutional networks: Visualising image classification models and saliency maps[J]. arXiv preprint arXiv:1312.6034, 2013. [3] Yosinski J, Clune J, Nguyen A, et al. Understanding neural networks through deep visualization[J]. arXiv preprint arXiv:1506.06579, 2015. [4] Mordvintsev A, Olah C, Tyka M. Inceptionism: Going deeper into neural networks[J]. 2015. [5] Nguyen A, Dosovitskiy A, Yosinski J, et al. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks[J]. Advances in neural information processing systems, 2016, 29. [6] Zeiler M D, Fergus R. Visualizing and understanding convolutional networks[C]//European conference on computer vision. Springer, Cham, 2014: 818-833. [7] Mahendran A, Vedaldi A. Understanding deep image representations by inverting them[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 5188-5196. [8] Mahendran A, Vedaldi A. Visualizing deep convolutional neural networks using natural pre-images[J]. International Journal of Computer Vision, 2016, 120(3): 233-255. [9]Dosovitskiy A, Brox T. Inverting visual representations with convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 4829-4837. [10] Wei, Donglai , et al. "Understanding Intra-Class Knowledge Inside CNN.", 10.48550/arXiv.1507.02379. 2015. [11] Zhou B , Khosla A , Lapedriza A , et al. Object detectors emerge in Deep Scene CNNs[J]. Computer Science, 2014. [12] Bau D , Zhou B , Khosla A , et al. Network Dissection: Quantifying Interpretability of Deep Visual Representations[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017. [13] Smilkov D, Thorat N, Kim B, et al. Smoothgrad: removing noise by adding noise[J]. arXiv preprint arXiv:1706.03825, 2017. [14] Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks[C]//International conference on machine learning. PMLR, 2017: 3319-3328. [15] Springenberg J T, Dosovitskiy A, Brox T, et al. Striving for simplicity: The all convolutional net[J]. arXiv preprint arXiv:1412.6806, 2014. [16] Zhou B , Khosla A , Lapedriza A , et al. Learning Deep Features for Discriminative Localization[C]// CVPR. IEEE Computer Society, 2016. [17] Selvaraju R R , Cogswell M , Das A , et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization[J]. International Journal of Computer Vision, 2020, 128(2):336-359. [18] Wang Z J, Turko R, Shaikh O, et al. CNN explainer: learning convolutional neural networks with interactive visualization[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 27(2): 1396-1406. [19] Liu M, Shi J, Li Z, et al. Towards better analysis of deep convolutional neural networks[J]. IEEE transactions on visualization and computer graphics, 2016, 23(1): 91-100. 第四章，輕量級模型設(shè)計參考資料如下： [1] Lin M, Chen Q, Yan S. Network in network[J]. arXiv preprint arXiv:1312.4400, 2013. [2] Zeiler M D, Fergus R. Visualizing and understanding convolutional networks[C]//European conference on computer vision. Springer, Cham, 2014: 818-833. [3] Iandola F N, Han S, Moskewicz M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size[J]. arXiv preprint arXiv:1602.07360, 2016. [4] Jin J, Dundar A, Culurciello E. Flattened convolutional neural networks for feedforward acceleration[J]. arXiv preprint arXiv:1412.5474, 2014. [5] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 2818-2826. [6] Sifre L , Mallat, Stéphane. Rigid-Motion Scattering for Texture Classification[J]. Computer Science, 2014. [7] Chollet F. Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1251-1258. [8] Howard A G, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017. [9]Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4510-4520. [10] Zhang X, Zhou X, Lin M, et al. Shufflenet: An extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 6848-6856. [11] Ma N, Zhang X, Zheng H T, et al. Shufflenet v2: Practical guidelines for efficient cnn architecture design[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 116-131. [12] Zhang T, Qi G J, Xiao B, et al. Interleaved group convolutions[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 4373-4382. [13] Xie G, Wang J, Zhang T, et al. Interleaved structured sparse convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 8847-8856. [14] Sun K, Li M, Liu D, et al. Igcv3: Interleaved low-rank group convolutions for efficient deep neural networks[J]. arXiv preprint arXiv:1806.00178, 2018. [15] Tan M, Le Q V. MixNet: Mixed Depthwise Convolutional Kernels[J]. arXiv preprint arXiv:1907.09595, 2019. [16] Chen C F, Fan Q, Mallinar N, et al. Big-little net: An efficient multi-scale feature representation for visual and speech recognition[J]. arXiv preprint arXiv:1807.03848, 2018. [17] Chen Y, Fang H, Xu B, et al. Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution[J]. arXiv preprint arXiv:1904.05049, 2019. [18] Gennari M, Fawcett R, Prisacariu V A. DSConv: Efficient Convolution Operator[J]. arXiv preprint arXiv:1901.01928, 2019. [19] Shang W, Sohn K, Almeida D, et al. Understanding and improving convolutional neural networks via concatenated rectified linear units[C]//international conference on machine learning. PMLR, 2016: 2217-2225. [20] Han K, Wang Y, Tian Q, et al. GhostNet: More Features from Cheap Operations.[J]. arXiv: Computer Vision and Pattern Recognition, 2019. [21] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700-4708. [22] Jin X, Yang Y, Xu N, et al. Wsnet: Compact and efficient networks through weight sampling[C]//International Conference on Machine Learning. PMLR, 2018: 2352-2361. [23] Zhou D, Jin X, Wang K, et al. Deep model compression via filter auto-sampling[J]. arXiv preprint, 2019. [24] Huang G, Sun Y, Liu Z, et al. Deep networks with stochastic depth[C]//European conference on computer vision. Springer, Cham, 2016: 646-661. [25] Veit A, Wilber M J, Belongie S. Residual networks behave like ensembles of relatively shallow networks[C]//Advances in neural information processing systems. 2016: 550-558. [26] Teerapittayanon S, McDanel B, Kung H T. Branchynet: Fast inference via early exiting from deep neural networks[C]//2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, 2016: 2464-2469. [27] Graves A. Adaptive computation time for recurrent neural networks[J]. arXiv preprint arXiv:1603.08983, 2016. [28] Figurnov M, Collins M D, Zhu Y, et al. Spatially adaptive computation time for residual networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1039-1048. [29] Wu Z, Nagarajan T, Kumar A, et al. Blockdrop: Dynamic inference paths in residual networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 8817-8826. [30] Wang X, Yu F, Dou Z Y, et al. Skipnet: Learning dynamic routing in convolutional networks[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 409-424. [31] Almahairi A, Ballas N, Cooijmans T, et al. Dynamic capacity networks[C]//International Conference on Machine Learning. PMLR, 2016: 2549-2558. [32] Wu B, Wan A, Yue X, et al. Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions[C]. computer vision and pattern recognition, 2018: 9127-9135. [33] Chen W, Xie D, Zhang Y, et al. All you need is a few shifts: Designing efficient convolutional neural networks for image classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 7241-7250. [34] He Y, Liu X, Zhong H, et al. Addressnet: Shift-based primitives for efficient convolutional neural networks[C]//2019 IEEE Winter conference on applications of computer vision (WACV). IEEE, 2019: 1213-1222. [35] Chen H, Wang Y, Xu C, et al. AdderNet: Do we really need multiplications in deep learning?[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 1468-1477. [36] You H, Chen X, Zhang Y, et al. Shiftaddnet: A hardware-inspired deep network[J]. Advances in Neural Information Processing Systems, 2020, 33: 2771-2783. [37] Li D, Wang X, Kong D. Deeprebirth: Accelerating deep neural network execution on mobile devices[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2018, 32(1). [38] Ding X, Zhang X, Ma N, et al. Repvgg: Making vgg-style convnets great again[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 13733-13742. [39] Li D, Hu J, Wang C, et al. Involution: Inverting the inherence of convolution for visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 12321-12330. [40] Denton E L, Zaremba W, Bruna J, et al. Exploiting linear structure within convolutional networks for efficient evaluation[C]//Advances in Neural Information Processing Systems. 2014: 1269-1277. 第五章，模型剪枝參考資料如下： [1] Wen W , Wu C , Wang Y , et al. Learning Structured Sparsity in Deep Neural Networks[J]. 2016. [2] Liu Z, Li J, Shen Z, et al. Learning efficient convolutional networks through network slimming[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 2736-2744. [3] Luo J, Wu J. AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference[J]. arXiv: Computer Vision and Pattern Recognition, 2018. [4] Huang Z, Wang N. Data-driven sparse structure selection for deep neural networks[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 304-320. [5] LeCun Y, Denker J S, Solla S A. Optimal brain damage[C]//Advances in neural information processing systems. 1990: 598-605. [6] Lee N, Ajanthan T, Torr P H S. Snip: Single-shot network pruning based on connection sensitivity[J]. arXiv preprint arXiv:1810.02340, 2018. [7] Han S, Pool J, Tran J, et al. Learning both weights and connections for efficient neural network[C]//Advances in neural information processing systems. 2015: 1135-1143. [8] Guo Y, Yao A, Chen Y. Dynamic network surgery for efficient dnns[C]//Advances In Neural Information Processing Systems. 2016: 1379-1387. [9] Anwar S , Hwang K , Sung W . Structured Pruning of Deep Convolutional Neural Networks[J]. Acm Journal on Emerging Technologies in Computing Systems, 2015. [10] Li H, Kadav A, Durdanovic I, et al. Pruning Filters for Efficient ConvNets[J]. arXiv: Computer Vision and Pattern Recognition, 2016. [11] Hu H, Peng R, Tai Y W, et al. Network trimming: A data-driven neuron pruning approach towards efficient deep architectures[J]. arXiv preprint arXiv:1607.03250, 2016. [12] He Y, Liu P, Wang Z, et al. Filter pruning via geometric median for deep convolutional neural networks acceleration[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 4340-4349. [13] He Y, Zhang X, Sun J. Channel pruning for accelerating very deep neural networks[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 1389-1397. [14] Luo J H, Zhang H, Zhou H Y, et al. Thinet: pruning cnn filters for a thinner net[J]. IEEE transactions on pattern analysis and machine intelligence, 2018. [15] Molchanov P, Tyree S, Karras T, et al. Pruning Convolutional Neural Networks for Resource Efficient Inference[C]. international conference on learning representations, 2017. [16] Zhuang Z, Tan M, Zhuang B, et al. Discrimination-aware Channel Pruning for Deep Neural Networks[C]. neural information processing systems, 2018: 883-894. [17] Liu Z, Sun M, Zhou T, et al. Rethinking the value of network pruning[J]. arXiv preprint arXiv:1810.05270, 2018. [18] Zhu M, Gupta S. To prune, or not to prune: exploring the efficacy of pruning for model compression[J]. arXiv: Machine Learning, 2017. [19] Yu R, Li A, Chen C F, et al. Nisp: Pruning networks using neuron importance score propagation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 9194-9203. [20] Lin J, Rao Y, Lu J, et al. Runtime Neural Pruning[C]. neural information processing systems, 2017: 2181-2191. [21] Ye J, Lu X, Lin Z, et al. Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers[J]. arXiv preprint arXiv:1802.00124, 2018. [22] Lee N, Ajanthan T, Torr P H, et al. SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY[C]. international conference on learning representations, 2019. 第六章，模型量化參考資料如下： [1] Courbariaux M, Bengio Y, David J, et al. BinaryConnect: training deep neural networks with binary weights during propagations[C]. neural information processing systems, 2015: 3123-3131. [2] Courbariaux M, Hubara I, Soudry D, et al. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1[J]. arXiv preprint arXiv:1602.02830, 2016. [3] Liu Z, Wu B, Luo W, et al. Bi-Real Net: Enhancing the Performance of 1-Bit CNNs with Improved Representational Capability and Advanced Training Algorithm[C]. european conference on computer vision, 2018: 747-763. [4] Rastegari M, Ordonez V, Redmon J, et al. Xnor-net: Imagenet classification using binary convolutional neural networks[C]//European conference on computer vision. Springer, Cham, 2016: 525-542. [5] Bulat A, Tzimiropoulos G. Xnor-net++: Improved binary neural networks[J]. arXiv preprint arXiv:1909.13863, 2019. [6] Li F, Zhang B, Liu B. Ternary weight networks[J]. arXiv preprint arXiv:1605.04711, 2016. [7] Zhu C, Han S, Mao H, et al. Trained ternary quantization[J]. arXiv preprint arXiv:1612.01064, 2016. [8] Ding R, Chin T, Liu Z, et al. Regularizing Activation Distribution for Training Binarized Deep Networks[C]. computer vision and pattern recognition, 2019: 11408-11417. [9] Darabi S, Belbahri M, Courbariaux M, et al. Regularized binary network training[J]. arXiv preprint arXiv:1812.11800, 2018. [10] Bulat A, Tzimiropoulos G, Kossaifi J, et al. Improved training of binary networks for human pose estimation and image recognition[J]. arXiv preprint arXiv:1904.05868, 2019. [11] Martinez B, Yang J, Bulat A, et al. Training binary neural networks with real-to-binary convolutions[J]. arXiv preprint arXiv:2003.11535, 2020. [12] Liu Z, Shen Z, Savvides M, et al. Reactnet: Towards precise binary neural network with generalized activation functions[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV 16. Springer International Publishing, 2020: 143-159. [13] Zhang Y, Pan J, Liu X, et al. FracBNN: Accurate and FPGA-efficient binary neural networks with fractional activations[C]//The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2021: 171-182. [14] Zhang Y, Zhang Z, Lew L. Pokebnn: A binary pursuit of lightweight accuracy[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 12475-12485. [15] Guo N, Bethge J, Meinel C, et al. Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticket[J]. arXiv preprint arXiv:2211.12933, 2022. [16] Jacob B, Kligys S, Chen B, et al. Quantization and training of neural networks for efficient integer-arithmetic-only inference[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 2704-2713. [17] Hubara I, Courbariaux M, Soudry D, et al. Quantized neural networks: Training neural networks with low precision weights and activations[J]. The Journal of Machine Learning Research, 2017, 18(1): 6869-6898. [18] Zhou S, Wu Y, Ni Z, et al. DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients[J]. arXiv: Neural and Evolutionary Computing, 2016. [19] Wang K, Liu Z, Lin Y, et al. HAQ: Hardware-Aware Automated Quantization with Mixed Precision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019: 8612-8620. [20] Micikevicius P, Narang S, Alben J, et al. Mixed precision training[J]. arXiv preprint arXiv:1710.03740, 2017. [21] Han S, Mao H, Dally W J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding[J]. arXiv preprint arXiv:1510.00149, 2015. [22] Zhang D, Yang J, Ye D, et al. LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks[C]. european conference on computer vision, 2018: 373-390. [23] Choi J, Wang Z, Venkataramani S, et al. Pact: Parameterized clipping activation for quantized neural networks[J]. arXiv preprint arXiv:1805.06085, 2018. [24] Zhou A, Yao A, Guo Y, et al. Incremental network quantization: Towards lossless cnns with low-precision weights[J]. arXiv preprint arXiv:1702.03044, 2017. [25] Zhu F, Gong R, Yu F, et al. Towards Unified INT8 Training for Convolutional Neural Network.[J]. arXiv: Learning, 2019. 第七章，模型蒸餾參考資料如下： [1] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015, 2(7). [2] Xu Z, Hsu Y, Huang J, et al. Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks.[J]. arXiv: Learning, 2017. [3] Ravi S. Projectionnet: Learning efficient on-device deep networks using neural projections[J]. arXiv preprint arXiv:1708.00630, 2017. [4] Romero A, Ballas N, Kahou S E, et al. Fitnets: Hints for thin deep nets[J]. arXiv preprint arXiv:1412.6550, 2014. [5] Huang Z, Wang N. Like What You Like: Knowledge Distill via Neuron Selectivity Transfer.[J]. arXiv: Computer Vision and Pattern Recognition, 2017. [6] Zagoruyko S, Komodakis N. Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer[C]. international conference on learning representations, 2017. [7] Yim J, Joo D, Bae J, et al. A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning[C]. computer vision and pattern recognition, 2017: 7130-7138. [8] Zhang Y, Xiang T, Hospedales T M, et al. Deep mutual learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 4320-4328. [9] Zhang L, Song J, Gao A, et al. Be your own teacher: Improve the performance of convolutional neural networks via self distillation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 3713-3722. [10] Furlanello T, Lipton Z C, Tschannen M, et al. Born Again Neural Networks[C]. international conference on machine learning, 2018: 1602-1611. [11] Cho J H, Hariharan B. On the efficacy of knowledge distillation[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 4794-4802. [12] Yuan L, Tay F E, Li G, et al. Revisit Knowledge Distillation: a Teacher-free Framework.[J]. arXiv: Computer Vision and Pattern Recognition, 2019. 第八章，自動化模型設(shè)計參考資料如下： [1] Cubuk E D, Zoph B, Mane D, et al. AutoAugment: Learning Augmentation Policies from Data.[J]. arXiv: Computer Vision and Pattern Recognition, 2018. [2] Zoph B, Cubuk E D, Ghiasi G, et al. Learning Data Augmentation Strategies for Object Detection[J]. arXiv preprint arXiv:1906.11172, 2019. [3] Ho D, Liang E, Stoica I, et al. Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules[J]. arXiv preprint arXiv:1905.05393, 2019. [4] Eger S, Youssef P, Gurevych I. Is it time to swish? comparing deep learning activation functions across NLP tasks[J]. arXiv preprint arXiv:1901.02671, 2019. [5] Luo P, Ren J, Peng Z, et al. Differentiable learning-to-normalize via switchable normalization[J]. arXiv preprint arXiv:1806.10779, 2018. [6] Bello I, Zoph B, Vasudevan V, et al. Neural optimizer search with reinforcement learning[C]//International Conference on Machine Learning. PMLR, 2017: 459-468. [7] Tan M, Le Q V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks[C]. international conference on machine learning, 2019: 6105-6114. [8] Tan M, Le Q V. MixNet: Mixed Depthwise Convolutional Kernels[J]. arXiv preprint arXiv:1907.09595, 2019. [9] Zoph B, Le Q V. Neural Architecture Search with Reinforcement Learning[J]. international conference on learning representations, 2017. [10] Zoph B, Vasudevan V, Shlens J, et al. Learning Transferable Architectures for Scalable Image Recognition[J]. computer vision and pattern recognition, 2018: 8697-8710. [11] Tan M, Chen B, Pang R, et al. MnasNet: Platform-Aware Neural Architecture Search for Mobile[J]. arXiv: Computer Vision and Pattern Recognition, 2018. [12] Xie L, Yuille A. Genetic cnn[C]//Proceedings of the IEEE international conference on computer vision. 2017: 1379-1388. [13] Real E , Aggarwal A , Huang Y , et al. Regularized Evolution for Image Classifier Architecture Search[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 33. [14] Liu H, Simonyan K, Yang Y, et al. DARTS: Differentiable Architecture Search[J]. arXiv: Learning, 2018. [15] Cai H, Zhu L, Han S. Proxylessnas: Direct neural architecture search on target task and hardware[J]. arXiv preprint arXiv:1812.00332, 2018. [16] He Y, Lin J, Liu Z, et al. Amc: Automl for model compression and acceleration on mobile devices[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 784-800. [17] Wang K, Liu Z, Lin Y, et al. HAQ: Hardware-Aware Automated Quantization with Mixed Precision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019: 8612-8620. [18] Ashok A, Rhinehart N, Beainy F, et al. N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning[J]. arXiv: Learning, 2017. [19] Pham H, Guan M, Zoph B, et al. Efficient neural architecture search via parameters sharing[C]//International conference on machine learning. PMLR, 2018: 4095-4104. 更多參考資料，請大家在學(xué)習(xí)過程中自行查找索引。 限量簽名版購買 如果你想要簽名版書籍，可以直接在我們課程平臺下單購買，【專屬簽名版書籍】如下。
購買地址如下，購買完成后【憑訂單記錄聯(lián)系有三本人微信Longlongtogo】，發(fā)送收貨地址。（發(fā)票可開）。 另外，即日起參加有三AI-CV中階-模型算法組，跟隨我們社區(qū)系統(tǒng)性地學(xué)習(xí)模型設(shè)計、優(yōu)化、部署相關(guān)內(nèi)容，本書為配套輔導(dǎo)教材，將隨組贈送，另外還有模型設(shè)計書籍一起贈送。 【一對一小組】2024年有三AI-CV中階-模型算法組發(fā)布，如何循序漸進(jìn)地學(xué)習(xí)好模型原理與部署落地
小提示：購買與贈送的【專屬簽名版書籍】當(dāng)日發(fā)貨，統(tǒng)一使用京東快遞。 更多模型設(shè)計與優(yōu)化部署的學(xué)習(xí)資料 由于作者的水平和時間有限，書籍出版的時間滯后，以及圖文形式本身的限制，因此在我們視頻課程平臺【地址為：https://appcdfgt3n15676.h5.】還有對應(yīng)的視頻學(xué)習(xí)資料，其中與本書相關(guān)的如下：
【視頻課】AI必學(xué)，超20小時，4大模塊，循序漸進(jìn)地搞懂CNN模型設(shè)計與簡單部署！
【視頻課】AI必學(xué)，超3小時，3大模塊，掌握模型分析核心技術(shù)！
【視頻課】AI必學(xué)，超10小時，4大模塊，掌握模型優(yōu)化核心技術(shù)！
【視頻課】AI必學(xué)，超9小時，6大框架，Android+NCNN+MNN+Tengine+TensorRT+小程序模型部署實戰(zhàn)！ 轉(zhuǎn)載文章請后臺聯(lián)系 侵權(quán)必究  |
|
|
