深度学习识别CIFAR10：pytorch训练LeNet、AlexNet、VGG19实现及比较（三）

2023年4月13日上午2:25 • 深度学习

VGGNet在2014年ImageNet图像分类任务竞赛中有出色的表现。网络结构如下图所示：

同样的，对32*32的CIFAR10图片，网络结构做了微调：删除了最后一层最大池化，具体参见网络定义代码，这里采用VGG19，并加入了BN：

 1 '''
 2 创建VGG块
 3 参数分别为输入通道数，输出通道数，卷积层个数，是否做最大池化
 4 '''
 5 def make_vgg_block(in_channel, out_channel, convs, pool=True):
 6     net = []
 7 
 8     # 不改变图片尺寸卷积
 9     net.append(nn.Conv2d(in_channel, out_channel, kernel_size=3, padding=1))
10     net.append(nn.BatchNorm2d(out_channel))
11     net.append(nn.ReLU(inplace=True))
12 
13     for i in range(convs - 1):
14         # 不改变图片尺寸卷积
15         net.append(nn.Conv2d(out_channel, out_channel, kernel_size=3, padding=1))
16         net.append(nn.BatchNorm2d(out_channel))
17         net.append(nn.ReLU(inplace=True))
18 
19     if pool:
20         # 2*2最大池化，图片变为w/2 * h/2
21         net.append(nn.MaxPool2d(2))
22 
23     return nn.Sequential(*net)
24 
25 
26 # 定义网络模型
27 class VGG19Net(nn.Module):
28     def __init__(self):
29         super(VGG19Net, self).__init__()
30 
31         net = []
32 
33         # 输入32*32，输出16*16
34         net.append(make_vgg_block(3, 64, 2))
35 
36         # 输出8*8
37         net.append(make_vgg_block(64, 128, 2))
38 
39         # 输出4*4
40         net.append(make_vgg_block(128, 256, 4))
41 
42         # 输出2*2
43         net.append(make_vgg_block(256, 512, 4))
44 
45         # 无池化层，输出保持2*2
46         net.append(make_vgg_block(512, 512, 4, False))
47 
48         self.cnn = nn.Sequential(*net)
49 
50         self.fc = nn.Sequential(
51             # 512个feature，每个feature 2*2
52             nn.Linear(512*2*2, 256),
53             nn.ReLU(),
54 
55             nn.Linear(256, 256),
56             nn.ReLU(),
57 
58             nn.Linear(256, 10)
59         )
60 
61     def forward(self, x):
62         x = self.cnn(x)
63 
64         # x.size()[0]: batch size
65         x = x.view(x.size()[0], -1)
66         x = self.fc(x)
67 
68         return x

其余代码同深度学习识别CIFAR10：pytorch训练LeNet、AlexNet、VGG19实现及比较（一）运行结果如下：

Files already downloaded and verified
VGG19Net(
(cnn): Sequential(
    (0): Sequential(
      (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace)
      (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU(inplace)
      (6): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (1): Sequential(
      (0): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace)
      (3): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU(inplace)
      (6): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (2): Sequential(
      (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace)
      (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU(inplace)
      (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (8): ReLU(inplace)
      (9): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (10): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (11): ReLU(inplace)
      (12): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (3): Sequential(
      (0): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace)
      (3): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (4): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU(inplace)
      (6): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (7): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (8): ReLU(inplace)
      (9): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (10): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (11): ReLU(inplace)
      (12): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (4): Sequential(
      (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace)
      (3): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (4): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU(inplace)
      (6): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (7): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (8): ReLU(inplace)
      (9): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (10): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (11): ReLU(inplace)
    )
)
(fc): Sequential(
    (0): Linear(in_features=2048, out_features=256, bias=True)
    (1): ReLU()
    (2): Linear(in_features=256, out_features=256, bias=True)
    (3): ReLU()
    (4): Linear(in_features=256, out_features=10, bias=True)
)
)
Train Epoch: 1 [6400/50000 (13%)]   Loss: 1.991934 Acc: 22.000000
Train Epoch: 1 [12800/50000 (26%)]   Loss: 1.851721 Acc: 27.000000
Train Epoch: 1 [19200/50000 (38%)]   Loss: 1.765295 Acc: 31.000000
Train Epoch: 1 [25600/50000 (51%)]   Loss: 1.708027 Acc: 33.000000
Train Epoch: 1 [32000/50000 (64%)]   Loss: 1.652181 Acc: 36.000000
Train Epoch: 1 [38400/50000 (77%)]   Loss: 1.597727 Acc: 38.000000
Train Epoch: 1 [44800/50000 (90%)]   Loss: 1.552660 Acc: 41.000000
one epoch spend: 0:01:08.269581
EPOCH:1, ACC:55.08

Train Epoch: 2 [6400/50000 (13%)]   Loss: 1.139670 Acc: 60.000000
Train Epoch: 2 [12800/50000 (26%)]   Loss: 1.099960 Acc: 61.000000
Train Epoch: 2 [19200/50000 (38%)]   Loss: 1.078881 Acc: 62.000000
Train Epoch: 2 [25600/50000 (51%)]   Loss: 1.054403 Acc: 63.000000
Train Epoch: 2 [32000/50000 (64%)]   Loss: 1.031371 Acc: 64.000000
Train Epoch: 2 [38400/50000 (77%)]   Loss: 1.011668 Acc: 64.000000
Train Epoch: 2 [44800/50000 (90%)]   Loss: 0.995242 Acc: 65.000000
one epoch spend: 0:01:08.220392
EPOCH:2, ACC:71.01

Train Epoch: 3 [6400/50000 (13%)]   Loss: 0.823265 Acc: 71.000000
Train Epoch: 3 [12800/50000 (26%)]   Loss: 0.799878 Acc: 73.000000
Train Epoch: 3 [19200/50000 (38%)]   Loss: 0.791265 Acc: 73.000000
Train Epoch: 3 [25600/50000 (51%)]   Loss: 0.790027 Acc: 73.000000
Train Epoch: 3 [32000/50000 (64%)]   Loss: 0.777267 Acc: 73.000000
Train Epoch: 3 [38400/50000 (77%)]   Loss: 0.771953 Acc: 74.000000
Train Epoch: 3 [44800/50000 (90%)]   Loss: 0.766835 Acc: 74.000000
one epoch spend: 0:01:08.485721
EPOCH:3, ACC:69.48

Train Epoch: 4 [6400/50000 (13%)]   Loss: 0.640418 Acc: 78.000000
Train Epoch: 4 [12800/50000 (26%)]   Loss: 0.637256 Acc: 78.000000
Train Epoch: 4 [19200/50000 (38%)]   Loss: 0.631245 Acc: 79.000000
Train Epoch: 4 [25600/50000 (51%)]   Loss: 0.629215 Acc: 79.000000
Train Epoch: 4 [32000/50000 (64%)]   Loss: 0.625925 Acc: 79.000000
Train Epoch: 4 [38400/50000 (77%)]   Loss: 0.618307 Acc: 79.000000
Train Epoch: 4 [44800/50000 (90%)]   Loss: 0.617456 Acc: 79.000000
one epoch spend: 0:01:08.289673
EPOCH:4, ACC:77.2

Train Epoch: 5 [6400/50000 (13%)]   Loss: 0.537330 Acc: 82.000000
Train Epoch: 5 [12800/50000 (26%)]   Loss: 0.529751 Acc: 82.000000
Train Epoch: 5 [19200/50000 (38%)]   Loss: 0.529389 Acc: 82.000000
Train Epoch: 5 [25600/50000 (51%)]   Loss: 0.528106 Acc: 82.000000
Train Epoch: 5 [32000/50000 (64%)]   Loss: 0.526467 Acc: 82.000000
Train Epoch: 5 [38400/50000 (77%)]   Loss: 0.525133 Acc: 82.000000
Train Epoch: 5 [44800/50000 (90%)]   Loss: 0.521847 Acc: 82.000000
one epoch spend: 0:01:08.272084
EPOCH:5, ACC:78.26

Train Epoch: 6 [6400/50000 (13%)]   Loss: 0.435377 Acc: 85.000000
Train Epoch: 6 [12800/50000 (26%)]   Loss: 0.431456 Acc: 85.000000
Train Epoch: 6 [19200/50000 (38%)]   Loss: 0.443582 Acc: 85.000000
Train Epoch: 6 [25600/50000 (51%)]   Loss: 0.442819 Acc: 85.000000
Train Epoch: 6 [32000/50000 (64%)]   Loss: 0.443313 Acc: 85.000000
Train Epoch: 6 [38400/50000 (77%)]   Loss: 0.442025 Acc: 85.000000
Train Epoch: 6 [44800/50000 (90%)]   Loss: 0.441722 Acc: 85.000000
one epoch spend: 0:01:10.725170
EPOCH:6, ACC:80.91

Train Epoch: 7 [6400/50000 (13%)]   Loss: 0.350214 Acc: 88.000000
Train Epoch: 7 [12800/50000 (26%)]   Loss: 0.351490 Acc: 88.000000
Train Epoch: 7 [19200/50000 (38%)]   Loss: 0.361328 Acc: 88.000000
Train Epoch: 7 [25600/50000 (51%)]   Loss: 0.362231 Acc: 87.000000
Train Epoch: 7 [32000/50000 (64%)]   Loss: 0.364318 Acc: 87.000000
Train Epoch: 7 [38400/50000 (77%)]   Loss: 0.367137 Acc: 87.000000
Train Epoch: 7 [44800/50000 (90%)]   Loss: 0.375220 Acc: 87.000000
one epoch spend: 0:01:09.395538
EPOCH:7, ACC:80.55

Train Epoch: 8 [6400/50000 (13%)]   Loss: 0.297754 Acc: 90.000000
Train Epoch: 8 [12800/50000 (26%)]   Loss: 0.303383 Acc: 89.000000
Train Epoch: 8 [19200/50000 (38%)]   Loss: 0.305170 Acc: 89.000000
Train Epoch: 8 [25600/50000 (51%)]   Loss: 0.311823 Acc: 89.000000
Train Epoch: 8 [32000/50000 (64%)]   Loss: 0.309851 Acc: 89.000000
Train Epoch: 8 [38400/50000 (77%)]   Loss: 0.310422 Acc: 89.000000
Train Epoch: 8 [44800/50000 (90%)]   Loss: 0.312672 Acc: 89.000000
one epoch spend: 0:01:08.041167
EPOCH:8, ACC:80.54

Train Epoch: 9 [6400/50000 (13%)]   Loss: 0.277638 Acc: 90.000000
Train Epoch: 9 [12800/50000 (26%)]   Loss: 0.276622 Acc: 90.000000
Train Epoch: 9 [19200/50000 (38%)]   Loss: 0.276465 Acc: 90.000000
Train Epoch: 9 [25600/50000 (51%)]   Loss: 0.278001 Acc: 90.000000
Train Epoch: 9 [32000/50000 (64%)]   Loss: 0.277109 Acc: 90.000000
Train Epoch: 9 [38400/50000 (77%)]   Loss: 0.277029 Acc: 90.000000
Train Epoch: 9 [44800/50000 (90%)]   Loss: 0.275243 Acc: 90.000000
one epoch spend: 0:01:08.143754
EPOCH:9, ACC:83.53

Train Epoch: 10 [6400/50000 (13%)]   Loss: 0.205785 Acc: 92.000000
Train Epoch: 10 [12800/50000 (26%)]   Loss: 0.210659 Acc: 92.000000
Train Epoch: 10 [19200/50000 (38%)]   Loss: 0.214871 Acc: 92.000000
Train Epoch: 10 [25600/50000 (51%)]   Loss: 0.218910 Acc: 92.000000
Train Epoch: 10 [32000/50000 (64%)]   Loss: 0.220843 Acc: 92.000000
Train Epoch: 10 [38400/50000 (77%)]   Loss: 0.220417 Acc: 92.000000
Train Epoch: 10 [44800/50000 (90%)]   Loss: 0.221100 Acc: 92.000000
one epoch spend: 0:01:08.333929
EPOCH:10, ACC:79.01

Train Epoch: 11 [6400/50000 (13%)]   Loss: 0.186917 Acc: 93.000000
Train Epoch: 11 [12800/50000 (26%)]   Loss: 0.183512 Acc: 93.000000
Train Epoch: 11 [19200/50000 (38%)]   Loss: 0.182561 Acc: 93.000000
Train Epoch: 11 [25600/50000 (51%)]   Loss: 0.186446 Acc: 93.000000
Train Epoch: 11 [32000/50000 (64%)]   Loss: 0.187314 Acc: 93.000000
Train Epoch: 11 [38400/50000 (77%)]   Loss: 0.185967 Acc: 93.000000
Train Epoch: 11 [44800/50000 (90%)]   Loss: 0.189130 Acc: 93.000000
one epoch spend: 0:01:10.476138
EPOCH:11, ACC:81.57

Train Epoch: 12 [6400/50000 (13%)]   Loss: 0.136427 Acc: 95.000000
Train Epoch: 12 [12800/50000 (26%)]   Loss: 0.147904 Acc: 95.000000
Train Epoch: 12 [19200/50000 (38%)]   Loss: 0.154502 Acc: 94.000000
Train Epoch: 12 [25600/50000 (51%)]   Loss: 0.155767 Acc: 94.000000
Train Epoch: 12 [32000/50000 (64%)]   Loss: 0.158346 Acc: 94.000000
Train Epoch: 12 [38400/50000 (77%)]   Loss: 0.159562 Acc: 94.000000
Train Epoch: 12 [44800/50000 (90%)]   Loss: 0.159924 Acc: 94.000000
one epoch spend: 0:01:10.779635
EPOCH:12, ACC:84.38

Train Epoch: 13 [6400/50000 (13%)]   Loss: 0.110026 Acc: 96.000000
Train Epoch: 13 [12800/50000 (26%)]   Loss: 0.113738 Acc: 96.000000
Train Epoch: 13 [19200/50000 (38%)]   Loss: 0.117731 Acc: 96.000000
Train Epoch: 13 [25600/50000 (51%)]   Loss: 0.123653 Acc: 95.000000
Train Epoch: 13 [32000/50000 (64%)]   Loss: 0.127138 Acc: 95.000000
Train Epoch: 13 [38400/50000 (77%)]   Loss: 0.128938 Acc: 95.000000
Train Epoch: 13 [44800/50000 (90%)]   Loss: 0.131382 Acc: 95.000000
one epoch spend: 0:01:09.020651
EPOCH:13, ACC:83.46

Train Epoch: 14 [6400/50000 (13%)]   Loss: 0.122690 Acc: 96.000000
Train Epoch: 14 [12800/50000 (26%)]   Loss: 0.114584 Acc: 96.000000
Train Epoch: 14 [19200/50000 (38%)]   Loss: 0.122652 Acc: 96.000000
Train Epoch: 14 [25600/50000 (51%)]   Loss: 0.123031 Acc: 95.000000
Train Epoch: 14 [32000/50000 (64%)]   Loss: 0.123427 Acc: 95.000000
Train Epoch: 14 [38400/50000 (77%)]   Loss: 0.123146 Acc: 95.000000
Train Epoch: 14 [44800/50000 (90%)]   Loss: 0.124063 Acc: 95.000000
one epoch spend: 0:01:10.294790
EPOCH:14, ACC:82.27

Train Epoch: 15 [6400/50000 (13%)]   Loss: 0.087797 Acc: 97.000000
Train Epoch: 15 [12800/50000 (26%)]   Loss: 0.086152 Acc: 97.000000
Train Epoch: 15 [19200/50000 (38%)]   Loss: 0.088446 Acc: 97.000000
Train Epoch: 15 [25600/50000 (51%)]   Loss: 0.093510 Acc: 96.000000
Train Epoch: 15 [32000/50000 (64%)]   Loss: 0.092870 Acc: 96.000000
Train Epoch: 15 [38400/50000 (77%)]   Loss: 0.092416 Acc: 96.000000
Train Epoch: 15 [44800/50000 (90%)]   Loss: 0.095187 Acc: 96.000000
one epoch spend: 0:01:10.375479
EPOCH:15, ACC:82.73

Train Epoch: 16 [6400/50000 (13%)]   Loss: 0.066554 Acc: 97.000000
Train Epoch: 16 [12800/50000 (26%)]   Loss: 0.079139 Acc: 97.000000
Train Epoch: 16 [19200/50000 (38%)]   Loss: 0.078223 Acc: 97.000000
Train Epoch: 16 [25600/50000 (51%)]   Loss: 0.076825 Acc: 97.000000
Train Epoch: 16 [32000/50000 (64%)]   Loss: 0.079679 Acc: 97.000000
Train Epoch: 16 [38400/50000 (77%)]   Loss: 0.081081 Acc: 97.000000
Train Epoch: 16 [44800/50000 (90%)]   Loss: 0.081967 Acc: 97.000000
one epoch spend: 0:01:09.971818
EPOCH:16, ACC:85.45

Train Epoch: 17 [6400/50000 (13%)]   Loss: 0.061477 Acc: 98.000000
Train Epoch: 17 [12800/50000 (26%)]   Loss: 0.066804 Acc: 97.000000
Train Epoch: 17 [19200/50000 (38%)]   Loss: 0.069621 Acc: 97.000000
Train Epoch: 17 [25600/50000 (51%)]   Loss: 0.068841 Acc: 97.000000
Train Epoch: 17 [32000/50000 (64%)]   Loss: 0.069220 Acc: 97.000000
Train Epoch: 17 [38400/50000 (77%)]   Loss: 0.071493 Acc: 97.000000
Train Epoch: 17 [44800/50000 (90%)]   Loss: 0.070973 Acc: 97.000000
one epoch spend: 0:01:10.599626
EPOCH:17, ACC:83.02

Train Epoch: 18 [6400/50000 (13%)]   Loss: 0.095195 Acc: 96.000000
Train Epoch: 18 [12800/50000 (26%)]   Loss: 0.081690 Acc: 97.000000
Train Epoch: 18 [19200/50000 (38%)]   Loss: 0.076400 Acc: 97.000000
Train Epoch: 18 [25600/50000 (51%)]   Loss: 0.073249 Acc: 97.000000
Train Epoch: 18 [32000/50000 (64%)]   Loss: 0.072114 Acc: 97.000000
Train Epoch: 18 [38400/50000 (77%)]   Loss: 0.073739 Acc: 97.000000
Train Epoch: 18 [44800/50000 (90%)]   Loss: 0.073761 Acc: 97.000000
one epoch spend: 0:01:11.619880
EPOCH:18, ACC:83.67

Train Epoch: 19 [6400/50000 (13%)]   Loss: 0.049970 Acc: 98.000000
Train Epoch: 19 [12800/50000 (26%)]   Loss: 0.051812 Acc: 98.000000
Train Epoch: 19 [19200/50000 (38%)]   Loss: 0.053814 Acc: 98.000000
Train Epoch: 19 [25600/50000 (51%)]   Loss: 0.054168 Acc: 98.000000
Train Epoch: 19 [32000/50000 (64%)]   Loss: 0.054138 Acc: 98.000000
Train Epoch: 19 [38400/50000 (77%)]   Loss: 0.055356 Acc: 98.000000
Train Epoch: 19 [44800/50000 (90%)]   Loss: 0.055334 Acc: 98.000000
one epoch spend: 0:01:10.397104
EPOCH:19, ACC:84.23

Train Epoch: 20 [6400/50000 (13%)]   Loss: 0.059795 Acc: 98.000000
Train Epoch: 20 [12800/50000 (26%)]   Loss: 0.059780 Acc: 98.000000
Train Epoch: 20 [19200/50000 (38%)]   Loss: 0.060332 Acc: 98.000000
Train Epoch: 20 [25600/50000 (51%)]   Loss: 0.057949 Acc: 98.000000
Train Epoch: 20 [32000/50000 (64%)]   Loss: 0.056517 Acc: 98.000000
Train Epoch: 20 [38400/50000 (77%)]   Loss: 0.055322 Acc: 98.000000
Train Epoch: 20 [44800/50000 (90%)]   Loss: 0.053375 Acc: 98.000000
one epoch spend: 0:01:10.407573
EPOCH:20, ACC:84.51

CIFAR10 pytorch LeNet Train: EPOCH:20, BATCH_SZ:64, LR:0.01, ACC:85.45
train spend time: 0:23:45.010363

Process finished with exit code 0

准确率达到85%，对比AlexNet的75%，提升了10%。

本站文章如无特殊说明，均为本站原创，如若转载，请注明出处：深度学习识别CIFAR10：pytorch训练LeNet、AlexNet、VGG19实现及比较（三） - Python技术站

深度学习

0 0 打赏

微信扫一扫

支付宝扫一扫

深度学习中卷积层和pooling层的输出计算公式(转)

上一篇 2023年4月13日

深度学习识别CIFAR10：pytorch训练LeNet、AlexNet、VGG19实现及比较（二）

下一篇 2023年4月13日

深度学习

深度学习–知识蒸馏介绍

以前理解的模型蒸馏就是模型“提纯”，这样说太宽泛了，应该说蒸馏是“提纯”的一种手段而已。知识蒸馏具体指：让小模型去学到大模型的知识。通俗的说，让student模型的输出接近(拟合)teacher模型的输出。知识蒸馏的重点在于拟合二字，即我们要定义一个方法去衡量student模型和teacher模型的接近程度，说白了就是损失函数。必备元素为：教师模型、学生…

2023年4月10日
000
win10下搭建深度学习–总结【学习笔记】

win10 下搭建深度学习开发环境总结: 1、本人环境如下：win10,GTX1050TI、i7,anaconda3,vs2015,cuda9.0,cudnn7.1.4,tensorflow-gpu=1.10.0,keras==2.2.2 2、先把anaconda3安装好，之后在安装vs2015，由于在window环境下安装cuda9.0的时候会检测是否安装…

深度学习 2023年4月12日
000
深度学习基本部件-激活函数详解

本文分析了激活函数对于神经网络的必要性，同时讲解了几种常见的激活函数的原理，并给出相关公式、代码和示例图。从机器学习的角度来看，神经网络其实就是一个非线性模型，其基本组成单元为具有非线性激活函数的神经元，通过大量神经元之间的连接，使得多层神经网络成为一种高度非线性的模型。神经元之间的连接权重就是需要学习的参数，其可以在机器学习的框架下通过梯度下降方法来进行学…

深度学习 2023年4月13日
000
谷歌Colab使用（深度学习）

Google Colaboratory是谷歌开放的一款研究工具，主要用于机器学习的开发和研究。这款工具现在可以免费使用，但是不是永久免费暂时还不确定。Google Colab最大的好处是给广大的AI开发者提供了免费的GPU使用！GPU型号是Tesla K80！你可以在上面轻松地跑例如：Keras、Tensorflow、Pytorch等框架。 2. 使用 2.…

深度学习 2023年4月10日
000
[Tensorflow实战Google深度学习框架]笔记3 – 王老头

[Tensorflow实战Google深度学习框架]笔记3 　　本系列为Tensorflow实战Google深度学习框架知识笔记，仅为博主看书过程中觉得较为重要的知识点，简单摘要下来，内容较为零散，请各位指出，共同进步。　　　　2017-11-06 　　[第四章] 深层神经网络　　1. 深度学习与深层神经网络　　维基百科对深度学习的精确定义为：一类通过…

深度学习 2023年4月12日
000
深度学习

深度学习Python实战：用Python实现第一个深度学习程序！

深度学习是 2018-22 年度最热门的话题之一。近几年该行业取得了非常大的进步，可以说机器或计算机程序真正取代人类的时代已经到来。在本文章，将使用 Python 进行深度学习的的实战，希望能帮助您了解深度学习到底是什么，以及它是如何实现的。我将在本文中介绍以下主题：理论篇深度学习的作用什么是深度学习？感知器和人工神经网络深度学习的应用为什么…

2022年12月6日
000
深度学习

使用horovod构建分布式深度学习框架

　　最近一直在尝试着分布式深度学习的架构，主要的原因一方面是几台机子全是1060卡，利用深度网络在较大数据样本上训练的效率极其低下，所以尝试着将几台机子做成分布式，看看能否提高训练效率；第二方面是有人习惯使用tensorflow，有人习惯使用keras，也有人喜欢使用pytorch等，虽然这些框架各自都有分布式的实现，但总的来说不能统一到一个平台上，造成使用…

2023年4月10日
000
一入侯门“深”似海，深度学习深几许（入门系列之一）

摘要：当你和女朋友在路边手拉手一起约会的时候，你可曾想，你们之间早已碰撞出了一种神秘的智慧–深度学习。恋爱容易，相处不易，不断磨合，打造你们的默契，最终才能决定你们是否在一起。深度学习也一样，输入各种不同的参数，进行训练拟合，最后输出拟合结果。恋爱又不易，且学且珍惜！【导言】目前人工智能非常火爆，而深度学习则是引领这一火爆现场的“火箭”。于是，有关“…

深度学习 2023年4月11日
000

深度学习识别CIFAR10：pytorch训练LeNet、AlexNet、VGG19实现及比较（三）

相关文章