编程

当前位置:澳门新莆京娱乐官网 > 编程 > 基于gluon的Inception结构澳门新莆京娱乐官网

基于gluon的Inception结构澳门新莆京娱乐官网

来源:http://www.drgigabytes.com 作者:澳门新莆京娱乐官网 时间:2020-01-24 13:24

本文公式比较多,由于简书不帮助公式渲染,公式完整版请移步个人博客

结构

初级英斯ption构造如下所示:

澳门新莆京娱乐官网 1inception_naive.png

其前向传来分为4个部分:

  • 通过1x1卷积
  • 通过3x3卷积,padding为1
  • 通过5x5卷积,padding为2
  • 通过3x3池化,为了保障图片大小与以上同等,stride应为1,padding应该为1

末尾,将上述四个部分在feature那风流洒脱维度堆积起来,即获取最后输出

分析

风流倜傥旦输入feature,每种层输出feature分别为$N_i,N_o$(即最后输出feature为$4 times N_o$);输入图片的尺寸为WxL,对于每风华正茂层,有:

  • 1x1卷积层:有参数$1 times 1 times N_i times N_o = N_iN_o$,须要举行总结的次数为$N_o times W times L times 1 times 1 times N_i = WLN_iN_o$
  • 3x3卷积层:有参数$3 times 3 times N_i times N_o = 9N_iN_o$,需求计算次数为$N_o times W times L times 3 times 3 times N_i = 9WLN_iN_o$
  • 5x5卷积层,同上,参数为$25N_iN_o$,供给计算次数为$25WLN_iN_o$

为此,总的参数量为$N_iN_o = 35N_iN_o$,要求的运算量为$WLN_iN_o = 35WLN_iN_o$。思考三个输入输出相仿尺寸的3x3卷积,要求的参数量为$3 times 3 times N_i times 4N_o = 36N_iN_o$,须求的运算量是$4N_o times W times L times 3 澳门新莆京娱乐官网,times 3 times N_i = 36WLN_iN_o$,能够开掘该协会在运算量和参数量近乎不改变的动静下达成了二种感想野的连年。

结构

改进的英斯ption构造如下图所示

澳门新莆京娱乐官网 2inception.png

长期以来具有四条前向传来通路,如下所示:

  • 1x1卷积
  • 先经过1x1卷积降维,再通过3x3卷积
  • 先经过1x1卷积降维,再通过5x5卷积
  • 先通过3x3maxpool,再经过1x1调解维度

说起底,将以上多个部分在feature那后生可畏维度聚成堆起来,即拿到最终输出

分析

若是同上有的,即便每一种降维将降维降低到原本维度的二分一,对每大器晚成局地宛如下所示:

  • 每一种降维的1x1层,供给的参数量是$cfrac{1}{2}N_i2$,运算参数量$cfrac{1}{2}WLN_i2$
  • 1x1卷积层:有参数$1 times 1 times cfrac{1}{2}N_i times N_o = cfrac{1}{2}N_iN_o$,供给张开总计的次数为$N_o times W times L times 1 times 1 times cfrac{1}{2}N_i = cfrac{1}{2}WLN_iN_o$
  • 3x3卷积层:有参数$3 times 3 times cfrac{1}{2}N_i times N_o = 9N_iN_o$,必要计算次数为$N_o times W times L times 3 times 3 times cfrac{1}{2}N_i = cfrac{9}{2}WLN_iN_o$
  • 5x5卷积层,同上,参数为$cfrac{25}{2}N_iN_o$,需求总结次数为$cfrac{25}{2}WLN_iN_o$

假设$N_i = N_o$,参数量豆蔻梢头共是$3 times cfrac{1}{2}N_i^2 + cfrac{35}{2}N_i^2 = 19N_i2$,要求运算的数额为$19WLN_i2$。能够窥见无论是运算量照旧参数量都低于原构造

import mxnet as mximport numpy as np

Inception结构

class inception(mx.gluon.Block): def __init__(self,out_channel): super(inception,self).__init__() with self.name_scope(): self.conv1 = mx.gluon.nn.Conv2D(out_channel,1,activation='relu') self.conv3_pre = mx.gluon.nn.Conv2D(out_channel//2,1) self.conv3 = mx.gluon.nn.Conv2D(out_channel,3,activation='relu',padding=1) self.conv5_pre = mx.gluon.nn.Conv2D(out_channel//2,1) self.conv5 = mx.gluon.nn.Conv2D(out_channel,5,activation='relu',padding=2) self.pool_post = mx.gluon.nn.Conv2D(out_channel,1,activation='relu') self.pool = mx.gluon.nn.MaxPool2D(pool_size=3,strides=1,padding=1) def forward: result = [ self.conv1, self.conv3(self.conv3_pre, self.conv5(self.conv5_pre, self.pool_post(self.pool] return mx.ndarray.concat(dim=1,*result)

英斯ption布局测量检验

inception_model = inceptionprint(inception_model)inception_model.collect_params().initialize(mx.init.Normal, ctx=mx.gpu

inception( (pool_post): Conv2D(None -> 10, kernel_size=, stride= : MaxPool2D(size=, stride=, padding=, ceil_mode=False) : Conv2D(None -> 10, kernel_size=, stride= : Conv2D(None -> 10, kernel_size=, stride=, padding= (conv5_pre): Conv2D(None -> 5, kernel_size=, stride= : Conv2D(None -> 10, kernel_size=, stride=, padding= (conv3_pre): Conv2D(None -> 5, kernel_size=, stride=

indata = mx.ndarray.zeros((1,5,10,10),mx.gpuinception_model.shape

(1, 40, 10, 10)

class network(mx.gluon.Block): def __init__: super(network,self).__init__() with self.name_scope(): self.conv1 = mx.gluon.nn.Conv2D(channels=8,kernel_size=3,padding=1) self.conv2 = inception self.conv3 = inception self.conv4 = inception self.fc = mx.gluon.nn.Dense self.pool = mx.gluon.nn.MaxPool2D(pool_size=3,strides=2) def forward: x = self.conv2(self.conv1 x = self.conv3(self.pool x = self.conv4(self.pool return self.fc

model = network()printmodel.collect_params().initialize(mx.init.Normal, ctx=mx.gpu

network: MaxPool2D(size=, stride=, padding=, ceil_mode=False) : Conv2D(None -> 8, kernel_size=, stride=, padding= : inception( (pool_post): Conv2D(None -> 16, kernel_size=, stride= : MaxPool2D(size=, stride=, padding=, ceil_mode=False) : Conv2D(None -> 16, kernel_size=, stride= : Conv2D(None -> 16, kernel_size=, stride=, padding= (conv5_pre): Conv2D(None -> 8, kernel_size=, stride= : Conv2D(None -> 16, kernel_size=, stride=, padding= (conv3_pre): Conv2D(None -> 8, kernel_size=, stride= : inception( (pool_post): Conv2D(None -> 8, kernel_size=, stride= : MaxPool2D(size=, stride=, padding=, ceil_mode=False) : Conv2D(None -> 8, kernel_size=, stride= : Conv2D(None -> 8, kernel_size=, stride=, padding= (conv5_pre): Conv2D(None -> 4, kernel_size=, stride= : Conv2D(None -> 8, kernel_size=, stride=, padding= (conv3_pre): Conv2D(None -> 4, kernel_size=, stride= : Dense(None -> 10, linear) : inception( (pool_post): Conv2D(None -> 16, kernel_size=, stride= : MaxPool2D(size=, stride=, padding=, ceil_mode=False) : Conv2D(None -> 16, kernel_size=, stride= : Conv2D(None -> 16, kernel_size=, stride=, padding= (conv5_pre): Conv2D(None -> 8, kernel_size=, stride= : Conv2D(None -> 16, kernel_size=, stride=, padding= (conv3_pre): Conv2D(None -> 8, kernel_size=, stride=

​```pythonindata = mx.ndarray.zeros((1,1,28,28),mx.gpumodel.shape

数据集——MNIST数据集

def transform(data, label): return mx.nd.transpose(data,axes=.astype(np.float32)/255, label.astype(np.float32)gluon_train_data = mx.gluon.data.DataLoader(mx.gluon.data.vision.MNIST(train=True, transform=transform),100, shuffle=True)gluon_test_data = mx.gluon.data.DataLoader(mx.gluon.data.vision.MNIST(train=False, transform=transform),100, shuffle=False)

代价函数——交叉熵

softmax_cross_entropy = mx.gluon.loss.SoftmaxCrossEntropyLoss()

优化器——sgd

trainer = mx.gluon.Trainer(model.collect_params(), 'sgd', {'learning_rate': .1})

精确率总结

def evaluate_accuracy: acc = mx.metric.Accuracy() for i, (data, lable) in enumerate(gluon_test_data): data = data.as_in_context lable = lable.as_in_context output = model predictions = mx.nd.argmax(output, axis=1) acc.update(preds=predictions, labels=lable) return acc.get()[1]evaluate_accuracy

0.050200000000000002

for _ in range: for i,(data,lable) in enumerate(gluon_train_data): data = data.as_in_context lable = lable.as_in_context with mx.autograd.record(): outputs = model loss = softmax_cross_entropy(outputs,lable) loss.backward() trainer.step(data.shape[0]) if i % 100 == 1: print(i,loss.mean().asnumpy

1 0.16968101 0.104242201 0.093354301 0.07079401 0.123301501 0.0868821 0.0325385101 0.0510763201 0.0242231301 0.0454984401 0.0788167501 0.0591589

evaluate_accuracy

0.98829999999999996

本文由澳门新莆京娱乐官网发布于编程,转载请注明出处:基于gluon的Inception结构澳门新莆京娱乐官网

关键词:

上一篇:没有了

下一篇:没有了