前期回顾:
深度学习实践系列(1)- 从零搭建notMNIST逻辑回归模型
深度学习实践系列(2)- 搭建notMNIST的深度神经网络
在第二篇系列中,我们使用了TensorFlow搭建了第一个深度神经网络,并且尝试了很多优化方式去改进神经网络学习的效率和提高准确性。在这篇文章,我们将要使用一个强大的神经网络学习框架Keras配合TensorFlow重新搭建一个深度神经网络。
官方对于Keras的定义如下:
“Keras: Deep Learning library for Theano and TensorFlow
You have just found Keras.
Keras is a high-level neural networks API, written in Python and capable of running on top of either TensorFlow orTheano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.
Use Keras if you need a deep learning library that:
- Allows for easy and fast prototyping (through user friendliness, modularity, and extensibility).
- Supports both convolutional networks and recurrent networks, as well as combinations of the two.
- Runs seamlessly on CPU and GPU.“
知乎上面对其的评价:如何评价深度学习框架Keras?
今年1月Keras被添加到TensorFlow被作为默认API:Keras 将被添加到谷歌 TensorFlow 成为默认 API
总结下来有以下几点:
1. Keras是基于TensorFlow和Theano更高级的封装框架,因此提供很多现成的功能更容易实现
2. 灵活度不够,并且由于封装对于外界是黑盒,所以定制也较难
3. 社区非常活跃,并且获得TensorFlow的认可,因此TensorFlow+Keras会成为初学者上手很好的一个平台
使用Keras搭建神经网络
环境准备
安装TensorFlow:See installation instructions.
安装Keras
sudo pip install keras
依赖包引入
引入了numpy, tensorflow, keras, six.moves
from __future__ import print_function import numpy as np import tensorflow as tf from six.moves import cPickle as pickle from six.moves import range np.random.seed(1337) # for reproducibility from keras.datasets import mnist from keras.models import Sequential from keras.layers.core import Dense, Dropout, Activation from keras.optimizers import SGD from keras.utils import np_utils from keras.optimizers import RMSprop from keras.optimizers import Adam from keras.regularizers import l2
读取数据
nb_classes = 10 pickle_file = 'notMNIST.pickle' with open(pickle_file, 'rb') as f: save = pickle.load(f) X_train = save['train_dataset'] y_train = save['train_labels'] X_valid = save['valid_dataset'] y_valid = save['valid_labels'] X_test = save['test_dataset'] y_test = save['test_labels'] del save # hint to help gc free up memory print('Training set', X_train.shape, y_train.shape) print('Validation set', X_valid.shape, y_valid.shape) print('Test set', X_test.shape, y_test.shape) X_train = X_train.reshape(200000, 784) X_valid = X_valid.reshape(10000, 784) X_test = X_test.reshape(10000, 784) X_train = X_train.astype('float32') X_valid = X_valid.astype('float32') X_test = X_test.astype('float32') X_train /= 255 X_valid /= 255 X_test /= 255 print(X_train.shape[0], 'train samples') print(X_valid.shape[0], 'valid samples') print(X_test.shape[0], 'test samples') # convert class vectors to binary class matrices Y_train = np_utils.to_categorical(y_train, nb_classes) Y_valid = np_utils.to_categorical(y_valid, nb_classes) Y_test = np_utils.to_categorical(y_test, nb_classes)
从以前系列中获得的训练文件”notMNIST.pickle“读取出数据,分为三组:training, validation, test。
原始的数据X_train的类型是(200000, 28, 28),将其转化成(200000, 784),用于训练输入。
X_train = X_train.reshape(200000, 784)
将数据类型转化成float32类型
X_train = X_train.astype('float32')
对数据进行Normalization,使得所有数据都在[0,1]的范围内
X_train /= 255
将数据进行转化成binary class matrix,用于后续训练。(Converts a class vector (integers) to binary class matrix.)
Y_train = np_utils.to_categorical(y_train, nb_classes)
设计神经网络模型
model = Sequential() model.add(Dense(512, input_shape=(784,))) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(512)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(10)) model.add(Activation('softmax')) model.summary()
Sequential是Keras的一种串行的层级模型。上述的模型解释如下:
1. 输入层:大小为784的数据集
2. hidden layer 1: 512个节点,使用ReLUs激活函数,0.5 Dropout
3. hidden layer 2: 512个节点,使用ReLUs激活函数,0.5 Dropout
4. 输出层: 10个节点,使用softmax
训练神经网络模型
batch_size = 128
nb_epoch = 20
model.compile(loss='categorical_crossentropy', #optimizer=SGD(lr=0.01), optimizer=Adam(), metrics=['accuracy']) history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, verbose=1, validation_data=(X_valid, Y_valid)) score = model.evaluate(X_test, Y_test, verbose=0) print('Test score:', score[0]) print('Test accuracy:', score[1])
compile函数里面的几个参数:
1. loss='categorical_crossentropy': 使用crossentropy作为loss function
2. optimizer=Adam(): SGD的高级版本,可以动态调整learning rate, 可以结合动量惯性
model.fix通过training data和validation data进行训练
model.evaluate通过在测试数据上进行计算得出最终的准确性
训练过程中的输入如下:
Epoch 1/20
200000/200000 [==============================] - 19s - loss: 0.6951 - acc: 0.7995 - val_loss: 0.5094 - val_acc: 0.8473
Epoch 2/20
200000/200000 [==============================] - 19s - loss: 0.5073 - acc: 0.8486 - val_loss: 0.4573 - val_acc: 0.8596
Epoch 3/20
200000/200000 [==============================] - 22s - loss: 0.4646 - acc: 0.8601 - val_loss: 0.4253 - val_acc: 0.8668
Epoch 4/20
200000/200000 [==============================] - 22s - loss: 0.4356 - acc: 0.8680 - val_loss: 0.3980 - val_acc: 0.8784
Epoch 5/20
200000/200000 [==============================] - 20s - loss: 0.4159 - acc: 0.8736 - val_loss: 0.3851 - val_acc: 0.8810
Epoch 6/20
200000/200000 [==============================] - 18s - loss: 0.3990 - acc: 0.8788 - val_loss: 0.3735 - val_acc: 0.8850
Epoch 7/20
200000/200000 [==============================] - 19s - loss: 0.3868 - acc: 0.8819 - val_loss: 0.3615 - val_acc: 0.8869
Epoch 8/20
200000/200000 [==============================] - 19s - loss: 0.3768 - acc: 0.8846 - val_loss: 0.3576 - val_acc: 0.8872
Epoch 9/20
200000/200000 [==============================] - 19s - loss: 0.3674 - acc: 0.8875 - val_loss: 0.3506 - val_acc: 0.8929
Epoch 10/20
200000/200000 [==============================] - 18s - loss: 0.3610 - acc: 0.8889 - val_loss: 0.3417 - val_acc: 0.8939
Epoch 11/20
200000/200000 [==============================] - 19s - loss: 0.3542 - acc: 0.8911 - val_loss: 0.3392 - val_acc: 0.8967
Epoch 12/20
200000/200000 [==============================] - 20s - loss: 0.3476 - acc: 0.8928 - val_loss: 0.3350 - val_acc: 0.8966
Epoch 13/20
200000/200000 [==============================] - 19s - loss: 0.3419 - acc: 0.8940 - val_loss: 0.3334 - val_acc: 0.8977
Epoch 14/20
200000/200000 [==============================] - 19s - loss: 0.3381 - acc: 0.8952 - val_loss: 0.3288 - val_acc: 0.9008
Epoch 15/20
200000/200000 [==============================] - 20s - loss: 0.3326 - acc: 0.8971 - val_loss: 0.3286 - val_acc: 0.8994
Epoch 16/20
200000/200000 [==============================] - 20s - loss: 0.3273 - acc: 0.8989 - val_loss: 0.3248 - val_acc: 0.9001
Epoch 17/20
200000/200000 [==============================] - 19s - loss: 0.3237 - acc: 0.8996 - val_loss: 0.3246 - val_acc: 0.8998
Epoch 18/20
200000/200000 [==============================] - 20s - loss: 0.3198 - acc: 0.9003 - val_loss: 0.3180 - val_acc: 0.9028
Epoch 19/20
200000/200000 [==============================] - 18s - loss: 0.3181 - acc: 0.9009 - val_loss: 0.3209 - val_acc: 0.9015
Epoch 20/20
200000/200000 [==============================] - 18s - loss: 0.3131 - acc: 0.9022 - val_loss: 0.3155 - val_acc: 0.9028
Test score: 0.139541323698
Test accuracy: 0.957
最终获得了大概95.7%的准确率,大家也可以不断去调整神经网络的结构,看看是否可以提高准确率,祝大家玩得开心。
附上最终的完整代码:
from __future__ import print_function import numpy as np import tensorflow as tf from six.moves import cPickle as pickle from six.moves import range import numpy as np np.random.seed(1337) # for reproducibility from keras.datasets import mnist from keras.models import Sequential from keras.layers.core import Dense, Dropout, Activation from keras.optimizers import SGD from keras.utils import np_utils from keras.optimizers import RMSprop from keras.optimizers import Adam from keras.regularizers import l2 batch_size = 128 nb_classes = 10 nb_epoch = 20 pickle_file = 'notMNIST.pickle' with open(pickle_file, 'rb') as f: save = pickle.load(f) X_train = save['train_dataset'] y_train = save['train_labels'] X_valid = save['valid_dataset'] y_valid = save['valid_labels'] X_test = save['test_dataset'] y_test = save['test_labels'] del save # hint to help gc free up memory print('Training set', X_train.shape, y_train.shape) print('Validation set', X_valid.shape, y_valid.shape) print('Test set', X_test.shape, y_test.shape) X_train = X_train.reshape(200000, 784) X_valid = X_valid.reshape(10000, 784) X_test = X_test.reshape(10000, 784) X_train = X_train.astype('float32') X_valid = X_valid.astype('float32') X_test = X_test.astype('float32') X_train /= 255 X_valid /= 255 X_test /= 255 print(X_train.shape[0], 'train samples') print(X_valid.shape[0], 'valid samples') print(X_test.shape[0], 'test samples') # convert class vectors to binary class matrices Y_train = np_utils.to_categorical(y_train, nb_classes) Y_valid = np_utils.to_categorical(y_valid, nb_classes) Y_test = np_utils.to_categorical(y_test, nb_classes) model = Sequential() model.add(Dense(512, input_shape=(784,))) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(512)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(10)) model.add(Activation('softmax')) model.summary() model.compile(loss='categorical_crossentropy', #optimizer=SGD(lr=0.01), optimizer=Adam(), metrics=['accuracy']) history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch, verbose=1, validation_data=(X_valid, Y_valid)) score = model.evaluate(X_test, Y_test, verbose=0) print('Test score:', score[0]) print('Test accuracy:', score[1])
本站文章如无特殊说明,均为本站原创,如若转载,请注明出处:深度学习实践系列(3)- 使用Keras搭建notMNIST的神经网络 - Python技术站