本篇blog的内容基于原始论文WassersteinGAN-GP(NIPs2017)和《生成对抗网络入门指南》第五章。
一、权重裁剪的问题(为什么要改进GP)
WGAN理论中前提条件是1-Liposchitz条件,而对应使用的方法是权重剪裁,希望把网络固定在一个大小范围内。
但是后来发现权重剪裁有许多问题,所以改进WGAN-GP,使用一种叫做梯度惩罚(gradient penalty)方法来代替权重剪裁。从实验中发现效果更稳定,生成的图像质量也越高。
这里权重裁剪问题有两个:
- 限制了网络的而表现能力,因为大小限制在了固定范围,神经网络很难再模拟出复杂的函数,而只能产生一些比较简单的函数。研究者使用了一些模拟数据重现了这些问题,并对之后的WGAN-GP做了对比,第一排是WGAN结果,第二排是WGAN-GP的结果。可以看到WGAN已经丢失了很多数据分布的高阶矩特征,而WGAN-GP则降低了这个问题。
- 梯度爆炸和梯度消失:因为限制了权值大小,所以会导致梯度爆炸或者梯度消失。如图所属,WGAN三种选择都产生了梯度爆炸或者消失的情况。
二、梯度惩罚(GP)
观察1-Liposchitz条件,所有满足该条件的函数在任意梯度都小于1,可以更换目标函数,在原有的WGAN基础上增加梯度惩罚项 。
具体实现方法是在真实数据分布和生成数据分布各进行一次采样,然后在两点的连线上再做一次随机采样,就是我们希望的惩罚项采样,且默认超参数取10。另外由于惩罚项无法使用BN,所以①我们不采用BN,发现效果依然很好。这里论文也②推荐使用层归一化(layer normalization)来代替BN。
③这里也重新采用了Adam,不存在WGAN中使用Adam方法稳定性不高的问题。
论文中最后的损失函数:
三、 WGAN-GP伪代码
1. 采样真实数据,隐含变量,以及一个随机数
2. 使用Adam进行训练判别器
3. 从前置随机分布采样
4. 使用Adam训练生成器
完整伪代码
四、WGAN-GP代码
1. 初始化超参数
使用Adam下降模型,但是官方给出的源码还是RMSProp
# Large amount of credit goes to:
# https://github.com/keras-team/keras-contrib/blob/master/examples/improved_wgan.py
# which I've used as a reference for this implementation
from __future__ import print_function, division
from keras.datasets import mnist
from keras.layers.merge import _Merge
from keras.layers import Input, Dense, Reshape, Flatten, Dropout
from keras.layers import BatchNormalization, Activation, ZeroPadding2D
from keras.layers.advanced_activations import LeakyReLU
from keras.layers.convolutional import UpSampling2D, Conv2D
from keras.models import Sequential, Model
from keras.optimizers import RMSprop
from functools import partial
import keras.backend as K
import matplotlib.pyplot as plt
import sys
import numpy as np
class RandomWeightedAverage(_Merge):
"""Provides a (random) weighted average between real and generated image samples"""
def _merge_function(self, inputs):
alpha = K.random_uniform((32, 1, 1, 1))
return (alpha * inputs[0]) + ((1 - alpha) * inputs[1])
class WGANGP():
def __init__(self):
self.img_rows = 28
self.img_cols = 28
self.channels = 1
self.img_shape = (self.img_rows, self.img_cols, self.channels)
self.latent_dim = 100
# Following parameter and optimizer set as recommended in paper
self.n_critic = 5
optimizer = RMSprop(lr=0.00005)
# Build the generator and critic
self.generator = self.build_generator()
self.critic = self.build_critic()
#-------------------------------
# Construct Computational Graph
# for the Critic
#-------------------------------
# Freeze generator's layers while training critic
self.generator.trainable = False
2. GP采样
在真实数据分布和生成数据分布各进行一次采样,然后在两点的连线上再做一次随机采样,就是我们希望的惩罚项采样
# Image input (real sample)
real_img = Input(shape=self.img_shape)
# Noise input
z_disc = Input(shape=(100,))
# Generate image based of noise (fake sample)
fake_img = self.generator(z_disc)
# Discriminator determines validity of the real and fake images
fake = self.critic(fake_img)
valid = self.critic(real_img)
# Construct weighted average between real and fake images
interpolated_img = RandomWeightedAverage()([real_img, fake_img])
# Determine validity of weighted sample
validity_interpolated = self.critic(interpolated_img)
# Use Python partial to provide loss function with additional
# 'averaged_samples' argument
partial_gp_loss = partial(self.gradient_penalty_loss,
averaged_samples=interpolated_img)
partial_gp_loss.__name__ = 'gradient_penalty' # Keras requires function names
self.critic_model = Model(inputs=[real_img, z_disc],
outputs=[valid, fake, validity_interpolated])
self.critic_model.compile(loss=[self.wasserstein_loss,
self.wasserstein_loss,
partial_gp_loss],
optimizer=optimizer,
loss_weights=[1, 1, 10])
#-------------------------------
# Construct Computational Graph
# for Generator
#-------------------------------
# For the generator we freeze the critic's layers
self.critic.trainable = False
self.generator.trainable = True
# Sampled noise for input to generator
z_gen = Input(shape=(100,))
# Generate images based of noise
img = self.generator(z_gen)
# Discriminator determines validity
valid = self.critic(img)
# Defines generator model
self.generator_model = Model(z_gen, valid)
self.generator_model.compile(loss=self.wasserstein_loss, optimizer=optimizer)
3. 构造损失函数
①GP损失
def gradient_penalty_loss(self, y_true, y_pred, averaged_samples):
"""
Computes gradient penalty based on prediction and weighted real / fake samples
"""
gradients = K.gradients(y_pred, averaged_samples)[0]
# compute the euclidean norm by squaring ...
gradients_sqr = K.square(gradients)
# ... summing over the rows ...
gradients_sqr_sum = K.sum(gradients_sqr,
axis=np.arange(1, len(gradients_sqr.shape)))
# ... and sqrt
gradient_l2_norm = K.sqrt(gradients_sqr_sum)
# compute lambda * (1 - ||grad||)^2 still for each single sample
gradient_penalty = K.square(1 - gradient_l2_norm)
# return the mean as loss over all the batch samples
return K.mean(gradient_penalty)
②Wasserstein损失
def wasserstein_loss(self, y_true, y_pred):
return K.mean(y_true * y_pred)
4. 构造生成器和判别器模型(同普通GAN)
①构造生成器模型
def build_generator(self):
model = Sequential()
model.add(Dense(128 * 7 * 7, activation="relu", input_dim=self.latent_dim))
model.add(Reshape((7, 7, 128)))
model.add(UpSampling2D())
model.add(Conv2D(128, kernel_size=4, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(Activation("relu"))
model.add(UpSampling2D())
model.add(Conv2D(64, kernel_size=4, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(Activation("relu"))
model.add(Conv2D(self.channels, kernel_size=4, padding="same"))
model.add(Activation("tanh"))
model.summary()
noise = Input(shape=(self.latent_dim,))
img = model(noise)
return Model(noise, img)
②构造判别器模型
def build_critic(self):
model = Sequential()
model.add(Conv2D(16, kernel_size=3, strides=2, input_shape=self.img_shape, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(Conv2D(32, kernel_size=3, strides=2, padding="same"))
model.add(ZeroPadding2D(padding=((0,1),(0,1))))
model.add(BatchNormalization(momentum=0.8))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(Conv2D(64, kernel_size=3, strides=2, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(Conv2D(128, kernel_size=3, strides=1, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(1))
model.summary()
img = Input(shape=self.img_shape)
validity = model(img)
return Model(img, validity)
5. 训练
def train(self, epochs, batch_size, sample_interval=50):
# Load the dataset
(X_train, _), (_, _) = mnist.load_data()
# Rescale -1 to 1
X_train = (X_train.astype(np.float32) - 127.5) / 127.5
X_train = np.expand_dims(X_train, axis=3)
# Adversarial ground truths
valid = -np.ones((batch_size, 1))
fake = np.ones((batch_size, 1))
dummy = np.zeros((batch_size, 1)) # Dummy gt for gradient penalty
for epoch in range(epochs):
for _ in range(self.n_critic):
# ---------------------
# Train Discriminator
# ---------------------
# Select a random batch of images
idx = np.random.randint(0, X_train.shape[0], batch_size)
imgs = X_train[idx]
# Sample generator input
noise = np.random.normal(0, 1, (batch_size, self.latent_dim))
# Train the critic
d_loss = self.critic_model.train_on_batch([imgs, noise],
[valid, fake, dummy])
# ---------------------
# Train Generator
# ---------------------
g_loss = self.generator_model.train_on_batch(noise, valid)
# Plot the progress
print ("%d [D loss: %f] [G loss: %f]" % (epoch, d_loss[0], g_loss))
# If at save interval => save generated image samples
if epoch % sample_interval == 0:
self.sample_images(epoch)
6. 可视化
def sample_images(self, epoch):
r, c = 5, 5
noise = np.random.normal(0, 1, (r * c, self.latent_dim))
gen_imgs = self.generator.predict(noise)
# Rescale images 0 - 1
gen_imgs = 0.5 * gen_imgs + 1
fig, axs = plt.subplots(r, c)
cnt = 0
for i in range(r):
for j in range(c):
axs[i,j].imshow(gen_imgs[cnt, :,:,0], cmap='gray')
axs[i,j].axis('off')
cnt += 1
fig.savefig("images/mnist_%d.png" % epoch)
plt.close()
7. 运行
if __name__ == '__main__':
wgan = WGANGP()
wgan.train(epochs=30000, batch_size=32, sample_interval=100)
0 [D loss: 8.065693] [G loss: -0.143634]
100 [D loss: -0.699471] [G loss: -2.357683]
200 [D loss: -0.720496] [G loss: -2.259384]
由于训练时间原因放出前3次训练结果。
WGAN-GP在CIFAR-10的表现明显十分优秀。
五、效果分析
对四种GAN在大量情况下实验对哦比,可以看到DCGAN和LSGAN(最小二乘GAN)大多数条件小之下已经无法很好的生成图像,而WGAN虽然稳定生成,后几组很模糊,但是WGAN-GP都保证了高质量表现。
本站文章如无特殊说明,均为本站原创,如若转载,请注明出处:[生成对抗网络GAN入门指南](6)WassersteinGAN-GP - Python技术站