import tensorflow as tf weights = tf.constant([[1.0, -2.0],[-3.0 , 4.0]]) >>> sess.run(tf.contrib.layers.l1_regularizer(0.5)(weights)) 5.0 >>> sess.run(tf.keras.regularizers.l1(0.5)(weights)) 5.0 >>> sess.run(tf.keras.regularizers.l1()(weights)) 0.099999994 >>> sess.run(tf.keras.regularizers.l1(1)(weights)) 10.0 >>> sess.run(tf.nn.l2_loss(weights)) 15.0 >>> sess.run(tf.keras.regularizers.l2(1)(weights)) 30.0 >>> sess.run(tf.keras.regularizers.l2(0.5)(weights)) 15.0 >>> sess.run(tf.contrib.layers.l1_regularizer(0.5)(weights)) 5.0 >>> sess.run(tf.contrib.layers.l2_regularizer(0.5)(weights)) 7.5 >>> sess.run(tf.contrib.layers.l2_regularizer(1.0)(weights)) 15.0
在tensorflow中,tf.nn中只有tf.nn.l2_loss,却没有l1_loss,于是自己网上查阅资料,了解到tf.contrib.layers中有tf.contrib.layers.l1_regularizer(),但是tf.contrib目前新版本已经被弃用了,后来发现tf.keras.regularizers下面有l1和l2正则化器,但是该正则化器的l2有点不一样,从上面的结果可以看出,scale都为1时,它要多一倍。可以查看源代码,tf.nn.l2_loss和 tf.contrib.layers.l2_regularizer 中都统一除以了2.所以值要少一半。
>>> sess.run(tf.nn.l2_loss(weights)) 15.0 >>> sess.run(tf.keras.regularizers.l2(1)(weights)) 30.0 >>> sess.run(tf.contrib.layers.l2_regularizer(1.0)(weights)) 15.0
将scale设为0.5后,可以得到一样的值,因此,以后在损失函数中可以使用这样的方式来求l2损失和l1损失。
>>> sess.run(tf.keras.regularizers.l2(0.5)(weights))
15.0
参考了 day-17 L1和L2正则化的tensorflow示例 - 派森蛙 - 博客园
https://www.cnblogs.com/python-frog/p/9416970.html
''' 输入: x = [[1.0,2.0]] w = [[1.0,2.0],[3,0,4.0]] 输出: y = x*w = [[7.0,10.0]] l1 = (1.0+2.0+3.0+4.0)*0.5 = 5.0 l2 = (1.0**2 + 2.0**2 + 3.0**2 + 4.0**2) / 2)*0.5 = 7.5 ''' import tensorflow as tf from tensorflow.contrib.layers import * w = tf.constant([[1.0,2.0],[3.0,4.0]]) x = tf.placeholder(dtype=tf.float32,shape=[None,2]) y = tf.matmul(x,w) with tf.Session() as sess: init = tf.global_variables_initializer() sess.run(init) print(sess.run(y,feed_dict={x:[[1.0,2.0]]})) print("=========================") print(sess.run(l1_regularizer(scale=0.5)(w))) #(1.0+2.0+3.0+4.0)*0.5 = 5.0 print("=========================") print(sess.run(l2_regularizer(scale=0.5)(w))) #(1.0**2 + 2.0**2 + 3.0**2 + 4.0**2) / 2)*0.5 = 7.5
本站文章如无特殊说明,均为本站原创,如若转载,请注明出处:tensorflow 中的L1和L2正则化 - Python技术站