I am going through the following blog on LSTM neural network:http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/

The author reshapes the input vector X as [samples, time steps, features] for different configuration of LSTMs.

The author writes

Indeed, the sequences of letters are time steps of one feature rather than one time step of separate features. We have given more context to the network, but not more sequence as it expected

What does this mean?

=========================================

I found this just below the [samples, time_steps, features] you are concerned with.

X = numpy.reshape(dataX, (len(dataX), seq_length, 1))

Samples - This is the len(dataX), or the amount of data points you have.

Time steps - This is equivalent to the amount of time steps you run your recurrent neural network. If you want your network to have memory of 60 characters, this number should be 60.

Features - this is the amount of features in every time step. If you are processing pictures, this is the amount of pixels. In this case you seem to have 1 feature per time step.

 

ASK：

can you explain the difference between : X = numpy.reshape(dataX, (len(dataX), 3, 1)) and X = numpy.reshape(dataX, (len(dataX), 1, 3)) How does this affect the lstm?

ANSWER：

(len(dataX), 3, 1) runs LSTM for 3 iterations, inputting a input vector of shape (1,). (len(dataX), 1, 3) runs LSTM for 1 iteration. Which means that it is quite useless to even have recurrent connections since there can't be any feedback from previous iterations. In this case input shape to RNN is of shape (3,)。

其实TimeSteps就是unfold的意思，就是tensorflow中的 NUM_STEPS 的意思。

Features其实就是输入的维度，也就是特征，一个维度一个特征。

 

 

The LSTM networks are stateful. They should be able to learn the whole alphabet sequence, but by default the Keras implementation resets the network state after each training batch.

LSTM网络本是状态传递的，这种网络本应该是学习整个序列的; 但是keras的默认实现却会在每个batch训练结束时重置网络的状态。