循环神经网络 H H H H t H_{t} H t H H H t t t H t H_{t} H t X t X_{t} X t H t − 1 H_{t-1} H t − 1 H t H_{t} H t H t H_{t} H t X t ∈ R n × d X_{t}\in \mathbb{R}^{n\times d } X t ∈ R n × d t t t H t ∈ R n × h H_{t}\in \mathbb{R}^{n\times h } H t ∈ R n × h H t = ϕ ( X t W x h + H t − 1 W h h + b h ) H_{t}=\phi(X_{t}W_{xh} +H_{t-1}W_{hh}+b_{h}) H t = ϕ ( X t W x h + H t − 1 W h h + b h ) W x h ∈ R d × h W_{xh}\in \mathbb{R}^{d\times h} W x h ∈ R d × h W h h ∈ R h × h W_{hh}\in \mathbb{R}^{h\times h} W h h ∈ R h × h b h ∈ R 1 × h b_{h} \in \mathbb{R}^{1\times h} b h ∈ R 1 × h ϕ \phi ϕ H t − 1 W h h H_{t-1}W_{hh} H t − 1 W h h H t H_{t} H t H t H_{t} H t H t − 1 H_{t-1} H t − 1 t t t O t = H t W h q + b q O_{t}=H_{t}W_{hq} +b_{q} O t = H t W h q + b q W h q ∈ R h × q W_{hq}\in \mathbb{R}^{h\times q} W h q ∈ R h × q b q ∈ R 1 × q b_{q}\in \mathbb{R}^{1\times q} b q ∈ R 1 × q
GRUH t = ϕ ( X t W x h + H t − 1 W h h + b h ) H_{t}=\phi(X_{t}W_{xh} +H_{t-1}W_{hh}+b_{h}) H t = ϕ ( X t W x h + H t − 1 W h h + b h ) R t = ϕ ( X t W x r + H t − 1 W h r + b r ) R_{t}=\phi(X_{t}W_{xr} +H_{t-1}W_{hr}+b_{r}) R t = ϕ ( X t W x r + H t − 1 W h r + b r ) Z t = ϕ ( X t W x z + H t − 1 W h z + b z ) Z_{t}=\phi(X_{t}W_{xz} +H_{t-1}W_{hz}+b_{z}) Z t = ϕ ( X t W x z + H t − 1 W h z + b z ) H t ~ = t a n h ( X t W x h + ( R h ⊙ H t − 1 ) W h h + b h ) \tilde{H_{t}}=tanh(X_{t}W_{xh} +(R_{h}\odot H_{t-1})W_{hh}+b_{h}) H t ~ = t a n h ( X t W x h + ( R h ⊙ H t − 1 ) W h h + b h ) H t = Z t ⊙ H t − 1 + ( 1 − Z t ) H t − 1 ⊙ H t ~ H_{t}=Z_{t}\odot H_{t-1} +(1-Z_{t})H_{t-1}\odot \tilde{H_{t}} H t = Z t ⊙ H t − 1 + ( 1 − Z t ) H t − 1 ⊙ H t ~
LSTM 长短期记忆long short-term memory : I t = ϕ ( X t W x i + H t − 1 W h i + b i ) I_{t}=\phi(X_{t}W_{xi} +H_{t-1}W_{hi}+b_{i}) I t = ϕ ( X t W x i + H t − 1 W h i + b i ) F t = ϕ ( X t W x f + H t − 1 W h f + b f ) F_{t}=\phi(X_{t}W_{xf} +H_{t-1}W_{hf}+b_{f}) F t = ϕ ( X t W x f + H t − 1 W h f + b f ) O t = ϕ ( X t W x o + H t − 1 W h o + b o ) O_{t}=\phi(X_{t}W_{xo} +H_{t-1}W_{ho}+b_{o}) O t = ϕ ( X t W x o + H t − 1 W h o + b o ) C t ~ = t a n h ( X t W x c + H t − 1 W h c + b c ) \tilde{C_{t}}=tanh(X_{t}W_{xc} +H_{t-1}W_{hc}+b_{c}) C t ~ = t a n h ( X t W x c + H t − 1 W h c + b c ) C t = F t ⊙ C t − 1 + I t ⊙ C t ~ C_{t}=F_{t}\odot C_{t-1} +I_{t}\odot \tilde{C_{t}} C t = F t ⊙ C t − 1 + I t ⊙ C t ~ H t = O t ⊙ t a n h ( C t ) H_{t}=O_{t}\odot tanh(C_{t}) H t = O t ⊙ t a n h ( C t )
参考资料：
本站文章如无特殊说明，均为本站原创，如若转载，请注明出处：循环神经网络知识要点笔记 - Python技术站
微信扫一扫 
支付宝扫一扫 