\(\nabla g_{t}=A_{t}^{M} \nabla_{\theta} d_{\cos }\left(s_{t+c}-s_{t}, g_{t}(\theta)\right)\)
where \(A_{t}^{M}=R_{t}-V_{t}^{M}\left(x_{t}, \theta\right)\) is the Manager’s advantage function, computed using a value function estimate \(V_{t}^{M}\left(x_{t}, \theta\right)\) from the internal critic; \(d_{\cos }\left(\alpha, \beta\right) = \alpha^T\beta /\left(|\alpha||\beta|\right)\) is the cosine similarity between two vectors.
\(\nabla g_{t}=A_{t}^{M} \nabla_{\theta} d_{\cos }\left(s_{t+c}-s_{t}, g_{t}(\theta)\right)\)
where \(A_{t}^{M}=R_{t}-V_{t}^{M}\left(x_{t}, \theta\right)\) is the Manager’s advantage function, computed using a value function estimate \(V_{t}^{M}\left(x_{t}, \theta\right)\) from the internal critic; \(d_{\cos }\left(\alpha, \beta\right) = \alpha^T\beta /\left(|\alpha||\beta|\right)\) is the cosine similarity between two vectors.
| status | not learned | measured difficulty | 37% [default] | last interval [days] | |||
|---|---|---|---|---|---|---|---|
| repetition number in this series | 0 | memorised on | scheduled repetition | ||||
| scheduled repetition interval | last repetition or drill |