Policy_freq = 2 # Policy network is updated every Xnd stepīatch_size = 512 # N of samples sampled from buffer to step of training Used to predict reward for a certain action made in a certain state. l3( x))Īgain 2 critic networks are implemented inside one pytorch Module.įC: (33 + 4) -> (400) -> (300) ->(1) with ReLu activations. You can do sigmoid and rescale if you prefer. Imporant to note, last operation in Tanh activation, that scales action output to required env. Why search for anything else? Neural networks detailsįC: (33) -> (400) -> (300) -> (4) with ReLu actiovations. It worked like a charm from the first attempt. As I was still in the exploratory phase, I switched to TD3. While I had some progress and reward was growing, training process was quite unstable. Game is considerend solved when mean reward from 20 agents in the last 100 episodes is >= + 30. So for 20 agent action space is (20,4).Īgent get rewarded if its end is localized in a moving spherical space. Action is defined by floats in range (-1,1) and every agent needs 4 actions. So in this case state is a (20, 33) vector.Īt every time step you have to perform an action for every agent, where action is a torque applied to the joint. Reacher20 is a Udacity version of Reacher enviroment, containing 20 simultaneous Reacher agents.Įach agent get a state vector of 33 floats, describing it's joints positions and speed as well as ball's position and speed. critic.pth - Saved weights from Critic networks from TD3.actor.pth - Saved weights for Actor network from TD3.replay_byffer.py - Replay Buffer implementation from OpenAI Baselines.networks.py - actor and critic Pytorch definitions.Solver.ipynb - reproduces the training procedure.Demo.ipynb - allows you to check enviroment and see working agent example.approach here, as enviroment isn't truly async. process, I choosed a TD3 algo that collects expirience 20x faster with 20 hands, treating them as sync. Proposed enviroment has 20 Reachers simultaneously. First reaches reward 30 in 21 episode, gets a mean of 30 in 100.
Repo with Implemenation of TD3 algo for Reacher 20 env.
Solution for Reacher enviroment in Unity ML.
0 kommentar(er)
