A reinforcement learning model that I trained locally using the unity’s Machine learning framework which controls a flying drone with 4 motors towards a goal in a physically accurate simulation, the model takes 15 inputs which consist of 5, three-dimensional vectors, the position of the drone, velocity of the drone, angular velocity of the drone, a vector pointing up based on the rotation of the drone (so that the model is aware of the drone’s rotation) and finally the current goal’s position.
The output of the model is 4 continuous numbers and each of them represents a motor’s force.
Training took about 10 hours and the rewards and punishments are as follows:
A big punishment is given when the drone hits the ground and then the corresponding learning session will reset. On each simulation update a small reward is given, the closer the drone is to the goal, the bigger the reward and at last, a big reward is given to the model whenever it hits the goal. I also tweaked the rewards each hour for better training, for example, in the beginning, each simulation update reward was much bigger so that the model just learns to not hit the ground, and later this reward was changed so that the drone starts reaching for the goal.
The number of hidden layers in the neural network is 2 and the number of units in the hidden layers of the neural network is 256 with a batch size of 1024 and buffer size of 32768.