Drone Machine Learning

Reinforcement learning flies a four-motor drone to a goal

Overview

Category Unity ML-Agents
Start date August 2022
Network 2 × 256 hidden
Batch · buffer 1024 · 32768
Training ~10 hours

A reinforcement learning model that I trained locally using Unity's Machine Learning framework. It controls a flying drone with 4 motors towards a goal in a physically accurate simulation.

The model takes 15 inputs — five three-dimensional vectors: the position, velocity and angular velocity of the drone, a vector pointing up based on the drone's rotation (so the model is aware of its orientation), and finally the current goal position. The output is 4 continuous numbers, each representing a motor's force.

Training took about 10 hours. The rewards and punishments:

A big punishment when the drone hits the ground — the learning session then resets.
A small reward on each simulation update; the closer the drone is to the goal, the bigger the reward.
A big reward whenever it reaches the goal.
Rewards were tweaked each hour: early on the per-update reward was much bigger so the model would simply learn not to hit the ground, and later it was changed so the drone starts reaching for the goal.

The neural network has 2 hidden layers of 256 units each, with a batch size of 1024 and a buffer size of 32768.