You can run the start.sh script to start the VSTS build agent on Ubuntu server. This script downloads the Visual Studio Online Build Agent, installs it on the machine, registers with the Visual Studio Online account, and adds to the specified build agent pool and starts the build agent to accept jobs.
You can also find some good explanation about how this script works over here.
The start.sh script requires env.sh, .configure & .token file availability and jq package running on the machine. The scripts creates the required files and install the jq package if it is not already installed.