Fix race condition segfault when trying to gracefull exit API server

There seems to be a problem when destroying the backend class. So going to delete this from main instead.