Attempting to do a TFS upgrade, the install was failing with an error: “The following Windows service is installed on your computer: elasticsearch-service-x64. Remove elasticsearch-service-x64 to continue”

I was required to:

Go to the TFS Search directory and remove the search folder.

Stop the elastic search service, and navigate to the program files folder for elastic search (C:\Program Files\Microsoft Team Foundation Server 15.0\Search\ES\elasticsearch-2.4.1\bin)

Run the command service.bat remove to uninstall the Elastic Search service.