I had setup vRealize Automation sometime back in a lab environment and this had been running fine for a few weeks.
I started seeing the HTTP 503 error without much changes to the environment.
From the logs on the vRA Appliance it looked like the IAAS machine did not respond to the status api call. - /var/log/vcac/catalina.out
2015-04-28 21:56:05,059 vcac: \[component="cafe:component-registry" priority="WARN" thread="registryServiceNotificationExecutor-21538" tenant=""\] org.springframework.web.client.RestTemplate.handleResponseE
rror:581 - GET request for "https://iaas.demo.local/WAPI/api/status" resulted in 503 (Service Unavailable); invoking error handler
2015-04-28 21:58:05,079 vcac: \[component="cafe:component-registry" priority="WARN" thread="registryServiceNotificationExecutor-21566" tenant=""\] org.springframework.web.client.RestTemplate.handleResponseE
rror:581 - GET request for "https://iaas.demo.local/WAPI/api/status" resulted in 503 (Service Unavailable); invoking error handler
In the IAAS machine, the following services were down:
- VMware vCloud Automation Center Service
- VMware vCloud Automation Center Agent - vSphereAgent
- VMware DEM-Worker - DEM
- VMware DEM-Orchestrator - DEO
What led to the services being down?
I recently reset the administrator account password for another solution that I was working on. The above services were configured to log on with the same administrator account.
Solution you ask?
Obviously, I updated the administrator user password for each of the services. The services started up fine, however, the error persisted. To resolve the issue, I had to reset the password at the application pools in IIS Manager.
To reset the password for the application pools:
- Launch IIS Manager, Start->run->inetmgr
- The password needs to be reset for the following application pools: RepositoryAppPool, vCACAppPool, WapiAppPool
- Right click the application pool, select Advanced Settings.
- Click into the identity section and select “…”
- Select set and reset the domain account credentials.
