Services and State with Ambari REST API

The Ambari management tool for Hadoop offers among other handy tools a comprehensive REST API for cluster administration. Logically a cluster is divided into hosts, services and service components. While the UI might not always has support for all needed scenarios sure the REST API can be used to achieve it. For example moving a master component of a service from one host to another.

In this post we are going to look a little closer at the way the Ambari API can be used to manage Hadoop services. At the end of this post you will find a list of all the currently supported Hadoop services with all the needed master, slave and client components that can be manged and administrated within your HDP stack. Also this posts contains the possible states and state transitions a component might have which could become useful when facing problems like Host config is in invalid state.

The possible states a service component can be in is listed below. To change or get the state of a service component the following API calls can be issued.

// Getting the state of the component
curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X GET
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/YARN_CLIENT 

// Setting the state of the component to INSTALLED
curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 
-d '{"RequestInfo":{"context":"Install YARN_CLIENT"},
"Body":{"HostRoles":{"state":"INSTALLED"}}}' 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/YARN_CLIENT

Among the need to change the state of a service component a very common issue is the need to actually move one service – most likely a master service – to different host. Users faced with this problems often need to issue the existing API due to a lack of support by the existing UI – at least with older releases of Ambari Web UI. This is an example of moving the OOZIE_SERVER to a new host.

// Stop the old component by putting it back in state INSTALLED
curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 
-d '{"RequestInfo": {"context": "Stop Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 
"Body":{"HostRoles": {"state": "INSTALLED"}}}' 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<OLD_HOST_FQDN>/host_components/OOZIE_SERVER

// Add component to host
curl -u admin:adming -H "X-Requested-By:ambari" -i -X POST 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

// Install component by achieving stated INSTALLED
curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 
-d '{"RequestInfo": {"context": "Install Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 
"Body":{"HostRoles": {"state": "INSTALLED"}}}' 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

// Delete old component
curl -u admin:admin -H "X-Requested-By:ambari" -i -X DELETE 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<OLD_HOST_FQDN>/host_components/OOZIE_SERVER

// Starting the new component
curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 
-d '{"RequestInfo": {"context": "Start Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 
"Body":{"HostRoles": {"state": "STARTED"}}}' 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

Removing a complete service from the cluster would require all the removal of all it’s components together with the service itself. For example for removing the Knox service we would need to remove the KNOX_GATEWAY component from the cluster, but before let’s get their state:

// Get service info
curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X GET
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/services/KNOX/

// Stop component KNOX_GATEWAY
curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X PUT 
-d '{"ServiceComponentInfo": {"state": "INSTALLED"}}'
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/services/KNOX/components/KNOX_GATEWAY

// Stop service KNOX
curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X PUT 
-d '{"ServiceInfo": {"state": "INSTALLED"}}'
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/services/KNOX

// Delete component
curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X DELETE
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/KNOX_GATEWAY

// Delete service
curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X DELETE
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/services/KNOX

Possible states of a component:

Allowed State transitions of a component:

Services

A Hadoop cluster contains of multiple services that individually have either master, slave or client components. Below you will find a list of all currently supported components as part of a HDP stack divided in master, slave, or client groups. The cardinality notates the number of components that can exist at the same time in one cluster.

Master Services

Slave Services

Clients

Further Readings

• Apache Ambari
• Ambari API Reference V1
• Host is in invalid state
• Ambari Workshop