Ambari Augeas Module and Configuration

Operating large to small size clusters needs automation; as well does installation. Going repeatedly through installation instructions, typing the same commands over and over again is really no fun. Worse it’s a waste of resources. The same holds true for keeping the configuration of 10 or 100 nodes in sync. A first step is to use a distributed shell and the next to turn to tools like Puppet, Cobblerd, Ansible, or Chef, which promise full automation of cluster operations.

As with most automation processes this tools tend to make the common parts easy and the not so common steps hard. The fact that every setup is different requires concepts of  adaptability that are easy to use. One of this tools for adapting configuration files could be seen in Augeas. Although hard to understand at time Augeas is easy to use, as it turns arbitrary configuration files into trees on which typical tree operations can be performed.

This post highlights the possibilities to us Augeas in a Puppet setup to install and configure a HDP Ambari cluster. Ambari agents are part of every node in a cluster, so when adding new nodes to a cluster in an automated fashion you would want to be prepared.

Ambari Agent Augeas Module

An Ambari setup mainly consists of properties, XML, and ini configuration files. For properties Files a properties lens exists. The same is also true for XML configuration files by a XML lens. Bot XML and properties file manipulation for Ambari setups will be covered below.

To manipulate ini files Augeas only provides a generic IniFile lens as a general interface. As ini files sometime follow their own specification one has to provide it’s own implementation base on that interface. The following lens is based on the  PHP.ini lens and once place under  /usr/share/augeas/lenses/ it can be used to configure  ambari-agent.ini files under /etc/ambari-agent/conf:

Place this under  /usr/share/augeas/lenses/ in a file named  ambariagent.aug and test it by running:

If nothing or an error happens, check the concrete output with:

Ambari Properties Configuration

The properties lens provided by Augeas is also just a generic one similar to the ini file lens. In contrast to providing a custom lense like we did above, we will in this case use the possibility to override autoload filters in the puppet module by providing an incl statement. This will automatically map it to a  /files/* mapping in augeas.

The following snippet in Puppet can be used to override the Ambari server user under /etc/ambari-server/conf/ambari.properties:

Ambari XML Configuration

One use case that would require the manipulation of a XML file in an Ambari setup is to change the repository URLs for your OS type. The following code snippet would exactly achieve this by using the Xml.lns in a puppet script:

 

Further Readings

2 thoughts on “Ambari Augeas Module and Configuration”

Leave a Reply