Custom inventory management using Ansible AWX Tower

In my first blog of the AWX series, I talked about how to set up auto-scaling infrastructure using Ansible AWX for AWS auto-scaling group configuration and deployment. You can also update existing auto scaling groups as well without touching user data. And in second blog I talked about value additions with Ansible AWX and how it helps you to manage infrastructure in better ways, adding Organizations, Users, and Teams to achieve delegations and processes within your infrastructure management, adding inventories hosts, and sources to fetch in inventories instead of maintaining old-fashioned host inventory file, which increases the complexity.

This time I will be talking about custom inventories - how it works, how we can write custom inventory for public and private cloud, how to create inventory scripts in AWX, and how to integrate custom inventory scripts in inventory to fetch host information.

What is Dynamic Inventory, and how it works?

Before Ansible came into the picture, you had some agent-based configuration management systems leading the market. In the agent-based approach, there was no need to keep inventory information stored. It worked on a server-client-based model, where the agent was configured with the server, and the client was getting configured with agent-based communication. But with Ansible’s agentless approach, it is necessary to store inventory information. So, ansible introduced a basic text-based file format - Inventory.

Generally, a DevOps engineer or ops engineer of a configuration management system wants to keep inventory in a centralized location or on a different software system, e.g., pulling inventory from a cloud provider, LDAP, Cobbler, or a costly enterprise-grade CMDB software. Ansible supports all these options via an external inventory system. The contrib/inventory directory already contains some of these, including options for EC2/Eucalyptus, Rackspace Cloud, and OpenStack. I will try to focus on some of them in this blog.

Here is an example of what an Ansible inventory file looks like:

[local]

server1

server2

server3
[web]
server4
server5
[web:vars]
ansible_connection=winrm
ansible_port=5986
ansible_user=Administrator
ansible_password=asasas
ansible_winrm_server_cert_validation=ignore
ansible_winrm_transport = ssl

Here you can see two groups, '[local]' and '[web].' I have various hosts underneath these two groups, including localhost and server1-5. Finally, I have a set of web and local group variables in the '[web:vars]' section.

Now, managing a large flat file can be a little unhandy. So, for a user who keeps on adding and removing devices frequently, maintaining the inventory can turn out to be a problematic and time-consuming affair. And there is always a possibility of duplicating information that is stored in an alternate system.

Ansible has an alternate way of managing inventory called dynamic inventory. With dynamic inventory, you can write code that dynamically fetches host/inventory information from your private/public cloud or data center.

How do we write Dynamic inventory scripts?

You can write inventory scripts in any dynamic language that you have installed on the AWX machine such as shell or python. But you need to start with a normal script shebang line like #!/bin/bash or #!/usr/bin/python, and it runs as the AWX user.

Most infrastructures can be managed with a custom inventory file or an off-the-shelf cloud inventory script, but there are many situations where more control is needed. Ansible will accept any kind of executable file as an inventory file, so you can build your own dynamic inventory the way you like, as long as you can pass it to Ansible as JSON.

Dynamic inventory script can do anything to get the data (call an external API, pull information from a database or file, etc.), and Ansible will use it as an inventory source as long as it returns a JSON structure like the one above when the script is called with the --list.

The --list option must enlist all the groups and associated hosts and group variables. The --host option must either return an empty dictionary or a dictionary of variables relevant to that host.

Here's a simple implementation of a dynamic inventory script in Python:

#!/usr/bin/env python
'''
Example custom dynamic inventory script for Ansible, in Python.
'''
import os
import sys
import argparse
try:
import json
except ImportError:
import simplejson as json
class ExampleInventory(object):
def __init__(self):
self.inventory = {}
self.read_cli_args()
# Called with `--list`.
if self.args.list:
self.inventory = self.example_inventory()
# Called with `--host [hostname]`.
elif self.args.host:
# Not implemented, since we return _meta info `--list`.
self.inventory = self.empty_inventory()
# If no groups or vars are present, return an empty inventory.
else:
self.inventory = self.empty_inventory()
print json.dumps(self.inventory);
# Example inventory for testing.
def example_inventory(self):
return {
'group': {
'hosts': ['192.168.28.71', '192.168.28.72'],
'vars': {
'ansible_ssh_user': 'vagrant',
'ansible_ssh_private_key_file':
'example_variable': 'value'
}
},
'_meta': {
'hostvars': {
'192.168.28.71': {
'host_specific_var': 'foo'
},
'192.168.28.72': {
'host_specific_var': 'bar'
}
}
}
}
# Empty inventory for testing.
def empty_inventory(self):
return {'_meta': {'hostvars': {}}}
# Read the command line args passed to the script.
def read_cli_args(self):
parser = argparse.ArgumentParser()
parser.add_argument('--list', action = 'store_true')
parser.add_argument('--host', action = 'store')
self.args = parser.parse_args()
# Get the inventory.
ExampleInventory()

As you can see in the above script, I have provided a list and host options, and its printing requires JSON format that Ansible uses to populate inventory.

How to configure custom scripts in Ansible AWX?

AWX also provides a database to store inventory results that are accessible via web and REST. AWX syncs with all Ansible dynamic inventory sources you might be using and also includes a graphical inventory editor. By having a database record of all of your hosts, it’s easy to correlate past events and see which ones had failures on their last playbook runs.

As described earlier, Ansible can pull inventory information from dynamic sources, including cloud sources.

To create custom Inventory scripts:

• Click on the tab on the left, “INVENTORY SCRIPTS.”
• Add new inventory scripts using ADD option.

• Fill in your Name, Description, and Organization, if any.
• Copy and paste your custom script like the one in the earlier part of the blog
• Save.

This adds inventory scripts.

How to use inventory scripts?

First, you need to add inventory, and I explained the process of adding new inventory in my last blog. Once new inventory is added:

• Edit newly added inventory using the edit option.
• Go to the sources option and add a new source.
• Fill in the Name, Description, and Select “Custom Script” for the source option.

• Select the Credentials required for your custom scripts (to know about Credentials and how to add them, please visit my first blog on AWX); e.g., if you are using AWS dynamic script, select the AWS credentials that we added in the first blog.
• Select Custom Inventory Script from List, the one we added in prior steps.

• Save the Source and sync it using the Sync option from the sources list.

• Once the sync process is finished, you can see the required hosts added to the inventory.

How to use inventory to run Ansible job?

Once you are done with the aforementioned steps, you are ready to use this inventory to create/run your template jobs. You can select added inventory while creating or running a template to run on added hosts.

There are lots of benefits in using feature-rich AWX to create your own custom inventory scripts. The biggest one of them is it will remove the overhead of creating/maintaining your own inventory files. You can write own script to sync your private/public cloud or datacenter. Some of the public cloud dynamic inventory scripts are already present on public Ansible git repo, which you can use to create a custom script in AWX. So, that was all about Ansible AWX and how you can leverage it and its features to provision auto-scaling infrastructure and more.