Right when I was starting out with SaltStack, I learned about the pillar. The pillar is an interface designed to offer values that the master distributes to (proxy) minions. Many of the examples floating around aim to show you how to use this pillar interface. This is mostly done by placing all sorts of data that is relevant to the infrastructure as a whole in the pillar for templating purposes or for use in states.

The examples worked really well and I started to use the pillar interface in a lot of templates. And since it worked so well, I started looking for more and more data I could put into the pillar. Whenever I came across something I needed for a template, I would convert the data to JSON or YAML and expose it to all the relevant (proxy) minions as pillar data.

After a while I noticed that everything in Salt was getting slow. States that would take 2-10 seconds in a lab suddenly took 7 minutes in production. After a some troubleshooting, I figured out that all of this was caused by the size of the pillar. After putting in more than 100.000 lines of JSON, YAML and Jinja, the environment basically came to a grinding halt. The master log showed that it was constantly rendering pillar data for minions, leaving little cycles for other tasks.

The solution to this problem was pretty straightforward: putting non-sensitive data into files and use that in states and templates. In the Salt-speak, such files are sometimes referred to as map files. The map file is a file that can contain YAML, JSON or JINJA and the data from the file can be imported into a state and/or into a template.

In this article, I will first put some data in the pillar and show you how to use it in a template. After this, I will move this data to a map file.

Using pillar data

When I set out working with the pillar, I had several scripts that would output JSON to files placed inside the pillar directory. The following is a shortened example of such a file:

/ $ cat /srv/pillar/data_file.sls 
{
    "hosts": {
        "switch01": "10.0.0.1",
        "router01": "10.0.0.2",
        "server01": "10.0.0.3"
    }
}

The file contains several hostnames and IP addresses. The file was added to the pillar in the top file using the following YAML:

base:
  '*':
    - data_file

With this in place, I was able to retrieve the data from the pillar on the command line like so:

/ $ salt minion pillar.item hosts

minion:
    ----------
    hosts:
        ----------
        router01:
            10.0.0.2
        server01:
            10.0.0.3
        switch01:
            10.0.0.1

This pillar data was accessible through states and templates using something like the following Jinja:

{% set router01 = pillar['hosts']['router01'] -%}
{{ router01 }}

Using slsutil.renderer, we can see what the previous template would output:

/ $ salt minion slsutil.renderer default_renderer='jinja' /var/tmp/example.j2
minion:
    10.0.0.2

Using a map file

After the amount of data in the pillar started slowing down the environment, I started moving pillar data into map files.

First, I moved the pillar file /srv/pillar/data_file.sls to /srv/salt/data/data_file.json. After moving the file, all the templates were changed to import the JSON file and lookup the data from there instead of using the pillar interface. This turned out to be a pretty minor change.

Let’s change the previous example template to make it use the map file:

{% import_json '/srv/salt/data/data_file.json' as data_file %}
{% set router01 = data_file['hosts']['router01'] -%}
{{ router01 }}

In the first line, the map file is imported as a dictionary called data_file. After this, we retrieve the value from that dictionary instead of getting it from the pillar. So we change pillar[‘hosts’][‘router01’] into data_file[‘hosts’][‘router01’].

When we render this new template, we get the following output:

/ $ salt minion slsutil.renderer default_renderer='jinja' /var/tmp/example.j2
minion:
    10.0.0.2

In this example, we used import_json to import JSON. In case you are dealing with YAML, you can simply use import_yaml.

Closing thoughts

Pillar data is expensive. The master needs to render the pillar for every individual minion, encrypt the pillar data and the message that it uses to send pillar data to the minion.

If you have a large environment where masters are controlling a lot of (proxy) minions, having a big pillar can impact performance. In some cases, you are better off storing non-sensitive data in a map file instead of the pillar.