Even though there are a lot of resources on rendering and building network configurations, I was missing content that describes the pattern that I have been using for some time now. This post outlines the pattern I have been using and provides a toy example where that pattern is implemented.
Building the network configuration
To configure a network device, you need to generate the configuration in a vendor specific way. This may be a string, XML or something else. What I have seen people do is write a Jinja template that is specific for a vendor. Sometimes, they also make child templates to make things manageable. Additionally, data is oftentimes coming from many different places. It is very inviting for someone to end up with a plethora of Jijna templates, YAML files and whatnot. This is especially the case with multiple people working a project for some time.
When start out, they start with a configuration, for instance:
set system host-name R1 set system root-authentication encrypted-password "$ABC123"
After examining the configuration, they come up with a template:
set system host-name {{ hostname }}
set system root-authentication encrypted-password "{{ root_password_hash}}"
Obviously there is more to a configuration. Interfaces, secrets, BGP neighbors, etc. YAML files are made, a secrets manager is installed and databases are setup, maybe Netbox.
An alternative approach would be to start modelling your network in code and follow something like this:
Before anything else, think about what data any given configuration would need.
You could start off with an interface:
from pydantic import BaseModel
import ipaddress
class Layer3Interface(BaseModel):
"""Standard Layer 3 interface"""
name: str
device_name: str
ipv4: Optional[ipaddress.IPv4Interface]
ipv6: Optional[ipaddress.IPv6Interface]
description: Optional[str]
Next, you would embed this interface into a device. While you are building said device, you would realize other characteristics are needed as well. So, you start adding things:
class Communities(BaseModel):
"""Model that represents all the communities"""
std_communities: Dict[str, str]
other_communities: Optional[Dict[str, str]] = None
class Config:
allow_mutation = False
class PLATFORM(str, Enum):
juniper = "juniper"
arista = "arista"
class MODEL(str, Enum):
"""Selection of possible models that can be used by the NetDevice class."""
QFX10008 = "QFX10008"
MX10008 = "MX10008"
DCS_7050CX3_32S_R = "DCS-7050CX3-32S-R"
DCS_7260CX3_64_R = "DCS-7260CX3-64-R"
class Layer2Interface(BaseModel):
"""Standard Layer 2 interface
TODO: further implement this class"""
name: str
class NetDevice(BaseModel):
"""Network device schema"""
name: str
serial: str
model: MODEL
platform: PLATFORM
mgmt: ipaddress.IPv4Interface
role: str
interfaces: List[Union[Layer3Interface, Layer2Interface]]
communities: Optional[Communities] = None
secrets: Optional[Dict[str, str]] = None
class Config:
use_enum_values = True
In the end, you realize that from a configuration point of view, the network is basically a list of these devices you have constructed:
class Network(BaseModel):
"""Model that represents all the netdevices"""
devices: List[NetDevice] = []
class Config:
use_enum_values = True
Data to populate these constructs can come from different places. We can write the functions to gather that information and have those functions return the proper model/value:
def load_communities() -> Communities:
"""Mocking data retrieval from 'something'."""
with open("communities.yaml") as f:
data = yaml.safe_load(f)
communities = Communities(**data)
return communities
def load_secrets_from_vault() -> Dict[str, str]:
"""Retrieves secrets information from Vault"""
with open("secrets.yaml") as f:
secrets_data = yaml.safe_load(f)
return secrets_data
def load_information_from_ssot() -> Network:
"""Retreives information from SSOT"""
with open("devices.yaml") as f:
data = yaml.safe_load(f)
network = Network(**data)
return network
We now have all the information we need.
The next thing that would be useful is something to put it all together, like a builder class:
class NetworkBuilder:
"""Class that builds the data schema and the network configuration"""
def __init__(self, network: Network, communities: Communities, secrets: dict):
self.network = network
self.communities = communities
self.secrets = secrets
Great, we have a class that has all the fields and data that is required to instantiate the configurations for the network. Nothing much is happening though.
What you could say is missing is something to produce the configurations. Before that though, we might also need to take care of certain things and manipulate the per device data. For instance, the spine routers do not need any communities, but the leafs do.
Wat we do is add 3 methods to the class:
- build_network(): this method will manipulate devices and put all the details in place. In the next example, we attach communities to the proper device object. More interesting things can be done here though. For instance, you could add all neighbouring devices or all servers to the device object for later processing.
- store_schema()`: we write the schema for the device to a place. In the example, we output it to a file on disk. We could also output it to a database. This would allow us to manage the entire configuration through this schema. In the future, we will be able to alter the schema, and then use the updated data to generate a new configuration.
- render_templates(): here we load the schema and render the template for every device. In this example we render a Jinja template. It just presents the hostname and community configuration, but I hope the point comes across:
class NetworkBuilder:
"""Class that builds the data schema and the network configuration"""
def __init__(self, network: Network, communities: Communities, secrets: dict):
self.network = network
self.communities = communities
self.secrets = secrets
self.build_network()
self.store_schema()
self.render_templates()
def build_network(self):
"""Implements additional build logic
In this example, only community data is added to the network device.
"""
for device in self.network.devices:
if device.role == "leaf":
device.communities = self.communities
device.secrets = self.secrets
return self.network
def store_schema(self):
"""Store the 'per device' schema."""
for device in self.network.devices:
with open(f"{device.name}.json", "w") as f:
f.write(device.json(indent=2))
def render_templates(self):
"""Render the templates and write the configuration to disk."""
for device in self.network.devices:
device_name = device.name
file_loader = FileSystemLoader("templates")
env = Environment(loader=file_loader)
template = env.get_template("template.j2")
output = template.render(data=device.dict())
if output:
# avoid the Jinja whitespace nonesense:
output = "\n".join(
[line for line in output.splitlines() if line.strip()]
)
with open(f"{device_name}.cfg", "w") as f:
f.write(output)
print(f"\n\nRendered {device_name}.cfg\n\n")
print(output)
else:
raise RuntimeError("No template output!!")
Next, we make a jinja template:
set system host-name {{ data.name }}
set system root-authentication encrypted-password "{{ data.secrets.root_hash }}"
{%if data.role == "leaf" %}
{% for community_name, community_value in data["communities"]["std_communities"].items() %}
set policy-options community {{community_name}} members {{community_value}}
{% endfor %}
{% endif %}
Finally, we drive the whole thing:
if __name__ == "__main__":
community_input = load_communities()
secrets_input = load_secrets_from_vault()
network_input = load_information_from_ssot()
network = NetworkBuilder(
network=network_input, communities=community_input, secrets=secrets_input
)
Closing thoughts
This approach has worked very well for me. Organizing things like this has offered a lot of advantages. To name a few:
- it is easy to reason about distinct parts of the process. Because of this, extending it, explaining it or troubleshooting it becomes something straightforward
- individual components can be tested during CI
- different people can work on different parts
- individuals focussing on the configuration aspect do not have to be burdened with the coding
- individuals focussing on the coding do not necessarily have to be burdened with the vendor specific configuration aspect
But I think most importantly, the models are driving everything. So swapping out individual components becomes (relatively) easy. Pydantic will start throwing up in case you forget something or make a mistake.
Also, as a sidenote, I chose to include pydantic in this overview even though I am not really using all the features it has to offer. There are a lot of things pydantic has to offer. Validators to just mention one.
‘But without pydantic it would be faster!’
Yes. But ‘pydantic enforces type hints at runtime and provides user friendly errors when data is invalid’. Generating hundreds of thousands of lines of config is nothing for modern CPUs and you generally generate them only every now and again. The thing that matters most is being precise about what you generate. Pydantic can really help up your game here. So including it here is my way of telling you to go check it out.
A working example is found here. You can clone the repo and run main.py.
Thing that could be added:
- a CICD pipeline that tests the models as well as the generated configuration
- additional focus on the relations between the models so devices can easily access the attributes of all the other devices
- link device interfaces with each other device interfaces so your configuration efforts become aware of the topology
- introduce server objects and interfaces that are linked to device interfaces. This way, an update in a server or interface port can be reflected in the device configuration