Parsing Network Device Output Part 2: Status or Configuration?

Reading Time: 3 minutes

A Brief Interlude

In my first post in this series, I dove into utilizing Regular Expressions (Regex) to parse network device output. Before I continue with some of the other parsing options, I thought it would be worthwhile to post a short blog laying out some definitions that I’ll be relying on. Specifically I’ll be using them for delineation among the different parsing options and their use cases.

At the heart of this problem, is that when interacting with traditional network devices there aren’t many programmatic options, such as a REST API, to return structured data. Sure, you could use SNMP for some things, but not for everything. In addition, working with SNMP can be it’s own type of nightmare, believe me, I’ve fought that fight many times in the past. This is changing as the industry matures, Cisco even has an entire certification track around utilizing their APIs now, however even today most Network Engineers’ first foray into automation uses SSH to query a device for information. Which brings us to output.

What Is Output?

For our purposes, output will be defined as any response that a network device returns when given a command via it’s Command Line Interface (CLI). This could be accomplished by many means; depending on the network device in question the CLI could be reached via Telnet, SSH, or even HTTPS. No matter the access method, the network device is still returning the same data, our output.

Execute show version on a Cisco-like device, and it will return some form of the status of the version as that device understands it. It will be structured, loosely, for human consumption with perhaps headings/titles or indentation/punctuation for legibility. It will not generally be structured in a way that would easily allow a computer program to understand and interact with it.

Just as you can issue show version and get output, you can similarly on most network devices execute a command akin to show running-config and get output. This would return some form of data about the configuration that was currently in use on that network device. Again, it will have some form of structure, usually either delineated by indentation or curly braces ({}), but mostly it will be in a form that is not easily understood or manipulated by a computer program that you are writing.

I make the specific call out that it may not easily be understood by a program that you are writing, because of course the network device itself can take that configuration and turn it into actionable information. However, that does us no good, as most vendors aren’t open sourcing their internal operating system parsing logic.

Status vs Configuration

In the two examples above, we received output from a network device after we issued a command via the CLI. One gave us some sort of status about the device, and the other gave us information about it’s configuration. Broadly speaking, all CLI commands on a network device fall into these two categories. You are either gathering or changing the status of the device with commands such as show version, show flash, or clear counters. Or you are gathering or changing the configuration of the device with commands such as show running-config, interface e1/0 <return> no shutdown.

It is vital to keep in mind which type of information you’re looking for or what changes you are trying to make, and whether it involves the device’s status or configuration. This will heavily influence how you parse/interact with the output that the device has given you. Some parsing methods or libraries are made specifically for turning configuration into something easy to manipulate/change in Python (CiscoConfParse, Heir-Config), some are more generic and have the ability to parse both status and configuration (Regex, TextFSM, Genie).

In the next post, I’ll dive into CiscoConfParse first, and we’ll see how it can make the act of understanding and manipulating traditional unstructured network device configuration much simpler.

Parsing Network Device Output Part 1: Regex

Reading Time: 4 minutes

In the Beginning…

Often when making their first steps into Network Automation, people may have an idea of what they want to do, but not exactly how to get there. For example, an engineer may want to simplify a single time-intensive task “Gather the SW version, serial numbers, and uptime from all of my Cisco ASAs.” However, upon getting the information, they’re unsure what to do with it or how to parse it and use it for something else. I’ve seen this question come up a few different times, and thought a series of blog posts was in order. Each of the posts (including this one) outline some sample parsing options. However, it is worth noting that Regex is not better or worse than using a parsing library, for example, they may just have different use cases.

  • First, in this post, I will outline the basics of gathering our information and parsing it directly with our own Regular Expressions (Regex).
  • Next, I will demonstrate parsing this information with the popular CiscoConfParse and HeirConfig libraries.
  • Third, I will demonstrate simple parsing with Google’s TextFSM and the awesome library of templates provided by Network To Code, ntc-templates.
  • Finally, I will demonstrate simple parsing with Cisco’s Genie/PyATS libraries.

Note that I am going to skip over a lot of the minutia of what Netmiko is, and how to use it to gather network device information, as there are many wonderful resources out there already on this. We’re going to focus primarily on what to do with the output once you have it.

In this series of posts, we’ll just use the simple network shown here. There is a Management Station with Python 3.8 installed, and two Cisco ASAv firewalls (ASAv1 and ASAv2) in a tiered setup.

Code samples and “requirements.txt” for this post can be found on my Github.

Environment Preparation

For this entire series of posts, I will be working the examples in a python virtual environment setup with the following dependencies installed. Note that we’re installing iPython, which is a much better alternative to the built in Python interactive shell. I highly recommend checking it out by executing ipython in this virtual environment. iPython allows us to do some more interactive introspection on objects and method/object doc_strings in real time, which we’ll use in later posts.

  • iPython
  • Netmiko
  • CiscoConfParse
  • HeirConfig
  • NTC Templates
  • Genie/PyATS

First, I create a virtual environment with whatever name you wish, and then activate that environment:

python3.8 -m venv nn_examples
cd nn_examples;source bin/activate

Next, I install the dependencies as discussed above:

pip install ciscoconfparse genie hier-config ipython netmiko ntc_templates pyats 

Gathering our Output

We need to instantiate a connection to our ASAs, and then execute the command “show version | inc So|Serial| up” against them to gather the needed output. Simple enough with Netmiko as shown below (or on Github):

#!/usr/bin/env python3

"""
Example code for Network-Notes.com entry on Parsing Network Device Output
"""

import getpass
import re
import sys


from netmiko import ConnectHandler


"""
Test parsing network device configuration
"""

# Gather the needed credentials
net_device_username = input("Username: ")
net_device_password = getpass.getpass()
# Set enable/secret = password for now
net_device_secret = net_device_password

# Setup a dict with our ASAvs in it, in real world this could be read
# from a CSV or the CLI or any other source
firewalls = {
    "ASAv1": {"ip": "10.10.10.50", "platform": "cisco_asa"},
    "ASAv2": {"ip": "10.10.60.2", "platform": "cisco_asa"},
}

# Setup an empty dict for our results:
results = {}

# Instantiate netmiko connection objects and gather the output of
# `show version | inc So|Serial| up` on these two firewalls
for fw_name, fw_data in firewalls.items():
    print(f"Connecting to {fw_name}...")
    fw_connection = ConnectHandler(
        ip=fw_data["ip"],
        device_type=fw_data["platform"],
        username=net_device_username,
        password=net_device_password,
        secret=net_device_secret,
    )
    results[fw_name] = fw_connection.send_command(
        command_string="show version | inc So|Serial| up"
    )

# Print our results
for fw_name, result in results.items():
    print(f"{fw_name} information:\n")
    print(result)

sys.exit()

Successfully executed with “python3.8 raw_gather.py”, this will output:

 Username: cisco
 Password: 
 Connecting to ASAv1…
 Connecting to ASAv2…
 ASAv1 information:

 Cisco Adaptive Security Appliance Software Version 9.12(1)2 
 ASAv1 up 23 mins 50 secs
 Serial Number: 123456789AB

 ASAv2 information:

 Cisco Adaptive Security Appliance Software Version 9.13(1)2 
 ASAv2 up 24 mins 5 secs
 Serial Number: 123456789AC

Now that we have an example of gathering the information, let’s parse it to python variables via Regex.

Parsing with Regex

Much like above, I will not exhaustively cover what Regex (Regular Expressions) are, as there are boundless resources on the internet (and good old O’Reilly books) that do this. Suffice to say for our purposes, that it is a way to match patterns in a string of text and gather output based on that, and that I strongly recommend finding a good test bed like https://regex101.com/ to help you along the way. You can get extremely complicated with Regex, and it can also bite you in the behind, just ask Cloudflare… Here is the three Regular Expressions I am using to match relevant output from the ASA.

Now here is what the program looks like utilizing the python re Regex module. The only difference is the addition of a section for parsing the output we’ve received, so that is all I’ll list below. The complete file is on Github as parse_with_regex.py:

# Parse our results:
print("Parsing Results...")
parsed_results = {}
for fw_name, result in results.items():
    parsed_results[fw_name] = {}
    parsed_results[fw_name]["version"] = re.search(
        r"Software Version (\d.\d{1,2}(?:\(?\d{1,2}?\)?\d{1,2}?)?)\W*$",
        result,
        re.MULTILINE,
    )
    parsed_results[fw_name]["uptime"] = re.search(
        r"up (.*)$", result, re.MULTILINE
    )
    parsed_results[fw_name]["serial"] = re.search(
        r"Serial Number: (\S*)$", result, re.MULTILINE
    )

# Print our results
for fw_name in results.keys():
    print(f"{fw_name} information:\n")
    print(f"\tVersion: {parsed_results[fw_name]['version'].group(1)}")
    print(f"\tUptime: {parsed_results[fw_name]['uptime'].group(1)}")
    print(f"\tSerial Number: {parsed_results[fw_name]['serial'].group(1)}")

This will give us the results of:

Username: cisco
 Password: 
 Connecting to ASAv1…
 Connecting to ASAv2…
 Parsing Results…
 ASAv1 information:
     Version: 9.12(1)2
     Uptime: 1 hour 25 mins
     Serial Number: 123456789AB
 ASAv2 information:
     Version: 9.13(1)2
     Uptime: 1 hour 25 mins
     Serial Number: 123456789AC

As you can see, this is a much more structured and useful output. We could also similarly output this to a CSV or other data source, now that we have parsed the data into something of more value than a raw string of all the data.

Next Up, Parsing Libraries!

In the next post I publish, I will utilize this same environment, but make use of some common configuration parsing libraries (CiscoConfParse and HeirConfig) to show how they could be useful for certain use cases.