New Backup Filters feature in Unimus 2.1.0

Tomas Kirnak -

To fully explain when and how the new Backup Filters are useful, let's start with an overview of how Unimus stores backups for your devices without using the new filters. Normally, the backup procedure Unimus performs works as follows:

  1. connect to the device
  2. switch to a desired CLI mode to perform backup
  3. retrieve configuration from the device (for example show running-config)
  4. remove pagination, perform formatting, etc.
  5. remove dynamic contents from the backup
  6. compare retrieved config to currently known config of the device
  7. create a new configuration point (backup) or update existing

The last step of this process is the most complex one. The "backups" that Unimus retrieves are used to build a versioned configuration history for your device. Or to simplify - if there are no changes to the config of your device, a new "backup" point is not required. So rather than show you individual backups, Unimus will show you configuration points - ranges of valid configurations on your device to build a configuration timeline of the device. Any point in this timeline is a valid backup, but you can also see when and how the config of your device changes over time. This is actually more difficult to explain than it is in reality - simply you see a timeline of how your device was configured over time, and you can use this to restore to any point in the past.

Here is an example of a configuration timeline for a device:

We can see that this particular device has had 9 unique configuration points since we added it to Unimus in Dec. 2019. The current configuration of this device (the one on top) has been applied to the device on Apr. 9th, and is valid up until today (Jun 9th).

To be able to do this, Unimus needs to be able to figure out if there was a config change on the device, which is not as trivial as it might seem. For example, if the configuration of the device contains timestamp (Cisco IOS will show the current timestamp of when show running-config was executed), this needs to be ignored so this "change" in the configuration contents doesn't create a new configuration point. Over the years, we have built a large set of what we call "dynamic content filters" - and actually for each device type we support in Unimus, we write these dynamic content filters as a part of adding support for a device.

While our built-in filters work properly in the vast majority of cases, sometimes you might have dynamic data as a part of your configuration - some output that is unique on each configuration printout. This can break the configuration timeline for your device, as Unimus will consider each backup run as a unique configuration point, which makes the configuration timeline pretty useless.

This is where the new Backup Filters come in - you can define rules which will tell Unimus that a part of the configuration should be ignored for the purposes of comparing the current known config to the new received config.

How the new Backup Filters work

There are 2 types of Backup Filters you can define:

  • Deleted data filters
  • Ignored data filters

How Deleted data filters work - before comparison of the new backup to the currently stored backup, Unimus will REMOVE the matched data from the "new" backup. This means that this data will not be stored as a part of the backup at all - as if this data was not received from the device at all.

How Ignored data filters work - during comparison of the new backup to the currently stored backup, Unimus will IGNORE the matched data for the comparison purposes. The data will still be stored as a part of the backup, but if changes in the data happen, those changes will be ignored.

The order of filtering is:

  1. Deleted data filters
  2. Built-in "dynamic content filters"
  3. Ignored data filters

In the previous section of the article we described how the Ignored data filters can be used to ignore changes to parts of the backup. The Deleted data filters on the other hand can be used to completely remove some parts of the backup. For example, if your device outputs the DHCP lease database as a part of it's configuration, but you don't want to store that in Unimus, you could create "Deleted data" filters that would filter out the appropriate lines from the config. In effect, Unimus would then never store this part of the config of the device.

How to configure Backup Filters

The Backup Filters can be added in the new Backup > Configuration window:

After clicking to add a new filter, you have multiple options of how and to what it should apply:

There are a few ways to filter:

  • Line starts with
  • Line ends with
  • Regex

For the "line" filters, if there is a match, the whole line is filtered / deleted. Regex is a bit more flexible. If there are no capture groups in the regex, the whole regex match is filtered / deleted. However if you use capture groups, only the content captured by the capture groups will be filtered / deleted. This allows for very precise and flexible matching using regex and capture groups, so you can filter / delete only exactly what you need.

There are also multiple ways on how the filter applies:

  • Vendor
  • Device Type
  • Tag

If a vendor is chosen, then the filter will apply to all devices from that vendor. If a device type is chosen, the filter will apply to all devices of that device type. Finally you can choose to apply the filter only to devices tagged with a specific Tag, for targeted filtering on specific devices.

Example

Filtering changes to IPSec secrets. In this example, we setup an IGNORE filter for IPSec passwords:

You will notice this is using a "Regex" filter, and using a capture group to only filter the actual password (note also the use of a non-capturing group for more flexible matching). As described earlier, this would cause Unimus to IGNORE and changes of IPSec password, and not create new configurations if these passwords are changed. HOWEVER, since this is an IGNORE filter, the configuration will still be present in your backups:

We could uncheck the Filter dynamic content checkbox to show the actual cleartext which was saved. If we wanted to completely remove these secrets from the backups, we could have used a DELETE filter instead of an IGNORE filter.

The new Backup Filters feature is available starting with Unimus 2.1.0. Please head over to the Download section to download the latest Unimus release.