Release Overview - Unimus 2.0.0

This article highlights the most significant changes and new major features in the Unimus 2.0.0 and Unimus Core release.

With each new release, we also upload a release overview video, so if you prefer a video format, you can find it here: Youtube - 2.0.0 Release Overview video

For those who prefer readable content, read on!

“Zones” feature and multi-tenancy

"Zones" add support for multi-tenancy, remote networks and distributed polling. You can have a central Unimus server to manage many remote networks, or you can split devices in your network across Zones, with each Zone using a separate Core to spread load from your server across multiple poller Cores.

Unimus Core and remote networks

The Core can serve as a remote poller / remote agent for Unimus, and the Zones you create in Unimus can either be polled directly from the Unimus server, or from a Unimus Core. You can check the new Architecture Overview and Zones articles on our Wiki for more details on Zones and Unimus Core.

Full Config Change Notifications over Slack

Previously Unimus would only send config change summaries over Slack, but due to community demand, we have implemented full diffs over Slack. You will need to reconfigure the Slack notification sender in Unimus to use a Slack Bot. Please check our blog article here on how to create and integrate a Slack Bot with Unimus.

Mass Config Push scheduling

You can now easily schedule Config Push jobs from you already existing Push Presets. This heavily extends the automation capabilities of Unimus, allowing you to schedule any configuration deployments. Push results are available in the Push Preset, and in a new “Config Push history” table on the Dashboard.

Support for binary backups

With 2.0.0, we have added support for storing binary backup files, and also extended all systems in Unimus to support binary backups. This means change detection, diffs, notifications, and everything else will properly work with binary backups.

Push binary backups into Unimus

With support for binary backups, we have also added a new API end-point, which allows you to push binary files to Unimus as device backups. This opens new use-cases for Unimus, as you can now push files to Unimus from external systems or scripts, and Unimus will perform change detection, notifications, and all other functions as expected.

Push text backups into Unimus

The new API endpoint also supports pushing text files to Unimus, which allows you to extend Unimus with support for any device even if we don't support it directly. You can script backup retrieval yourself, and push the resulting backup file to Unimus for processing, storage, notifications, etc...

PRTG and Observium in NMS Sync

We continue to add new sync connectors to NMS sync, and with this release, you can now adopt devices into Unimus from PRTG or Observium. We are adding new connectors with each Unimus release, so if your NMS doesn't yet have a connector available, please keep an eye on our Roadmap and future changelogs.

UI and UX improvements

There are also MANY UI and user experience improvements in this release. Some of the more notable ones are the new "Device Info" table in Devices, last job status indicator in Devices, much more detailed job failure logs for failed Discoveries, new result history tables on the Dashboard, and the "Export backups" functionality in the "Backup" screen.

Performance improvements

We have worked hard to improve the UI performance of Unimus in 2.0.0. The UI should now be much more responsive when working with many devices. Specifically large tables should load much faster - such as Devices, Backups, etc. Config Search server-side logic has been also improved to deliver search results faster.

Bug squashing and issue fixing

We have fixed more than 50 various bugs, issues and UI problems in 2.0.0, some of which have been present since 1.0.0. There are also security fixes for users with Tag-based access restrictions. A big thank-you here goes to everyone testing the 2.0 Beta and RC releases, and helping us iron out all of these.

With each new release, we add support for new network vendors and devices. This time around, we are releasing support for 22 new device types, across multiple networking vendors.
           
The Changelog for 2.0.0 is quite large, and this article doesn't cover it completely. If you want to see the full scope of changes in this release, please check the full Changelog below:

Full changelog:

= Version 2.0.0 =
Important:
  Slack integration has been migrated from a Webhook to a Slack App. Your Slack notifications will not work without reconfiguration.
  Manual migration and reconfiguration is required, please see more in the "Migration warnings" section.

Features:
  New "Zones" feature for support of remote networks - includes a new "Unimus Core" that serves as the remote proxy / remote agent
  Configuration Change Notifications with full diffs are now supported over Slack (if Slack Notification Sender enabled)
  Failed discovery logs now show full details of discovery and why it failed (Dashboard > Latest Failed Jobs)
  Added visual indicator (grey/green/red) of last job result to the Devices table
  "Devices > Info" window completely reworked, now shows much more useful information about the device
  Added Last Backup Date to device info window ("Devices > Info")
  Added REST endpoint to upload backup (Push backup into Unimus)
  Added support for binary backups (currently only possible with API Backup Push)
  Added an "Export backups" button to the "Backups" view - allows to export all or only latest backups for all devices
  Added support for specifying a CRON expression for Schedules (in addition to current options)
  Changed pagination on the "Config Search" view to 500 (up from 10)
  NetXMS client API updated to version 3.1 (NMS Sync)
  Zabbix importer will now import nodes with only Agent-type interfaces
  Added a new help link on the Backups view, "How does Unimus store backups?"
  Added a new Backup Retention Policy - "Number of backups" (will only keep last x backup for device)
  Added a new "Send Diff" and "Send Backup" popup that replaces the old email input form
  The "Send Diff" and "Send Backup" features now also supports sending diffs over Slack
  New global notification options to control where the system FQDN is displayed in notifications (title or body)
  Added system FQDN to notifications which were missing it (all notifications now contain system FQDN)
  Improved system FQDN lookup for notifications on Windows
  Improved message formatting in all Email and Slack notifications
  Improved UX in all sections of the Notification view ("Save" buttons now only active on change, added "Discard" button, etc.)
  Added retention cleanup jobs to the "Show scheduled tasks" window
  Added new "http.proxyType" and "https.proxyType" settings to configure proxy type when running Unimus behind a HTTP(S) proxy
  Improved responsiveness in multiple views in Mass Config Push
  Added a new Easter Egg (hint: "mike", also, Hi Mike!)
  Improved handling of CLI mode changes, many previously unhandled edge-cases now work properly
  Added support for empty password (just press enter) CLI mode changes (enable, configure)
  Improved detection of "Press any key to continue" and "Press enter to continue" prompts
  Added support for "Do you accept this statement [yes/no]" prompts during login
  Added support for shortened prompts on Cisco IOS in Configure mode
  Added support for line-break prompts in Cisco IOS when using tclsh
  Improved support for Cisco ASA Thread Defense and Cisco FirePOWER TDM
  Added support for read-only user accounts on ExtremeOS
  Improved support for Enhanced Security Mode on HP/HPE ProCurve/Provision/ArubaOS
  Added output of "show bof" to TiMOS backup
  Improved support for ArubaOS Wireless Controllers in various edge-cases
  Improved banner detection during CLI login process

  "Zones" feature for support of remote networks and distributed polling
    - you can create as many Zones as required, each zone signifying a unique network
    - new top level "Zones" view for Zone management
    - Zones can be polled directly from Unimus, or using the new Unimus Core serving as the remote proxy / remote agent for the Zone
    - architectural overview: https://wiki.unimus.net/display/UNPUB/Architecture+overview
    - more info about Zones: https://wiki.unimus.net/display/UNPUB/Zones

  "Debug Mode" options moved to the "Zones" menu
    - Unimus allows debugging remote cores directly from the Unimus UI
    - you can also download logs from Remote Cores directly in Unimus
    - this requires setting debug options per-zone, so Debug Mode moved to "Zones"

  Mass Config Pushes can now be scheduled
    - You can now schedule Config Push jobs for more automation power
    - More details on Push result notifications and Push result history below

  Other Mass Config Push improvements:
    - Added "Config Push History" table to the Dashboard
    - Added new "Config Push Result" notifications (enabled by default)
    - Push job status is displayed for each Push preset in Mass Config Push Home view
    - Improved the responsiveness (UI scaling) of the Config Push view

  PRTG importer was added to the "NMS Sync" view
    - uses PRTG API to sync devices from PRTG to Unimus
    - sync possible based on Tags, or by node hierarchy in the device tree

  Observium importer was added to the "NMS Sync" view
    - uses Observium API to sync devices from Observium to Unimus
    - sync only specific devices from Groups, or all devices in Observium

  Updated dynamic (runtime) data filtering from backups in diffs:
    - improved filtering of dynamic (runtime) data from backups in all diff views
    - whenever possible, filtering will no longer make a backup invalid by changing it's syntax
    - this only influences diffs - in Unimus and in Config Change notifications
      (View, Download and Send Backup features were always sending raw, unfiltered backups)
    - See more info below in the "Migration warnings" section

  Network Scan improvements:
    - Added "Network Scan History" table to the Dashboard
    - Added new "Network Scan Result" notifications (disabled by default)

  Added support for:
    - ArubaOS-CX devices (Aruba / HPE 8320)
    - more variants of AudioCode devices
    - Blonder Tongue CMTS
    - Casa CMTS
    - Cisco ASA TD
    - Cisco IE (industrial ethernet) switches
    - CTS switches (FOS-3128)
    - more variants of Dell PowerConnect switches
    - Draytek Vigor (Discovery and Config Push only, Backup not supported)
    - Exinda devices
    - Fortinet FortiAnalyzer
    - Fortinet FortiOS v6
    - Harmonics CMTS
    - HPE StoreFabric devices
    - HPE VirtualConnect
    - Huawei Eudemon
    - Huawei VRP in HRP mode
    - Huawei VRP multi-context
    - LANCOM switches (Discovery and Config Push only, Backup not supported)
    - more variants of Mellanox switches
    - Moxa switches
    - Omnitron RuggedNet switches
    - Ubiquiti AirOS CS (custom script) firmwares
    - Ubiquiti UFiber OLT
    - Zhone MXK

Fixes:
  Fixed discovery not running for undiscovered devices when credential was added and discovery should run according to system settings
  Fixed the password of a High Security credential being visible in "Device -> Show Info -> Show credentials"
  Fixed Mass Config Push not working when it contained Un-managed or Undiscovered devices
  Fixed Mass Config Push not working when it contained devices with all connectors disabled
  Fixed Config Search showing only first 500 backups that matched the search
  Fixed Config Search "Expand all" not working
  Fixed wrong (empty) config change notifications on Calix OccamOS based devices
  Fixed device selection selecting devices randomly if they were imported from "Address Importer" or ".csv File Importer"
  Fixed Zabbix importer not importing nodes which only had Agent-type interfaces
  Fixed .csv importer sometimes importing the file header even when "Ignore header" was enabled
  Fixed wrong config change notification for Cisco WLC caused by CDP peer changes
  Fixed wrong config change notification for FortiOS caused by dynamic certificate key output
  Fixed Mass Config Push status showing "Scan Status" instead of "Push Status"
  Fixed multiple extremely rare bugs where Config Search did not show some backups that matched (normally this would never happen)
  Fixed scheduling services to run on Schedule deletion, even if no Push or Scan presets were scheduled (did not schedule jobs, just enabled service)
  Fixed a very rare bug that could cause backup failing with very short backups
  Fixed change of backup retention only being applied after service restart
  Fixed not being able to delete schedules in the Deployment Wizard
  Fixed multiple UI inconsistencies and UX pain-points
  Fixed multiple rare edge-case failures when switching CLI modes (enable, configure)
  Fixed ExtremeOS devices not working when used with read-only accounts
  Fixed FS S3900 switching being discovered as Allied Telesis
  Fixed some HP 1910 models not being discovered
  Fixed Network Scan very slow when DNS requests were timing out
  Fixed Network Scan subnets import incorrectly accepting invalid some subnets as valid
  Fixed DNS timeout configuration being ignored
  Fixed some Mellanox switch models not being discovered
  Fixed backup not working on specific TelcoSys T-Marc firmwares
  Fixed backup not working on a few specific Brocade devices
  Fixed very rare login failure on devices with extremely slow data output during login
  Fixed Config Push that required Configure mode not working on some Fiberstore switches
  Fixed Patton/Inalp devices not working (discovery/backup/push) in certain cases
  Fixed ArubaOS Wireless Controllers not working in very rare edge-cases
  Fixed extremely rare login failure on devices with a post-login menu
  Fixed multiple other extremely rare login failures in various edge-cases
  Fixed backup failing on Adtran Total Access with extremely long configurations
  Fixed discovery failing on newer AudioCodes Mediant devices / firmwares
  Fixed extremely rare cases where VT100 control sequences were not properly stripped from backups

  Fixed a bug that could cause ~1% of scheduled backups to fail on slow, or heavily loaded devices
    - each scheduled backup, a small random subset of devices would fail their scheduled backups
    - slow (older) devices, devices under sufficient load to slow down the control plane, or devices with slower external AAA were most affected
    - in the long run, all devices would be properly backed up, as the subset was usually different for each scheduled run
    - running backups manually would work, only scheduled backups were affected

Security fixes:
  Fixed issue that caused imports from HTTPS URLs in "NMS Sync" to not check HTTPS certificates even if "Do not check HTTPS certificates" was not checked
  Fixed users being able to change "Other settings > Sensitive data stripping" even for Tags they didn't have access to
  Fixed users being able to change "Other settings > Per-Tag connectors" even for Tags they didn't have access to

Embedded Core version:
  2.0.0

Migration warnings:
  Slack integration has been migrated from a Webhook to a Slack App. This is due to the addition of sending Configuration
  Change Notifications over Slack. The Webhook API did not support sending Snippets, which Config Notifications require.
  You will need to setup a new Slack App for Unimus, and reconfigure Unimus Slack sender in "Notifications > Slack".

  For some devices, there may be a single config-change notification after the first backup following the 2.0.0 upgrade.
  This will show a change occurred inside a comment or a non-config line. This is expected due to changes to the dynamic
  (runtime) backup content filtering mentioned in the "Features" section. This is caused by changes to what Unimus
  considers as dynamic (runtime) data inside backups, and you can safely ignore this change notification.

Known issues:
  ISSUE: under rare circumstances, when a Unimus Core disconnects due to packet loss, some jobs may become stuck in Unimus
  WORKAROUND: restarting Unimus is necessary
  STATUS: fixing already in progress - fix coming in 2.0.1

  ISSUE: when you delete the "Device Output Log" file in "Debug mode", any jobs that started before deletion, but finish after
         deletion will recreate the file and write their output to the file
  WORKAROUND: delete "Device Output Log" after all jobs finish / no jobs are running
  STATUS: issue scheduled for fix in 2.0.1

  ISSUE: session timeout doesn't work in certain situations when browser tab is not closed - user's web session can remain logged-in forever
  WORKAROUND: close all tabs in which Unimus is opened, or log-out manually
  STATUS: issue scheduled for fix in 2.0.1

  ISSUE: Importing is possible even with accounts that don't have access to the Default Zone due to Tag-based access restrictions
  WORKAROUND: none, account can be made read-only
  STATUS: issue scheduled for fix in 2.0.1

  ISSUE: Unable to export all backups when two zones have devices with same addresses
  WORKAROUND: none
  STATUS: issue scheduled for fix in 2.0.1

  ISSUE: with higher latency, when writing text into an input box, a desync may occur that causes a character to get lost,
         and the cursor to jump to the start of the input box
  WORKAROUND: none
  STATUS: we are investigating

  ISSUE: unable to set connection timeout in Core - this doesn't influence Core functionality in any way
  WORKAROUND: none
  STATUS: currently no ETA, framework limitations

  ISSUE: special characters can be replaced by '?' under specific circumstances
  WORKAROUND: none
  STATUS: currently no ETA, framework limitations