New features — base Vyatta NOS
VRRP VRF support with RFC-compatibility
Virtual Router Redundancy Protocol (VRRP) Virtual Internet Protocols (VIP) addresses are unaware of the Virtual Routing Function (VRF) they are created within, operating in the default/global routing namespace. While this does not affect the functionality of the protocol, it does affect attempting to ping the virtual address. This feature introduces the ability to make specific VRRP groups VRF-aware, creating interfaces in the parent interface's VRF. It is then possible to ping the IP addresses.
Operational mode commands
The show vrrp detail output command contains a new line when a group is VRF-aware, for example:
vyatta@vm-log-2# run show vrrp detail
--------------------------------------------------
Interface: dp0p1s1
--------------
Group: 1
----------
State: MASTER
Last transition: 2s
Version: 3
RFC Compliant
VRF Aware: true
Virtual MAC interface: dp0vrrp1
Address Owner: no
Source Address: 10.1.1.22
Configured Priority: 100
Effective Priority: 100
Advertisement interval: 1000 milli-sec
Preempt: enabled
Accept: disabled
VIP count: 1
10.10.10.100/32
Group: 2
----------
State: MASTER
Last transition: 2s
Version: 3
RFC Compliant
VRF Aware: false
Virtual MAC interface: dp0vrrp1
Address Owner: no
Source Address: 10.1.1.22
Configured Priority: 100
Effective Priority: 100
Advertisement interval: 1000 milli-sec
Preempt: enabled
Accept: disabled
VIP count: 1
10.10.11.100/32
Configuration data model
The following command must be used with the rfc-compatibility configuration option. Without it, the configuration fails validation and is not committed.
set interfaces intf_type intf_name vrrp vrrp-group N vrf-aware Syslog event detection and notification
Some customers use the syslog content to monitor what is happening on vRouter. Although it is possible to send these logs to a central location for processing, where an analysis of the logs could result in the triggering of some actions, it isn't always possible to do so with a suitable response time, or due to some connectivity issue with the box. This feature provides a way to trigger on-box actions in response to the content of log messages or perhaps certain logs occurring more frequently than a defined threshold. This is achieved by the addition of new syslog-enhanced actions, which can trigger the calling of an arbitrary script. Some examples of what the scripts do is to capture the output of some show commands, modify the configuration, or reboot the box.
A syslog event consists of an event configuration, detailing the script to be called to handle the event, and an event syslog-enhanced rule action. The following example shows defining three events with two rules that have event actions.
system {
syslog-enhanced {
event-handler emergency {
call-script "emerg-alert-action"
rate-limit {
interval 300
}
user vyatta
}
event-handler interface-state-change {
call-script "interface-event"
rate-limit {
interval 0
}
}
}
rule 100 {
match {
msg {
posix-match "(Link|admin) (up|down)"
}
}
then {
event {
handler interface-state-change
}
}
}
rule 200 {
match {
severity {
equals crit
}
}
then {
event {
handler emergency
arguments "--critical"
}
}
}
} Event handler call script
The event call-script specifies a script to call. The only scripts that can be called are scripts that are present in the managed event-script location. If the named script is not present in the managed location, the event is logged, but no further action occurs. The call-script can include arguments that are passed to the script, as shown below.
event-handler event-one {
call-script "event_script --arg1 one --arg2 two"
}
Event rate-limit interval
An event is rate-limited to avoid bursts of logs triggering the same event in quick succession and causing a flood of the same script being called an excessive number of times. The rate-limit interval configures the number of seconds before the event script is called again. Configuring a value of 0 seconds results in no rate-limiting of the event. The default rate-limit is every 5 seconds.
In the following example, the event handler for the notify-engineer event is restricted to once per hour (3600 seconds).
system {
syslog-enhanced {
event-handler notify-engineer {
call-script "emerg-alert-action"
rate-limit {
interval 3600
}
}
}
}
Event user
A username can be specified for an event. The script is invoked as the named user. This caters for running scripts with appropriate permissions, either to allow access to privileged operations or resources or to perform some show or configuration operation with appropriate authorization, perhaps to hide sensitive information. Where no user is specified, the script is called as root.
In the following example, the collect-debug event handler calls the script "do_show_commands", and it is run as user "admin143ED". This ensures that command and data access are authorized using the credentials of "admin143ED".
system {
syslog-enhanced {
event-handler collect-debug {
call-script "do_show_commands"
rate-limit {
interval 30
}
user admin143ED
}
}
Event action
A rule has a new event action, available as both "then" and "otherwise" actions. The event must be configured in the event list system syslog-enhanced events event-name.
In this example, when a log message contains "memory error", the collect-debug event handler is called to capture the output of some show commands, to capture the system state near the point at which the log was captured. The event handler itself restricts this information gathering to once every 30 seconds.
system {
syslog-enhanced {
event-handler collect-debug {
call-script "do_show_commands"
rate-limit {
interval 30
}
user admin143ED
}
}
rule 100 {
match {
msg {
posix-match "memory error"
}
}
then {
event {
handler collect-debug
}
}
}
}
Per-event action arguments
An event rule action can specify arguments, which is a text string that is made available to the called script via the environment variable SYSLOG_EVENT_ARGS. This provides a way for an event action to pass additional arguments or context to the called script.
In this example, the emergency event action is called by rules 200 and 210. However, rule 200 includes the additional argument –emergency for logs of level emerg.
system {
syslog-enhanced {
event-handler handle-emergency {
call-script "process-emergency"
rate-limit {
interval 300
}
user vyatta
}
rule 200 {
match {
severity {
equals emerg
}
}
then {
event {
handler handle-emergency
arguments "--emergency"
}
set-flag emergency-level
}
}
rule 210 {
match {
severity {
at-least crit
}
without-flag emergency-level
}
then {
event {
handler handle-emergency
}
}
}
}
}
Script environment
Scripts are called using the Linux environment. Information about the syslog message that triggered the script to be called is json encoded and made available to the script on stdin.
Aug 20 14:31:45 vm-evnt2-1 dataplane[1996]: DATAPLANE: dp0p1s1 changed state to admin up{
"syslog-tag" : "dataplane[1996]:",
"timestamp" : "2021-08-20T14:31:45.451640+00:00",
"msg" : "DATAPLANE: dp0p1s1 changed state to admin up",
"host" : "vm-evnt2-1",
"severity" : 5,
"facility" : 22,
"source" : "dataplane"
}
Anything seen on stdout from the script is logged at the DEBUG level to the journal. This provides a mechanism to debug scripts during development or as a logged record that the script has been called and what actions it has taken. This can be controlled by the configuration set system syslog-enhanced event-handler name log output true/false.
Anything seen on stderr from the script is logged at the ERR level to the journal. This provides a method to log any errors that occurred during the execution of the script.This can be controlled by the configuration set system syslog-enhanced event-handler name log errors true/false.
Operational mode commands
name is assigned to the script, where name is the name referenced by the syslog event action. If a name is not specified, the current name of the script is used.add syslog event-script script-location [ to name ]delete syslog event-script nameConfiguration data model
The new commands are added in this release.
set system syslog-enhanced event-handler event-name call-script text
set system syslog-enhanced event-handler event-name rate-limit interval time-in-seconds
set system syslog-enhanced event-handler event-name user text
set system syslog-enhanced event-handler event-name log errors true/false
set system syslog-enhanced event-handler event-name log output true/false
set system syslog-enhanced rule 1..65535 otherwise event handler event-name
set system syslog-enhanced rule 1..65535 otherwise event argument textNeighbor Unreachability Detection model for ARP
Neighbor Unreachability Detection (NUD) in IPv6 Neighbor Discovery Protocol (RFC 4861) allows monitoring of bidirectional reachability during traffic transmission, which is absent in ARP in IPv4 (RFC 826). The objective of this feature is to implement a similar process for ARP.
- Hints from the upper layer protocols that indicate a connection is making "forward progress", when available. This information becomes available to the dataplane if Bidirectional Forwarding Detection (BFD) is activated.
- Receipt of a Neighbor Advertisement message that is a response to a Neighbor Solicitation message.
NUD ensures fast detection of failure and therefore recovery mechanisms can take place in a timely manner when reachability is lost. This process can improve packet delivery when dealing with unreliable routers, partially failing or partitioned links and mobile nodes.
In ARP, once an entry is created, no bidirectional reachability checks are performed until it ages out and there is a need to recreate the entry. As a result, in case of change of route to a destination node, an existing ARP entry is not updated to reflect that (unless a gratuitous ARP is received, which is not always guaranteed) until the entry ages out and gets recreated. This results in packet loss. Therefore, implementing a process similar to NUD for ARP would be beneficial.
Operational mode commands
An additional parameter NUD state is shown in brackets under the Flags field (except for the STATIC entries) in the output of the command show dataplane arp.
For
the command show
arp, the
NUD state is shown in brackets under the Dataplane field (except for STATIC
entries). Controller or kernel ARP already has the NUD state set, but it
was not displayed in the outputs. Now it is displayed in brackets under
the Controller
field.
| NUD State | Description |
|---|---|
| INCOMPLETE | Address resolution is in progress. An ARP request has been sent, but no reply has yet been received. |
| REACHABLE | Forward-direction communication has been verified within the past 30 seconds (or user configured stale-time). |
| STALE | An entry in the ARP cache has not been verified as reachable within the past 30 seconds (or user configured stale-time). A gratuitous ARP message adds an entry to the cache (if gratuitous ARP processing is allowed in settings) for the sender of the message, with state STALE. No action is required until traffic is sent to the STALE entry. |
| DELAY | No reachable verification has been received within the past 30 seconds, and a packet has been sent to the specified neighbor within the past 5 seconds. If no positive confirmation is received within 5 seconds of entering DELAY state, send an ARP request and change the state to PROBE. |
| PROBE | An ARP request has been sent to verify reachability. No ARP reply has yet been received. |
Flow-Accounting Support on Tunnel and VTI Interfaces
This feature extends the flow-monitoring interface support to include the "tunnel" and "vti" interface types.
This feature extends the flow-monitoring interface support to include the "tunnel" and "vti" interface types. Note that the packets reported on are those without the outer tunnel/VTI headers (that is the inner headers). Therefore, NetFlow is run before adding the tunnel/VTI headers for the egress direction (that is entering the tunnel), and after removing the tunnel/VTI headers for the ingress direction (that is leaving the tunnel). It reports DPI information if ndpi is enabled and running on the tunnel/VTI interface that NetFlow is configured on.
NetFlow is supported on all platforms that contain tunnel and VTI interface support.
Configuration Data Model
The new commands for configuring NetFlow on tunnel interfaces are as follows:
set interfaces tunnel tunN flow-monitoring selector selector-name
set interfaces tunnel tunN flow-monitoring aggregator aggregator-name
set interfaces tunnel tunN flow-monitoring exporter exporter-name
The new commands for configuring NetFlow on VTI interfaces are as follows:
set interfaces vti vtiN flow-monitoring selector selector-name
set interfaces vti vtiN flow-monitoring aggregator aggregator-name
set interfaces vti vtiN flow-monitoring exporter exporter-name
SNMP listening on multiple VRFs
In the previous versions, the SNMP daemon listened for incoming requests from one VRF, either the default routing instance or a non-default routing instance. This feature adds SNMP listening on multiple VRFs, that includes both default and non-default VRFs, and at specific addresses within a VRF.
Operational Mode Commands
The command show snmp routing-instance, which lists the one global VRF configured, now displays all the routing instances and the listen addresses and ports configured. For example:
vyatta@vyatta:~$ show snmp routing-instance
Routing Instance SNMP Agent is Listening on for Incoming Requests:
Routing-Instance Port Listen Address
----------------- ---- --------------
blue 161 30.1.0.11
red 161
green 161 20.1.0.11
Configuration Data Model
set service snmp routing-instance vrfThe restriction of configuring only one global VRF with command set service snmp routing-instance has been removed. Multiple VRFs can be configured with this feature, which include both default and non-default VRFs.
set service snmp listen-address address routing-instance vrfset service snmp listen-address has been modified to specify the routing-instance for the listen-address. - When no routing instance is specified, the address is assumed to belong to the default routing instance.
- If only one global routing instance is configured with
set snmp routing-instancecommand, then the listen-address with no routing-instance specified is assumed to belong to the global routing-instance and not the default routing instance for backward compatibility. - When multiple global VRFs are configured, then the listen-address with no routing-instance specified belongs to the default routing instance.
Per Interface Filtering for show interfaces extensive
This extends the command show interfaces extensive with an optional interface name.
- When provided, only the extensive information for the provided interface is output. The interface supplied must be a dataplane interface.
- When not provided, the extensive information for all dataplane interfaces is output.
BGP peer-group support for BFD fall-over
BFD fall-over is used for fast detection of dead or unreachable BGP peers.
Upon a loss of BFD connectivity, BGP is informed of a DOWN event, whereupon the BGP best-path may be recalculated, excluding routes learned from the unreachable peer. This allows for faster convergence to other working BGP paths upon topology changes and is much quicker than if we waited for the BGP hold timer to expire for the BGP TCP session.
Currently it is possible to enable BFD fall-over at the BGP neighbor configuration level. With a large number of neighbors, the per-neighbor configuration for BFD fall-over quickly becomes repetitive and leads to a potentially much larger configuration. BGP peer-groups are a way to configure features across a range of BGP neighbors to avoid such problems.
This feature allows BFD fall-over to be configured at the BGP peer-group and the (previously implemented) BGP per-neighbor level. If both configurations, peer-group and per-neighbor are present, then the BGP per-neighbor level will take precedence.
Configuration Data Model
set protocols bgp ASN peer-group PEER-GROUP fall-over bfdEnables fall-over BFD for all neighbors that are currently configured for this peer-group. May be configured via global BGP and per route instance BGP.
set protocols bgp ASN peer-group PEER-GROUP fall-over bfd-strict holdtimeEnables fall-over BFD strict mode for all neighbors that are currently configured for this peer-group. May be configured via global BGP and per route instance BGP.
File system monitoring
This feature provides support for monitoring of file systems, and generates syslog messages if the file system is remounted with the read-only option, or the file system space usage exceeds the configured limit.
The read-only remount of a file system may happen due to disk or SSD errors to avoid further corruption of the disk. This read-only remount may also be triggered manually with SysRq-U on systems.
When configured, the monitoring service checks the file system at a regular interval. If the file system monitor detects a read-only remount, it generates a specific syslog message.
This service also monitors the file system space usage and emit a syslog message if file system usage exceeds the configured limit.
To capture these syslog messages remotely, users must have a reliable remote syslog server configured. If the local partitions are read-only or the file system is full, the system cannot write any local logs.
Configuration Data Model
set system storage monitor file-system mountpoint read-only-remount
set system storage monitor file-system mountpoint read-only-remount syslog level level
set system storage monitor file-system mountpoint read-only-remount syslog facility facility
Use this command to watch for read-only remount of the file system.
In most cases, using the "/" (root) directory as the mount point would be sufficient, but other mounted file systems like /var/log may also be monitored in addition to the root file system.
This command is only useful for file systems that are backed by a permanent storage block device. Monitoring memory-based file system like tmpfs, or other kernel file systems like /proc, /sys etc. are allowed but not useful as these file sytems are excluded by the kernel during emergency remount.
For remote monitoring, users must configure syslog to send these logs to a remote syslog receiver. As the file systems are already read-only when these messages are generated, these messages are unlikely to be saved in the local syslog.
set system storage monitor file-system mountpoint space-usage threshold percent syslog
set system storage monitor file-system mountpoint space-usage threshold percent syslog level level
set system storage monitor file-system mountpoint space-usage threshold percent syslog facility facility
Use this command to watch file system usage and emit a log when the usage exceeds the given threshold.
You should configure syslog to send the logs to a remote syslog receiver.
Native 3G, 4G, LTE support
This feature adds support for wireless broadband modems. The supported modems must support either QMI or MBIM. Currently, all of these type of modems use a USB interface for connection.
Operational Mode Commands
show modemsConfiguration Data Model
set system modems connection-settings name apn access-point
set system modems connection-settings name interface ifname
set system modems connection-settings name number telephone
set system modems connection-settings name password pw
set system modems connection-settings name pin sim-pin
set system modems connection-settings name user username
Routing performance and scalability improvements
Routing performance and scalability has been improved, along with the following changes:
- OSPFv2 and OSPFv3 rate limiting the number of concurrent connections
- New diagnostic and observability commands for troubleshooting
Both OSPFv2 and OSPFv3 currently limit the number of concurrent neighbors allowed to transition from ExStart state. This feature allows configuring this limit maximum-concurrent-dd, where selecting a more conservative number can improve overall convergence. When a large number of OSPF neighbors are connecting at the same time, reducing the number of transitioning active connections avoids a possible churn of neighborships, due to the ever increasing number of LSAs being flooded to all the neighbors. Consequently, configuring a lower maximum-concurrent-dd can avoid issues where OSPF neighbors fall out of established state.
Additional OSPF operational commands have been added, providing improved diagnostics of OSPF operating at scale. The commands relate to displaying OSPF statistics, including OSPF routing protocol traffic.
Following a network change, it is helpful to observe convergence in the RIB and dataplane by verifying a next hop change. In particular, for eBGP at scale, it is helpful to count the number of a specific next hop installed in the RIB and the dataplane, in order to determine how long convergence takes. An additional set of commands is provided to count the number of next hops used by RIB and dataplane entries, avoiding the need to extract the full table and count them separately, as this takes significant time and resources, e.g., a BGP full table is roughly 800k routes.
Operational Mode Commands
show protocols rib memory [ detail ]
show protocols bgp memory [ detail ]
show protocols ospf memory [ detail ]
show protocols ospfv3 memory [ detail ]
These commands display the current protocol process memory statistics:
- allocated: (stats.allocated) the number of bytes currently allocated for the process
- active: (stats.active) the total number of bytes in the active pages allocated by the application
- mapped: (stats.mapped) the number of bytes in mapped chunks, corresponds to the process resident set size (RSS)
- metadata: (stats.metadata) the number of bytes held as meta data
show protocols ospf [ routing-instance Name process ID ] [ process ID ] statistics
show protocols ospfv3 [ routing-instance Name process ID ] [ process ID ] statistics
show protocols ospf [ routing-instance Name process ID ] [ process ID ] traffic [ IFLIST ]
show protocols ospfv3 [ routing-instance Name process ID ] [ process ID ] traffic [ IFLIST ]
clear protocols ospf statistics traffic [ interface ]
clear protocols ospf routing-instance Name statistics traffic
clear protocols ospfv3 statistics traffic [ interface ]
clear protocols ospfv3 routing-instance Name statistics traffic
show ip route route next-hop nexthop [ summary ]
show ipv6 route route next-hop nexthop [ summary ]
show ip route routing-instance name next-hop nexthop [ summary ]
show ipv6 route routing-instance name next-hop nexthop [ summary ]
show dataplane route route next-hop nexthop
show dataplane route6 route next-hop nexthop
show dataplane route routing-instance name route next-hop nexthop
show dataplane route6 routing-instance name route next-hop nexthop
Configuration Data Model
set protocols ospf maximum-concurrent-dd 1..65535
set routing routing-instance Name protocols ospf maximum-concurrent-dd 1..65535
set protocols ospfv3 maximum-concurrent-dd 1..65535
set routing routing-instance Name protocols ospfv3 maximum-concurrent-dd 1..65535
maximum-concurrent-dd, which can improve OSPF convergence for a large number of neighbors. - Increasing the
maximum-concurrent-ddallows for a higher number of simultaneous connections. This can lead to OSPF neighbor flaps, where neighbors drop from Established state due to the large number of LSAs being flooded to a large set of neighbors. - Decreasing the
maximum-concurrent-ddto a low value, leads to fewer simultaneous connections. This can lead to a longer convergence time before all neighbors reach established state.
The default value of maximum-concurrent-dd is 64, which allows a total combination of 128 incoming and outgoing connections transiting ExStart to Established state simultaneously.
Show total WRED dropped and tail dropped packets for all virtual interfaces
This feature adds new commands to aid troubleshooting QoS.
If a network engineer notices unexpected packets being dropped in the network, they need to determine which box is dropping the packets. If a box is dropping packets, a network operator will then want to determine which interface is dropping the most packets. This feature adds a simple command to show the total packets being dropped by QoS for the whole box and per interface.
Operational Mode Commands
show policy qos summary
monitor policy qos summary