dCloud Topology
Here is the guide topology I will build into EVE-NG with the Nexus 7000 Titanium image:
Management Network
This is what I ended up doing in EVE-NG:
192.168.0.0/24 is the management network, with mgmt0 configured on each switch:
- 192.168.0.10/24 = NXOS1
- 192.168.0.20/24 = NXOS2
- 192.168.0.30/24 = NXOS3
There is a Layer 2 switch running with an SVI of 192.168.0.1 and from here I can ssh to each switch on the mgmt0 IP address with the default admin / admin credentials.
Layer 3 Connectivity
Each routed link will be setup as per the topology:
3 x segments used between each switch:
- 10.10.1.0/24 (e2/2 -> e2/2)
- 10.10.2.0/24 (e2/1 -> e2/1)
- 10.10.3.0/24 (e2/3 -> e2/3)
Full IP connectivity between each switch is working as expected. Also to point out that the default VRF is being used for the /24 networks and interfaces above. One other thing that I was stuck on, but seemed quite obvious was that all the interfaces had the same MAC address statically assigned.. so my connectivity didn’t work.. so in the end I deleted the mac-address with a ‘no mac-address’ and surprise surprise it was then working fine. Not sure if this was a lab / Titanium thing of if you will see that in the real world of hardware Nexus. Until that day!
Scenario 1: System Configuration
Let’s use some of the below basic commands to explore the NX-OS:
| Intent | Command | |
| Verify hardware configuration | show module | |
| Check software version | show version | |
| Currently running configuration | show running-config | |
| Currently running and defaults | show running-config all | |
| Currently running section only | show running-config all | section mgmt0 |
Commands and Outputs
NX-OS is composed of two images:
- A kickstart image that contains the Linux Kernel
- A system image that contains most of the NX-OS software components
- They both show up in the configuration. Currently the modular NX-OS only includes the plug-ins Core and Ethernet.
- In future releases there will be additional plug-ins, like the “Storage” plug-in for FCoE.
show running-config (A bit long with all of the ethernet interfaces, but worth seeing and keeping an eye on anything different from traditional IOS or IOS-XE. Also note the default MAC address on each interface is the same!)
NXOS1# show running-config !Command: show running-config !Time: Sun Mar 7 11:50:54 2021 version 7.3(0)D1(1) power redundancy-mode redundant hostname NXOS1 vdc NXOS1 id 1 limit-resource module-type m1 m1xl m2xl f2e allocate interface Ethernet2/1-48 allocate interface Ethernet3/1-48 allocate interface Ethernet4/1-48 limit-resource vlan minimum 16 maximum 4094 limit-resource vrf minimum 2 maximum 4096 limit-resource port-channel minimum 0 maximum 768 limit-resource u4route-mem minimum 96 maximum 96 limit-resource u6route-mem minimum 24 maximum 24 limit-resource m4route-mem minimum 58 maximum 58 limit-resource m6route-mem minimum 8 maximum 8 username admin password 5 $5$Otc7T0NC$K.ulnSZnSyXLrTGNBdtLgZJXEa8EeNx.BrdZ98XyK2 C role network-admin no password strength-check ip domain-lookup vlan dot1Q tag native system default switchport system jumbomtu 0 no logging event trunk-status enable copp profile strict snmp-server user admin auth md5 0x328945d53e05e8e7207f8c20b142f0b7 priv 0x328945 d53e05e8e7207f8c20b142f0b7 localizedkey engineID 128:0:0:9:3:0:0:0:0:0:0 rmon event 1 log description FATAL(1) owner PMON@FATAL rmon event 2 log description CRITICAL(2) owner PMON@CRITICAL rmon event 3 log description ERROR(3) owner PMON@ERROR rmon event 4 log description WARNING(4) owner PMON@WARNING rmon event 5 log description INFORMATION(5) owner PMON@INFO snmp-server enable traps link vlan 1 vrf context management interface mgmt0 vrf member management interface Ethernet2/1 no switchport ip address 10.10.2.1/24 no shutdown interface Ethernet2/2 no switchport ip address 10.10.1.1/24 no shutdown interface Ethernet2/3 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/4 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/5 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/6 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/7 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/8 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/9 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/10 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/11 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/12 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/13 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/14 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/15 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/16 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/17 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/18 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/19 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/20 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/21 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/22 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/23 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/24 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/25 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/26 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/27 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/28 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/29 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/30 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/31 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/32 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/33 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/34 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/35 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/36 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/37 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/38 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/39 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/40 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/41 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/42 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/43 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/44 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/45 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/46 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/47 shutdown no switchport mac-address 0000.0000.002f interface Ethernet2/48 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/1 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/2 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/3 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/4 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/5 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/6 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/7 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/8 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/9 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/10 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/11 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/12 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/13 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/14 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/15 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/16 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/17 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/18 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/19 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/20 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/21 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/22 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/23 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/24 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/25 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/26 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/27 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/28 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/29 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/30 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/31 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/32 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/33 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/34 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/35 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/36 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/37 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/38 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/39 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/40 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/41 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/42 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/43 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/44 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/45 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/46 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/47 shutdown no switchport mac-address 0000.0000.002f interface Ethernet3/48 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/1 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/2 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/3 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/4 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/5 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/6 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/7 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/8 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/9 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/10 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/11 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/12 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/13 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/14 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/15 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/16 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/17 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/18 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/19 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/20 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/21 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/22 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/23 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/24 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/25 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/26 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/27 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/28 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/29 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/30 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/31 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/32 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/33 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/34 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/35 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/36 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/37 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/38 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/39 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/40 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/41 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/42 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/43 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/44 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/45 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/46 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/47 shutdown no switchport mac-address 0000.0000.002f interface Ethernet4/48 shutdown no switchport mac-address 0000.0000.002f line console line vty boot kickstart bootflash:/titanium-d1-kickstart.7.3.0.D1.1.bin boot system bootflash:/titanium-d1.7.3.0.D1.1.bin no system default switchport shutdown
show running-config all | section mgmt0 (Shows defaults as well as configured items)
NXOS1# show running-config all | section mgmt0 interface mgmt0 no description speed auto duplex auto snmp trap link-status no shutdown cdp enable spanning-tree port-priority 128 spanning-tree cost auto spanning-tree link-type auto no spanning-tree bpduguard no spanning-tree bpdufilter vrf member management
To compare with a show run | section mgmt0:
NXOS1# show running-config | section mgmt0 interface mgmt0 vrf member management
Quite a difference!
Management VRF
The management interface is by default part of the management VRF. This particular VRF is part of the default configuration and the management interface “mgmt0” is the only interface allowed to be part of this VRF.
The philosophy behind Management VRF is to provide total isolation to the management traffic from the rest of the traffic flowing through the box by confining the former to its own forwarding table.
NXOS1# show vrf interface mgmt0 Interface VRF-Name VRF-ID Site-of-Origin mgmt0 management 2 --
Interface mgmt0 is associated with the VRF name ‘management’. The management VRF interface is part of the default configuration and the management interface “mgmt0” is the only interface that can be made a member of this VRF.
Everything else in the switch is part of the default VRF called ‘default’ (VRF ID1):
NXOS1# show vrf interface Interface VRF-Name VRF-ID Site-of-Origin Null0 default 1 -- Ethernet2/1 default 1 -- Ethernet2/2 default 1 -- Ethernet2/3 default 1 -- Ethernet2/4 default 1 -- Ethernet2/5 default 1 -- Ethernet2/6 default 1 -- Ethernet2/7 default 1 -- Ethernet2/8 default 1 -- Ethernet2/9 default 1 -- Ethernet2/10 default 1 -- Ethernet2/11 default 1 -- Ethernet2/12 default 1 -- Ethernet2/13 default 1 -- Ethernet2/14 default 1 -- Ethernet2/15 default 1 -- Ethernet2/16 default 1 -- Ethernet2/17 default 1 -- Ethernet2/18 default 1 -- Ethernet2/19 default 1 -- Ethernet2/20 default 1 -- Ethernet2/21 default 1 -- **snipped all other ports
CLI Familiarization
As you may have already noticed NX-OS gives the user a very IOS look and feel sensation when configuring the system. However, there are differences, which should be considered improvements:
- One of the main differences consists in NX-OS implementing a hierarchy independent CLI.
- Every command can in fact be issued from anywhere in the configuration.
Example using the ping command in global configuration mode:
NXOS1# conf t Enter configuration commands, one per line. End with CNTL/Z. NXOS1(config)# ping Vrf context to use [default] :
CLI Piping Functionality
The output piping has also been improved. It can now be used in a similar way to Linux.
NXOS1# sh run | grep ? WORD Search for the expression count Print a total count of matching lines only ignore-case Ignore case difference when comparing strings invert-match Print only lines that contain no matches for <expr> line-exp Print only lines where the match is a whole line line-number Print each match preceded by its line number next Print <num> lines of context after every matching line prev Print <num> lines of context before every matching line word-exp Print only lines where the match is a complete word
For example the grep command is available for use.
Question Mark vs Tab
- Example 1 using the ?
- Example 2 using the Tab key
NXOS1(config)# interface ? breakout Configuring the breakout for an interface ethernet Ethernet IEEE 802.3z loopback Loopback interface mgmt Management interface port-channel Port Channel interface NXOS1(config)# interface breakout ethernet loopback mgmt port-channel
The [TAB] button does not only complete the command, but also it shows the keywords that are available.
RBAC
RBAC stands for Role Based Access Control. Every account is assigned to a role which defines the privileges of the user who will access the system with the corresponding account. NX-OS, through the RBAC feature, provides a very flexible and powerful framework to create roles for any type of user. In this context, a role can be seen as a group of rules that permit or deny a set of operations on NX-OS components.
Lets look at:
- Display the default role
- Display the role features and the feature-groups
- Create a new role and apply the role to a newly created user
- Test the role
show role
Worth paying attention to the 1st 4 roles: (Essentially RW / RO for the switch or a particular VDC instance – more to come on this later on this Nexus journey)
- network-admin
- network-operator
- vdc-admin
- vdc-operator
NXOS1# show role Role: network-admin Description: Predefined network admin role has access to all commands on the switch ------------------------------------------------------------------- Rule Perm Type Scope Entity ------------------------------------------------------------------- 1 permit read-write Role: network-operator Description: Predefined network operator role has access to all read commands on the switch ------------------------------------------------------------------- Rule Perm Type Scope Entity ------------------------------------------------------------------- 1 permit read Role: vdc-admin Description: Predefined vdc admin role has access to all commands within a VDC instance ------------------------------------------------------------------- Rule Perm Type Scope Entity ------------------------------------------------------------------- 1 permit read-write Role: vdc-operator Description: Predefined vdc operator role has access to all read commands within a VDC instance ------------------------------------------------------------------- Rule Perm Type Scope Entity ------------------------------------------------------------------- 1 permit read Role: priv-15 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) ------------------------------------------------------------------- Rule Perm Type Scope Entity ------------------------------------------------------------------- 1 permit read-write Role: priv-14 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) ------------------------------------------------------------------- Rule Perm Type Scope Entity ------------------------------------------------------------------- 1 permit read-write Role: priv-13 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-12 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-11 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-10 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-9 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-8 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-7 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-6 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-5 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-4 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-3 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-2 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-1 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) Role: priv-0 Description: This is a system defined privilege role. Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) ------------------------------------------------------------------- Rule Perm Type Scope Entity ------------------------------------------------------------------- 10 permit command traceroute6 * 9 permit command traceroute * 8 permit command telnet6 * 7 permit command telnet * 6 permit command ping6 * 5 permit command ping * 4 permit command ssh6 * 3 permit command ssh * 2 permit command enable *
All users when they login are associated to a particular role. It can be one of the default pre-configured roles or a user-made role. A role is a set of rules that define what operations the user can perform on individual CLI commands, features, and feature-groups basis. Feature-groups are essentially groups of related features, such as the L3 feature group (defined by default). You can group features in feature-groups and assign read/read-write permission to the whole group of features.
To see the set of features and the feature groups available to be defined as part of a role, issue the following commands:
NXOS1# show role feature aaa (AAA service related commands) access-list (IP access list related commands) arp (ARP protocol related commands) callhome (Callhome configuration and show commands) cdp (Cisco Discovery Protocol related commands) crypto (Security related commands) diagnostics (Gold diagnostics related commands) install (Software install related commands) l3vm (Layer 3 virtualization related commands) license (License related commands) ping (Network reachability test commands) platform (Platform configuration and show commands) radius (Radius configuration and show commands) scheduler (Scheduler configuration and show commands) snmp (SNMP related commands) syslog (Syslog related commands) tacacs (TACACS configuration and show commands) tcap (Terminal settings related commands) tcpudp (TCP/UDP related commands) bridge (BRIDGE-MIB access control) cts (CTS related commands) dot1x (DOT1X related commands) eou (EAP over UDP related commands) eth-port-sec (Ethernet port security related commands) glbp (Gateway Load Balancing Protocol related commands) hsrp (Hot Standby Router Protocol related commands) igmp (Internet Group Management Protocol related commands) interface (Interface configuration commands) ipfib (IP Forwarding Information Base related commands) msdp (Multicast Source Discovery Protocol related commands) pong (Pong related commands) ptp (PTP (IEEE 1588) related commands) qbridge (Q-BRIDGE-MIB access control) qosmgr (Quality of Service related commands) router-bgp (Border Gateway Protocol related commands) router-eigrp (Enhanced Interior Gateway Routing Protocol related commands) router-isis (ISIS protocol related commands) router-ospf (Open Shortest Path First protocol related commands) router-rip (Routing Information Protocol related commands) spanning-tree (Spanning Tree protocol related commands) svi (Interface VLAN related commands) vlan (Virtual LAN related commands) vtp (CISCO-VTP-MIB access control) vtpmib-auth (CISCO-VTP-MIB vtpAuthenticationTable access control) wccp (Web Cache Communication Protocol related commands) acl (FC ACL related commands) cloud (Cloud discovery related commands) fc-qos (FC Quality of Service related comamnds) fcanalyzer (FC analyzer related commands) fcns (Fibre Channel Name Server related commands) fcsp (Fibre Channel Security Protocol related commands) ficon (Ficon related commands) fspf (Fabric Shortest Path First protocol related commands) iscsi (ISCSI related commands) isns (Internet Storage Name Service related commands) ivr (InterVsan Routing protocol related commands) mpls-tunnel (FC tunnel related commands) rlir (Registered Link Incident Report related commands) rscn (Registered State Change Notification related commands) san-ext-tuner (IP Network Simulator related commands) sme (Storage Media Encryption feature related commands) sme-kmc-admin (SME commands authorized to kmc admin) sme-recovery-officer(SME commands authorized to recovery officer) sme-stg-admin (SME commands authorized to storage admin) span (SPAN session relate commands) vsan (VSAN configuration and show commands) vsan-assign-intf(Assign interfaces to vsan) wwnm (WorldWide Name related commands) zone (Zone related commands) NXOS1# show role feature-group feature group: L3 router-bgp (Border Gateway Protocol related commands) router-eigrp (Enhanced Interior Gateway Routing Protocol related commands) router-isis (ISIS protocol related commands) router-ospf (Open Shortest Path First protocol related commands) router-rip (Routing Information Protocol related commands)
Adding a role
NXOS1# show run | section role role name ROLE-NXOS1LAB rule 3 permit command ping * rule 2 permit read-write feature cdp rule 1 permit read
Verifying the role
NXOS1# show role name ROLE-NXOS1LAB Role: ROLE-NXOS1LAB Description: new role Vlan policy: permit (default) Interface policy: permit (default) Vrf policy: permit (default) ------------------------------------------------------------------- Rule Perm Type Scope Entity ------------------------------------------------------------------- 3 permit command ping * 2 permit read-write feature cdp 1 permit read
This role ‘ROLE-NXOS1LAB’ enables the ping command, CDP and read operations.
Create a user and attach the role
NXOS1(config)# username nxosroletest password C1sco12345 role ROLE-NXOS1LAB
Test the role
NXOS1 login: nxosroletest Password:
So I am logged in, but what can I do?
NXOS1# ? clear Reset functions configure Enter configuration mode debug Debugging functions ping Test network reachability show Show running system information end Go to exec mode exit Exit from command interpreter NXOS1# debug ? cdp Configure CDP debugging NXOS1# ping 10.10.1.1 PING 10.10.1.1 (10.10.1.1): 56 data bytes 64 bytes from 10.10.1.1: icmp_seq=0 ttl=255 time=3.009 ms 64 bytes from 10.10.1.1: icmp_seq=1 ttl=255 time=0.706 ms 64 bytes from 10.10.1.1: icmp_seq=2 ttl=255 time=0.691 ms 64 bytes from 10.10.1.1: icmp_seq=3 ttl=255 time=0.802 ms 64 bytes from 10.10.1.1: icmp_seq=4 ttl=255 time=0.668 ms --- 10.10.1.1 ping statistics --- 5 packets transmitted, 5 packets received, 0.00% packet loss round-trip min/avg/max = 0.668/1.175/3.009 ms NXOS1# conf t Enter configuration commands, one per line. End with CNTL/Z. NXOS1(config)# ? cdp Configure CDP parameters end Go to exec mode exit Exit from command interpreter
So I can debug cdp only, ping and in conf t only use cdp. Success with RBAC.
Also:
NXOS1# copy run start % Permission denied for the role
No cigar on that one.
Configuration Rollback
NX-OS fully supports Configuration Rollback. This functionality allows you to revert to a previous configuration state, effectively rolling back configuration changes.
- These are the steps for this:
- Create a checkpoint for the current configuration
- Modify the configuration for an interface
- Rollback the configuration and verify the interface configuration
Here we go:
NXOS1# checkpoint ? <CR> WORD Checkpoint name (Max Size 80) description Checkpoint description for the given checkpoint file Create configuration rollback checkpoint to file NXOS1# checkpoint CHECK1 .Done NXOS1# show checkpoint summary 1) CHECK1: Created by admin Created at Sun, 12:21:27 07 Mar 2021 Size is 27,865 bytes User Checkpoint Summary -------------------------------------------------------------------------------- Description: None
So now lets make a change to the running configuration:
NXOS1# sh run int eth2/10 !Command: show running-config interface Ethernet2/10 !Time: Sun Mar 7 12:22:59 2021 version 7.3(0)D1(1) interface Ethernet2/10 description ROLLBACKTEST shutdown no switchport mac-address 0000.0000.002f
I have simply added the description to the eth2/10 interface.
Lets now attempt to rollback:
NXOS1# rollback running-config checkpoint CHECK1 Note: Applying config parallelly may fail Rollback verification Collecting Running-Config #Generating Rollback Patch Executing Rollback Patch Generating Running-config for verification Generating Patch for verification Verification is Successful. Rollback completed successfully.
Has it worked?
NXOS1# sh run int eth2/10 !Command: show running-config interface Ethernet2/10 !Time: Sun Mar 7 12:24:17 2021 version 7.3(0)D1(1) interface Ethernet2/10 shutdown no switchport mac-address 0000.0000.002f
Yep! What a great feature. 🙂
Configuration Session
NX-OS offers a new way of configuring ACLs and QoS: the Configuration Session mode.
- This new mode allows to “dry-run” the configuration against the system resources availability.
- For “dry-run” we mean a process that allows the user to check whether the hardware resources are available without actually performing any modification on them.
- These are the steps for this scenario:
- Create a new configuration session
- Create a simple access-list and apply the access list to an interface
- “Verify” the configuration
- “Commit” the configuration
NXOS1# configure session TESTSESSION Config Session started, Session ID is 1 Enter configuration commands, one per line. End with CNTL/Z. NXOS1(config-s)#
‘TESTSESSION’ created. Note also you are dropped into the session ready to add configuration.
So lets create a basic ACL and apply it to an interface:
NXOS1(config-s)# ip access-list 1 NXOS1(config-s-acl)# permit tcp 1.1.1.1/24 any NXOS1(config-s-acl)# permit tcp 2.2.2.2/24 any NXOS1(config-s-acl)# permit tcp 3.3.3.3/24 any NXOS1(config-s-acl)# exit NXOS1(config-s)# interface eth4/1 NXOS1(config-s-if)# ip access-group 1 in
Now lets view our session:
NXOS1(config-s-if)# show configuration session config session TESTSESSION 0001 ip access-list 1 0002 permit tcp 1.1.1.1/24 any 0003 permit tcp 2.2.2.0/24 any 0004 permit tcp 2.2.2.2/24 any 0005 permit tcp 3.3.3.3/24 any 0006 interface Ethernet4/1 Number of active configuration sessions = 1
Let us now verify our configuration. During the verification process, the system checks the configuration against the hardware and software resources for their availability.
NXOS1(config-s-if)# verify Verification Successful NXOS1(config-s)# verify verbose `ip access-list 1` `permit tcp 1.1.1.1/24 any` `permit tcp 2.2.2.0/24 any` `permit tcp 2.2.2.2/24 any` `permit tcp 3.3.3.3/24 any` `exit` Verification Successful
Success! (Note there is also a verbose option with more detail)
The configuration can fit in the hardware table. Again, until this point the ACL TCAM has not been touched yet.
We are now ready to commit the configuration. If the commit process will succeed, the session will be considered completed and will be terminated. This is done with the session:
NXOS1(config-s)# commit Commit Successful
What a great process for checking the configuration before it is applied.
Features
NX-OS is a fully modular operating system; most software modules do not run unless the correspondent service is enabled. We like to refer to these features that need to be specifically enabled as “conditional services”.
NXOS1(config)# feature bfd lisp privilege bgp lldp ptp bulkstat msdp rip cable-management msrp scheduler cts mvrp scp-server dhcp netflow sftp-server dot1x ngoam sla eigrp ntp ssh ethernet-link-oam nv tacacs+ evb nxapi telnet evc ospf tunnel evmed ospfv3 udld fabric otv vmtracker glbp password vn-segment-vlan-based hsrp pbr vni imp pim vpc interface-vlan pim6 vrrp isis pong vrrpv3 lacp port-security vtp ldap private-vlan wccp
Let’s enable the OSPF service for example:
NXOS1(config)# license grace-period NXOS1(config)# feature ospf LAN_ENTERPRISE_SERVICES_PKG license not installed. ospf feature will be shutdown after grace period of approximately 120 day(s)
Notice I had to enable grace-period licensing for this work. Otherwise I was stuck here:
NXOS1(config)# feature ospf Feature grace period is disabled
The CLI also tells me this in syslog:
2021 Mar 7 12:40:55 NXOS1 %LICMGR-2-LOG_LIC_NO_LIC: No license(s) present for feature LAN_ENTERPRISE_SERVICES_PKG. Application(s) shut down in 119 days. 2021 Mar 7 12:40:55 NXOS1 %LICMGR-2-LOG_LICAPP_NO_LIC: Application ospf running without LAN_ENTERPRISE_SERVICES_PKG license, shutdown in 119 days
So we have 120 days to play with this feature. 🙂
Process Restartability
NX-OS is a modern operating system. NX-OS continuously checks the health of each software module making sure that if a process crashes or hangs the right action is taken to allow service continuity and availability. NX-OS has been designed around the concept of zero service destruction.
In this scenario we will demonstrate the non-stop forwarding capabilities of OSPF:
- In a first step, a crash of the OSPF process will be simulated. This will cause a stateful restart, which uses our PSS (Persistent Storage Service) architecture, so that the system recovers in a seamless way. You will see how the connected Core Layer router will not notice that the process has crashed and been restarted.
- In a second step, we will perform a graceful restart of OSPF. This will utilize the Non-Stop Forwarding (NSF) feature of OSPF, as defined in RFC 3623, to recover the routing table on the local node by resynchronizing it with the neighbor.
NOTE
The process monitoring feature of NX-OS will also constantly monitor the number and frequency of process restarts and will escalate the situation accordingly:
- Should the OSPF process crash a second time within four minutes a graceful restart will be performed instead of a stateful restart.
- Should you trigger a second graceful restart within four minutes a supervisor switchover will be triggered. As our Titanium boxes only have one simulated Supervisor, this will render the machine that you are using unusable.
*** I am not doing this my lab as the crash is done in dCloud using a script which I don’t have in my lab***
However… when the crash occurs it is worth noting when we gracefully restart the crashed process, this graceful restart will be non-disruptive to the forwarding plane. Therefore:
- The state changes from FULL to EXSTART and not to DOWN as NFS is used to acquire the OSPF routing table.
- No pings are lost between the 2 OSPF neighbors
Licensing
NX-OS enforces licensing for some of its features. However, the licensing scheme has been made very easy to understand and simple to use. There are three levels of enforced software licensing:
- The Base license which contains a complete set of Layer2 and management features
- The Enterprise Services license which contains the Layer3 routing protocols
- The Advanced Services license for Virtual Device Context (VDC) and Cisco Trusted Security (CTS)
The Base license is free and comes with the Nexus hardware. The Enterprise Services and Advance Services licenses can be purchased and used independently.
There is a grace period of 120 days, so the users can test out the features before buying. The grace period is calculated on active features instead of absolute time. So, if a user tries out a licensed feature for a few days and then disabled it, the countdown of the grace period will stop until a licensed feature within the same license gets turned on again.
These are the steps for this scenario:
- Enable the grace period feature (Already done earlier in the lab)
- Show current license usage
NXOS1# show license usage
Feature Ins Lic Status Expiry Date Comments
Count
--------------------------------------------------------------------------------
MPLS_PKG No - Unused -
STORAGE-ENT No - Unused -
VDC_LICENSES No 0 Unused -
ENTERPRISE_PKG No - Unused -
FCOE-N7K-F132XP No 0 Unused -
FCOE-N7K-F248XP No 0 Unused -
FCOE-N7K-F312FQ No 0 Unused -
FCOE-N7K-F348XP No 0 Unused -
ENHANCED_LAYER2_PKG No - Unused -
SCALABLE_SERVICES_PKG No - Unused -
TRANSPORT_SERVICES_PKG No - Unused -
LAN_ADVANCED_SERVICES_PKG No - Unused -
LAN_ENTERPRISE_SERVICES_PKG No - In use Grace 119D 23H
-------------------------------------------------------------------------
Note that the ‘Enterprise Services’ license is in use (as we enabled the OSPF feature) and are running in the 120 day grace period.
So how do we disable a feature? How do we stop the clock on the grace license? Here is how:
NXOS1(config)# no feature ospf
NXOS1(config)# end
NXOS1# show license usage
Feature Ins Lic Status Expiry Date Comments
Count
-------------------------------------------------------------------------
MPLS_PKG No - Unused -
STORAGE-ENT No - Unused -
VDC_LICENSES No 0 Unused -
ENTERPRISE_PKG No - Unused -
FCOE-N7K-F132XP No 0 Unused -
FCOE-N7K-F248XP No 0 Unused -
FCOE-N7K-F312FQ No 0 Unused -
FCOE-N7K-F348XP No 0 Unused -
ENHANCED_LAYER2_PKG No - Unused -
SCALABLE_SERVICES_PKG No - Unused -
TRANSPORT_SERVICES_PKG No - Unused -
LAN_ADVANCED_SERVICES_PKG No - Unused -
LAN_ENTERPRISE_SERVICES_PKG No - Unused Grace 119D 23H
-------------------------------------------------------------------------
Note now that the Enterprise Services license is ‘Unused’, whereas before it was ‘In Use’ therefore the clock has stopped. Way cool!
Summary
In these labs we have:
- Have got familiar with the NX-OS Operating System which will power the Nexus7000 switch
- Learned some of the aspects of NX-OS and some of its difference from classical IOS
- General
- OS Images: NX-OS consists of two images: Kickstart + System
- Management VRF: Separate Management VRF for total isolation of management traffic
- Modular OS: Non-core Features – called Conditional Services need to be enabled
- Process Restartability: Monitoring of system service health and stateful/graceful restarts
- Licensing: Enforced Licensing with grace-period for testing features
- CLI
- Hierarchy Independence: Non-config commands can be issued from everywhere. E.g. ping, show running-config
- Default Config: Display defaults of the running-config
- Interface Types: Only one interface type Ethernet. No distinction between 10MB, 100MB, 1GB, 10GB interface type
- Slash Notation: For the IP address configuration the slash notation (e.g. x.x.x.x/24) can be used
- Rollback Mode: Rollback of the entire configuration to pre-defined checkpoints
- ACLs & QoS
- ACL Types: No more ACL types such as Standard or Extended
- Configuration Sessions: “Dry-run” mode for checking hard- and software capabilities
- Access-Control
- RBAC: Role-based-access-control
- L3 Forwarding/Protocols
- IGP routing protocols: Interface centric configuration (e.g. for OSPF)
- Interface
- HSRP – Sub-Interface: Configuration for HSRP is performed in a sub- interface mode
(**I didn’t lab the OSPF and HSRP configurations as I wanted to focus more on pure Nexus basics, however still worth noting this in the summary)
CCIEME 
You must be logged in to post a comment.