CCIE R&S Written Overview: MPLS

MPLS Layer 3 VPNs

MPLS Overview

• Multiprotocol Label Switching

• Open standard per RFC 3031

“Multiprotocol Label Switching Architecture”

• Previously Cisco proprietary Tag Switching

MPLS Overview – Multiprotocol

• Can transport different payloads

• Layer 2

– Ethernet, HDLC, PPP, Frame Relay, & ATM

• Layer 3

– IPv4 & IPv6

MPLS Overview – Label Switching

• Traffic is switched between interfaces based on locally significant label values

• Similar to how a Frame Relay or ATM switch uses input/output DLCIs and VPI/VCIs

MPLS Label Format

• 4 byte header used to switch packets

• RFC 3032 – “MPLS Label Stack Encoding”

– 20 bit Label = Locally significant to router

– 3 bit EXP = Class of Service

– S bit = Bottom of Stack

• If 1, label is last in the stack

– 8 bit TTL = Time to Live

How Labels Work

• MPLS Labels are bound to FECs

Forwarding Equivalency Class

– Mainly IPv4 prefix for our purposes

– Could also be IPv6 prefix or layer 2 circuit

• Router uses MPLS LFIB to switch traffic

– Essentially CEF table + Label

• Switching logic

– If traffic comes in if1 with label X send it out if2 with label Y

MPLS Device Roles

• PE / LER Provider Edge Router / Label Edge Routers

• Connects to Customer Edge (CE) devices

• Receives unlabeled packets and adds label

– AKA “label push” or “label imposition”

• In L3VPN performs both IP routing & MPLS lookups

• P / LSR devices Provider Router / Label Switch Routers

• Connects to PEs and/or other P routers

• Switches traffic based only on MPLS label

Label Push / Pop / Swap

• PE and P routers perform three major operations

Label push

– Add a label to an incoming packet

– AKA label imposition

• Label swap

– Replace a label on an incoming packet

• Label pop

– Remove a label from an outgoing packet

– AKA label disposition

Label Distribution

• Adjacent P/PEs must agree on label per FEC

• Label binding can be dynamic through…

Tag Distribution Protocol (TDP)

• Cisco proprietary and legacy

Label Distribution Protocol (LDP)

Resource Reservation Protocol (RSVP)

• Used for MPLS Traffic Engineering (MPLS TE)

Multiprotocol BGP (MP-BGP)

Label Distribution Protocol (LDP)

• Standard per RFC 3036 “LDP Specification”

• Neighbor discovery

– UDP port 646 to

• Neighbor adjacency

– TCP port 646 to remote LDP Router-ID

• Label advertisement

– Advertise FEC for connected IGP interfaces

– Advertise FEC for IGP learned routes

Penultimate Hop Popping (PHP)

• Penultimate means next to last

• Normally last hop must…

– Lookup MPLS Label

– Pop MPLS Label

– Lookup IPv4 destination

• PHP avoids extra lookup on last hop

• Accomplished through Implicit NULL label advertisement for connected prefixes

MPLS Layer 3 VPNs

• RFC 4364 – BGP/MPLS IP Virtual Private Networks (VPNs)


• Combines logic of MPLS Tunnels with separation of layer 3 routing information

– PEs learns customer routes from CEs

– PEs advertises CEs routes to other PEs via BGP

– BGP next-hops point to MPLS tunnels

• E.g. Loopbacks of PE routers

How MPLS L3VPNs Work

• MPLS L3VPNs have two basic components

• Separation of customer routing information

– Virtual Routing and Forwarding (VRF) Instance

– Customers have different “virtual” routing tables

• Exchange of customer routing information

– MP-BGP over the MPLS network

– Traffic is label switched towards BGP next-hops

Virtual Routing and Forwarding

• Each VRF has its own routing table

show ip route vrf [name | * ]

• Interfaces not in a VRF are in the global table

show ip route

• VRF and global routes are separate

– Implies addressing can overlap in different VRFs

– Implies VRFs can’t talk to each other because they have no routes to each other

• VRFs without MPLS is considered “VRF Lite”

VRF Aware Routing

• Routing inside a VRF can be through…

– VRF aware static routes

– VRF aware IGPs



– Policy Routing

VRF Lite vs. MPLS VPNs

• In VRF Lite all devices in transit must carry all routes

– Same as normal IP routing logic

• In MPLS VPNs only PE routers need customer routes

• Accomplished through…

VPNv4 Route

• RD + Prefix makes VPN routes globally unique


• PE routers exchange label for each customer route via VPNv4

Transport Label

• Label towards PE’s BGP next-hop

Multiprotocol BGP

• RFC 4364 “BGP/MPLS IP Virtual Private Networks (VPNs)”

– MP-BGP defines AFI 1 & SAFI 128 as VPN-IPv4 or “VPNv4”

• 8 byte Route Distinguisher (RD)

– Unique per VPN or per VPN site

– ASN:nn or IP-address:nn

• 4 byte IPv4 address

– Unique per VPN

– Implies globally unique routes

Controlling VPNv4 Routes

• Route distinguisher used solely to make route unique

• New BGP extended community “route-target” used to control what enters/exits VRF table

• “export” route-target

– What routes will be go from VRF into BGP

• “import” route-target

– What routes will go from BGP into VRF

• Allows granular control over what sites have what routes

– “import map” and “export map” allow control on a per prefix basis

VPNv4 Route Target

• 8 byte field per RFC 4360 “BGP Extended Communities Attribute“

• Format similar to route distinguisher

– ASN:nn or IP-address:nn

• VPNv4 speakers only accept VPNv4 routes with a route-target matching a local VRF

– Route reflection exception

– no bgp default route-target filter

• VPNv4 routes can have more than one route target

• Allows for complex VPN topologies

– Full mesh

• Import and export same everywhere

– Hub and Spoke

• Spokes import only hub’s routes

– Central services

• Multiple VPNs can import routes from a central site or from a central server

– Management VPNs

• Management Loopback on CE routers can be exported into special management VPN


Some screencaps from the MPLS Layer 3 VPN Overview videos:

IMG_0441 IMG_0443 IMG_0445 IMG_0446 IMG_0448 IMG_0449 IMG_0451 IMG_0463


BGP: Route Aggregation Options and BGP Synchronization


We know that we can aggregate/summarise multiple network statements that are sequential/contigious into a single advertisement. This is how it is done using BGP..

Within the BGP AS/process, we will inject the aggregated route using the ‘aggregate-address’ command:

router bgp 100

The router sees the aggregate and places it into its BGP table. Note that by default, the more-specific routes are not removed from the BGP table. With our IGP protocols, the specific networks disappeared from the routing table, NOT with BGP.

To suppress the advertisement of the more-specific routes, use the summary-only option with the aggregate-address command. (on the end of the command)

To further extend this command… If you also use the as-set option, the path advertised for this route will be an AS_PATH that was traveled by all of the more-specific paths being aggregated. Cisco recommends that you do not use this option when a great number of paths are being aggregated, since the aggregate may be removed, updated, and replaced as AS-path reachability changes.

Why aggregate routes in the first place? For the same reason we did so with other protocols – route aggregation lessens the load on router resources by making the routing tables smaller while still being complete and accurate.

Resetting and Clearing The BGP Peer Connections

Sometimes you’ll find it necessary to reset the TCP session between BGP speakers. Not all changes require this. For example, the route aggregation we just performed required no such reset. There is a “hard reset” and a “soft reset”. The clear ip bgp* command performs a hard reset where the TCP session itself is reset:

R1#clear ip bgp *
09:18:36: %BGP-5-ADJCHANGE: neighbor Down User reset
09:18:36: %BGP-5-ADJCHANGE: neighbor Down User reset
09:18:36: %BGP-5-ADJCHANGE: neighbor Down User reset

With this command, the BGP sessions themselves are reset and the neighbor adjacencies are lost. The adjacencies you see here came back within 20 – 40 seconds, but BGP reachability was lost during that time.

To clear the sessions without resetting the sessions, use the soft option, as shown here;

R1#clear ip bgp * soft

Internal BGP: Synchronization, Full Meshes, and Route Reflectors

Rules and Guidelines when working with iBGP:

  • iBGP neighbors do not have to be directly connected. The connection between iBGP routers is on TCP port 179.
  • It’s common practice to use a remote router’s loopback address in the neighbor statement, rather than the closest physical address. This allows us to keep our BGP adjacencies in situations where losing the physical address would result in losing that adjacency.
  • iBGP routers do not send updates to every single neighbor. The only way an iBGP router will advertise a route to its neighbors is if the route was created by the transmitting router via the network command, by static route redistribution, IGP route redistribution, or if the advertised route is a connected route in the first place. (KEY POINT!!!)

*This means that when a iBGP speaker learns about a route from an iBGP peer, the only kind of BGP router that route can then be advertised to is an eBGP router. iBGP routers do not advertise routes received from one iBGP neighbor to other iBGP neighbors.

The only way round this is to have a full mesh in each AS, which is not realistic on the internet! BGP has a solution to this dilemma.

The BGP Rule Of Synchronisation

The BGP rule of synchronization only matters when an AS is going to serve as a transit area, and if there are non-BGP speakers in the transit area.

The BGP Rule Of Synchronization states that a transit AS will not advertise a route until every router in the transit AS has the route in its IGP routing table.

BGP Synchronization’s major benefit is that packets that can’t possibly reach the desired remote network will not even be sent, reducing both the amount of unnecessary traffic and the unnecessary strain on router resources.

BGP Synchronization is turned off in many deployments, though, and as of IOS version 12.2(8) it’s turned off by default. There are three scenarios under which it’s safe to turn synchronization off:

1. If all the routers in the AS are running BGP.

2. If a full mesh exists in the AS.

3. If the AS is not a transit AS to begin with.

To do so, simply run the BGP command no synchronization.

R1(config)#router bgp 100
R1(config-router)#no synchronization

The Problem With BGP Full Mesh Deployments

BGP Split Horizon is much different that what we are used to with routing in the CCNA.

BGP Split Horizon states that one iBGP peer can’t learn about a path from one iBGP peer and then advertise it to another iBGP peer.

Therefore, we would need a logical full mesh among all iBGP speakers in an autonomous system.

Reasons against a Full Mesh with BGP:

  • Any full-mesh deployment of BGP is going to have a large cost on the router’s resources (memory, CPU).
  • A full mesh is going to require a large number of TCP connections, and the more routers you have, the more connections you’ll need.
  • A full mesh setup will use a lot of bandwidth.
  • Time consuming to setup and admin overhead is a nightmare.

BGP: Changing the Weight & Intro to Route Aggregation


If you’re in an all- Cisco environment, it makes sense to change the weight of a route to make it the preferred route, since that is the first criteria checked.

The default weight is 0, so we’ll use the neighbor command to set the weight for all routes learned for a specific network with a custom weight value.

Since the weight attribute is Cisco proprietary, so in a multi-vendor environment we’d change the local pref to get a similar result.

*This change to a route’s weight is locally significant only

Route Aggregation

The Atomic Aggregate Attribute

When BGP paths are aggregated, this well-known attribute indicates the router that performed the aggregation. This attribute gives notice to downstream routers that more-specific BGP routing information was lost at the point of aggregation. Aggregation = Summarization

The Aggregator Attribute

This optional attribute gives the BGP Router ID and AS number of the router that performed the aggregation. The aggregator attribute will also include a list of all the AS numbers that these aggregated routes passed through.

BGP: Changing the Local Pref attribute

If we have a scenario whereby 2 routers with iBGP peers, have 2 ways to break out of the AS, we can influence this exit traffic by changing the local pref attribute to something higher than the default of 100.

When the local preference of a path is changed, all routers in the AS will learn about it.

Always run show ip bgp followed by the network number when you want to examine local preferences:

One way of doing this is to change the default local preference for a router as a whole, which means that every update the router sends out to other devices in the same AS will carry this new local preference value.

Required configuration:

R1(config)#router bgp 12
R1(config-router)#bgp default ?
ipv4-unicast Activate ipv4-unicast for a peer by default
local-preference local preference (higher=more preferred)
route-target Control behavior based on Route-Target attributes
R1(config-router)#bgp default local-preference 200

BGP: Advertising Routes + Intro to Attributes

Advertising BGP Routes

Example of adding a Loopback interface into BGP:

R3(config)#router bgp 200
R3(config-router)#network mask

The BGP network mask must match the IP routing table’s mask exactly in order for the route to be successfully advertised via BGP.

BGP Path Attributes

There are two classes of BGP Path Attributes, well-known and optional.

Here are the two categories of well-known attributes, both mandatory and discretionary:

  • Well-known mandatory: AS_PATH, origin, next-hop
  • Well-known discretionary: local preference, atomic aggregate

There are also optional attributes, both transitive and non-transitive.

  • Optional transitive: aggregator, community
  • Optional non-transitive: MED (multi-exit discriminator)

A BGP path carrying an unrecognized transitive optional attribute will be accepted; if this path is advertised to other routers, the Partial bit will be set and the attribute advertised to the neighboring router. Basically, marking an attribute as partial is the equivalent of the advertising router saying “I didn’t understand this attribute, but here is it anyway.”

An unrecognized non-transitive optional attribute will not be passed on to other BGP speakers.

The Origin Attribute

The source of the routing update itself can be viewed with show ip bgp.

R1#show ip bgp
BGP table version is 2, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 0 0 200 i

There are three possibilities for the Origin code:

  • “i” — path originated from an IGP via the network command
  • “e” — path originated from an Exterior Gateway Protocol (EGP)
  • “?” — Actual origin unclear; learned via route redistribution.

*Those are shown in order of most preferred to least preferred, from top to bottom.

The AS_PATH Attribute

This attribute shows the autonomous systems along the path to the destination network, including the AS the destination network resides in. The shortest AS path is the preferred path.

The AS_PATH attribute helps to prevent routing loops; if a router receives an update and sees its own AS number in the path to a destination, that route will be discarded.

In this example, the only AS shown in the path is the AS the network resides in, AS 200.

R1#show ip bgp
BGP table version is 2, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i -
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 0 0 200 i

BGP: Using Loopbacks For Adjacencies

We know that it is Cisco best practice for eBGP neighbours to be directly connected, however we can setup neighbour adjacencies when not directly connected via eBGP using loopback interfaces.

Loopback key point: Loopback interfaces are not considered directly connected even if they share a common subnet.

Setting up eBGP relationships when neighbours are not on the same network segment:

  1. The ebgp-multihop command is required to specify how many hops away the neighbour is.
  2. The update-source command is used to specify the loopback as the source interface.
  3. eBGP neighbours will require a static route to reach the loopback interface, as by default this loopback address will not be in the routing table.



Static Routes:

R1 - ip route (next hop IP Address or exit interface)
R2 - ip route (next hop IP Address or exit interface)

Example configuration to setup an eBGP neighbour:

R1(config)#int loopback1
R1(config-if)#ip address
R1(config-if)#router bgp 100
R1(config-router)#no auto
R1(config-router)#no synch
R1(config-router)#neighbor remote-as 200
R1(config-router)#neighbor ebgp-multihop 2
R1(config-router)#neighbor update-source loopback1

3 Common Reasons why a eBGP adjacency will fail..
  1. The AS number is incorrectly identified in the config. If you do this, trust me, you’re not the first and you won’t be the last! 🙂
  2. A peering has been configured for an eBGP router that is not directly connected, and the ebgp-multihop option has been omitted.
  3. An ACL is blocking TCP port 179.

BGP: Fundamentals & Neighbour Relationships

BGP is an external routing protocol used primarily by Internet Service Providers (ISPs).

BGP Definition

An Internet protocol that enables groups of routers (called autonomous systems) to share routing information so that efficient, loop-free routes can be established. BGP is commonly used within and between Internet Service Providers (ISPs).

An AS will use Exterior Gateway Protocols (EGP) to exchange updates with other ASes.

Interior Gateway Protocols such as OSPF and EIGRP have their place as well, and that place is inside an AS. Routes learned via BGP can be redistributed into an IGP, and vice versa.

BGP Characteristics

  • BGP supports VLSM and summarization.
  • BGP will send full updates when two routers initially become neighbors and will send only partial updates after that.
  • BGP does create and maintain neighbor relationships before exchanging routes, and keepalives are sent to keep this relationship alive.
  • BGP is a path-vector protocol
  • BGP path information comes in the form of attributes, and these path attributes are contained in the updates sent by BGP routers.
  • Attributes themselves are broken up into two classes, well-known and optional.
  • BGP also keeps a routing table separate from the IP routing table.

When Should BGP Not Be Used?

  • Some general guidelines on when not to use BGP:
  • When there is a single connection to the Internet or to another autonomous system.
  • No redundant link to the internet is present.
  • Situations where you don’t really care what path is used to reach a route in another AS.
  • When router resources are a concern (memory and CPU).
  • When there is a low-bandwidth connection between multiple autonomous systems. In this situation, static and default routing may be a better choice if any of these circumstances exist.

The BGP Peering Process

Like TCP, BGP is connection-oriented (“reliable”). An underlying connection between two BGP speakers is established before any routing information is exchanged. This connection takes place on TCP port 179.

As with EIGRP and OSPF, keepalive messages are sent out by the BGP speakers in order to keep this relationship alive.

Hint: TCP port 179 is a good port to leave unblocked by ACLs.

Once the connection is established, the BGP speakers exchange routes and synchronize their tables. After this initial exchange, a BGP speaker will only send further updates upon a change in the network topology.

The IGP protocols that use Autonomous Systems, IGRP and EIGRP, require prospective neighbors to be in the same AS. This is not true with BGP. Routers can be in different Autonomous Systems and still exchange routes.

A BGP peer that is in the same AS as the local router is an Internal BGP (iBGP) peer, where a BGP peer in another AS is an External BGP (eBGP) peer. That little “i” or “e” makes a big difference when it comes to advertising routes and other BGP behaviours.

A sample iBGP configuration (same AS):

Router bgp 100
Neighbor remote-as 100

A sample eBGP configuration (different AS):

Router bgp 100
Neighbor remote-as 200

Cisco recommends that eBGP peers be directly connected. iBGP peers are not required to be directly connected and generally aren’t.

The neighbor statement is used to statically define neighbors. BGP is not capable of discovering neighbors dynamically.

To verify that the remote BGP speaker has become a peer, run show ip bgp neighbor.

R1#show ip bgp neighbor
BGP neighbor is, remote AS 200, external link
BGP version 4, remote router ID
BGP state = Active
Last read 00:01:39, hold time is 180, keepalive interval is 60 seconds
Received 0 messages, 0 notifications, 0 in queue
Sent 0 messages, 0 notifications, 0 in queue
Route refresh request: received 0, sent 0
Default minimum time between advertisement runs is 30 seconds

If you note a connection has gone to idle and stayed there, check three things:

  1. The IP address in the neighbor statement (this is usually the issue).
  2. Make sure you have a neighbor statement for that remote router.
  3. Make sure your local router knows how to get to that same address.

Another handy BGP command:

R1#show ip bgp summary
BGP router identifier, local AS number 100
BGP table version is 2, main routing table version 2
1 network entries and 1 paths using 133 bytes of memory
1 BGP path attribute entries using 60 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP activity 1/1 prefixes, 1/0 paths, scan interval 60 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 4 100 83 84 2 0 0 01:19:03 0 4 200 104 103 2 0 0 01:39:24 1

BGP States

Idle is the initial state of a BGP connection. The BGP speaker is waiting for a start event, generally either the establishment of a TCP connection or the re-establishment of a previous connection. Once the connection is established, BGP moves to the next state.

There’s nothing wrong with this state, but we don’t want to stay there. If you note a connection has gone to idle and stayed there, check three things:

  1. The IP address in the neighbor statement (this is usually the issue).
  2. While we’re at it, make sure you have a neighbor statement for that remote router.
  3. Make sure your local router knows how to get to that same address.

Connect is the next state. In this state, a TCP connection request has been sent but a response has not yet been received. If the TCP connection completes, BGP will move to the OpenSent stage; if the connection does not complete, BGP goes to Active.

Active indicates that the BGP speaker is continuing to create a peer relationship with the remote router – basically, this is the halfway point of the connection. The local router has successfully sent a BGP Open packet to the address in the neighbor statement, but it hasn’t heard anything back yet.

As with Idle, there’s nothing wrong with this state – unless your connection stays there. If the connection goes Active and stays there, it’s really a mirror image of the Idle issue we spoke of earlier, so… Check the remote router’s neighbor statement

Be sure the remote router knows how to get the OpenConfirm packet back to the local router (OpenConfirm is BGP’s ACK)

And my personal favorite – make sure your AS numbers are correct, especially if the connection is flapping between Idle and Active.

OpenSent indicates that the BGP speaker has received an Open message from the peer. BGP will determine whether the peer is in the same AS (iBGP) or a different AS (eBGP) in this state.

In OpenConfirm state, the BGP speaker is waiting for a keepalive message. If one is received, the state moves to Established, and the neighbour relationship is complete. It is in the Established state that update packets are actually exchanged.