In this first article on MPLS Layer 3 VPN, we will be having a step by step lab on how to setup MPLS VPN network in both the service provider and customer perspectives. Below are the following tasks and requirements that we need to accomplish in Part 1:
- Configure MPLS LDP in the Service Provider network.
- Configure VRF in the Provider Edge (PE) routers.
- Configure BGP VPNv4 peering between R1 and R4.
- Configure Peering between PE routers R1 and R4 to customer routers CUST_A-R1 and CUST-A-R2. Announce Loopback 10 and 100 in the CE routers. Verify connectivity.
OSPF and BGP were preconfigured. The IOS used was a 3745 with Advanced Enterprise feature.
MPLS VPN might sound complicated but in fact the basics are easy. Before you proceed further with this lab, reading an introduction to MPLS will be extremely helpful. My colleague had written a very good article entitled “The Complete Guide to MPLS” which can be found here.
Figure 1: Lab Topology
Before we move on to the first task, let us review some concepts and terminologies related to MPLS. MPLS operates in the middle of the data link layer (Layer 2) and the network layer (Layer 3) hence it is considered to be a Layer 2.5 protocol. It operates using the protocol called LDP (Label Distribution Protocol) which assigns labels ranging from 16 to 1,048,575 (0-15 reserved and cannot be used in Cisco routers) to IP prefixes/subnets in the routing table. LDP relies on the routing table in order for it to form its LIB (Label Information Base) and LFIB (Label Forwarding Information Base). LSR (Label Switch Routers) are routers in the middle of the Service provider network that uses label to perform routing. LER (Label Edge Routers) are routers that are entry and exit points of the network. They are generally the Provider Edge (PE) routers.
The three general operations of LDP when dealing with labeling packets are PUSH, SWAP and POP. PUSH means that the incoming packet has no label and has to assign a new label to it. SWAP is basically changing the label to a different label. POP on the other hand, is to remove the label. LERs are usually the ones who PUSH labels to packets, LSRs do SWAP of the labels and the penultimate hop routers (Adjacent LSRs to the LERs, in our case Routers R2 and R3) do the POP operation. The feature where LSRs POP labels before it sends out to LERs is called PHP (Penultimate Hop Popping) with Implicit-null.
The LDP router-id needs to be reachable through the global routing table in order for LDP to form neighborship. The router-id election for LDP is the highest IP address of any loopback interface. If no loopback interfaces exist, it will be the highest IP address of any physical interface that is operational.
Let’s proceed to configure Task 1.
Task 1: Configure MPLS LDP in the Service Provider network
R1(config)#int fa0/0 R1(config-if)#mpls ip R1(config-if)#exit R1(config)#mpls label ? protocol Set platform default label distribution protocol range Label range R1(config)#mpls label range ? Minimum label value R1(config)#mpls label range 1000 1999 % Label range changes will take effect at the next reload. R1(config)#mpls ldp router-id lo0 force R2(config)#int fa0/0 R2(config-if)#mpls ip R2(config-if)#int f0/1 R2(config-if)#mpls ip R2(config-if)#mpls label range 2000 2999 R3(config)#int fa0/0 R3(config-if)#mpls ip R3(config-if)#int fa0/1 R3(config-if)#mpls ip R3(config-if)#mpls label range 3000 3999 R4 R4(config)#int fa0/0 R4(config-if)#mpls ip R4(config-if)#mpls label range 4000 4999
The command “mpls ip” is required to form LDP neighbors. It is only configured in interfaces that are inside the service provider network. Any interfaces such as loopbacks or those facing the customer are not required to be configured because LDP is not required between customer and PE routers. Though the customer is connected to the MPLS network, it is a common practice for service providers not to make their network visible to the customer.
The “mpls label range” command in the routers sets the number of labels only. I configured it that way so it will be easier to explain later how LDP works. In the example configuration above, the number of labels that can be assigned for each router only amounts to 1000. If the network has more than 1000 prefixes, the rest of the prefixes will not be labeled and will be routed using IP.
The “mpls ldp router-id loopback0 force” command enforces the LDP to use the IP address of Loopback0 as its ID. The “force” keyword will tear down existing LDP sessions and clear all the current bindings and applies the changes to the LDP ID. If “force” is not used, the router will wait until the current interface of the LDP ID goes down before it applies the new LDP ID specified in the command.
The routers have been restarted. Let’s check the LDP neighborship. Let’s use two examples for brevity.
R1#sh mpls ldp neigh Peer LDP Ident: 220.127.116.11:0; Local LDP Ident 18.104.22.168:0 TCP connection: 22.214.171.124.18805 - 126.96.36.199.646 State: Oper; Msgs sent/rcvd: 46/46; Downstream Up time: 00:31:24 LDP discovery sources: FastEthernet0/0, Src IP addr: 188.8.131.52 Addresses bound to peer LDP Ident: 184.108.40.206 220.127.116.11 18.104.22.168 R3#sh mpls ldp neigh Peer LDP Ident: 22.214.171.124:0; Local LDP Ident 126.96.36.199:0 TCP connection: 188.8.131.52.646 - 184.108.40.206.42778 State: Oper; Msgs sent/rcvd: 67/66; Downstream Up time: 00:49:52 LDP discovery sources: FastEthernet0/1, Src IP addr: 220.127.116.11 Addresses bound to peer LDP Ident: 18.104.22.168 22.214.171.124 126.96.36.199 Peer LDP Ident: 188.8.131.52:0; Local LDP Ident 184.108.40.206:0 TCP connection: 220.127.116.11.59644 - 18.104.22.168.646 State: Oper; Msgs sent/rcvd: 67/67; Downstream Up time: 00:49:51 LDP discovery sources: FastEthernet0/0, Src IP addr: 22.214.171.124 Addresses bound to peer LDP Ident: 126.96.36.199 188.8.131.52 R1#show mpls forwarding-table Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 1000 Pop tag 184.108.40.206/32 0 Fa0/0 220.127.116.11 1001 Pop tag 18.104.22.168/24 0 Fa0/0 22.214.171.124 1002 2001 126.96.36.199/24 0 Fa0/0 188.8.131.52 1003 2002 184.108.40.206/32 0 Fa0/0 220.127.116.11 1004 2003 18.104.22.168/32 0 Fa0/0 22.214.171.124
As mentioned, the LDP ID will be the highest loopback IP address that is operational in the router. The LDP routers, before they form an LDP session, elect which router will be active and passive. The router chosen as active will initiate the LDP TCP connection. In our case, R2 initiated a connection using a random number which in this case is 18805, R1 responds back with the port 646, the TCP port that is assigned to LDP. The “Addresses bound to peer LDP Ident:” section specifies that the routes below are directly connected to the LDP neighbor. Directly connected routes to the neighbor by default will not have any label assigned in the LIB (Label Information Base).
The “show mpls forwarding-table” also called the LFIB, shows the actions which LDP will take when it receives a specific label. As you can see, it doesn’t put any labels to directly connected routes of its adjacent LDP neighbor which is R2.
Task 2: Configure VRF in the Provider Edge (PE) Routers
VRF (Virtual Routing and Forwarding) is comparable to a VLAN in a switch. VRF is used to create different routing tables that are separated from each other. Since one VRF can’t see what routes are in another VRF, the same IP prefix can exist in different VRFs. However, duplicate IP prefixes will have an issue when it comes to route-leaking between VRFs. This will be discussed in the next article.
R1(config)#ip vrf CUST-A R1(config-vrf)#rd 65002:1 R1(config-vrf)#route R1(config-vrf)#route-target import 65002:1 R1(config-vrf)#route-target export 65002:1 R4(config)#ip vrf CUST-A R4(config-vrf)#rd 65002:1 R4(config-vrf)#route-target import 65002:1 R4(config-vrf)#route-target export 65002:1
Let’s apply the VRF into the interface facing the CE (customer edge) router.
R1(config-if)#ip vrf forwarding CUST-A % Interface FastEthernet0/1 IP address 126.96.36.199 removed due to enabling VRF CUST-A R1(config-if)#ip address 188.8.131.52 255.255.255.0 R4(config-if)#int fa0/1 R4(config-if)#ip vrf forwarding CUST-A % Interface FastEthernet0/1 IP address 184.108.40.206 removed due to enabling VRF CUST-A R4(config-if)#ip address 220.127.116.11 255.255.255.0
The VRF name is locally significant. It is not a transitive attribute that will be shared between routers. In fact, in an MPLS VPN network, as long as the RD (Route Distinguisher) and the RT (Route Target) values are configured correctly but the VRF names are different, the MPLS VPN service will work.
RD is what Multiprotocol BGP uses to distinguish and makes the route unique. The standard telco practice is to assign a unique RD for every customer. RT on the other hand, is an extended BGP community that marks, tags or classifies the prefix. The “export” keyword in the command means that the route will be marked and announced out with that value; “import” means put all the routes with that mark, into the VRF’s routing table specified above the command.
Task 3: Configure BGP VPNv4 peering between R1 and R4
VPNv4 is an address-family of Multiprotocol BGP. To explain it simply, VPNv4 is a collection of all routes from different VRFs that were marked with the extended community route-target. This is the address-family where route-leaking can be performed. Route-leaking is simply sharing a route from one VRF to another. Common application for this is, one company wants to connect to another company’s servers and they happen to be connected to the same MPLS provider. The different techniques of route-leaking will be discussed in Part 2.
R1(config)#router bgp 65001 R1(config-router)#address-family vpnv4 R1(config-router-af)#neigh 18.104.22.168 activate R1#sh run | inc router bgp | address-family vpnv4|neigh router bgp 65001 bgp log-neighbor-changes neighbor 22.214.171.124 remote-as 65001 neighbor 126.96.36.199 update-source Loopback0 neighbor 188.8.131.52 activate neighbor 184.108.40.206 next-hop-self address-family vpnv4 neighbor 220.127.116.11 activate neighbor 18.104.22.168 send-community extended R4(config-if)#router bgp 65001 R4(config-router)#address-family vpnv4 R4(config-router-af)#neigh 22.214.171.124 activate R4#sh ip bgp vpnv4 all sum BGP router identifier 126.96.36.199, local AS number 65001 BGP table version is 1, main routing table version 1 Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 188.8.131.52 4 65001 116 116 1 0 0 00:01:09 0
In VPNv4 address-family configuration, you simply issue the neighbor statement and the keyword “activate”. The BGP peering configuration needs to be done outside the address-family. The router understands that VPNv4 peering needs to activate extended communities so it automatically configured the statement highlighted above. In regards to the VPNv4 BGP peering, we can’t see any prefixes for now since there is no peering yet between the PE s and CEs.
Task 4: Configure Peering between PE routers R1 and R4 to customer routers CUST_A-R1 and CUST-A-R2. Announce Loopback 10 and 100 in the CE routers. Verify connectivity.
R1(config)#router bgp 65001 R1(config-router)#address-family vpnv4 R1(config-router-af)#address-family ipv4 vrf CUST-A R1(config-router-af)#neighbor 184.108.40.206 remote-as 65002 R1(config-router-af)# neighbor 220.127.116.11 activate R1(config-router-af)# neighbor 18.104.22.168 as-override CUST_A-R1(config)#router bgp 65002 CUST_A-R1(config-router)#neighbor 22.214.171.124 remote-as 65001 CUST_A-R1(config-router)# network 126.96.36.199 mask 255.255.255.255 CUST_A-R1(config-router)# network 188.8.131.52 mask 255.255.255.255 R4(config)#router bgp 65001 R4(config-router)# address-family ipv4 vrf CUST-A R4(config-router-af)# neighbor 184.108.40.206 remote-as 65002 R4(config-router-af)# neighbor 220.127.116.11 activate R4(config-router-af)# neighbor 18.104.22.168 as-override CUST_A-R2(config)#router bgp 65002 CUST_A-R2(config-router)#network 22.214.171.124 mask 255.255.255.255 CUST_A-R2(config-router)# network 126.96.36.199 mask 255.255.255.255 CUST_A-R2(config-router)# neighbor 188.8.131.52 remote-as 65001
The PE is configured with an “address-family ipv4 vrf” when peering with the CE routers. The “as-override” command replaces the AS of the route to circumvent the BGP loop prevention. BGP loop prevention blocks any route that it receives from an eBGP peer with its own AS (65002 in this case) inside it. The AS for the customer is 65002, but notice the output below, the PE’s replaced the AS to 65001 to enable communication between these two routers with the same AS inside an MPLS cloud. CUST_A-R2 is now able to see the CUST_A-R1 routes but with a different AS. Another way to do this is to configure a neighbor statement with “allowas-in” keyword.
R1#show ip bgp vpnv4 vrf CUST-A BGP table version is 7, local router ID is 184.108.40.206 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 65002:1 (default for vrf CUST-A) *> 220.127.116.11/32 18.104.22.168 0 0 65002 i *>i22.214.171.124/32 126.96.36.199 0 100 0 65002 i *> 188.8.131.52/32 184.108.40.206 0 0 65002 i CUST_A-R2#sh ip bgp BGP table version is 5, local router ID is 220.127.116.11 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 18.104.22.168/32 22.214.171.124 0 65001 65001 i *> 126.96.36.199/32 0.0.0.0 0 32768 i *> 188.8.131.52/32 184.108.40.206 0 65001 65001 i *> 220.127.116.11/32 0.0.0.0 0 32768 i
Lets check and verify connectivity.
CUST_A-R2#traceroute 18.104.22.168 source l100 Type escape sequence to abort. Tracing the route to 22.214.171.124 1 126.96.36.199 24 msec 28 msec 20 msec 2 188.8.131.52 88 msec 92 msec 100 msec 3 184.108.40.206 108 msec 92 msec 80 msec 4 220.127.116.11 80 msec 68 msec 72 msec 5 18.104.22.168 88 msec 88 msec 80 msec
As we can see, there is a full reachability between the CE routers but the traceroute shows the path it took inside the service provider core network. This is not an advisable behavior, normally service provider from the customer any information about its core network. Let’s configure a way to do that.
R1(config)#no mpls ip propagate-ttl R4(config)#no mpls ip propagate-ttl
Let’s test that again.
CUST_A-R2#traceroute 22.214.171.124 source l100 Type escape sequence to abort. Tracing the route to 126.96.36.199 1 188.8.131.52 28 msec 16 msec 20 msec 2 184.108.40.206 80 msec 80 msec 80 msec 3 220.127.116.11 108 msec 104 msec 96 msec
Now, the service provider network has been hidden through the “no mpls ip propagate-ttl” command.
Hopefully this has been an informative article. The next part will focus on how to share routes between different VRFs inside an MPLS network. Stay tuned.