Part 2 of my experiments with VRF-Lite, which provides more thoughts on whether the customer edge (CE) device should run one or more VRFs. (Part 1 is available here CE Design Options When Using VRF-Lite End-to-End, and discusses using the CE as a “fusion” router to support dynamic routing the customer routing process(es) with the shared service VRFs by using multiple EIGRP processes on the CE without any VRFs.)
As you may recall, the environment is a simple shared services scenario, where a small service provider or large organization is providing access to the Internet, as well as on-network Voice over IP (VoIP) service between some locations. In this scenario, there are two VRFs, Internet and Voice offered to multiple locations. The provider is running VRF-Lite end-to-end across their backbone, and is not using BGP. The backbone routing protocol is EIGRP, and the customer sites have non-overlapping IP address allocations. The network connects each remote campus or site as ‘customers’ to a pair of CE routers, and the CEs to PE routers in the core. Two of the PE routers connect to a cloud of VoIP servers – such as a Unified Communication Manager, a Unity server, … Two of the PE routers connect to the Internet. The provider allocates IP addressing to the customers, with network 10.0.0.0/9 is reserved for data, and network 10.128.0.0/9 is reserved for voice. (By convention, the voice service is 10.d + 128.x.x, where “d” is the block assigned for data service.) Each customer runs EIGRP to interconnect their campus locations.
In the diagram, the Voice VRF is shown in blue, the Internet VRF is in Red. The global routing for the infrastructure management is in black.
The 10.8.0.0/16 addresses are allocated for the Internet infrastructure, and the 10.136.0.0/16 addresses are allocated for the voice infrastructure, with the 10.136.248.0/248 addresses allocated for the network management infrastructure. (Note: We reserved a block of the Voice infrastructure addresses for network management, since we wanted them to have more protection than the Internet infrastructure.)
CE Running the Shared Services VRFs
Another design option is to have the CE also run the shared services VRFs. If the remote locations run different EIGRP processes, what are the options at the CE for interconnecting customers to the shared servies? Can dynamic routing be supported between the customer and the CE?
Note: I will focus on CE-1 and PE-1 configurations. The diagram shows the VLANs used for the network virtualization.
This design will require three main configuration tasks:
- Connect shared services VRFs between the PEs and the CEs
- Connect the CE with the customer
- Provide connectivity to the shared VRFs to the customer
1. Connecting the Shared Services VRFs with the PEs and CEs
The backbone VRF configuration is straight-forward, and the same on all devices supporting the VRFs:
! ip vrf Internet
rd 8:4 ! ip vrf Voice
rd 136:2 ! router eigrp 248 network 10.136.248.0 0.0.7.255 no auto-summary ! address-family ipv4 vrf Voice
network 10.136.0.0 0.0.255.255
no auto-summary
autonomous-system 136
exit-address-family ! address-family ipv4 vrf Internet
network 10.8.0.0 0.0.255.255
no auto-summary
autonomous-system 8
exit-address-family !
For example, the CEs are supporting the VRFs on specific interfaces:
! CE-A1 interface Vlan2 description CE-B2 ip address 10.136.250.149 255.255.255.252 ! interface Vlan3 description VoIP:CE-B2 ip vrf forwarding Voice ip address 10.136.4.149 255.255.255.252 ! interface Vlan4 description Inet:CE-B2 ip vrf forwarding Internet
ip address 10.8.4.149 255.255.255.252 !
We can verfiy that the CEs are receiving routes from the PEs:
! CE-A1#sh ip ro vrf Voice
Routing Table: Voice
Codes: C – connected, S – static, R – RIP, M – mobile, B – BGP
D – EIGRP, EX – EIGRP external, O – OSPF, IA – OSPF inter area
N1 – OSPF NSSA external type 1, N2 – OSPF NSSA external type 2
E1 – OSPF external type 1, E2 – OSPF external type 2
i – IS-IS, su – IS-IS summary, L1 – IS-IS level-1, L2 – IS-IS level-2
ia – IS-IS inter area, * – candidate default, U – per-user static route
o – ODR, P – periodic downloaded static route
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
C 10.136.4.148/30 is directly connected, Vlan3
D 10.136.4.144/30 [90/3072] via 10.136.4.150, 00:01:42, Vlan3
C 10.136.4.140/30 is directly connected, Vlan12
D 10.128.0.0/9 [90/3072] via 10.136.4.141, 00:01:42, Vlan12
CE-A1#
!
2. Connecting the CEs and the Customer – Dynamic Routing?
So, how can the CustomerA devices be connected? First choice – can we use dynamic routing with the customer?
In this example, if the customer is running both AS 88 and AS 216 to support their voice and data, you can put the customer interfaces into the appropriate VRFs with the ip vrf forwarding XXX command, with the links to customer added in the network statements for each address-family. However, it is extremely unlikely that the customer is running separate EIGRP processes to support their voice and data traffic that match the AS number of the address familes. (Note: The provider is running different AS numbers in each address-family so that two routers need to be on the same subnet as well as running the same EIGRP process to become neighbors.)
So what happens if we try to run a second EIGRP process and match the customer’s EIGRP AS?
! CE-A1 router eigrp 1 network 10.100.1.248 0.0.0.3 network 10.228.1.248 0.0.0.3 exit !
!
CE-A1#sh ip ro eigrp 1
10.0.0.0/8 is variably subnetted, 49 subnets, 8 masks
D 10.129.1.128/30 [90/130816] via 10.100.1.249, 00:00:31, Vlan88
D 10.1.1.0/25 [90/130816] via 10.228.1.249, 00:00:31, Vlan216
D 10.1.1.0/24 [90/130816] via 10.100.1.249, 00:00:31, Vlan88
D 10.66.1.1/32 [90/130816] via 10.228.1.249, 00:00:31, Vlan216
[90/130816] via 10.100.1.249, 00:00:31, Vlan88
D 10.67.1.1/32 [90/130816] via 10.228.1.249, 00:00:31, Vlan216
[90/130816] via 10.100.1.249, 00:00:31, Vlan88
D 10.117.32.0/25 [90/130816] via 10.228.1.249, 00:00:32, Vlan216
D 10.111.0.0/16 [90/130816] via 10.100.1.249, 00:00:32, Vlan88
. . .
!
The CE routers can receive dynamic routes from the customer.
Can we get these routes into the VRF tables with redistribution?
! CE-A1(config)#router eigrp 248 CE-A1(config-router)#address ipv4 vrf Voice CE-A1(config-router-af)#redist ? bgp Border Gateway Protocol (BGP) connected Connected eigrp Enhanced Interior Gateway Routing Protocol (EIGRP) metric Metric for redistributed routes odr On Demand stub Routes ospf Open Shortest Path First (OSPF) rip Routing Information Protocol (RIP) route-map Route map reference static Static routes <cr> CE-A1(config-router-af)#redist eigrp 1 ? metric Metric for redistributed routes route-map Route map reference <cr> CE-A1(config-router-af)#redis eigrp 1 %VRF specified does not match AS CE-A1(config-router-af)# !
No, we can not redistribute from EIGRP 1 into the address-family vrf Voice.
Can we re-use the same address-family in EIGRP 1?
! CE-A1(config)#router eigrp 1 CE-A1(config-router)#address-family ipv4 vrf Voice % VRF Voice is already associated with another eigrp process CE-A1(config-router-af)# !
No, we can not use the same adress-family for two EIGRP routing processes.
What happens if we put a customer interface into the Voice VRF?
! CE-A1(config)#int vlan 136 CE-A1(config-if)#ip vrf for Voice % Interface Vlan136 IP address 10.228.1.250 removed due to enabling VRF Voice CE-A1(config-if)# 04:23:21: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 10.228.1.249 (Vlan136) is down: interface down
CE-A1(config-if)#ip add 10.228.1.250 255.255.255.252 CE-A1(config-if)#do sh ip eigrp 1 int IP-EIGRP interfaces for process 1
Xmit Queue Mean Pacing Time Multicast Pending
Interface Peers Un/Reliable SRTT Un/Reliable Flow Timer Routes
Vl8 1 0/0 416 0/1 2080 0
CE-A1(config-if)
!
If the interface to the customer is placed into the Voice VRF, it is removed from EIGRP 1 process, and the CE will no longer form an adjacency with the customer across this subnet.
Conclusion: With this end-to-end VRF-LITE configuration where you want to share two VRFs with multiple customer locations, the CEs can not run the VRFs and redistribute dynamically learned customer routes.
3. Connecting the Customer and the Shared VRFs – Static Routing
Since dynamic routing to the customer does not allow us to redistribute dynamically learned routes into the VRF, we can put the customer facing interfaces into the appropriate VRFs, remove the EIGRP 1 routing process from the previous test, and set up static routes to point to the customer’s existing networks.
! CE-A1 no router eigrp 1 ! interface Vlan8 ip vrf forwarding Internet ip address 10.100.1.250 255.255.255.252 ! interface Vlan136 ip vrf forwarding Voice ip address 10.228.1.250 255.255.255.252 ! ip route vrf Internet 10.100.0.0 255.255.0.0 10.228.1.249
ip route vrf Internet 10.117.0.0 255.255.0.0 10.228.1.249
ip route vrf Internet 10.118.0.0 255.255.0.0 10.228.1.249
ip route vrf Internet 10.119.0.0 255.255.0.0 10.228.1.249
! ip route vrf Voice 10.228.0.0 255.255.0.0 10.228.1.249
ip route vrf Voice 10.245.0.0 255.255.0.0 10.228.1.249
ip route vrf Voice 10.246.0.0 255.255.0.0 10.228.1.249
ip route vrf Voice 10.247.0.0 255.255.0.0 10.228.1.249 !
Note: The customer interfaces must be placed in the VRFs in order for the static routes to be placed in the VRF tables. You need to re configure the IP address after placing the interfaces in the VRF.
These static routes can then redistributed into the address families using route-maps for control:
! CE-A1 ! ip prefix-list A-Data description Only use Data summaries from customer ip prefix-list A-Data permit 10.100.0.0/16 ip prefix-list A-Data permit 10.117.0.0/16 ip prefix-list A-Data permit 10.118.0.0/16 ip prefix-list A-Data permit 10.119.0.0/16 !
ip prefix-list A-Voice description Only use Voice summaries from customer ip prefix-list A-Voice permit 10.228.0.0/16 ip prefix-list A-Voice permit 10.245.0.0/16 ip prefix-list A-Voice permit 10.246.0.0/16 ip prefix-list A-Voice permit 10.247.0.0/16 !
route-map A-Data-in permit 10
match ip address prefix-list A-Data
!
route-map A-Voice-in permit 10
match ip address prefix-list A-Voice
! router eigrp 248 network 10.136.248.0 0.0.7.255 no auto-summary ! address-family ipv4 vrf Voice redistribute static metric 10000 100 255 1 1500 route-map A-Voice-in
network 10.136.0.0 0.0.255.255 network 10.228.1.248 0.0.0.3
no auto-summary autonomous-system 136 exit-address-family ! address-family ipv4 vrf Internet redistribute static metric 10000 100 255 1 1500 route-map A-Data-in
network 10.8.0.0 0.0.255.255
network 10.100.1.248 0.0.0.3
no auto-summary autonomous-system 8 exit-address-family !
We can verify that the PEs now see the redistributed customer summary routes from the CE:
! PE-1#sh ip ro vrf Voice | inc Vlan12 C 10.216.4.140/30 is directly connected, Vlan12 L 10.216.4.141/32 is directly connected, Vlan12 D EX 10.228.0.0/16 [170/281856] via 10.216.4.142, 00:01:25, Vlan12 D EX 10.245.0.0/16 [170/281856] via 10.216.4.142, 00:01:25, Vlan12 D EX 10.246.0.0/16 [170/281856] via 10.216.4.142, 00:01:25, Vlan12 D EX 10.247.0.0/16 [170/281856] via 10.216.4.142, 00:01:25, Vlan12 PE-1# ! PE-1#sh ip ro vrf Internet | inc Vlan13 C 10.88.4.140/30 is directly connected, Vlan13 L 10.88.4.141/32 is directly connected, Vlan13 D EX 10.100.0.0/16 [170/281856] via 10.88.4.142, 00:01:52, Vlan13 D EX 10.117.0.0/16 [170/281856] via 10.88.4.142, 00:01:52, Vlan13 D EX 10.118.0.0/16 [170/281856] via 10.88.4.142, 00:01:52, Vlan13 D EX 10.119.0.0/16 [170/281856] via 10.88.4.142, 00:01:52, Vlan13 PE-1# !
The remote customer also needs to define static routes that point to the correct CE interfaces:
! Cust1 ! for traffic going to the Internet ip route 0.0.0.0 0.0.0.0 10.100.1.250
! ! for VoIP traffic supporting access to the Call Manager and on-net calls ip route 10.128.0.0 255.128.0.0 10.228.1.250 !
We can the verify that the customer has connectivity to devices in the two service VRFs:
! Cust1#sh ip ro stat 10.0.0.0/8 is variably subnetted, 21 subnets, 7 masks S 10.128.0.0/9 [1/0] via 10.228.1.250
S* 0.0.0.0/0 [1/0] via 10.100.1.250
Cust1 Cust1#ping 1.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
!!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms Cust1#ping 10.216.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.216.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms Cust1#
Conclusion:
This implementation extends the basic PE VRF configuration on the CE with static routes for each customer address block connencted to the CE. The customer facing interfaces need to be in the shared services VRF. One drawback to this configuration is that since the allowed subnets are explicitally defined for each customer at the CE, we might black hole traffic if the subnets are not available at the customer site. This design needs to be carefully implemented so that the network engineer properly configures the correct static routes to the customer. It is also harder to verify whether all customers routes are available, or just preconfigured on an interface/site that is not yet up.
Which design to use may depend on how many address blocks each customer location will need to support, and whether the network administrators can understand and support the redistribution used for “fusing” the routing processes in the previous example, or are more comfortable carefully managing the configuration of a limited number of static routes per CE per customer.
____________________________________________________________________________________________________________
More on VRF-Lite
Other recent NetCraftsmen blogs on VRF-Lite include:
CE Design Options When Using VRF-Lite End-to-End (Part1)
Using VRF-Lite, EIGRP, and Static Routes