Using BGP with VRF-Lite for Shared Service Support

You probably know a couple of ways to use BGP in an enterprise environment. Most commonly, BGP has been used in the enterprise to provide connectivity to the Internet, supporting multihoming solutions, traffic policies, and summarization. BGP has also been used in the core of  large enterprise environments as a tool to enhance network scalability and support separate administrative control domains. However, another way to use BGP in the enterprise is on a single router supporting  networking virtualization with VRF-Lite. (VRF-Lite is also known as Multi-VRF).

VRF-Lite is good technology  for providing path isolation capabilities in the enterprise, and it uses an Interior Gateway Protocol (IGP) for the control plane. However, in some VRF-Lite deployments you may need to establish some shared services to be used by different routing instances. In these cases,  BGP can be enabled locally on one device (perhaps in the data center) to support communication from selected VRFs or ranges of addresses in a VRF to the shared services network. Note that in this case, the BGP process will not need to establish any BGP neighbor relationships with another device.

In the following example, I will use BGP to allow access to the Shared Service subnet from two user subnets in separate VRFs. The following diagram shows the core network where a Shared Service is to be provided, and this service should be accessible by one subnet in the Red VRF and one subnet in the Blue VRF.

PE-T1 and PE-T2 are representative ‘provider edge’ devices in this enterprise that support the VRF-Lite implementation. PE-S is the router that will support access to the Shared Service subnet while maintaining the separation between the Red and the Blue VRFs. The PEs are all Cisco 7600s. CE-A1 and CE-A2 are ‘customer edge’ Layer 3 switches (actually ME-3400s) used to connect in one or more customers. The highlighted subnets on CE-A1 and CE-A2 are the only subnets in their respective VRFs that will be allowed connectivity into the Shared Services subnet.

The basic VRF configuration on is straight-forward, and is the same on PE-T1,  PE-T2, CE-A1, and CE-A2 :

!
ip vrf Blue
 rd 88:2
!
ip vrf Red
 rd 26:4
!
router eigrp 24
 network 10.24.0.0 0.0.255.255
 no auto-summary
 !
 address-family ipv4 vrf Blue
 network 10.88.0.0 0.0.255.255
 no auto-summary
 autonomous-system 88
 exit-address-family
!
 address-family ipv4 vrf Red
 network 10.26.0.0 0.0.255.255
 no auto-summary
 autonomous-system 26
 exit-address-family
!

The VRF configuration on PE-S extends the previous configuration to include the Shared Services VRF:

!
ip vrf Blue
 rd 88:2
!
ip vrf Red
 rd 26:4
!
ip vrf Shared
 rd 16:16
!
router eigrp 24
 network 10.24.0.0 0.0.255.255
 no auto-summary
!
 address-family ipv4 vrf Blue
 network 10.88.0.0 0.0.255.255
 no auto-summary
 autonomous-system 88
 exit-address-family
!
 address-family ipv4 vrf Red
 network 10.26.0.0 0.0.255.255
 no auto-summary
 autonomous-system 26
 exit-address-family
!
 address-family ipv4 vrf Shared
 network 10.16.32.0 0.0.0.255
 no auto-summary
 autonomous-system 16
 exit-address-family
!

Initially, the shared service subnet is only known on PE-S in the Shared VRF:

PE-S#sh ip ro vrf Shared 10.16.32.0

Routing Table: Shared
Routing entry for 10.16.32.0/24
Known via “connected”, distance 0, metric 0 (connected, via interface)
Redistributing via eigrp 16
Routing Descriptor Blocks:
* directly connected, via Loopback16
Route metric is 0, traffic share count is 1
PE-S#sh ip ro vrf Blue 10.16.32.0

Routing Table: Blue
% Subnet not in table
PE-S#sh ip ro vrf Red 10.16.32.0

Routing Table: Red
% Subnet not in table
PE-S#

PE-T1#sh ip ro vrf Shared 10.16.32.0
% IP routing table vrf Shared does not exist
PE-T1#sh ip ro vrf Blue 10.16.32.0

Routing Table: Blue
% Subnet not in table
PE-T1#sh ip ro vrf Red 10.16.32.0

Routing Table: Red
% Subnet not in table
PE-T1#

PE-T2#sh ip ro vrf Shared 10.16.32.0
% IP routing table vrf Shared does not exist
PE-T2#sh ip ro vrf Blue 10.16.32.0

Routing Table: Blue
% Subnet not in table
PE-T2#sh ip ro vrf Red 10.16.32.0

Routing Table: Red
% Subnet not in table
PE-T2#

We can use BGP to provide local route leaking on PE-S, which is the device where the shared service network is attached. (I simulate the Shared Service network with a /24 Loopback.)  All the new configuration will be done on the PE-S router.

Updating the VRF Configuration

To get the route leaking working, we will need to update the VRF configurations to import and export routes using route targets. The Shared Service subnet needs to be exported from the Shared VRF, and imported into the Blue and the Red VRFs.  In addition, the two user subnets need to be exported from their default VRFs and imported into the Shared Services VRF.

PE-S(config)#ip vrf Shared
PE-S(config-vrf)# rd 16:16
PE-S(config-vrf)# route-target export 30:30
PE-S(config-vrf)#route-target import 10:10
PE-S(config-vrf)#route-target import 20:20
PE-S(config-vrf)#
PE-S(config-vrf)#ip vrf Blue
PE-S(config-vrf)# rd 88:2
PE-S(config-vrf)# route-target export 20:20
PE-S(config-vrf)#route-target import 30:30
PE-S(config-vrf)#
PE-S(config-vrf)#ip vrf Red
PE-S(config-vrf)# rd 26:4
PE-S(config-vrf)# route-target export 10:10
PE-S(config-vrf)#route-target import 30:30
PE-S(config-vrf)#

Configuring the Access Lists and Route Maps for Control of the Route Exchange

I will need to configure BGP so that it uses the VRF configurations and exchanges the appropriate routes. I defined some route maps to control what routes will be exchanged.

PE-S(config)#
PE-S(config)#access-list 16 permit 10.16.32.0 0.0.0.255
PE-S(config)# ! allow only the Shared Service subnet
PE-S(config)#access-list 26 permit 10.26.34.0 0.0.0.255
PE-S(config)# ! allow only the designated Red subnet
PE-S(config)#access-list 88 permit 10.88.35.0 0.0.0.255
PE-S(config)# ! allow only the designated Blue subnet
PE-S(config)#route-map Allowed-Blue permit 10
PE-S(config-route-map)# match ip address 88
PE-S(config-route-map)#
PE-S(config-route-map)#route-map Allowed-Red permit 10
PE-S(config-route-map)# match ip add 26
PE-S(config-route-map)#
PE-S(config-router-map)#route-map Shared-Services permit 10
PE-S(config-route-map)# match ip add 16
PE-S(config-route-map)#

Configuring the BGP Process

Then I configured the BGP process so that the Shared Services subnets were redistributed into the VRFs. Since I am  running BGP as a local process only, there will be no neighbor or network statements.

PE-S(config)#
PE-S(config)#router bgp 300
PE-S(config-router)#no synch
PE-S(config-router)#
PE-S(config-router)#address-family ipv4 vrf Shared
PE-S(config-router-af)# redist connect route-map Shared-Services
PE-S(config-router-af)# no synch
PE-S(config-router-af)#
PE-S(config-router-af)#address-family ipv4 vrf Blue
PE-S(config-router-af)# redist eigrp 88 route-map Allowed-Blue
PE-S(config-router-af)# no synch
PE-S(config-router-af)#
PE-S(config-router-af)#address-family ipv4 vrf Red
PE-S(config-router-af)# redist eigrp 26 route-map Allowed-Red
PE-S(config-router-af)# no synch
PE-S(config-route-map)#

Verifying the Shared Services Network is in All the PE-S VRFs

Some quick show commands will verify that the Shared Services network is in all the VRFs on PE-S:

PE-S#sh ip ro vrf Shared 10.16.32.0

Routing Table: Shared
Routing entry for 10.16.32.0/24
Known via “connected”, distance 0, metric 0 (connected, via interface)
Redistributing via eigrp 16, bgp 300
Advertised by bgp 300 route-map Shared-Services
Routing Descriptor Blocks:
* directly connected, via Loopback16
Route metric is 0, traffic share count is 1
PE-S#sh ip ro vrf Blue 10.16.32.0

Routing Table: Blue
Routing entry for 10.16.32.0/24
Known via “bgp 300”, distance 20, metric 0 (connected, via interface), type external
Routing Descriptor Blocks:
* directly connected, via Loopback16
Route metric is 0, traffic share count is 1
AS Hops 0
PE-S#sh ip ro vrf Red 10.16.32.0

Routing Table: Red
Routing entry for 10.16.32.0/24
Known via “bgp 300”, distance 20, metric 0 (connected, via interface), type external
Routing Descriptor Blocks:
* directly connected, via Loopback16
Route metric is 0, traffic share count is 1
AS Hops 0
PE-S#

Checking the Status of the Shared Services Network in PE-T1 and PE-T2 VRFs

If you look for the Shared Services network on PE-T1 and PE-T2, you will not see it because the route leaking is currently local to PE-S.

PE-T1#sh ip ro vrf Shared 10.16.32.0
% IP routing table vrf Shared does not exist
PE-T1#sh ip ro vrf Blue 10.16.32.0

Routing Table: Blue
% Subnet not in table
PE-T1#sh ip ro vrf Red 10.16.32.0

Routing Table: Red
% Subnet not in table
PE-T1#

PE-T2#sh ip ro vrf Shared 10.16.32.0
% IP routing table vrf Shared does not exist
PE-T2#sh ip ro vrf Blue 10.16.32.0

Routing Table: Blue
% Subnet not in table
PE-T2#sh ip ro vrf Red 10.16.32.0

Routing Table: Red
% Subnet not in table
PE-T2#

Advertising the Shared Services Network in the Red and Blue VRFs

To get PE-T1 and PE-T2 to learn about the Shared Services network, we need to redistribute the route into the EIGRP process.

PE-S(config)#router eigrp 24
PE-S(config-router)# address-family ipv4 vrf Red
PE-S(config-router-af)# redist bgp 300 metric 100000 1 255 1 1500
PE-S(config-router-af)# exit-address-family
PE-S(config-router)# !
PE-S(config-router)# address-family ipv4 vrf Blue
PE-S(config-router-af)# redist bgp 300 metric 100000 1 255 1 1500
PE-S(config-router-af)# exit-address-family
PE-S(config-router)#!

Re-checking the Status of the Shared Services Network in PE-T1 and PE-T2 VRFs

After the BGP redistribution into EIGRP is configured, the Shared Services network will be in the VRFs on both PE-T1 and PE-T2.

PE-T1#sh ip ro vrf Blue 10.16.32.0

Routing Table: Blue
Routing entry for 10.16.32.0/24
Known via “eigrp 88”, distance 170, metric 26112, type external
Redistributing via eigrp 88
Last update from 10.88.4.6 on Vlan7, 00:00:25 ago
Routing Descriptor Blocks:
* 10.88.4.6, from 10.88.4.6, 00:00:25 ago, via Vlan7
Route metric is 26112, traffic share count is 1
Total delay is 20 microseconds, minimum bandwidth is 100000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
PE-T1#sh ip ro vrf Red 10.16.32.0

Routing Table: Red
Routing entry for 10.16.32.0/24
Known via “eigrp 26”, distance 170, metric 26112, type external
Redistributing via eigrp 26
Last update from 10.26.4.6 on Vlan6, 00:00:25 ago
Routing Descriptor Blocks:
* 10.26.4.6, from 10.26.4.6, 00:00:25 ago, via Vlan6
Route metric is 26112, traffic share count is 1
Total delay is 20 microseconds, minimum bandwidth is 100000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
PE-T1#

PE-T2#sh ip ro vrf Red 10.16.32.0

Routing Table: Red
Routing entry for 10.16.32.0/24
Known via “eigrp 26”, distance 170, metric 26112, type external
Redistributing via eigrp 26
Last update from 10.26.4.14 on Vlan9, 00:25:41 ago
Routing Descriptor Blocks:
* 10.26.4.14, from 10.26.4.14, 00:25:41 ago, via Vlan9
Route metric is 26112, traffic share count is 1
Total delay is 20 microseconds, minimum bandwidth is 100000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
PE-T2#sh ip ro vrf Blue 10.16.32.0

Routing Table: Blue
Routing entry for 10.16.32.0/24
Known via “eigrp 88”, distance 170, metric 26112, type external
Redistributing via eigrp 88
Last update from 10.88.4.14 on Vlan10, 00:25:51 ago
Routing Descriptor Blocks:
* 10.88.4.14, from 10.88.4.14, 00:25:51 ago, via Vlan10
Route metric is 26112, traffic share count is 1
Total delay is 20 microseconds, minimum bandwidth is 100000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
PE-T2#

Verifying Connectivity to the Shared Services Network

Although all the PE routers see the Shared Services network, traffic is only supported from the two user networks defined in the Allowed-Red and Allowed-Blue route maps.  The user subnets are simulated on loopbacks on CE routers that are also running VRF-Lite.

CE-A1#sh ip vrf
 Name Default RD Interfaces
 Blue 88:2 Lo88
 Vl13
 Red 26:4 Lo26
 Vl12
CE-A1#ping vrf Red 10.16.32.3 source loopback 26

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.16.32.3, timeout is 2 seconds:
Packet sent with a source address of 10.26.34.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CE-A1#ping vrf Blue 10.16.32.3 source loopback 88

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.16.32.3, timeout is 2 seconds:
Packet sent with a source address of 10.88.34.1
…..
Success rate is 0 percent (0/5)
CE-A1#

CE-A2#sh ip vrf
 Name Default RD Interfaces
 Blue 88:2 Lo88
 Vl13
 Red 26:4 Lo26
 Vl12
CE-A2#
CE-A2#ping vrf Blue 10.16.32.3 source loopback 88

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.16.32.3, timeout is 2 seconds:
Packet sent with a source address of 10.88.35.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CE-A2#ping vrf Red 10.16.32.3 source loopback 26

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.16.32.3, timeout is 2 seconds:
Packet sent with a source address of 10.26.35.1
…..
Success rate is 0 percent (0/5)
CE-A2#

Note: Since all the PEs learn about the Shared Services network, as needed I can allow additional customer subnets to reach the shared services network by updating the access lists on PE-S.

Verifying No Connectivity Between the VRFs

One last check, I wanted to verify that there is no connectivity between the Red and Blue VRFs at the customer sites.

CE-A1#sh ip ro vrf Red

Routing Table: Red
. . .
Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 11 subnets, 3 masks
D 10.26.4.20/30 [90/3072] via 10.26.4.129, 00:35:38, Vlan12
D 10.26.4.4/30 [90/3072] via 10.26.4.129, 00:32:03, Vlan12
D 10.26.1.1/32 [90/130816] via 10.26.4.129, 00:40:29, Vlan12
D 10.26.1.3/32 [90/131072] via 10.26.4.129, 00:31:57, Vlan12
D 10.26.1.2/32 [90/131072] via 10.26.4.129, 00:35:35, Vlan12
D 10.26.4.12/30 [90/3328] via 10.26.4.129, 00:32:04, Vlan12
C 10.26.34.0/24 is directly connected, Loopback26
D 10.26.35.0/24 [90/131328] via 10.26.4.129, 00:35:35, Vlan12
D EX 10.16.32.0/24 [170/26368] via 10.26.4.129, 00:15:44, Vlan12
D 10.26.4.132/30 [90/3328] via 10.26.4.129, 00:35:35, Vlan12
C 10.26.4.128/30 is directly connected, Vlan12
CE-A1#
CE-A1#sh ip ro vrf Blue

Routing Table: Blue
. . .
Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 11 subnets, 3 masks
D EX 10.16.32.0/24 [170/26368] via 10.88.4.129, 00:16:21, Vlan13
D 10.88.4.20/30 [90/3072] via 10.88.4.129, 00:36:16, Vlan13
D 10.88.1.3/32 [90/131072] via 10.88.4.129, 00:32:35, Vlan13
D 10.88.1.2/32 [90/131072] via 10.88.4.129, 00:36:11, Vlan13
D 10.88.4.4/30 [90/3072] via 10.88.4.129, 00:32:41, Vlan13
D 10.88.1.1/32 [90/130816] via 10.88.4.129, 00:41:08, Vlan13
D 10.88.4.12/30 [90/3328] via 10.88.4.129, 00:32:43, Vlan13
D 10.88.35.0/24 [90/131328] via 10.88.4.129, 00:36:11, Vlan13
C 10.88.34.0/24 is directly connected, Loopback88
D 10.88.4.132/30 [90/3328] via 10.88.4.129, 00:36:11, Vlan13
C 10.88.4.128/30 is directly connected, Vlan13
CE-A1#

CE-A2#sh ip ro vrf Red

Routing Table: Red
. . .
Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 11 subnets, 3 masks
D 10.26.4.20/30 [90/3072] via 10.26.4.133, 00:36:03, Vlan12
D 10.26.4.4/30 [90/3328] via 10.26.4.133, 00:32:27, Vlan12
D 10.26.1.1/32 [90/131072] via 10.26.4.133, 00:35:59, Vlan12
D 10.26.1.3/32 [90/131072] via 10.26.4.133, 00:32:23, Vlan12
D 10.26.1.2/32 [90/130816] via 10.26.4.133, 00:36:31, Vlan12
D 10.26.4.12/30 [90/3072] via 10.26.4.133, 00:32:28, Vlan12
D 10.26.34.0/24 [90/131328] via 10.26.4.133, 00:36:00, Vlan12
C 10.26.35.0/24 is directly connected, Loopback26
D EX 10.16.32.0/24 [170/26368] via 10.26.4.133, 00:16:08, Vlan12
C 10.26.4.132/30 is directly connected, Vlan12
D 10.26.4.128/30 [90/3328] via 10.26.4.133, 00:36:00, Vlan12CE-A2#sh ip ro vrf Blue
CE-A2#
CE-A2#sh ip ro vrf Blue
Routing Table: Blue
. . .
Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 11 subnets, 3 masks
D EX 10.16.32.0/24 [170/26368] via 10.88.4.133, 00:15:55, Vlan13
D 10.88.4.20/30 [90/3072] via 10.88.4.133, 00:35:50, Vlan13
D 10.88.1.3/32 [90/131072] via 10.88.4.133, 00:32:10, Vlan13
D 10.88.1.2/32 [90/130816] via 10.88.4.133, 00:36:16, Vlan13
D 10.88.4.4/30 [90/3328] via 10.88.4.133, 00:32:15, Vlan13
D 10.88.1.1/32 [90/131072] via 10.88.4.133, 00:35:44, Vlan13
D 10.88.4.12/30 [90/3072] via 10.88.4.133, 00:32:16, Vlan13
C 10.88.35.0/24 is directly connected, Loopback88
D 10.88.34.0/24 [90/131328] via 10.88.4.133, 00:35:45, Vlan13
C 10.88.4.132/30 is directly connected, Vlan13
D 10.88.4.128/30 [90/3328] via 10.88.4.133, 00:35:45, Vlan13
CE-A2#

Good – only the appropriate VRF routes and the Shared Service network are in the routing tables.

More on VRF-Lite and BGP

Other recent NetCraftsmen blogs on VRF-Lite topics  and BGP topics include:

Using VRF-Lite, EIGRP, and Static Routes

IP Multicast in a VRF

BGP Route Reflector Design Issues