Come with me on a ride-along about “front door VRFs”.  Front-door VRFs in a tunneled environment are really quite cool but often scare people new to them. 

SIMPLE GRE Tunnel Example

Let’s take Foxtrot13 and Foxtrot14.  I hand you 2 tasks ->

1. create a GRE tunnel between Foxtrot13’s 10.13.2.1 physical interface and Foxtrot14’s 10.4.14.2 physical interface
2. over this tunnel have an EIGRP neighbor

You go and configure Foxtrot13 and Foxtrot14 as below.  Nothing overly exciting in this simplified GRE configuration.

Tunnel Endpoint Connectivity

Of course…. for this Tunnel to come up the physical endpoints of the tunnel (10.13.2.1 & 10.4.14.2) must be able to reach each other.

Let’s look at the physical layer under the tunnel. As you can see, we are going to need some form of  “underlying” routing to be working so these 2 IP addresses can communicate with each other.   As you can see below, I used OSPF.

So does Foxtrot13 know how to get to its Tunnel11 GRE tunnel endpoint IP address 10.4.14.2?  Does Foxtrot14 know how to get to its Tunnel11 GRE tunnel endpoint IP address 10.13.2.1?  From what we can see below, the answer is yes.

• Foxtrot13 has learned about 10.4.14.2 from Echo2 (10.13.2.2) via OSPF.
• Foxtrot14 has learned about 10.13.2.1 from Echo4 (10.4.14.1) via OSPF.

Are our tunnel11s up?  As you can see below they are.

Adding Routing Over the Tunnel

So I have already put the EIGRP configs on Foxtrot14.

Let’s put the EIGRP on together on Foxtrot13.

Huh?.. Wait… WHAT?!  Not only did my EIGRP go up and then go down…. but my tunnel also went down. Okay.. so …

• Tunnel11 was up/up and solid
• Configure both routers for EIGRP to neighbor up over tunnel11
• EIGRP neighbor goes up over Tunnel11
• Seconds later Tunnel11 goes down/down
• EIGRP (obviously) then goes down

But look at that other error that was reported — “Tunnel temporarily disabled due to recursive routing.”

For those of you who have been “bit” by this one you already know exactly what is happening.  Let’s step it through to see if I can help rope in the rest of you.

1) Here we see 2 things

• the interface Tunnel11 goes up at timestamp 21:41:30.77
• less than 2 seconds later (21:41:32.32) we see the EIGRP neighbor come up over the Tunnel

2) Foxtrot13 is now learning the IP address (10.4.14.2) used as its destination IP address for the tunnel via EIGRP over the tunnel.  As you can guess… this is just NOT going to work.

3) Not surprisingly we see that at timestamp 21:41:40.777 (just 8 seconds after the EIGRP neighbor came up) we get that error message telling us that “Tunnel11 temporarily disabled due to recursive routing”.  Followed by Tunnel11 getting torn down… which… of course… tears down the EIGRP neighbor going over it.

4) With the EIGRP neighbor torn down we now see the OSPF route to get to 10.4.14.2 is in the RIB (routing information base).  In a few seconds more… if we hung around… the entire process would repeat all over again.  Flap…. flap… flap.

Front-Door VRFs

Let’s say that —

• Foxtrot13 is a router of yours at a Branch
• Echo2 and Echo4 are SP routers
• Foxtrot14 is a router of yours at your Hub Site
• Tunnel over the SP is what you are going to use exclusively for your routing

This then makes the physical IPs and routing underneath just a “Pipe” the GRE Tunnel uses.  So let’s treat it as a pipe.  🙂   What do I mean?

• On our routers (Foxtrot13 and Foxtrot14) let’s silo the WAN physical interfaces towards the service providers off into their own VRF (virtual routing forwarding instance).
• Leave the tunnel interfaces on our routers in the global (default) routing table.
• Enter a single command on the tunnel 11 interfaces on Foxtrot13 and Foxtrot14 to let them know the physical underneath piping to get us to the other end of the tunnel is in a VRF.

The below 4 steps are how I converted my lab environment over to a front door VRF.  We don’t need to keep “like for like” but to keep things consistent we will.  We will keep the underlying OSPF.

1. Create a VRF named “Pipe” on both Foxtrot13 and Foxtrot14
2. Put the “WAN” physicals on both Foxtrot13 and Foxtrot14 into this VRF
3. Modify the OSPF in Foxtrot13 and Foxtrot14 to be in VRF Pipe
4. Modify the Tunnel interface to “stitch” the tunnel to the front door VRF

#1: Create a VRF Named “Pipe”

We configure the above in both Foxtrot13 and Foxtrot14. Done.

#2: Configure “WAN” Physicals on Foxtrot13 and Foxtrot14 to be in VRF Pipe

Be careful here.  You can see that when I go to configure gig0/0/3 to be in VRF Pipe… my existing IPv4 address is removed from the interface.  Just an FYI here.

So not only will you need to configure the WAN physicals on both Foxtrot13 and Foxtrot14 to be in VRF Pipe… you will also need to enter the WAN physical IP addresses again.

#3: Modify the OSPF in Foxtrot13 and Foxtrot14 to be in VRF Pipe

This is probably not going to be as simple for you as it was for me in the lab. Since (as you can see from my diagram) I have nothing to the “left” of Foxtrot13 and nothing to the “right” of Foxtrot14… my OSPF on those routers is ONLY running on the “WAN” interfaces.  So for me I just blew away my previous “router ospf 100″ and created a new one tied to vrf Pipe.

PREVIOUS

router ospf 100
router-id 13.13.13.13

REMOVE

no router ospf 100

NEW

router ospf 100 vrf Pipe
router-id 13.13.13.13

#4:  Modify the Tunnel interface to “stitch” the tunnel to the front door VRF

As you can see we did NOT move the Tunnel11 interface from the global routing table to the routing table for VRF Pipe.  All we did is “stitch” them together.  Via one command “tunnel vrf Pipe” we “told” the router to look for the underlying “piping” not in the global routing table but in the VRF named “Pipe”

Putting It All Together

The WAN physical interfaces and the OSPF neighbors and any routing table built out of the OSPF advertisements will ALL remain “removed” and off in their own little “silo’d” world.

It is the single command “tunnel vrf Pipe” under the tunnel11 interface that makes all the magic happen.

What magic? 🙂

The magic of the tunnel being up, functional, and with an EIGRP neighbor across it despite the fact that the routing instance (global/default) that the tunnel11 interface lives in does NOT know how to get to the tunnel end point.

And that ladies and gentlemen is the absolute beauty of front-door VRFs!  Protects your network and routing over the tunnel from the “transit” you are using underneath.  I am now a HUGE fan of this for DMVPN environments!

Hope you had great fun again playing in the lab with me.  See ya next time!

Addl Note:

Darren O’Conner (author of Day One: MPLS for Enterprise Engineers) let me know that Juniper also has a similar feature for tunneled environments. He mentioned this design also being good for security – see tweet below.

*This blog post was updated from the original blog post.

Categories: DMVPN, Fun in the Lab, IWAN

Tags: DMVPN, VRFs