Warning: count(): Parameter must be an array or an object that implements Countable in /homepages/29/d134578137/htdocs/wp-includes/post-template.php on line 284

L2 tunneling is a topic that has more less been haunting me for some time. Ever since I worked on IPexpert volume 1, lab 5 which is dedicated to this technology, there were some things I just didn’t understand. Mainly these things were related to loop avoidance. I’ve been having some good discussion on this recently with a fellow blogger, and have been reading up on the subject. Some of thing things in the solutions guides just didn’t seem to add up, so I decided to go in and just make something up myself to see if I could get it to work the way I thought. Here is a diagram of what I was working with:

l2protocol-tunnel

l2protocol-tunnel

I setup all the links between switches as regular old dot1q trunks.  I also set each switch to tag the native vlan with the “vlan dot1q tag native vlan” command. For my first trick, I wanted something pretty simple. I wanted R1 to see R2 in the output of show cdp neighbor. This doesn’t require you do anything on R1 or R2, only on Cat1 in this case

On Cat1:

int range fa0/1-2
no cdp enable
l2protocol tunnel cdp

after doing “clear cdp table” on R1 and R2 and waiting about 30 seconds I had success!

R1>sh cdp neigh
Capability Codes: R – Router, T – Trans Bridge, B – Source Route Bridge
S – Switch, H – Host, I – IGMP, r – Repeater

Device ID Local Intrfce Holdtme Capability Platform Port ID
R2 Eth 0/0 132 R S I 3640 Eth 0/0

R2>sh cdp neigh
Capability Codes: R – Router, T – Trans Bridge, B – Source Route Bridge
S – Switch, H – Host, I – IGMP, r – Repeater

Device ID Local Intrfce Holdtme Capability Platform Port ID
R1 Eth 0/0 127 R 2611 Eth 0/0

My next trick was to make Cat1 trunk to Cat2 over Fa0/21 by taking the path Cat1 –> Cat3 –> Cat4 –> Cat2. This was where I had gotten caught up before. I spent a good deal of time on this tonight to really understand it. I ran into an issue you will see here in a minute that cost me a lot of hair!

OK, since I would be tunneling a trunk through more than 1 switch I decided to implement Q-Q tunneling between Cat3 and Cat4, basically making Cat3 and Cat4 my “service provider” switches. So, Cat1 fa0/21 was to be a regular old dot1q trunk, nothing special. I set cat 3 fa0/21 to access vlan 12 and mode dot1q-tunnel. Fa0/23 on Cat3 and Cat4 is setup as a regular dot1q trunk as well. Fa0/21 on Cat4 was also a dot1q-tunnel port in access vlan 12 while Fa0/21 on Cat2 was a regular dot1q trunk. The basic idea here guys is that when Cat3 gets a frame from Cat1, that frame will be tagged by Cat1. Since Cat3 has access VLAN 12 and dot1q-tunnel on port fa0/21 it sends the frame over to Cat4 double tagged. The Cat1 “client” vlan is preserved inside our little Cat3/Cat4 “service provider” tag of 12. When Cat4 gets the double-tagged frame, it removes the first tag before sending the frame over to Cat2. The same thing happens the other way around from Cat2/Cat4/Cat3/Cat1.

Since we are using VLAN 12 as our tunneling VLAN we need to make sure it only exists where need be, or we can run into loops. Therefore, I manually pruned VLAN 12 from every single trunk except the Fa0/23 trunk between Cat3 and Cat4. For this trunk I also made VLAN 12 the only allowed vlan. Here are the configurations going clockwise starting with Cat1

Cat1(config-if)#do sh run | beg 0/19
interface FastEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/21
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-9,11,13-4094
switchport mode trunk
!
interface FastEthernet0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk

Cat2(config-if)#do sh run | beg 0/19
interface FastEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/21
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-9,11,13-4094
switchport mode trunk
!
interface FastEthernet0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk

Cat3(config-if)#do sh run | beg 0/19
interface FastEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/21
switchport access vlan 12
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel stp
l2protocol-tunnel vtp
no cdp enable
!
interface FastEthernet0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport trunk native vlan 12
switchport trunk allowed vlan 12
switchport mode trunk
no cdp enable
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk

Cat4(config-if)#do sh run | beg 0/19
interface FastEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/21
switchport access vlan 12
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel stp
l2protocol-tunnel vtp
no cdp enable
!
interface FastEthernet0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport trunk native vlan 12
switchport trunk allowed vlan 12
switchport mode trunk
no cdp enable
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk

Aaaaaaaaaaaaaaand…… Walla!

Cat1#sh int trunk | i Port|Fa0/21
Port Mode Encapsulation Status Native vlan
Fa0/21 on 802.1q trunking 1
Port Vlans allowed on trunk
Fa0/21 1-9,11,13-4094
Port Vlans allowed and active in management domain
Fa0/21 1,20,30,40,50
Port Vlans in spanning tree forwarding state and not pruned
Fa0/21 1,20,30,40,50

Cat1#sh cdp neigh | i Cat2
Cat2 Fas 0/21 148 S I WS-C3550- Fas 0/21
Cat2 Fas 0/24 151 S I WS-C3550- Fas 0/24
Cat2 Fas 0/23 151 S I WS-C3550- Fas 0/23

Now, if you have a very keen eye, you may have noticed that I manually pruned VLAN 10 from the fa0/21 trunks on Cat1 and Cat2. Why? This is what I was beating my head into the wall for all night! Every time I had the above setup configured, Ports fa0/21 would keep shutting down due to loop detection and going err-disabled. I couldn’t figure it out for the longest time. I had only allowed VLAN 12 on that one trunk, but still every time, err-disabled. The reason a port will go err-disabled in an l2tunnel config is because an l2tunnel port is receiving a frame from another l2tunnel port. hmmmmmmm.

Remember magic trick number 1 earlier between R1 and R2? Just for grins I went ahead and removed “l2tunnel-protocol cdp” from fa0/1-2 on Cat1 and BOOM my above configuration worked! So I got to thinking about this. Why was this happening? My theory is that R1 sent a CDP frame. The frame arrived on fa0/1 on Cat1, which is in vlan 10. That port was configured for l2tunnel-protocol cdp and also for no cdp enable. Since the switch is not participating in CDP on that port, it sees the CDP frame as just another multicast frame….so it forwards it out every port in VLAN 10. This includes our trunk over to Cat3 on fa0/21 ! According to the documentation, when a switch has a port configured for l2protocol-tunnel, and it receives a frame on that port, it changes the destination MAC address to a cisco proprietary tunneling MAC and send it on it’s way. Remember, on Cat3 fa0/21 we had it configured as an l2protocol-tunnel port also. So, Cat1 gets the CDP frame from R1, it changes the destination MAC to the cisco proprietary MAC and forwards it out every port in VLAN 10, including our trunk to Cat3. Cat3 gets the frame, sees it is destined for that proprietary L2tunnel MAC address and goes “NOOOOOOOOOOPE!!!!, I can’t receive an L2tunnel frame on an l2tunnel port SHUT ME DOWN!”
I should also note, the exact same behavior was happening between Cat2/Cat4 on the other side.

So how to fix this? I just manually pruned VLAN 10 from the trunks going between Cat1/Cat3 and Cat2/Cat4 on those l2 tunnel ports. Since we have a very redundant topology here, there are other unblocked paths along VLAN 10. I verified this by making sure R1/R2 could ping R6 down on Cat2.

Problem solved!

OK now for the last bit of insanity…something that will truly make your brain hurt if you have not seen this stuff before. OK, so we now have 3 logical links between Cat1 and Cat2 — Fa0/23, Fa0/24 and our tunneled link Fa0/21. We have 3 logical links, can we make a 3 link port-channel?! Sure, why not?!!! You can also tunnel PaGP and LACP with just one more line of configuration

Cat3/Cat4
———–

int fa0/21
l2protocol-tunnel point-to-point pagp

Cat1/Cat2
———-

int range fa0/21 , fa0/23-24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-9,11,13-4094
switchport mode trunk
channel-group 12 mode desirable

int po12
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-9,11,13-4094
switchport mode trunk

Bam!

Cat1(config-if-range)#do sh etherchan sum | beg Group
Group Port-channel Protocol Ports
——+————-+———–+———————————————–
12 Po12(SU) PAgP Fa0/21(P) Fa0/23(P) Fa0/24(P)

Yeah I know, it’s fucking insane right? Thats part of becoming a Cisco Jedi Master I suppose 😉

I think I came a long way with l2tunneling tonight, and it feels good to finally have a better grasp on this.

– Joe A

Comments

2 Responses to “Tutorial: Mucking About With L2 Tunneling”

  1. Hobbs on February 24th, 2009 5:38 pm

    right on man, you are a tunneling guru now!

  2. jstout on November 23rd, 2011 5:35 am

    randomly stumbled upon this… this just helped clear up a lot about this that cisco’s documentation lacked… feel good knowing you helped someone else in their CCIE journey 😀

Leave a Reply