L2 tunneling is a topic that has more less been haunting me for some time. Ever since I worked on IPexpert volume 1, lab 5 which is dedicated to this technology, there were some things I just didn’t understand. Mainly these things were related to loop avoidance. I’ve been having some good discussion on this recently with a fellow blogger, and have been reading up on the subject. Some of thing things in the solutions guides just didn’t seem to add up, so I decided to go in and just make something up myself to see if I could get it to work the way I thought. Here is a diagram of what I was working with:

l2protocol-tunnel

l2protocol-tunnel

I setup all the links between switches as regular old dot1q trunks.  I also set each switch to tag the native vlan with the “vlan dot1q tag native vlan” command. For my first trick, I wanted something pretty simple. I wanted R1 to see R2 in the output of show cdp neighbor. This doesn’t require you do anything on R1 or R2, only on Cat1 in this case

On Cat1:

int range fa0/1-2
no cdp enable
l2protocol tunnel cdp

after doing “clear cdp table” on R1 and R2 and waiting about 30 seconds I had success!

R1>sh cdp neigh
Capability Codes: R – Router, T – Trans Bridge, B – Source Route Bridge
S – Switch, H – Host, I – IGMP, r – Repeater

Device ID Local Intrfce Holdtme Capability Platform Port ID
R2 Eth 0/0 132 R S I 3640 Eth 0/0

R2>sh cdp neigh
Capability Codes: R – Router, T – Trans Bridge, B – Source Route Bridge
S – Switch, H – Host, I – IGMP, r – Repeater

Device ID Local Intrfce Holdtme Capability Platform Port ID
R1 Eth 0/0 127 R 2611 Eth 0/0

My next trick was to make Cat1 trunk to Cat2 over Fa0/21 by taking the path Cat1 –> Cat3 –> Cat4 –> Cat2. This was where I had gotten caught up before. I spent a good deal of time on this tonight to really understand it. I ran into an issue you will see here in a minute that cost me a lot of hair!

OK, since I would be tunneling a trunk through more than 1 switch I decided to implement Q-Q tunneling between Cat3 and Cat4, basically making Cat3 and Cat4 my “service provider” switches. So, Cat1 fa0/21 was to be a regular old dot1q trunk, nothing special. I set cat 3 fa0/21 to access vlan 12 and mode dot1q-tunnel. Fa0/23 on Cat3 and Cat4 is setup as a regular dot1q trunk as well. Fa0/21 on Cat4 was also a dot1q-tunnel port in access vlan 12 while Fa0/21 on Cat2 was a regular dot1q trunk. The basic idea here guys is that when Cat3 gets a frame from Cat1, that frame will be tagged by Cat1. Since Cat3 has access VLAN 12 and dot1q-tunnel on port fa0/21 it sends the frame over to Cat4 double tagged. The Cat1 “client” vlan is preserved inside our little Cat3/Cat4 “service provider” tag of 12. When Cat4 gets the double-tagged frame, it removes the first tag before sending the frame over to Cat2. The same thing happens the other way around from Cat2/Cat4/Cat3/Cat1.

Since we are using VLAN 12 as our tunneling VLAN we need to make sure it only exists where need be, or we can run into loops. Therefore, I manually pruned VLAN 12 from every single trunk except the Fa0/23 trunk between Cat3 and Cat4. For this trunk I also made VLAN 12 the only allowed vlan. Here are the configurations going clockwise starting with Cat1

Cat1(config-if)#do sh run | beg 0/19
interface FastEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/21
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-9,11,13-4094
switchport mode trunk
!
interface FastEthernet0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk

Cat2(config-if)#do sh run | beg 0/19
interface FastEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/21
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-9,11,13-4094
switchport mode trunk
!
interface FastEthernet0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk

Cat3(config-if)#do sh run | beg 0/19
interface FastEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/21
switchport access vlan 12
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel stp
l2protocol-tunnel vtp
no cdp enable
!
interface FastEthernet0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport trunk native vlan 12
switchport trunk allowed vlan 12
switchport mode trunk
no cdp enable
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk

Cat4(config-if)#do sh run | beg 0/19
interface FastEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/21
switchport access vlan 12
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
l2protocol-tunnel stp
l2protocol-tunnel vtp
no cdp enable
!
interface FastEthernet0/22
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport trunk native vlan 12
switchport trunk allowed vlan 12
switchport mode trunk
no cdp enable
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-11,13-4094
switchport mode trunk

Aaaaaaaaaaaaaaand…… Walla!

Cat1#sh int trunk | i Port|Fa0/21
Port Mode Encapsulation Status Native vlan
Fa0/21 on 802.1q trunking 1
Port Vlans allowed on trunk
Fa0/21 1-9,11,13-4094
Port Vlans allowed and active in management domain
Fa0/21 1,20,30,40,50
Port Vlans in spanning tree forwarding state and not pruned
Fa0/21 1,20,30,40,50

Cat1#sh cdp neigh | i Cat2
Cat2 Fas 0/21 148 S I WS-C3550- Fas 0/21
Cat2 Fas 0/24 151 S I WS-C3550- Fas 0/24
Cat2 Fas 0/23 151 S I WS-C3550- Fas 0/23

Now, if you have a very keen eye, you may have noticed that I manually pruned VLAN 10 from the fa0/21 trunks on Cat1 and Cat2. Why? This is what I was beating my head into the wall for all night! Every time I had the above setup configured, Ports fa0/21 would keep shutting down due to loop detection and going err-disabled. I couldn’t figure it out for the longest time. I had only allowed VLAN 12 on that one trunk, but still every time, err-disabled. The reason a port will go err-disabled in an l2tunnel config is because an l2tunnel port is receiving a frame from another l2tunnel port. hmmmmmmm.

Remember magic trick number 1 earlier between R1 and R2? Just for grins I went ahead and removed “l2tunnel-protocol cdp” from fa0/1-2 on Cat1 and BOOM my above configuration worked! So I got to thinking about this. Why was this happening? My theory is that R1 sent a CDP frame. The frame arrived on fa0/1 on Cat1, which is in vlan 10. That port was configured for l2tunnel-protocol cdp and also for no cdp enable. Since the switch is not participating in CDP on that port, it sees the CDP frame as just another multicast frame….so it forwards it out every port in VLAN 10. This includes our trunk over to Cat3 on fa0/21 ! According to the documentation, when a switch has a port configured for l2protocol-tunnel, and it receives a frame on that port, it changes the destination MAC address to a cisco proprietary tunneling MAC and send it on it’s way. Remember, on Cat3 fa0/21 we had it configured as an l2protocol-tunnel port also. So, Cat1 gets the CDP frame from R1, it changes the destination MAC to the cisco proprietary MAC and forwards it out every port in VLAN 10, including our trunk to Cat3. Cat3 gets the frame, sees it is destined for that proprietary L2tunnel MAC address and goes “NOOOOOOOOOOPE!!!!, I can’t receive an L2tunnel frame on an l2tunnel port SHUT ME DOWN!”
I should also note, the exact same behavior was happening between Cat2/Cat4 on the other side.

So how to fix this? I just manually pruned VLAN 10 from the trunks going between Cat1/Cat3 and Cat2/Cat4 on those l2 tunnel ports. Since we have a very redundant topology here, there are other unblocked paths along VLAN 10. I verified this by making sure R1/R2 could ping R6 down on Cat2.

Problem solved!

OK now for the last bit of insanity…something that will truly make your brain hurt if you have not seen this stuff before. OK, so we now have 3 logical links between Cat1 and Cat2 — Fa0/23, Fa0/24 and our tunneled link Fa0/21. We have 3 logical links, can we make a 3 link port-channel?! Sure, why not?!!! You can also tunnel PaGP and LACP with just one more line of configuration

Cat3/Cat4
———–

int fa0/21
l2protocol-tunnel point-to-point pagp

Cat1/Cat2
———-

int range fa0/21 , fa0/23-24
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-9,11,13-4094
switchport mode trunk
channel-group 12 mode desirable

int po12
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-9,11,13-4094
switchport mode trunk

Bam!

Cat1(config-if-range)#do sh etherchan sum | beg Group
Group Port-channel Protocol Ports
——+————-+———–+———————————————–
12 Po12(SU) PAgP Fa0/21(P) Fa0/23(P) Fa0/24(P)

Yeah I know, it’s fucking insane right? Thats part of becoming a Cisco Jedi Master I suppose 😉

I think I came a long way with l2tunneling tonight, and it feels good to finally have a better grasp on this.

– Joe A

With the economy the way it is right now this may help a lot of people out in some savings :). Good time to take advantage of offers that may come your way.

Your friends at IPexpert recognize that these are difficult economic times for so many of you. We have heard from CCIE hopefuls around the world that all say that their training budgets have tightened and travel budgets have been cut.

We understand that, although the economy has weakened, your CCIE goals remain strong… and we want to help you push forward.

Today, we are beginning to roll out “Stimulus Pricing” which will include significant discounts on IPexpert’s world-class training you need to succeed. Already, you can see all-time-low registration fees available for the popular Boot Camp courses that IPexpert is known for – courses that have helped more CCIEs pass the CCIE lab exam than any other training company in the world! Now is the time to take advantage of this special ILT pricing – and continue to push forward with your CCIE certification quest. Distinguishing yourself amongst your peers – and earning your CCIE certification is what will keep your career afloat through these challenging economic times!

Full link here

LOL a what you say? Yea I know! I have been slowly chugging away at my Soup to Nuts labs and reading material. I was going to go just take my written again soon, but decided against it. I will just take it when I feel I am ninety days out from taking the lab. No sense in taking it again now and have the clock start once more. Plus it hasn’t even expired yet.

I am trying to figure out my dates for the bootcamps as well. I am still trying for the March Chicago camp Narbik has scheduled. The class is still not close to full though. It is getting too close to crunch time for having to put in for vacation. These might become a summer thing since the wife will be off for the summer. I should of been a teacher. Why am I going through all of this when I could have summers off? I still have yet to figure out when to head off to Internetwork Expert’s 5 day camp as well. I really need to get this in and take my lab by the end of Fall this year. My wife is starting to get that look in her eyes with Jake on the verge of starting to crawl. it won’t be long until she starts crying he isn’t a baby anymore :).

Looking back I am wondering if I should have grabbed material/camp from IP Expert since they have their camps in Columbus OH which is only a 4 – 5 hour drive or so from Buffalo. As this flying will not make me happy…

I did redo my dynamips server on my mac mini a few weeks ago. Since I had a test rack at work setup for Internetwork Expert’s topology I bought more usb nics and rewired my server at home for Narbik’s topology. It is alot easier doing his labs when you don’t have to wait for hardware reloads! It runs like a champ as well. Once I redid all the usb nics to the real switches reworking the .net file was easy and off I went.

I will try to get back to these posts every week again. The site was always great motivation for me, and I could use as much as I can right now. It is either crank this stuff out or the wife will make me put siding up on the house…

From Internetworkexpert.com:

The Security section of Internetwork Expert’s CCIE Routing & Switching Lab Workbook Volume 1 Version 5.0 is completed and available on the members site. As of now the fully completed and posted sections are Bridging & Switching, Frame Relay, IP Routing, RIP, EIGRP, OSPF, QoS, Security, System Management, and IP Services. BGP, Multicast, and IPv6 remain, and will be incrementally posted next.

Read more

It’s official! No matter what the outcome, Cisco Systems now has $1400 of my hard earned money. Just wanted to make that quick note. It feels a lot more real now I suppose. I need to book my flight and hotel soon.

On another note, I am going on vacation tomorrow! I read a post on GS I recently about a guy that had passed his lab. He said a key part of his success was going on a cruise vacation with his family and relaxing for a bit. It just so happens I already had a cruise vacation planned with my family, so that works out :) For the next week, I’ll be in the warm beautiful weather relaxing with my wife, brother, parents, and sister in-law. A splendid time is guaranteed for all!

I figure after I get back I will have about 12 weeks to finish preparation for my lab. I am just finishing up IPexpert Volume 3 labs (I have 3 left to go). I really have not had the time to post detailed accounts of all of them. After I finish volume 3 I plan on redoing some labs. Also, after the volume 3 labs, I plan on dedicating a large chunk of time to just reading. Most of that will be the DocCD. I guess I plan on just starting with the routing config guide and working my way down. Maybe hitting the command references if I have time. In addition, I’ve been playing with the idea of picking up Narbik’s soup to nuts book. I have heard nothing but good things about it from the community at large. Also, if I can afford it and have the time, I’d like to book at least 1 of the 2 Cisco assessor labs, and maybe a mock lab or a handful from another refutable vendor.

– Joe A.

Send a congrats over to him on Ethan’s site!