Saturday, August 31, 2013

ASA capturing packets

Part of our firewall engineer diagnostic duties are; " to trouble-shoot". This requires any one of the following activities;

  • execute show commands
  • debug  
  • monitor logs ( syslog , show log , grep............)
  • or on ocassion, we do a  packet capture

In my day to day duties, I'm typically doing any or all the four above, & when trouble-shooting issues.

On the ASA with the newer code, it's very simple to conduct a packet diagnostics. I will walk you thru a typical packet capture episode

1: Build a access-list to match on just traffic of interest  
( very important if you have a busy link, don't try to capture all traffic, you might missed the traffic of interest and  waste memory space & time....... use a ACL )

!!!   BE SPECIFIC AS POSSIBLE in your ACL  !!!

access-list myacl standard permit

Will capture traffic for that host only.

2: you need to specify a capture name

3:monitor active  captures with the "show cap" cmd

4: delete any access-list and capture at the conclusion of the t-shoot event.

here's a few screen shots of a capture on within  a asa.

( validating my ACL and then applying the capture )

 ( showing active or non-active captures )

( removing captures )

( capture based on ethernet frame type no ip )

( copying a capture to disk0 for later downloading )

So now you have the option to copy the saved capture, &  to a device of your pick'ins for off appliance analysis or deliver to let's cisco TAC.

Ken Felix
Freelance Network / Security Engineer
kfelix  ----a---t---socpuppets ---d---o---t---com

     ^      ^
=(  @   @ )=
       /     \

why friends don't let friends , buy cisco ASA part#2

This is part#2,  to part#1 on why cisco licenses can be confusing at best,  & why friends don't let friends buy cisco ASAs :)

If you actually read the below link and try to follow the licenses, and restrictions that cisco has in place for the ASAs, your head will really hurt;

I just recently ran into a cisco ASA5558-X  & lack of 10gige io ports.

Guess what ?  we need a license to just enable 10gige ports.

If you try to use the ten10ge ports, the unit will tell you it's unlicensed.

It's funny that cisco wil let you configured the interfaces, but you don't have the ability to use them,  until you apply the licenses.

Ken Felix
Freelance Network/ Security Engineer

    ^       ^
=( @   @ )=
      /    \

Monday, August 12, 2013

t-mobile advancement towards ipv6 in the mobile cell provider arena

Here's a few good pieces for  information for T-Mobile customer & who has questions on ipv6 support and usage ;

Support apps that's been tested;


I reached out to a  t-mobile ipv6 engineer  (Bryne.Cameron)  awhile back on statistics in their network, and this was relayed to me.

"We dont publish data but isoc does , T-Mobile usa is listed  "

Google +Android and more ipv6 stuff ( Success and Rant )

A Google Nexus4 handset  and with it's integrated wifi-interface,  natively supports ipv6.

To double check, I did a capture on  a  fortinet FWF60D looking for my ipv6 address & it's request on the wifi interface.

( capture fortinet fortigate  firewall )

NOTE: Here you can monitor packets,  and write a pcap capture file,  that could be read back within tcpdump, tshark/wireshark.

In this screen shot; "  my phone is enable for  dual-stack operations  on the wifi interface ", so after  I found my  ipv4 address and arp  entry on my  testing macbook, I'm now able to drill-in and focus my efforts on the discovery for ipv6 address & any ND discovery.

These one-on-top screenshots; "shows my  layer2 ether_address and snippet of a analyze ipv6 packet using tshark of my ping and ping request "

I've circled the corresponding ipv6 entries,  and the ether frame mac_address.  

I than  used  tcpdump for this post  and this cmd line capture-filter, once I identified my adapter's mac_address

I could have filter out any thing non-Ipv6  by adding the <ip6>

So their you have it, we found my ipv6 phone  & it's address :)

Google, good job, we have ip6 aware interfaces for  the WiFi, but no ipv6  details in the phone displays. Just only ipv4 information.

 I'm leaving you,  with some screen-shots of how this looks & from the phone display.

One interesting tidbit. If you check around  the internet forums, you will see many discussions about if your try to engage a ipv6-only network.  The Android phone will not establish a session if using the wifi interface ONLY for ipv6 addressing. ( It needs to have a ipv4 address enabled also )

Here's my phone associated to the my Fortigate AP.

Ipv6 RouterSolicitation reflects the "33:33:00:00:00:01"  address  during  host solicitation for a ipv6 router.

And here's the 2nd SSID configured as a  ipv6-only interface on a Fortigate FWF60D running latest  5.0.4 code.

So if you should ever enter a  ipv6-only  access wifi  location, you will probably NOT get an ipv6 assignment.

Why Android OS  has this stipulation is ridiculous,  and more so when you think deep about it.

So in order to  combat this flawed logic, you can either statically install an address on the phone wifi or place a APIPA a dhcp-server or some othert bogus network on the interface serving the phone. I know this is stupid, but this is google being just ....... Google :)

A interface that's  precipitating  within a ipv6-only address space, should not have a dependency with the need for a  ipv4 address. That's the whole point of trying to move into a ipv6 realm. Maybe google missed this simple concept, &  in their promotion of ipv6 world day.

As a matter of fact, if you recall the earlier post of mine; " where we took this same google nexus 4 phone , and enabled the dual stack on the GSM cellular interface  "?

This same phone works as a ipv6-only phone ( no ipv4 support ),  when using my celluar provider network.

( notice no ipv4 addressing?  )

So why can't google allow the same principle for the wifi interface on their google Nexus4 phone ?

Also one more tidbit,  if you happen to come across a ipv6-only GSM celluar phone, T-Mobile is most likely doing some NAT64  to allow you access to the various  ipv4 portions of the internet. Check out this email header that  I harvest from a mail message sent to myself ;

T-Mobile, a good job !

Okay, I'm done with ipv6 testing on my  Android............for now. Stay tune, I will have some new things that are ipv6 related, next week.

Ken Felix
Freelance Network / Security Engineer
kfelix  ----a---t---socpuppets ---d---o---t---com

     ^      ^
=(  @   @ )=
       /     \

Thursday, August 8, 2013

bgp confederation ( cisco with fortigates peers )

In this post, we will look at a simple BGP confederations.

1st here's my topology;

FGT200A AS#2
cisco 3825 BGP confederation AS#100/65051
cisco 3825 BGP confederation AS#100/65052
FWF60D  AS#1

Okay,  so we have a AS#100  that's sub-divided into 2 unique ASes ( 65051 and 65052 ). External routes from  fortigates devices  (  FWF60D & FGT200A ) , will be populated across this bhp-confederation.

So let's  start out with the  cisco router bp configs for bgp confederations # 650051/650052

router pe1

router# pe2

Key points;

  • your router bgp "AS#" is your bgp confederation ID
  • You must define the other peer members sub-ASes, you can add multiple AS#s to this line as indicated with my type ( 650051 ).
  • You define your bgp neighbors like any legacy bgp configurations

For the  fortigate devices, we have the following;


NOTE:  For the FWF60D , we do a little of metrics additions , to validate if the bhp confederation passes any metrics across the federation and to other bgp-confed-peers or external bgp peers

A simple route-map with a prefix-list;


Similar to the  FWF60D, but we will inject 2 prefixes ( & ) and with no route-map applied.

Both Fortigates are peering to a router in AS#100.

Okay,  see how simple that was ?

This took approx all of 10mins to configure  & I spent more time putting this post together  than the configuration :)

Okay, once all bhp neighbors are up, we can review the individual  bgp tables to witness how the AS-PATH will look within this bgp confederation.

1st cisco3825  pe1

NOTE1: As you can see, our AS# is 65051 within the bgp-confederation, but anybody not identified as  bgp-confed-peer would see us as AS#100

NOTE2: We have neighborship  to a eBGP peer outside of the confederation and one within the bgp confederation

Moving on,

 Let's take a look at the 2nd  cisco router bgp table for cisco pe2;

As you can see, it  has a local learned router from a eBGP peer at ( FWF60D ) and a neighbor ship to another bhp-confederation peer from AS#65051 ( router1 c3825 ). This neighbor passed the prefixes 192.0.2.{1-2} from AS#2 ( FGT200A )

I want to stop here and point out the obvious;

  • bgp confederation peer <AS_PATH>  information is always reflected as  ( AS##### ) and these sub-AS# are dropped when the path is advertised outside any of the bgp-confederation-peers
  • The 2x fortinet devices has no clue they are peering to bgp-confederations members, nor the size of the confederation, since as_path is drop for the outside AS#100. This confederation could have been 4 sub-AS deep, we would never know outside of the sub-ASes
  • The  metric of #666 is being carried within the bgp-confederation and amongst it's  members, but dropped outside of the confederation and parent AS100

So let's see what the  fortigates BGP tables show?


Let's stop and look at this closely. The FWF60D prefix that was advertised on the other side.

It came across from  AS_PATH   ^1_100_2$     --> into the  FGT200A  (AS2).

The bgp-metric was dropped, as normal expected behavior for any intra-as-metric ( remember the earlier post, "metrics are not carried outside of a  AS"  and is considered a optional / non-transitive path_attribute ).

My FGT200A is peer'd directly to ( cisco 3825  aka pe2 ) ,  but that router is really configured in sub-AS#65052. The bgp-confederation, uses it's  bgp-confederation-identifier for a non-BGP confederation peers.

In closing, BGP confederations is just an alternate strategy for breaking up a hug he AS into smaller subset. This allows for smaller full meshing of iBGP peers, and scale much better as your network grows in size and number of iBGP peers. Trying to maintain  more than 12-20 iBGP in a fullmesh or with route-reflectors, get's to be a major challenge & does not scale well.

If you didn't know;  "BGP has a rule that all  bgp routers in a AS, must be fully meshed or connected to a route-reflector(s) that are fully-meshed or use a bgp-confederation as alternative approach".

( quote )

RFC 5065                                                     August 2007

1. Introduction

As originally defined, BGP requires that all BGP speakers within a single AS must be fully meshed. The result is that for n BGP speakers within an AS, n*(n-1)/2 unique Internal BGP (IBGP) sessions are required. This "full mesh" requirement clearly does not scale when there are a large number of IBGP speakers within the autonomous system, as is common in many networks today.


In my example, AS#100 was broken into 2 sub-ASes, but in a true servivce provider network, they  might have a few dozen or more sub-ASes and  any where from 10, 100  or more bgp-routers installed.

The path in this case, was carried over 2 sub-ASes ( 65051/65052 ). The fortigates, both eBGP peers and non-confederation peers, see them as AS100. BGP confederation in this blog example, was overkill and I would not even consider using a BGP-confederation, unless you had  more than 10+ routers.

As a  interesting topic, almost all commercial routers has support for BGP confederations. Even the fortigate firewall supports bhp-confederations;

The size of the sub-ASes peers, has to be designed around the memory,cpu and  network topology being deployed. And we usually uses private-ASes in the 64512-65535 range. You have to be careful of this range to not overlap with any private-as that's might be used by a client. I myself like to use AS numbers closer to  the bottom of this range ( i.e  65412-65499 ) this give you plenty of room and almost never collide with any other private-as usage YMMV

Most ISPs that deploys bgp-confederations, tries to keep the sub-ASes within the same regional area. And run fully meshed links to other subASes.

Take this example of a private bgp network that I worked on back in the early 2000s using junipers. They had a design based similar to this diagram and layout across the US. It scaled well and had very little problems. Management of any part of the confederation was straight forward and simple. Each color semi-circle would be a sub-AS.

I hope this has been helpful for understanding the basic of building bgp confederations.

Ken Felix
Freelance Network / Security Engineer
kfelix  ----a---t---socpuppets ---d---o---t---com

     ^      ^
=(  @   @ )=
       /     \

Forward-Address not equal = ( the why? )

I've been helping  trouble-shooting some  weird ospf E1 external networks issues over the last 2 days, and found something interesting that I would like to share.


after identifying the problem  of  additional metric-cost for type Ext1 ospf routes. I started scratching my head , & checked to see if cisco had anything posted and found this article. This link explain a little bit of the same thing that I encountered with a client of mine.

( i'm highlighting the following from the above link  )

Note: Router 3 and Router 4 do not include network in the OSPF process; therefore, the type 5 LSAs generated by both routers have the forwarding addresses set to

Keep the above in mind as you continue to read this post.

The normal behavior for any ospf redistributed routes, is to install these as External-type2 . By default the asbr that originates the redistribution does  this regardless if it was statical or from another dynamic routing protocol ( i.e BGP, EIGRP, rip, etc,)

These routes will be listed in the ospf database  ( external ) with the forward-address set to if NO network statement matches the next-hop of the redistributed routes within the "router ospf" .


If you have any network statements that matches the next-hop of the route to be installed within that  ASBR, it will populate the Forward Address entry  &  with that value of the next-hop and your metric is now computed as following ;

redistribute-metric-cost  + cost-2-the-forward_address + link-cost for external-type 1.

okay let demonstrate this concept, once you see it executed it would be quite clear. Keep in mind, this is more of an issues with E1 type routes. Without knowing the above, you could be chasing your tail around in circles with regards to weird metric or your routing not as optimized as what you wanted or expected.

Here's my  OSPF topology;

R1+R2 are receiving the redistributed route via Ospf As a E1 route. All three routers are in area

R3 configuration:

R2 configuration:

R1 configuration:

Let's look at the ospf database as it stands now & for  the prefix.

1st up:

R1 & R2

Notice the  Forward Address   = ?


If we change the  configuration on R3 by adding a network statement that matches the  next-hop of, than the forward address will be  changed in our ospf database;

Here's R3 new configuration ( nothing was changed on R1 or R2 btw  )

And let's take a new  look at the both ospf database on r1 and r2.

Notice how we now have a Forward_Address that's not & equals the next-hop address ? The ospf metric will now reflect this change as well.

B4 we modified the network statements here's how our ospf route table looked for the;

show ip route

After the network statement was entered;

show ip route

The over all end2end metric was increased because of the forward_address included in the cost analysis.

Keep these thoughts in mind when using E1 types in your ospf redistributions ?

  • This is mainly a big issues if you use E1 vrs E2 external ospf routes

  • The Forward Address,  is changed if you used E2 ( the default redistribution type ) and the same for E1, if you have a network statement entry that matches the next-hop

  • The Froward Address cost is NOT calculated in E2 type of redistributions

NOTE: This diagnostics was kinda of interesting, and similar to what you will find in a CCIE R/S lab. So if your dusty in your OSPF skills, you might want  lab some examples or explore  OSPF  and with redistributions methods and concepts.

For reference, I'm going to show the difference of the ospf database and route table once we redistribute  the same  static route, but this time as an External-Type 2.

R3 new configuration ( nothing was changed in R1 or R2 configurations ):

( btw default redistribution type = E2 )

R1 & R2  ospf database  + route table ;

( with the network statement under  my router ospf  )

R1 & R2  ospf database  + route table ;

( without  the network statement  on R3)

I underlined a something in the last displayed snapshot from above.

  • When you see the "forward metric" in any  external ospf route display, that means it's a E2 route and that would be the cost to the ASBR, which is not included in the route ospf metric 
  • It's what's not added to the final metric cost for the Type5 LSA ( external summary )  only 

I love messing around with OSPF. It's an strange and not overly complex routing protocol, but the behavior is predictable if you understand it. Outside of NSSA, ospf is very simple to use and once you take a few stabs at design and deployment, it's rather easy to trouble-shoot.

I've seen a lot of IT staffer struggle with fixing OSPF and 10 out of 10 times, it because they didn't have an handle on the issue(s) and used the wrong t-shoot logic

I hope this blog helps you out with any ospf redistribution issues

Ken Felix
Freelance Network / Security Engineer
kfelix  ----a---t---socpuppets ---d---o---t---com

     ^      ^
=(  @   @ )=
       /     \