cisco bug has finally posted

Cisco TAC is still looking and researching my case but  the earlier bug has now posted;

https://tools.cisco.com/bugsearch/bug/CSCus95063

NOTE: I was surprise to see that we still have no workaround. 

One  thing to point out as a plus for cisco TAC, bugs are immediately made available to the public knowledge via the bugwatcher. They don't hidden things from the General Public as soon as it's been repeated and vetted in-house.

Checkpoint, Fortinet, and Juniper don't have anything  that closely emulated the  cisco bugwatch. The big search tool is pretty detailed in selections and researching bugs.


The only bad thing, you need an active support contract in-order to search  using the tool.

Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \

I found out my IPS reloading on a ASA 5558-X is a normal issue, CISCO has 2 bugs pertaining to this action and pretty much they tell you pretty much just to ignore it ( CSCub28854 / CSCts98836 )

http://www.cisco.com/c/en/us/support/docs/security/ips-sensor-software-version-71/116099-productqanda-ips-00.html

 

Okay so now we are still investigating why card in slot#1 sometimes drops all  interfaces and the 2  x RMA linecards from cisco TAC are NOT recognized in my  ASA 5558-X.

Stay tuned

Ken

Year of the Goat 2015

Okay have a Happy Chinese new year. Not sure if it's a goat or sheep but either way;



baa....Baaaa......Baaaaa.......baaaaaa

Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \

xz to the extreme

If you recall the post about xz vrs bzip2/gzip, xz has one more item  up it's bag of tricks. The  -e options allows for extreme tightness of your compressed file.

Now how much you save will depends on the file type that your compressing. Here's a ls -lR on my macbook.  The file1 used  xz -9 and file2 used  xz -e

So you will need to determine if you can give up some time for cpu-process and gain a few more %s with your compression ratio for the data to compressed.

http://socpuppet.blogspot.com/2015/01/bzip2-vrs-xz-should-we-be-using-it.html



Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  +  + )=
       o 
     /  \

ASA 9.3.2 memory resources issues

My problem with the ASA and memory utilization resulted in a bug  "CSCus95063" which hasn't posted yet  on cisco bugwatch. 

https://tools.cisco.com/bugsearch/bug/CSCus95063

Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \

Ikev2 and why we should be using it

I'm going to discuss my thoughts on IKE version2 and the benefits of using it.  

1st ,  what is IKE?


IKE  Internet Key Exchange, is one first building block for IPSEC vpns. It's  allows vpn  peers to authenticate and negotiate  security-association for encrypting data.

IKEv2 is support  by most modern ipsec vpn gateways.  The following vendors has support for IKEv2;

1.    cisco
2.    juniper
3.    fortinet
4.    sonicwall
5.    checkpoint
6.    openstrong
7.    pfsense
8.    others

2nd , IKE advantages ?

IKEv2 has host of benefits over the older IKEv1.

• resistances to IKE protocol DoS attacks, where IKEv1 was more prone & exposed to these attacks
• support NAT-T directly
• more secured and quicker SAs setup
• support for SCTP
• support active ACKs and Replies between peers
• dual or uni direction authentication parameters

3rd , A  few  IKE vpn-clients?

• forticlient
• microsoft
• shrewnet ( has not been confirm )
• green bow

   

Ken Felix
Freelance Network/Security Engineer
kfelix  -----a----t---- socpuppets ---dot---com

    ^    ^
=(  !   ! )=
      @
      /   \

uRPF cisco ASR

Unicast Verifications on IOS-XR is quite simple to deploy and to verify. Here's the simple command to deploy loose mode  uRPFs checks

And to verify the status you can use the following show command;

 

To verify that  packets are being matched and dropped you can use the following command;

show cef interface <interface name>   rpf-statistics

Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \

Ipv6 bgp table

Still small and growing as a snail rate;


routes@socpuppets.net> show route protocol bgp table inet6.0

inet6.0: 21117 destinations, 314572 routes (21117 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

This year in 2015 we should see more ipv6 networks becoming active as more migrations and new networks are populated over ipv6 vrs ipv4.

Also keep in the back of your mind that the  ipv4 table is way over half-million routes so you might need to adjust your l3 profiles for ipv4-unicast routes received. Smaller platforms with less memory are more effected.

In IOS-XR you need to be in admin and config t and  you can adjust the scale;

hw-module profile scale

hw-module profile scale ?
  default  Default scale profile
  l3       L3 scale profile
  l3xl     L3 XL scale profile

You can monitor  the  usages by using the show cef comamnds;

e.g


show cef summary location 0/0/CPU0 

show cef resource location

and

show cef platform resource summary location 0/0/CPU0

Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \

Problems with ping/ssh allowaccess secondary-ip fortigate

I was doing some investigations with a FGT110C and why  allowaccess is broken. The device is out of contract and runs the 4.3.18 build. Check this  out;

Port2 is configured with a secondary address only;

FGT110C # show sys interface port2
config system interface
    edit "port2"
        set vdom "root"
        set type physical
        set secondary-IP enable
            config secondaryip
                edit 1
                    set ip 1.0.0.1 255.255.255.252
                    set allowaccess ping ssh
                next
            end
    next
end


We can ping out of this interface with no problems.

But inbound pings or ssh access is broke. Take a look at  this diagnostic  flow for icmp and ssh;

FGT110C # get sys status | grep Vers
Version: Fortigate-110C v4.0,build0689,140731 (MR3 Patch 18)
Release Version Information: MR3 Patch 18


So I tried the same setup under FortIOS5.2.2  running under a  FGT60D;

Interesting so it seems like a problem in 4.3.18.


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  $  #  )=
        o 
       /  \

9.3.2 woes ( scp copy erros and memory issues )

Will the cisco ASA software 9.3.2 has been surprising okay,  but when it has problems it has problems;

1st up  scp copy a file does not complete, but the status shows it was copied. But the target has no file whatsoever.

NOTE: ftp attempts from off the unit completely failed 

2nd,  another  ASA 5558-X decide to start dropping pings and traffics and then I notice in the logs the following errors;

I've been issuing a lot of { show tech-support  file disk0:MYDUMP_FILENAMEt  detail } commands lately. This is no the norm for the cisco ASA .

For now my  boot variables are ALL going back to the following;

BOOT variable = disk0:/asa931-smp-k8.bin;disk0:/asa922-4-smp-k8.bin
Current BOOT variable = disk0:/asa931-smp-k8.bin;disk0:/asa922-4-smp-k8.bin
CONFIG_FILE variable =
Current CONFIG_FILE variable =

Ken Felix
Freelance Network/Security Engineer
kfelix  -----a----t---- socpuppets ---dot---com

    ^    ^
=( #  # )=
      @

The ruination for cisco ASA5558-X again with sot#1

My slot #1 for the  ASA 5558-X firewall, what a disaster!

Cisco sent me the new card. This encompass the  IPS and upper ports known as Gi1/X  and TenGige1/X

A power down  insertion of the new card did not come up upon reboot. In fact a lot of things did not come up from; the failover link, interfaces and detection for failover link by the failover standby peer

So back to cisco  & why my IPS-module reload and why  upper ports on  this chassis  goes down  with no warning and requires a reboot or power down/reset for slot#1. Also the million down  mystery ; "is it by design that shutdown of module one is suppose to shutdown the ports on that card " my case engineer is investigating that.

To refresh, read the following;

http://socpuppet.blogspot.com/2015/02/cisco-asa-5558-x-slot0-and-slot1-beware.html
http://socpuppet.blogspot.com/2015/01/asa-ips-modules-reloads-732-e4.html

I'm sure ciscoTAC will make it all right  in the end. Friends shouldn't  let friends buy a cisco ASA

Ken Felix
Freelance Network/Security Engineer
kfelix  -----a----t---- socpuppets ---dot---com

    ^    ^
=( #  # )=
      @
      /   \

Finding SerialNumber ASR IOS-XR

One more tip for finding the chassis  serial number on a ASR9K. When the router boots up, the console will briefly show the chassis serial number.

e.g

http://socpuppet.blogspot.com/2014/10/heres-new-thing-i-found-out-playing.html


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

     ^      ^
=(  @  @ )=
         o 
        /  \

SSH v2 security cisco devices ( ASA / ROUTER / SWITCH )

With securing SSH servers on cisco devices, it's ideal to use SSHv2 protocol. Most software versions support SSHv2 by default, while others are bi-mode supporting
version 1 and 2 & at the same time

1:
With in cisco ASA you can test the support of  sshv1  by setting the client to  use SSHv1 and you can disable SSHv1 via config t ; ssh version 2

2:
 
With in cisco IOS routers you can test the support of  sshv1  by setting the client to  use SSHv1 and you can disable SSHv1 via config t ; ip ssh version 2

3:
With in cisco IOS-XR routers you can test the support of  sshv1  by setting the client to  use SSHv1 and you can disable SSHv1 via config t ; ssh version 2 ; commit

4:
With in cisco NX-OS ,  " I believe SSHv2 is the only protocol supported "

It's a good time to audit your network devices and disable SSHv1, by forcing your ssh_client to use version2


Here's some screenshot of various cisco devices and ssh details


ciscoASA

cisco IOS router

cisco IOS-XR router

cisco NX-OS
( still investigating the best way  and means )


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

     ^      ^
=(  @  @ )=
         o 
        /  \

pfsense upgrade left me with a weird message

I did a upgrade for pfsense a few weeks back and was stuck with this image upon logging into the host.

 

So I did a system upgrade to  the latest version.

Everything seems to have work correctly with 2.1.5, but my hosting contact  me to tell me my "address" was listed as doing email-phlishing.  So we will see if 2.2rc have any wear issues.


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

     ^      ^
=(  @  @ )=
         o 
        /  \

diag debug flow from a fortigate ( local vrs interface )

Diag debug flow is the #1 trouble-shooting tool that should always be deployed from a fortigate. In this example,  I will show you how to determine if your diag debug flow caught packets that where generated locally from the unit


1st a simple filter

Now here's a  trace where packets crossed a inside to outside interface

Now here's a trace where the packets where generated locally ( in my a case a ping from the FGT100D device )

NOTE  Do you happen to notice the "from local" 

So yes diagnostic debug flow will show you any and all packets regardless if it crossed interfaces or are locally generated.

To learn more revisit one of my earlier threads.

http://socpuppet.blogspot.com/2013/06/diag-debug-flow-troubleshooting.html
http://socpuppet.blogspot.com/2014/08/fortigate-connectivity-diagnostic-steps.html


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

     ^      ^
=(  @  @ )=
         o 
        /  \

HOWTO: fortigate tos/dscp markup

In this post we will look at how easy it is to classified  QoS within the layer3 header of a IP datagram on a fortigate.

1st a little background, there's 8 bits allowed in ip_header  for QoS, but the 8th bit is unused. So this leaves us really with 7 bits. This 8th bit should always be "0" btw.

So in IP_Precedence  the 1st 3 bits are used for classification of traffic and setting traffic in one of the 8 precedences.

With DSCP you now have  6 bits total that can be used for classification, with 3 levels & with 4 drop-class.

So this gives you more room with fine tuning your QoS classifications and markups.
BTW: The 1st 3 bits in DSCP are class-selectors and  reflects the classes  of 1-thru-4  in the above snapshot

Now for DSCP on a fortigate, you  needs to 1st enabled it for the firewall-policy and in the direction.

e.g enabling  a dscp  value of  3F binary 111 111


Here's I'm demonstrating a DSCP value of 63 0x3F which is not a common DSCP value. And will use the diagnostic sessions to validate  my fwpolicy by id#.

If you want to  know the real values for DSCP use a cheat-sheet, similar to the following link.

 http://www.netcontractor.pl/download/QoS%20Values%20Calculator%20v3.pdf

Tip I marked off a few of the common values used everyday by VoIP solutions. 0x0 is BE ( best effort ) or simply known as the default.

Yeap, it's that easy for you to enable  DSCP on a fortigate. Most carriers will give you a QoS contract and tell you what markings it will expect and the bandwidth and prioritization for the traffic that you markup.

I've seem various QoS agreements from ATT, Paetec and Sprint,but  they all work about the same. A QoS policy could be similar to the below xls snapshot with any traffic exceeding the limits reclassified to Best Effort or drop if bandwidth is not available. Your provider should explain the terms of the QoS contract and any re-classifications.

http://en.wikipedia.org/wiki/Differentiated_services


Ken Felix
Freelance Network/Security Engineer
kfelix  -----a----t---- socpuppets ---dot---com

    ^    ^
=( #  # )=
      @
      /   \

A life of a Packet ( fortigate )

In this thread, I wanted to post a reminder of the life-of-a-packet ( by fortinet )  and what and where actions are taken in regards to a flow or connection between 2 interfaces.

In almost are firewalls, the object is to allow packets to flow across 2 interfaces regardless if the interface are L2 ( transparent mode ) or L3 ( routed aka NAT mode ) and a firewall-policy  has been configured to allow such activity aka  "accept action".

Take a look at this;

 I highlighted both DNAT and SNAT .

A DNAT  ( destination NAT ) for all practical reasons is a VIP. In  linux iptables , it's known as  pre-routing due to this action takes place before we looking into the routing information base.

Where as SNAT ( source NAT ) is always a process after we determine where/what interface to route out of ( post-routing ).


In all  cases regardless of direction, advance-security  features are applied after we found the matching policy and advance feature has been enabled per the policy. This could be a IPS sensor or URL filter,  etc....

uRPF checks is also critical since a modern firewall will drop packets that don't have a loose or strict route for the "source", but keep in mind  that unicast-routing is always determine by the "destination". A router/firewall without uRPF does not care too much about the source-address in the routing determination.




Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \

ESP replay window enabling & disable Fortigate

To  set the vpn tunnel with ESP replay checks, you need to configure the following command under your phase2 definitions.

set replay enable

What this does; "  is to set the ESP anti-replay window to a default size of 1024 bytes ". The default is for the  esp-replay to be disabled.

By using the diag vpn tunnel list commands, you can validate if the window is set.


( enabled )

( disabled )

TIP: To get an ideal of what happens when replay has taken place, use a program like tcpreplay to re-inject  capture ESP  from a packet dump  and check your vpn-ipsec logs.

Most modern firewall have a means to enable and set the size of the window, but the fortigate does not give you this option that I'm aware of.

By monitoring the  sequence numbers ( seqno= ) and using a capture techniques, you can determine if a ESP replay attack is underway.


(A example of monitoring w/ESP-seq#s &  tshark )


tshark -n -tad -i eth0 -T Fields -e esp.sequence -e frame.time 

You can take this information and place this received/sent  sequence# into a graph to discover anomalies. For all packets sent or received, the sequence number should increment by one if traffic was encrypted or decrypted.


Ken Felix
Freelance Network/Security Engineer
kfelix  -----a----t---- socpuppets ---dot---com

    ^    ^
=( #  # )=
      @
      /   \

HOWTO: Packet capture PA firewall PaloAlto

Like on a juniper SRX you can conduct packet captures within PAN-OS. I will show you how.

1st it's ideal to specify a filter  this limits the information you capture to just the traffic that you want. If your working with support or a sysadmin it's ideal to set capture filters for what your looking at.


e.g ( to look at src/dst of 192.0.2.1 192.0.0.244 )

debug dataplane packet-diag set filter match source 192.0.2.1
debug dataplane packet-diag set filter match destination 192.0.0.244
debug dataplane packet-diag set filter on 


Now you can prepare the capture;
  
debug dataplane packet-diag set capture stage drop file <filename> 
debug dataplane packet-diag set capture stage transmit file <filename> 
debug dataplane packet-diag set capture stage receive file <filename> 
debug dataplane packet-diag set capture stage firewall file <filename> 

Now you can enable the capture;

debug dataplane packet-diag   set capture on

Now you can view the name capture file or export the capture via SCP or TFTP

(view)

view-pcap follow yes filter-pcap



 
(exportation )

scp export filter-pcap from <filename> to username@<host IP>:/path 
tftp export filter-pcap from <filename> to <host IP>


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \

A peek at cisco ASA & IOS software authenticity + digital signature checks

Here's a quick means for  signature and authenticity checks in cisco ASA software.  1st to get an ideal of the running code you can execute the following cmd.

show software  authenticity  running

NOTE: you don't have to be in enable to execute this cmd

To see keys and certificate details;

As you can see, cisco implements digitally signed software on cisco routers, you have the option to verify any image running or stored within local flash

e.g ( Here's a cisco 6509E L2/3 switch  )

show software authenticity file bootdisk:<filename>

This also allows for you to verify the digital signature before loading the code









As indicated by the show outputs,  all certificates uses a 2048bit  RSA public-key. The private-key is always private.


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \

snmp tip cisco ios

I wanted to share a tip/trick for snmp gets and what oids are being hit on a cisco switch. Take this 6509 output from the following command

show snmp stats oid

The above output will show you what oids are being hit and the last time it was hit.

Great when your working with cacti/mrtg, Nagios, OpenNMS, or any other  SNMPquery tool and your looking to see if your  SNMPget/walk is being process on  a cisco gear without being in debug mode.

I believe this command is available for the cisco 6500/7600  series switches only and a few of the 3900 routers. Every time the list of oid are hit, the counters will increase & the timestamp updated


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

    ^     ^
=(  *  * )=
        o 
       /  \