Showing posts sorted by relevance for query diag debug flow. Sort by date Show all posts
Showing posts sorted by relevance for query diag debug flow. Sort by date Show all posts

Wednesday, March 20, 2013

Flow Diagnostic (fortigate)

-->
In these next series of posts, I will go over some basic diagnostic  methods for netscreen, fortigate and cisco ASA. As firewall admins, we need to know how to aid in t-shooting. Blindly changing rules, rebooting, and host of other steps are typically a hail mary and accomplish nothing.


1st up,

Flow diagnostic  on the Fortigates series are simple. 


Why we do this is mainly to trouble-shoot L3/L4 fwpolicies rules. And to validate what’s being hit and the policy-id involved, any nat'ing or if encryption is taking place.



First  check the debug  options , too see if any one left something running or to clear any post debugs sessions;


SomeFWFGT01 # diag debug info

debug output:                       disable

console timestamp:   disable

console no user log message:           disable

CLI debug level:        3


if anything is set you might want to clear it;


diag debug reset



2nd, we what to set up debug to be enabled before we start;



SomeFWFGT01 # diag debug enable



SomeFWFGT01 #


Third, we want to set up the flow diagnostic and similar to my flow diag with netscreens,  we have  filters options in the FortiOS;



SomeFWFGT01 # diag debug flow

filter     trace packet with filter

filter6    trace packet with ipv6 filter

show       whether to display trace on console

trace      start/stop trace



SomeFWFGT01 # diag debug flow filter

addr      ip address

clear     clear filter

daddr     dest ip address

dport     destination port

negate    inverse filter

port      port

proto     protocol number

saddr     source ip address

sport     source port

vd        index of virtual domain



note: filter6 are for ipv6 filters

Here I’m going to be very specific,  and place a flow filter for the google dns server
8.8.8.8 and port 53

 diag debug flow filter daddr 8.8.8.8
diag debug flow filter dport 53


Third, Now that we have the flow filter-set. We need to tell the firewall to display the traces on the console




SomeFWFGT01 # diag debug flow show console enable

show trace messages on console

And lastly, we start the trace;



SomeFWFGT01 # diag debug flow  trace start 1000


 NOTE: The above says to start , and display the  1st 1K traces


Now when interesting traffic matches our filter we will get a trace;

SomeFWFGT01 # id=36871 trace_id=217 msg="vd-root received a packet(proto=17, 10.10.100.9:46883->8.8.8.8:53) from WEB_NET01."
id=36871 trace_id=217 msg="allocate a new session-301981cb"
id=36871 trace_id=217 msg="find a route: gw-192.0.2.1 via EXT_NET01"
id=36871 trace_id=217 msg="find SNAT: IP-192.0.2.2, port-30639"
id=36871 trace_id=217 msg="Allowed by Policy-10: SNAT"
id=36871 trace_id=217 msg="SNAT 10.10.100.9->192.0.2.2:30639"
id=36871 trace_id=217 msg="run helper-dns-udp(dir=original)"
id=36871 trace_id=218 msg="vd-root received a packet(proto=17, 10.10.100.9:52631->8.8.8.8:53) from WEB_NET01."
id=36871 trace_id=218 msg="allocate a new session-30198221"
id=36871 trace_id=218 msg="find a route: gw-192.0.2.1 via EXT_NET01"
id=36871 trace_id=218 msg="find SNAT: IP-192.0.2.2, port-40219"
id=36871 trace_id=218 msg="Allowed by Policy-10: SNAT"
id=36871 trace_id=218 msg="SNAT 10.10.100.9->192.0.2.2:40219"
id=36871 trace_id=218 msg="run helper-dns-udp(dir=original)"
id=36871 trace_id=219 msg="vd-root received a packet(proto=17, 10.10.100.9:51360->8.8.8.8:53) from WEB_NET01."
id=36871 trace_id=219 msg="allocate a new session-3019827d"
id=36871 trace_id=219 msg="find a route: gw-192.0.2.1 via EXT_NET01"
id=36871 trace_id=219 msg="find SNAT: IP-192.0.2.2, port-26668"
id=36871 trace_id=219 msg="Allowed by Policy-10: SNAT"
id=36871 trace_id=219 msg="SNAT 10.10.100.9->192.0.2.2:26668"
id=36871 trace_id=219 msg="run helper-dns-udp(dir=original)"
id=36871 trace_id=220 msg="vd-root received a packet(proto=17, 10.10.100.9:45227->8.8.8.8:53) from WEB_NET01."
id=36871 trace_id=220 msg="allocate a new session-3019ab4d"
id=36871 trace_id=220 msg="find a route: gw-192.0.2.1 via EXT_NET01"
id=36871 trace_id=220 msg="find SNAT: IP-192.0.2.2, port-45095"
id=36871 trace_id=220 msg="Allowed by Policy-10: SNAT"
id=36871 trace_id=220 msg="SNAT 10.10.100.9->192.0.2.2:45095"
id=36871 trace_id=220 msg="run helper-dns-udp(dir=original)"
id=36871 trace_id=221 msg="vd-root received a packet(proto=17, 10.10.100.9:59618->8.8.8.8:53) from WEB_NET01."
id=36871 trace_id=221 msg="allocate a new session-3019ab63"
id=36871 trace_id=221 msg="find a route: gw-192.0.2.1 via EXT_NET01"
id=36871 trace_id=221 msg="find SNAT: IP-192.0.2.2, port-47214"
id=36871 trace_id=221 msg="Allowed by Policy-10: SNAT"
id=36871 trace_id=221 msg="SNAT 10.10.100.9->192.0.2.2:47214"
id=36871 trace_id=221 msg="run helper-dns-udp(dir=original)"



Each trace is marked with a trace_id, and the key take aways are the following;
  
    
  •   Interface dst/src ( directionality ) 
  •    Fwpolicy ID
  •     Session allocation
  •   is SNAT/DNAT  ( NAT )
  •   Allowed or Denied ( action )

As with the  netscreen, we need to remind our selves to reset the diag debug fuction to conserve on memory.

 diag debug reset

Next post will be cisco ASA and then I will wrapped up with PaloAlto series. With these last three, we have  the means to capture data  and run pcaps for exporting. I will cover that in unique post.

Ken Felix Freelance Network/Security Engineer
Kfelix at hyperfeed  <d-o-t> com

Wednesday, June 26, 2013

diag debug flow troubleshooting

In this blog, we will look at the FortiGate  diag debug  flow output messages & what they are trying to tell you.

I've been dabbing with Fortigates ever since 3.0 came out ( since 2005 ) and it still surprises me  on the pure amount of  individuals,  that struggles with diagnostics and those that don't even use the  diag debug flow.

In this post I'm going to show you some output messages and how to interpet them.

1st  up:

You try to ping , ssh or web into a fortigate and you are denied. Diag debug flow will show you something of the following;



So this means; " whatever interface your trying to access did now have the correct < set allowaccess> statement "

e.g ( webgui interface configurations )




or via the cli;



2nd:

This  message  below is "10 out of 10" times one of the following;



1: a deny firewall policy ( it even tells you so :) )
2: firewall policy missed-ordered or sequence#  ( our most specific and denys should be 1st in the order )
3: error in your firewall policy ( typo , wrong address, wrong  interface , service or a combination of erros )
4: or a missing firewall policy ( if it's not allowed it's denied )

Bottom line check your firewall policies and go thru them with a fine tooth comb.


3rd and final: ( my favorite )

When your working with  sslvpn or tunnels this is a common  error. It's also seen when you have internal LANs being routed behind another device like a internal router & you have no route for the source(s) in the FGT route table.


In the above 10 out of 10 times it's any of the following;

1: routes installed  incorrectly
2: no route installed ( it's missing )

Bottom line ;"your routing is screwed up" . Monitor your route table and validate the next-hop gateway is correct & the correct interface.




Diagnostics with diag debug flow is simple and straight.



It frustrate me to see  junior/senior  Fortigate engineers struggling to diagnostic connections problems, and they DON'T ever bother to use the built-in diagnostics tools or just bypass the process of making a packet capture.

The fortigate is one of the best firewalls on the market to trouble-shoot ( period ). You will not find anything as simple or as easy.

These commands should be routine in our everyday  activities and before you waste time throwing things into the mix;

diag debug flow
diag sniffer

And finally, we have  the means to make  packet  captures on most any newer fortigates.

 So instead of guessing, shoot-n-pray, or just using the "trial--n--error" process, start using the diag debug flow  cmds :)



Ken Felix
Freelance Network/Security Engineer
kfelix  ---a---t--- hyperfeed --d-o-t-- com

    ^        ^
=( @   @ ) =
         ~
      /     \

Thursday, February 5, 2015

diag debug flow from a fortigate ( local vrs interface )

Diag debug flow is the #1 trouble-shooting tool that should always be deployed from a fortigate. In this example,  I will show you how to determine if your diag debug flow caught packets that where generated locally from the unit


1st a simple filter



Now here's a  trace where packets crossed a inside to outside interface



Now here's a trace where the packets where generated locally ( in my a case a ping from the FGT100D device )



NOTE  Do you happen to notice the "from local" 

So yes diagnostic debug flow will show you any and all packets regardless if it crossed interfaces or are locally generated.

To learn more revisit one of my earlier threads.

http://socpuppet.blogspot.com/2013/06/diag-debug-flow-troubleshooting.html
http://socpuppet.blogspot.com/2014/08/fortigate-connectivity-diagnostic-steps.html


Ken Felix
NSE ( Network Security Expert) and Route/Switching Engineer.
kfelix  -----a----t---- socpuppets ---dot---com

     ^      ^
=(  @  @ )=
         o 
        /  \

Wednesday, November 12, 2014

Fortigate identity policies trouble-shooting

With fwpolicies that uses identity-based , you have a few means for diagnostics.

The 1st thing you need to do is to ensure that the expected-traffic is matching the policy that a user is having problems authenticating with. And ensure that traffic is arriving to the fortigate firewall. The easiest way for determining  this;

diag debug flow

or

diag sniffer packet 

The former is more acceptable.


If traffic is NOT hitting your policy,  than "Stop" and don't proceed until you ensure that any other network routing or filtering problems has been fixed.

I 've seen now on 1-to-2 dozen occasions or more, that a firewall engineer stumbles around just to find out a inside  interior  firewall or router ACL was preventing the traffic destine to the  identity-based firewall policies.

NOTE: With the diag debug flow, if you see "Denied by forward policy check",  than that  means you hit a policy with the action either set to  disable or  you have no policy to begin with.

If you are seeing a policy-id match that matches your traffic,  the following will be logged via the diag debug flow. This will show the policy-id  that's being matched, now you can dive in and find the problem.

e.g

msg="Allowed by Policy-124:"


Next, is this the policy-id that you have identity-based authentication set for? And does the group have the proper user defined ?

config firewall policy
    edit 124
        set srcintf "wan1"
        set dstintf "port1"
            set srcaddr "all"            
            set dstaddr "XXX_XXX_XXX_XXX"            
        set action accept
        set identity-based enable
            config identity-based-policy
                edit 1
                    set schedule "always"
                        set groups "consultant"                        
                        set service "TELNET"                        
                next
            end
    next
end



Next, you will need to inspect the user  and the  user-group that's defined  for the above firewall-policy-id. Here it would be my group named "consultant" and it has one member  "kfelixsslvpn"

config user group
    edit "consultant"
        set sslvpn-portal "full-access"
            set member "kfelixsslvpn"            
    next
end


and

config user local
    edit "kfelixsslvpn"
        set type password
        set passwd ENC VLrvImdmeB0GbZs5ZG+D5Wzhyhy0xNHyfo0lnX0YmATrFc2pAiZ7T8dcAbOfbi+k2WA1i3ydO2m4jyHLusuWU0kVJot8e518qzyIqzi1IvewXug1
    next
end


In my case we are using local authentication So this user is authenticated locally against the user database. If the user doesn't exist or does not exist for the group, you will fail authentication. Also if the password is incorrect, you will fail authentication.

So if you suspect problems with the end-user,  have her/him double check the login/password or create a new password for the login.

If the user is  authenticated off-appliance, you need to  diagnostic  the  remote authentication services.
 (   radius, ldap, etc... ). Most AAA authentication platforms have local tools for testing  user validation or credentials. Research your platform for any local tools or diagnostics.

NOTE: If you see a group with multiple users & only ONE user is having problems that you can narrow down your diagnostics with just that "user".

Lastly, you can use the displayed output when your intercepted for authentication. If your getting that the authentication block that you know the identity policy is being matched.


You can also use the diag debug app authd -1 to report the status also.


( telnet  )


( http )


So based on the policy and identity the  fortigate will challenge the user with a login.



config firewall policy
    edit 124
        set srcintf "wan1"
        set dstintf "port1"
            set srcaddr "all"            
            set dstaddr "XXX_XXX_XXX_XXX"            
        set action accept
        set identity-based enable
            config identity-based-policy
                edit 1
                    set schedule "always"
                        set groups "consultant"                        
                        set service "HTTP"                        
                next
            end
    next
end







With authentication  & identity-based policies, if the original session was via an un-encrypted services , than the authentication would also be un-encrypted. 

 What this means,  my above HTTP identity-based policy would authenticated me via "HTTP" and not "HTTPs".  The fortigate is actually intercepting the traffic src/dst and spoof'ing the  challenge on behalf of the destination.  Your NOT authenticating via the target but via the firewall and the packet spoof'ing.



Here's a tshark decode


No pun intended , but the user credentials clearly are in the clear ( username = sssss password = ssssssss )

This is bad thing ( lack of security ) but for diagnostics, you can packet capture the traffic and inspect the user credentials that's being presented.

A few useful authentication commands;

Diagnostic

 diag debug reset 
 dag debug en
 diag debug application authd -1

Operations & monitoring

 diag firewall iprope authuser
 diag firewall iprope resetauth

The former shows who authentication & from where. It will now show what policy-id trigger the authentication. The latter clears all authentications. Use with caution.

  diag firewall auth list

This will show you details about the user, timeout and policy-id that was triggered.


For my  very last item, you can adjust the authentication timeout. This can be done  from the webGu.

 user > device > authentication> setting 


or via the CLI;

config user setting
    set auth-cert "self-sign"
    set auth-timeout 436
end



NOTE: default timeout == 300secs


And for stricter and harsher controls, you can set the  authenticating parameters for blackout and the max failed-attempts. This is great protection for any  brute-force login attacks.

config user setting
    set auth-blackout-time 5

    set auth-invalid-max 3          < after 3  failures blacklist at above value 5mins.
end


NOTE: various  fortiOS exhibits failures on the blackout time value and it's ignored YMMV


I hope this gives you some ideals on identity-based authentications. IMHO  security methods exist such as 802.1x, FSSO , Radius SSO, Proxies or a combination with identity-based.

Ken Felix
Freelance Network/Security Engineer
kfelix  -----a----t---- socpuppets ---dot---com

     ^    ^
 =( #  # )=
      @
      /   \


Saturday, October 26, 2013

Site-2-Site ROUTED VPN Trouble-shooting & Guide Fortigate

In my past postings, where we configured a lan2lan vpn between a fortigate and juniper-SRX, this is a continuation on t-shooting.

If you are familiar with the webGUI,  you will have ran across this ipsec-monitor at some point and time.


Okay, so what do we do and what do we look at, while trouble-shooting ipsec vpns?

In this blog, we will look at a few various means for  the diagnostics of VPN problems. This post is for troubleshooting vpn-tunnels created in a  interface-mode  ( aka routed  vrs policy ) & those using  PSK for peer-authentication.



PHASE1

We will now look at some of the  troubleshooting and show commands, that can be executed on a fortigate to help diagnose vpn problems.

1st, I want to state,

90% of the problems with these VPNs can be traced back to a mismatch in our configurations. So always double check your configurations, and this mean all aspects of the configurations.

e.g
  • Your Pre-Shared-Key  or  Certificate 
  • IKE proposal ( 3des md5/sha1/sha256 or whatever your using )
  • IPSEC proposal ( 3des md5/sha sha256  or whatever your using )
  • peer address  ( is it correct , is it the correct interface )
  • expected identification ( FQDN or ip_address or Hybrid )
  • fwpolicies
  • route reachability 
  • etc.......

The  very 1st common error  encountered, typically pertains to  our PSK key entries;  "be cautious of any leters, numbers, lower and uppercase". These are simple to spot in the logs, due to we commonly see "mis-match pre-shared-key" or something to that nature. So always re-key if you believe the PSK is not correct.  This is probably 99.99% of the problems that prevent phase1 setup.

Using the combination of the following cmds;
  1. diag debug app ike filter name "phase1-name"
  2. diag debug app ike -1 
  3. diag debug enable

Will always give you clues to any PSK and other proposal issues;

here's a possible mis-match within our PSK;



Moving on, when the PSK has been confirmed & is correct, then the next most common problem develops around the initial  proposals that's being offered between both parties.

I ( just me personally ) do not like to deploy multi-proposals. It's not required,  can cause confusion by some various  remote-vpn devices. Unless it's a dynamic-vpn, you  DON"T NEED MULTI proposals. Yeap, I'll re say it here again, YOU DON'T NEED MULTI PROPOSALS in a site2site vpn.

Once again  the diag debug app ike  will show you both yours,  and theirs proposals, & what's being  present  by both parties. There has to be a match and agreement before phase1 can ever become established.


e.g ( spotting a mis-match proposals blue is the far-end and what they sent and purple is what I have configured )





NOTE: With a packet capture only, you will also notice very little traffic after the initial connection from the initiator & responder ( why would  2 vpn appliance attempt any further communication if  the keys don't match ? or the proposal didn't agreed in the 1st place )

So even without logging, you can follow the ip datagrams and determine if it's a possible PSK issue exists, & by looking at the ip datagrams from a packet capture. I always suggest doing  this ( packet capturing )  for  a few other reasons;
  • You can validate the peers  ip_address that are configured
  • are they negoiating via NAT transversal
  • confirm that no ACLs are blocking traffic between peers

Next, you need to be aware of  IKE and that it uses src_port #500/udp and dst_port #500/udp unless we are being NAT. If NAT-transversal ( aka NAT -T ) is involved, the dst_port becomes  #4500/udp.


The integral  diag sniffer will expose problems if the initiator or responder are not being reached or fail to respond

e.g ( cmdline using wan1 interface )


diag sniffer packet wan1 "udp and dst port 500"

or

diag sniffer packet wan1 "udp and st port 4500"

Next, once you have confirmed the  PSK and initiator/responder are talking, than we want to ensure the IKE +IPSEC proposal are matching.

With the IKE+IPSEC proposals, the authentication proposal has to match, and the Diffie-Hellman group must match. This determine the key strength during the negoiation of the peers over a unsecured channel.

  



Why such a big harp on proposals and mis-match? 

Will look at  just how many proposals that a modern  Fortigate Appliance running 5.0.X software supports;


What this means? That's a lot of @$#$%  proposals.

Even a juniper or cisco don't support ALL of the listed proposals. Even some firewall have proposals, that even a fortigate don't support.

e.g 

Juniper SRX supports both, PSK and RSA/DSA signatures  certificates, where-as a  Fortigate supports only PSK and RSA

Juniper SRX supports md5 , sha1 and sha256, where-as a fortigate support all of these and  sha 364, 512

Linux's Strongswam support twofish and blowfish ciphers, both of which are not part of a Fortigate lingo or supported ipsec  ciphers  by commercial firewall that I know of

Until recently,  cisco firewall didn't support  IKEv2, where-as fortigate has  had ikev2 support for probably 7+ years or more if I had to guess

Some cisco routers support cisco-only cipher available with cisco routers or phase1 authentication methods that could be totally different.

Eg

 here what a cisco firewall  support ( crack, rsa-sig,psk );


yes Crack :)


Eg 

a common cisco router ;









Next  ( this is  VERY VERY VERY IMPORTANT  PAY ATTENTION! )

" The Phase IKE SAs keylife should always match, and be pretty darn close at it!  "





A mis-match here , will typically not be a show-stop within the initial setup for our phase1 security-association, but the 2 ends not matching, will cause interruptions and problematic issues upon re-keying.

This is due to the keylife expiring out on one vpn device, while the other side still think  it's all is good.

NOTE: DPD and ike-keepalive ( similar but they are not the same ) can help with regards to remote-peers & detection of a failure in connectivity.

Take this SRX and FGT, one party has the keylife set to an exaggerated  120sec ( FGT ) and the other to 180sec ( SRX ). That's the two lowest values that support by these firewall devices btw

Look at what happens when we ping across the vpn tunnel & when we have a mis-match in IKE keylife ?




One last items about phase1 trouble-shooting

If NAT-T is involved, you might need to pin the PAT/NAT session table, or use some of kind of NAT-T keepalive to avoid the expiration of your PAT/NAT translation.

What this means; "  if  the  NAT-T session is maintain by a NAT'ing device such as a cisco router , an upstream firewall, or some other network translation device", any traffic that's does not maintain a certain interval, could cause the IKE session to get hung up and stale. The NAT-T keepalives prevents this from happening.

You can enable NAT-T support,. by each per-peer under your fortigate configurations for phase1 ;

e.g




NAT-T support is passed along during the IKE phase1 negotiations between peers, if they  both support it.


Let's look at another diag debug output from a fortigate  " diag debug app ike 255 ";



Here's we can see the output shows a phase2 proposal mis-match and it's clearly indicated by the above output & using the "diag debug app ike " command.


Let's look at some more diag vpn outputs. One of the most important  option within fortigate diagnostics is our phase1 filters. In a hub or a service provider arena, you might havd a dozen or more  vpn-tunnels. It's not at all uncommon to see 100-1k vpn-ipsec-tunnels nailed to a fortigate.





NOTE:  Use the filtering option at your hub vpn-device,  to avoid the over-saturation of diagnostics outputs & filter on the gateway of interest



Here a listing of my vpn tunels. It provides good  basic information on any IKE and IPSEC SAs, uptime and  peer information. If DPD was enabled we would have DPD details.



and for comparison, here's an active  VPN I have where dpd is enabled &  on just one side;



So sometimes we can quickly find out if both parties are using  DPD,  and if we are going across a NAT-d connection via the remote/local peers destination ports of 500/udp vrs 4500/udp.



NOTE:  DPD DOES NOT NEED TO BE ENABLE BOTH WAY for a vpn tunnel to work, but it should be enable mutually imho.




For negotiation, either parties can negotiate the  Phase1 SA, But let's say you don't want to do this or maybe  your a passive vpn-termination device like those  that I worked within a a few 3 letter gov agencies sub-contractors in the past.

Will have no fear, we can tell your device to never initiate the connection;





The above two screenshots stacked one on top of the other,  shows the Juniper/SRX being the initiator and the  Fortinet/Fortigate acting as a  responder.

PHASE2

Now let's say we think we have our IKE SAs established, but we are still having problems, remember the quickmode selectors ( aka  QM or Proxy-ids ) ? They have to match for both parties & for the IPSEC-SA that are establish under phase2.

Here's our configuration for the FGT<>SRX using a static defined proxy-ids.







NOTE: The local and remote subnets where defined. If you should build your  phase2 configuration like this &  between all devices, than you will never have a phase2 proxy-id mismatch imho. Also if you should happen to change the device on one end to another vpn-hardware appliance, you will not have to any  changes to make on your fortigate vpn configuration.

But in reality, fortinet made a easy button, &  that has been a source of many  major problems & confusion.



By default the proxy-ids are defined as 0.0.0.0/0:0 for both the dst and src subnets:protocol. But this would not work for 99.99% of the vpn-tunnels that terminates to a non Fortigate VPN appliance device.

Yes, try a QuickMode selector of 0.0.0.0/0:0  to a;  cisco ASA/Router, Strongswan, Pfsense,  Checkpoint, etc.....

Your tunnel will not come up ( period ).

Okay we know the easy button  does not fit all applications,  and does not fit all vpn-appliances. I only recommend the 0.0.0.0/0:0  quickmode selectors,  when you have a only-Fortigates on the other end of the vpn-tunnel and you have multiple of networks that you are carrying over the vpn-tunnel.

In reality, I almost never do this and  prefer to build my phase2-interface SA  per dst/src subnet and let me tell you my logic why?

When you build multiple ipsec-phase2 SA, and define each local/remote subnets details, you get statistics per each IPSEC-SA. This can help you with your diagnostics of why a SA number ###  does not work , or what direction ( remember  SA are uni-directional   )

Take this design;




With the above method#1 and #2 & static defined SAs, we will have statistics & for 2ea SAs,  using our diagnostic commands.




And now for some diagnostics output for review;




As you can clearly see, the 2nd proxy-id name srx-p2-1 is not encrypting any data. ( see the text above for our redline underline? )

If you had used method#2 & the one single 0.0.0.0/0:0 define for both src/dst, it would not have been easy to determine the above problem.

PFS the what and why

PFS is one of those configuration items,  that can cause a whole heck of problems. Will have no fear, make sure if you enable it on one vpn-peer, that the other peer also has it enabled. What PFS does in short, is to initialize the IPSEC-SA from a key sessions  that's not generated from the  parent key within the IKE SA.This provides more tighter security against any compromised ike keys.

Keep in mind that with PFS enabled, your ipsec SAs could  use a completely different  encryption schema and ciphers  from that of the phase1 IKE. What that means, you could enable 3DES+grp5 for IKE phase1 and AES128+grp14 for the IPSEC SA phase2.

A complete lack of  any  connectivity at our phase 2 ipsec SAs, could very well be a mismatch in your  IPSEC proposals, or worst you have one SA encrypting data and the other side not de-encrypting or vice-versa. These are hard to identify without mutual eyes, and troubleshooting from both end of the vpn via the firewall engineers. BUT the diag vpn tunnel list , does shed some light into these types of problems.


Moving on,

When the tunnel are up, we have a useful means for gathering statistical data ( the number of packet encrypted/decrypted, SPI identifier , bytes sent/received ) via the "diagnose vpn tunnel  list " command and by following the output. This command is always my 1st command issued, while trouble-shooting vpn diagnostics problems.


One last things that comes up, a VPN ipsec-tunnel  has some overhead. So a 1500 byte MTU is not going to fit with the overhead of the ESP-header, including the additional ip_header,etc.......



Unix offers the means to test  max MTU size by using the DontFrag option and slowly increasing pings with the DF bit set to "1", at some point of the pings failing, this would be your max MTU.

e,g

ping -D ( macosx ) or  ping -M ( linux ), for windows you have a few utilities out that accomplish the same





Conducting packet captures at the src/dst subnets can validation the TCP max-segement size and ip-DF ( don't frag bit  ) settings, these  can shed light  within any path-mtu related issues.  Ping sweeps starting at a low to high packet size, can also some shed light to a vpn-tunnel mtu issues.




A review of the diag commands that are useful for all firewall engineers using a Fortigate security appliance;


diag debug enable
diag packet sniffer
diag debug app ike
diag vpn tunnel list 




NOTE: These are only available from the command line



MISC

Interface based vpns or not?

In this posting we didn't  even tap the various different modes of  vpns.  Within the fortiOS you have 2 possible means for crafting vpns;


1: policy base where we use the key wording "encrypt or ipsec" within a firewall policy action   ( fwiw I NEVER EVER  use this mode & it's highly recommended not to build you vpns in this fashion )


2: routed mode or commonly known as interface mode ( this the best-common-practice and mode most engineers rely on )

Here's the PROs for the latter ( routed mode )


  1.  you can run a routing protocol over the vpn
  2.  you have a interface to nail policies to just like your ports or wan interfaces
  3.  since we have a interface, we can dump or conduct packet captures over that interface using diagnosic sniffer command
  4. you can assign ip-address to the inteface
  5. you have snmp-ifindex , hence now we can graph and monitor with cacti, solarwinds, OpenNMS, and against a interface.


Guess what, if you created the phase1/2 definitions with the phase1-interface and phase2-interface, you are creating a routed-vpn. And the key word, "routed" means you need a static or routing protocol to learn the remote location dst-subnet(s). So this means you need a route to reach the far end dst-subnet(s)



DIAG SNIFFER

The name you give in that phase1-interface configuration, is the same interface you defined for the route gateway and the same interface you can dump on, with the sniffer to capture packets b4 or after encryption.






NPU issues

You can always disable the NPU from the vpn tunnels if you suspect some weirdness or have concerns with advance  acceleration or other off-chip performance. I never recommend this as a permannet fix, since it drives the encrypted traffic directly at the  cpu.

Using the full keyword in our show, we can see that we are off-loading to the NPU.



The following screenshot shows some of the  crypto stats and these are  helpful with eyeballing traffic at the NPU if your model has any VPN acceleration ;

( continuation )







Show-Full 

Don't forget, that In all configuration details, the "show full" command within the sublevels, provides greater details and default settings.


Clearing IKE-gateways

Some times we need to clear a IKE gateway. For good practices, any changes done in you pre-existing vpn tunnels, should be followed by a clearing



By specifying the "clear " and within the diag vpn ike gateway, we can  flush our  IKE SA.

And lastly & final

Use the diag debug and flow commands. I 'm providing a few link to my earlier posts & with concerning the easy to execute diagnostics debug flows

http://socpuppet.blogspot.com/2013/03/flow-diagnostic-fortigate.html

http://socpuppet.blogspot.com/2013/06/diag-debug-flow-troubleshooting.html

http://socpuppet.blogspot.com/2013/06/diag-system-session-quick-way-find.html


Execution of the diagnostic debug flow, gives you about 90% of the time, the proper direction to take based on the output followed by the diagnostic flow


  • the lack of a fwpolicy
  • lack of a forwarding route or uRPF checks failures
  • policy ordering issues ( maybe you over look policy ID that denying the traffic & ahead of your fwpolicies for your VPN )


By using the diagnostic debug flow commands, you can save yourself time, frustration, and wasted tickets to fortinet-support,  or posts to online forums.



Ken Felix
Freelance Network/Security Engineer
kfelix  at  socpuppets .....dot....com

   ^      ^
=( *  * )=
       o
      /  \