Thursday, February 27, 2014

Dual VPC terminate on Fortigate firewall

If you remember this earlier posting on my cisco 1941,  and with dual AMAZON VPC-VPNs, will here's the same setup for the fortigate  series of firewalls.

1st To recap the design;


NOTE: This is not ideal in that both tunnels are terminated to the same  device, but due to the satefull inspection we are limited on a firewall. You could terminate the tunnels on 2 unique firewall and run a iBGP session between, but you will need to be cautious on  asymmetrical routing issues and the statefull inspection failures. Please see import/export route policies included in my bgp configuration section YMMV


The configuration is broken down to interfaces, crypto, bgp and firewall  & misc configurations. YMMV and your outcome could be different. This was tested on 5.0.X code

1st the interfaces;

==================== INTERFACES ======================

config system interface
    edit "wan1"
        set vdom "root"
        set ip 12.126.x.x 255.255.255.252

        set allowaccess ping 
        set ident-accept enable
        set type physical
        set alias "WAN-ISP1 ATT-pl"
    next

    edit "wan2"
        set vdom "root"

        set ip 38.x.x.x 255.255.255.252
        set allowaccess ping 
        set ident-accept enable
        set type physical
        set alias "WAN-ISP2 PSINET"
    next

 end

tunnels  configurations;

config system interface
    edit "AWSvpc1"
        set vdom "root"
        set ip 169.254.255.2  255.255.255.255
        set type tunnel
        set remote-ip 72.21.209.225
        set interface "wan1"

        set tcp-mss  1412       
        set alias "VPC_TUN1"
     next
    edit "AWSvpc2"
        set vdom "root"
        set ip 169.254.255.6  255.255.255.255
        set type tunnel
        set remote-ip 169.254.255.6
        set interface "wan2"      

        set tcp-mss  1412
        set alias "VPC_TUN2"
     next
end




==================== VPN CRYPTO STUFF  ======================


WARNING: It's suggested to use  interface mode. If you deviated from this, than your on your own DO NOT contact me !


( interface mode vpns  aka  route-based )

config vpn ipsec phase1-interface
    edit "AWSvpc1"
        set interface "wan1"
        set dpd enable
        set dhgrp 2
        set proposal aes128-sha1
        set remote-gw 72.21.209.225
        set psksecret 1blahblahblahmyPSKkey
        set dpd-retryinterval 15
    next


    edit "AWSvpc2"
        set interface "wan2"
        set dpd enable
        set dhgrp 2
        set proposal aes128-sha1
        set remote-gw 72.21.209.193
        set psksecret 2blahblahblahmyPSKkey
        set dpd-retryinterval 15
    next
end







NOTE: phase2 quick-modes selectors defaults to 0.0.0.0/0:0 which is okay for AWS

config vpn ipsec phase2-interface
    edit "AWSvpc1"
        set dhgrp 2
        set pfs enable
        set phase1name "AWSvpc1"
        set proposal aes128-sha1
        set replay enable
    next
    edit "AWSvpc2"
        set dhgrp 2
        set pfs enable
        set phase1name "AWSvpc2"
        set proposal aes128-sha1
        set replay enable
    next
end


==================== BGP CONFIGURATIONS  ======================

1st we build a few import/export policies via  a common route-maps that we will call-out for our peers


We need to adjust the local_pref on 1 peer only
We need to set metrics on the 1 peer that we want to be carrying our traffic


config router route-map
    edit "import"
        set comments "my import policy apply to one peer"
            config rule
                edit 1
                    set set-local-preference 600
                next
            end
    next
    edit "export1"
        set comments "my export policy apply to one peer only and on the primary tunnel peer"
            config rule
                edit 1
                    set set-metric 1147400
                next
            end
    next
  edit "export2"
        set comments "my export policy apply to one peer only on the path you don't want to use"
            config rule
                edit 1
                    set set-metric 2147400
                next
            end
    next

 end


And now for  the fun stuff our  bgp configs ;


config router bgp

    set as 6500

        config neighbor

            edit "169.254.255.1"

                set description "VPC1"

                set interface " AWSvpc1"

                set remote-as 7224

                set route-map-in "import"

                set route-map-out "export1"

            next

            edit "169.254.255.5"

                set description "VPC2"

                set interface " AWSvpc2"

                set remote-as 7224

                set route-map-out "export2"

            next
          end

          config network
            edit 1
                set prefix 10.10.0.0 255.255.255.0
            next
        end

NOTE:  If it's not clear here, we are setting the local_preference higher on tunnel1 to force our traffic out tunnel1 always. 

Likewise we are forcing metrics to ensure AWS steer traffic down tunnel1 over tunnel2. This helps maintain  our statefull checks. If we didn't do this, we would have to enable asymmetrical routing which is a bad bad things for firewalls


If tunnel1 fails, goes down or if AWS conduct maintenance on the datacenter that houses the  VPN termination, we revert to  tunnel2.



==================== FWPOLCIES  ======================

For the firewall-policies,  it's all up to you & on how you want to define traffic to allow. You even can opt to not advertise anything via  BGP,  and just  SNAT all traffic behind the 169.254.X.X APIPA address. I haven't tried this approach, but it should work

( a simple fwpolicies )

config firewall policy
    edit 0
        set srcintf "internal"
        set dstintf "AWSvpc1"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 0
        set srcintf "AWSvpc1"
        set dstintf "internal"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 0
        set srcintf "internal"
        set dstintf "AWSvpc2"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 0
        set srcintf "AWSvpc2"
        set dstintf "internal"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
end



==================== STATIC ROUTES  ======================

And lastly, it's best practices to place static /32 routes to the AWS vpn-terminate for the respective ISP interfaces;


config router static
    edit 99
        set device "wan1"
        set dst 72.21.209.225 255.255.255.255

        set gateway 12.x.x.x
    edit 100
        set device "wan2"
        set dst 72.21.209.193 255.255.255.255

        set gateway  38.x.x.x
    next
end
 

 

NOTE: Amazon seems strange in not wanting to support you if you use a device that's not a router.  So YMMV on how well your approach and design  & the support you get from the open community.


I hope to have a working for a Juniper/SRX in a few days,  so stay tuned :)


If your in the need for a cloud based solution,  please reach out to these fine folks at  Turing Group. They have been providing cloud based solutions with Amazon for over a year and have over  8 years within the cloud datacenter technologies and strategies

http://www.turinggroup.com
info  @   turinggroup  <dot>  com

If you need remote access solutions please contact me at email below.

Ken Felix
Freelance Network / Security Engineer
kfelix  ----a---t---socpuppets ---d---o---t---com

     ^      ^
=(   ^   ^  )=
          o
       /     \

10 comments:

  1. Great article. Did you ever get to set this up on a Juniper SRX? I'm facing the same issue with asymmetric routing if I want the tunnels to be redundant and BGP seem to be the only way to solve this.
    I'm curious to know why you choose "set metric" vs. as-path prepend to attract the traffic on the right link (if there is a reason).

    ReplyDelete
  2. Nicolas
    1st thanks for the reply.

    2nd yes, I did some stuff with juniper & AWS and I will post ablog. Juniper has a few more quirks and issues but the process is about the same.

    To answer the question

    [QUOTE]I'm curious to know why you choose "set metric" vs. as-path prepend to attract the traffic on the right link (if there is a reason).[/QUOTE]

    Traffic within the same AS 7224 would not be effected by the as-path-prepend. The this bgp attribute prepends your AS# X amount of times in the advertisement outbound.

    In order to select one uplink path over the other, we set the metric for the path we want to use. This will allow us to route over one link over the other.

    Also, for traffic intra-AS, we could advertise to that AS peer, what path to set by metrics when we have a multi-exits. It's aka as MEDs. This metric is typically not carried out of the peer AS.

    Please see the follow blog posting

    http://socpuppet.blogspot.com/2013/08/non-transitive-path-attributes-bgp.html

    ReplyDelete
    Replies
    1. Thanks for the extra explanation (I'm not familiar with Fortigate, so please forgive my cluelessness ;-)) so if I understand correctly you're using MED (set metric) to influence inbound routing but this doesn't seem to be supported by AWS (based on this conversation https://forums.aws.amazon.com/message.jspa?messageID=470375 and the linked doco http://docs.aws.amazon.com/AmazonVPC/latest/NetworkAdminGuide/Introduction.html#MultipleVPNConnections where MED isn't mentioned). I'm guessing that it worked as expected because the lowest BGP peer IP address belonged to tunnel1 which is your primary tunnel. So it sounds like AS-PATH prepending is the way to go to control inbound routing (and for outbound we use local-preference, no issue there).
      I'm figuring this out as I go along so I might still be missing part of the picture.

      Delete
  3. Nicolas thanks for your reply. I will do some testing hopefully this wekend to see if AWS vpc actually honors the metrics. MEDs are typically use intra-as and is not carried out of the peering AS.

    I will post what I find

    ReplyDelete
  4. Is it possible to setup the IPSec tunnel with AWS without using BGP to advertise the local subnets to the VGW?

    I'm using static routes on both ends and while I have the tunnels up and running, I can't pass any traffic because there are no routes being 'advertised' as per amazon support. It's almost as if they only use dynamically learned routes in their gateways.

    ReplyDelete
    Replies
    1. you can add static route in AWS routing table. so just add your LAN subnet and it will be fine.
      fyi, AWS does support static route IPsec tunnel. in fact they will ask you whether its dynamic or static when you first create the tunnel.

      Delete
  5. As of Jan 2016 AWS no longer honors metrics. Don't want anyone to be led to a dead end following these (excellent) instructions. Instead you need to use set as-path-prepend and prepend your AS multiple times to influence less desirable routes. We are also finding lots of the formentioned gotchas with asymetric routing since these are firewalls, its almost mandatory to set the asymetric routing enable command on the Fortigates depending on your design.

    ReplyDelete
  6. dear sir,

    Thank you so much. I also have some problem with AWS Assym routing and dont want to enable assym routing.
    I follow exactly as your tutorial. Its working fine and OK.

    Regards

    ReplyDelete
  7. There is a chance you are qualified to get a $1,000 Amazon Gift Card.

    ReplyDelete