Thursday, December 19, 2013

A Cisco ASA , breaking the SMTP_TLS security function ( why friends don't let friends, buy the cisco ASA )



Why friends, don’t let friends buy cisco ASA J

The ASA security appliance is one of the most common firewall in use by a lot of enterprise networks, but can be one of the most frustrating  unit on planet earth. It has been the root of a lot of problems, both performance related and inspection related.  It's also very pricey for the performance that the ASA chassis provides and encompass a whole slew of license restrictions.


Here’s the problem;

1: I’ve spent approx 3 days trying to debug why my fortimail 3000D didn’t allow for TLS  on  inbound mail connections

2: All of my test showed that TLS was disabled when testing from the outside

3: TLS was enable within the webGUI of the Fortimail AS/AV appliance

4: I never thought of testing from the inside and validating any differences in behavior ( I was single focus in that it was the fortinet Fortimail appliance since it was quite new )

5: Fortinet TAC gave me the  ideal of  using the exec smtptest 127.0.0.1  command locally on the  fortimail ( I’ve always used it from an external testing, never realized we had a listener  on 127.0.0.1:25/tcp  )

6: Looking at the problem, I suspected the FortiMAIL was the culprit, but it turned out to be our choice of firewall and it’s inspection features ( cisco ASA )

7: Other fortimail appliances that I managed &  that works great, are not behind a cisco ASA security appliance

How can the  Adaptive Security Appliance ( aka ASA ) & with it’s wonder  inspection features, actually weaken a security feature for mail, does not make any sense.

Sadly, this problem with the cisco ASA,  & with it  dis-allowing a common security function ( TLS encryption for mail ), is just down right disturbing.   Adaptive my ASS!. 

The  ASA in my dictionary, means AnotherSorryAppliance !



Moving on, let’s look at the  diagnostics & the issues encountered;

Using the mxtoobox against the public facing address of my fortimail appliance, which is operating in gateway mode we get the following;



 and with out the inspect esmtp enabled on the ASA we get the following;

]




So now what? 

Why did this failed when the inspection policy was enabled in the global policy?


My initial suspicions where;  that the ASA was inspecting the  EHLO output between the client ( mxtoolbox ) and the server ( Fortimail 3000D )  twice,  and this is causing the problem with the ASA fixing up the commands between the two parties. But this was not the case.

A side by side comparison when using telnet show you what the ASA is doing ( leftside was done locally and right remotely )



It really boils down to the cisco ASA is masking the EHLO reply & the command options it was presenting to the remote client.


So how do we remediate this?


What first  take a look at the follow cisco link about inspect  to get an understanding of what the purpose of the XXX;



You could  easily disable esmtp inspection, but that would allow for bad things to happen to your mailserver if something bad where too actually happen

ie
  • a DoS attack
  • the handling of non ESMTP or SMTP commands
  • exposing your  mail banner
  • etc………

Here’s how socpuppets fixed  & remediated the problem,  and for allowing the  fortimail security appliance to actually support TLS encryption.

1:  we had to  change the inspect esmtp to use a custom profile.



2: in this inspection,  we allow for TLS

"allow-tls"

3:  we apply this profile to our inspect esmtp statement



4:  and  then re-test and monitor





Now after the above changes, any with our  follow up testing; " we can easily see that STARTTLS is now indeed working ". 

So any external mail clients that want to provide encryption, can easily do so & the ASA will not strike the support for STARTTLS option in the reply.

We can also issue the show command to monitor statistics or use debug esmtp to see our inspection in action.

( sample debug output )

 SMTP: Initial state:10
 SMTP: EHLO REPLY - Reply len:15, match_len:15, reply_re_state:0
 SMTP: REPLY DONE - eid: 19
 SMTP: State changed to:10
 SMTP: Initial state:10
 SMTP: EHLO REPLY - Reply len:74, match_len:74, reply_re_state:49
 SMTP: EHLO REPLY - match id:41
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:23, match_len:23, reply_re_state:174
 SMTP: EHLO REPLY - match id:35
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:14, match_len:14, reply_re_state:165
 SMTP: EHLO REPLY - match id:37
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:12, match_len:12, reply_re_state:152
 SMTP: EHLO REPLY - match id:30
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:8, match_len:8, reply_re_state:131
 SMTP: EHLO REPLY - match id:38
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:7, match_len:7, reply_re_state:113
 SMTP: EHLO REPLY - match id:34
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:8, match_len:8, reply_re_state:125
 SMTP: EHLO REPLY - match id:31
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:12, match_len:12, reply_re_state:157
 SMTP: EHLO REPLY - match id:39
 SMTP: - Mask EHLO reply 512
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:15, match_len:15, reply_re_state:49
 SMTP: EHLO REPLY - match id:41
 SMTP: CHECK EHLO REPLY - eid:19
 SMTP: State changed to:10
 SMTP: EHLO REPLY - Reply len:10, match_len:10, reply_re_state:49
 SMTP: EHLO REPLY - match id:41
 SMTP: CHECK EHLO REPLY - eid:8
 SMTP: State changed to:1
%ASA-3-106014: Deny inbound icmp

 and

(output  show service-policy global inspect esmtp  ) 



I hope this post comes in handy for others, who happen to run into the same issues and with a cisco ASA in front of your  mailserver or mail firewall.


For more information about fortimail



For purchasing, support and unstallation, contact my friends at maxis360




Ken Felix
Freelance Network / Security Engineer
kfelix  ----a---t---socpuppets ---d---o---t---com

     ^      ^
=(  @   @ )=
          o
       /     \




1 comment:

  1. I have just spent a day battling with the same issue. Thank you for this post.

    ReplyDelete