Common External web performance monitoring sites

Gomez like  Alertsite;  installs their name in the user_agent  field of the http headers.

To find  gomez  you can do something similar to ;
tshark -n -i eth5 -R 'http.user_agent contains "omez"'

for  alertSite;
tshark -n -i eth5 -R 'http.user_agent contains "AlertSite"'

and for Keynote
tshark -n -i eth5 -R 'http.user_agent contains "KTXN"'

A sample  output;
Hypertext Transfer Protocol
    GET /css/search-nav-mq.1.0.2.min.css HTTP/1.1\r\n
        Request Method: GET
        Request URI: /css/search-nav-mq.1.0.2.min.css
        Request Version: HTTP/1.1
    Accept: */*\r\n
    Accept-Language: en-us\r\n
    Accept-Encoding: gzip, deflate\r\n
    User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GomezAgent 3.0)\r\n
    Host: www.overstock.com\r\n
    Connection: Keep-Alive\r\n
    [truncated] Cookie: SSLB=B; mxcsurftype=2; mxclastvisit=20121022; ostk_aggr_year=mxcuserseed^5874340604601831424|mxcskupage^120|language^en|pageresult^120|currency^USD|country^US; se_list=se_list^0|2|55|; ostk_aggr_year2=""; ostk_aggr_sess
    \r\n
Hypertext Transfer Protocol
    GET /foresee/foresee-surveydef.js HTTP/1.1\r\n
        Request Method: GET
        Request URI: /foresee/foresee-surveydef.js
        Request Version: HTTP/1.1
    Host: www.overstock.com\r\n
    User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GomezAgent 3.0)\r\n
    Accept: */*\r\n
    Accept-Language: en-us,en;q=0.5\r\n
    Accept-Encoding: gzip, deflate\r\n
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
    Connection: keep-alive\r\n
      [truncated] Cookie: SSLB=B; mxclastvisit=20121022; ostk_aggr_year=mxcuserseed^4448573595864717312|language^en|currency^USD|country^US; se_list=se_list^0|1|; ostk_aggr_year2=""; ostk_aggr_session=gcr^false|cart.item-count^0|dlp^k|billingcou
    \r\n
and
Hypertext Transfer Protocol
    GET / HTTP/1.1\r\n
        Request Method: GET
        Request URI: /
        Request Version: HTTP/1.1
    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; AlertSite)\r\n
    Accept-Language: en-us,en;q=0.5\r\n
    Host: www.aramco.com\r\n
    Connection: close\r\n
    \r\n
and
Hypertext Transfer Protocol
    GET / HTTP/1.1\r\n
        Request Method: GET
        Request URI: /
        Request Version: HTTP/1.1
    Accept: */*\r\n
    Accept-Language: en-US\r\n
    Accept-Encoding: gzip, deflate\r\n
    User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; KTXN B498853546A48324T1414872)\r\n
Hypertext Transfer Protocol
    GET / HTTP/1.1\r\n
        Request Method: GET
        Request URI: /
        Request Version: HTTP/1.1
    Accept: */*\r\n
    Accept-Language: en-US\r\n
    Accept-Encoding: gzip, deflate\r\n
    User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; KTXN B498853546A48324T1414872)\r\n
    Host: www.tdameritrade.com\r\n
    Connection: Keep-Alive\r\n
    \r\n


These common external testing sites User_Agents helps distinguish that a monitor site is hitting your URLs. Keep in mind, these could be spoof'd by a unethical hacker.

Ken Felix

Freelance network/security engineer
Ken a t hyperfeed.com