I’m just not a Networking Guru

Acquired a Fortinet Fortigate 100A (pdf link) firewall/internet-gateway device for the office. Overall impression: pretty decent product. Configuration and administration are simple, whether through the Web interface or command console. Works well for our purposes too (small business network, 20 users) and has handled everything we’ve been able to throw at it. And then…

The hardware architect (my counterpart on the hardware team) noticed that a certain group of our mobile devices were connecting sporadically to our network. He setup the same type of modem on his laptop and would try to open a simple TCP connection and it would sometimes succeed and sometimes fail. We noticed that this started happening around the time we installed the new device.

Oh crap.

Couple of calls to our supplier and we spent a Saturday with their top tech hunting down this issue. The weird part is that it wasn’t showing up in the device’s memory logs, nor in the syslog (after we set up a syslog server). These connections were getting mysteriously bounced.

The network tech swapped in a Cisco 506e (roughly equivalent). Same behaviour!? Hmmm… Packet sniffing time. I had no idea what was in the logs. They were forwarded to Cisco and Fortinet and we’d hear back later.

In the meantime, our hardware guru sets up a Suse Linux 9.3 box with basic gateway/firewalling features turned on. Things were looking good. Then things started to deteriorate almost immediately and we were back to square one.

Cisco and Fortinet get back to our supplier’s tech guy and say that the issue is with the handshake between the particular network gateway our mobile devices originate from and our device. Apparently we’re getting packets with their ACK flags set, from an ip/port pair that hasn’t established a connection; they’re getting bounced WAY before the firewall even analyzes anything. *phew* the devices aren’t nutty. So what now?

Reconfigure the Linux box. Add some logging. Then do a lot of testing from our undeployed modems. Survey says:

A connection will be initiated from the external network. The source-ip/port are logged by the FW. The transaction continues as per normal, then it terminates normally. A short time later (40sec – 2min or so) we’ll start to receive packets, with their ACK flags set, from the closed source-ip/port.

Time to open up a conversation with the mobile-device network tech-support.

Grr…

What fun!

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: