Recently my wonderful ISP, Brighthouse networks of Central Florida began having issues with their network. it bagan with a handful of websites not coming up, which is not really that big of a deal, but when my wife was unable to reach her work (she's a medical transcriptionist) it became a big issue. So I did some diggin and found that we were getting out just fine, but we were just losing packets and never getting a response. Way to go Brighthouse!
My brain immediately told me that if I could reach one of my Linux boxes on another network that I might be able to set up a tunnel and create and artificial route and bypass whatever routes Brighthouse was taking. However it's Christmas weekend so I was not too excited about taking the time to figure out how to actually implement my theories. I left it up to the wife and she said don't take the time.
The problem started on Friday, Christmas, and even though I knew I would get some joker unlucky enough to be working on Christmas, I decided to call them and let them know they were having problems. The call went as suspected...I told the tech that Brighthouse was having routing issues and that I wanted to open a ticket so it could get escalated before the issue got worse. He asked to me to turn my cable modem off and then back on. I'll leave out the rest, but needless to say it was a pathetic display of network engineering and I simply hung up after trying to explain it to him. Come Saturday around 6 oclock it was worse. Now my wife wanted to work and I was sleeping, so she called Brighthouse again and this time they actually admitted to the routing issues and owed up to "over a thousand customers affected". This was a nice admission but did not solve the problem.
Well I have had a great weekend. I have played Call Of Duty Modern Warfare2 for just about the whole weekend on my new PS3 so I figured I might as well take a shot at my tunnel theory. The first order of business was deciding what TYPE of tunnel to use. There are plenty of software packages for VPNs, but All I wanted was traffic to particular hosts that Brighthouse could not reach, to go to a host they could reach that had a route to the desired destinations.
Here is how I did it:
1) I chose the GRE protocol.
2) On my home linux box I set up a tunnel to my remote linux box like so:
ip tunnel add TUNNELNAME mode gre remote REMOTEIP local LOCALIP ttl 255
TUNNELNAME is arbitrary. REMOTEIP should be public routable ips respectively.
3) Next activate the new tunnel
ip link set TUNNELNAME up
4) Now give the new interface an ip address.
ip addr add 10.0.1.1 dev TUNNELNAME
5) Next we do the same steps with slightly different info on the remote box
ip tunnel add TUNNELNAME mode gre remote REMOTEIP(LOCALIP FROM PREVIOUS) local LOCALIP(REMOTEIP FROM PREVIOUS) ttl 255
ip link set TUNNELNAME up
ip addr add 10.0.2.1 dev TUNNELNAME
6) The ip addresses I gave to these tunnel interfaces were completely random, they could be anything, however I used a
non routable range to simulate joining two LANs.
Everything
from this point on is manipulating the routing tables and a little iptables magic:
On my home router:
ip route add 10.0.2.0/24 dev TUNNELNAME
ip route add 192.168.0.0/24 dev TUNNELNAME (This is my home subnet)
I continued these for every network that was having issues.
On the remote router:
ip route add 10.0.1.0/24 dev TUNNELNAME
ip route add 192.168.0.0/24 TUNNELNAME
iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu (this is to compensate for the extra overhead of encapsulation)
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
Thats it! With those tunnels in place I was able to get to all the sites that Brighthouse was unable to route to.
The downfall is the maintenance. I currently have to add networks every time I hit one that is broken via Brighthouse.
What I would like to research how to do is to place a timeout on a request so that if a packet goes out the default route,
but no reply is received within say 5 seconds it tries the tunnel route. If anyone knows how to do that
please comment and let me know. Lastly this whole set up is based upon he fact that the issues are NOT global.
If Brighthouse were unable to get to my remote server all this would be moot as the tunnel could not be made.
