Elvar started with all the correct elements to setup a functioning multrouting gateway. Two connections from two different providers, eth1, eth2. Running on an Ubuntu box with eth0 as internal. Both internet connections working on their own.

But alas. Whenever both eth1 and eth2 were active on the host, outgoing packets just would not go out. I don’t know if incoming packets were being replied to as we were unable to check that.

If just eth1 or eth2 was active than everything traversed ok. But we wanted it to work with both connections.

After a LONG time of diagnosing we noticed there were two default routes in iproute2 (ip’s fudged, but you get the idea):

firewall# ip route show
...
default via 1.1.2.1 dev eth2  metric 100
default via 1.1.3.1 dev eth1  metric 100

My firewall often has two default routes listed on the main table (ppp0, and ppp1) until the cleanup script fixes it. Without any negative side effect. I may just be lucky though.

Upon removing the eth2 line from table main, everything started working correctly. Incoming, outgoing, forwarding, balancing.

I also noticed the output of ifconfig eth2 looked a bit screwed too, but there was not much we could do about that as it was assigned by dhcp.

eth2      Link encap:Ethernet  HWaddr 00:11:22:33:44:55
    inet addr:1.1.2.1  Bcast:255.255.255.255  Mask:255.255.255.0

See it? No not the MAC address. The broadcast address. A quick ipcalc 1.1.2.1/24 gives me a broadcast of 1.1.2.255. But once we removed the eth2 default route line, it all started working again and didn’t get to dig into it to see if the broadcast actually affected it.

So that’s all. Just remember there are a lot more things to go wrong in a multigateway setup, including things outside of iptables.

This only balances outgoing traffic as incoming traffic balanced via DNS RR and the firewall just returns the traffic on the interface it arrived on as per the previous post.

On the whole, I prefer the iptables solution. It seems to balance the traffic better. ip route balances outgoing connections based on nexthop of the route to that host is not already in it’s routing cache. While iptables balances traffic by alternate outgoing connections. The only downside I have seen is occasional connection drops to the BlackBerry servers.

After 24 hours of iptables balancing:

ppp0      Link encap:Point-to-Point Protocol
          RX bytes:1186783900 (1.1 GB)  TX bytes:1290603327 (1.2 GB)
ppp1      Link encap:Point-to-Point Protocol
          RX bytes:1109227490 (1.1 GB)  TX bytes:1140565429 (1.1 GB)

This is using inclusion rules for determining balanced traffic. These are the rules that ended up on the production server:

# Load balancing rules (Split 50/50 between fwmark 1/2)
iptables -t mangle -A balance1 -d 192.168.0.0/16      -j RETURN
iptables -t mangle -A balance1 -d 10.0.0.0/8          -j RETURN
iptables -t mangle -A balance1 -m connmark ! --mark 0 -j RETURN
iptables -t mangle -A balance1 -m state --state ESTABLISHED,RELATED -j RETURN
iptables -t mangle -A balance1 -m statistic --mode nth --every 2 --packet 0 -j CONNMARK --set-mark 1
iptables -t mangle -A balance1 -m statistic --mode nth --every 2 --packet 1 -j CONNMARK --set-mark 2

# Check to see if we have already marked a packet
iptables -t mangle -A PREROUTING  -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT      -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark

# Mark incoming connections to return on the interface they came in on
iptables -t mangle -A PREROUTING          -i ppp0                     -m state --state NEW  -j CONNMARK --set-mark 1
iptables -t mangle -A PREROUTING          -i ppp1                     -m state --state NEW  -j CONNMARK --set-mark 2

# New outgoing packets
iptables -t mangle -A PREROUTING  -i eth0          -p tcp --dport  22 -m state --state NEW  -j balance1
iptables -t mangle -A PREROUTING  -i eth0          -p tcp --dport  25 -m state --state NEW  -j balance1
iptables -t mangle -A PREROUTING  -i eth0          -p tcp --dport  80 -m state --state NEW  -j balance1
iptables -t mangle -A PREROUTING  -i eth0          -p tcp --dport 443 -m state --state NEW  -j balance1
iptables -t mangle -A OUTPUT                       -p tcp --dport  80 -m state --state NEW  -j balance1

# Choose our route and save the mark
iptables -t mangle -A PREROUTING  -m connmark --mark 1 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING  -m connmark --mark 2 -j MARK --set-mark 2
iptables -t mangle -A PREROUTING  -m state --state NEW -m connmark ! --mark 0  -j CONNMARK --save-mark

That’s all of the CONNMARK and MARK related rules I use.

The new outgoing packets section is where I choose what packets should be balanced and accounts for about 95% of our outgoing traffic.

The balance1 chain just has some checks at the beginning to catch further traffic that should not be balanced in case some rule gets messed up.

Of the new outgoing packets rules, the PREROUTE lines are for forwarded traffic and the OUTPUT rule is for traffic generated on that host by a transparent squid proxy.

Hope somebody finds that useful one day.

Running on Ubuntu-9.10 with two internet connections ppp0 and ppp1 both with static IP’s from two different internet providers in Australia (iiNet and Internode).

Preperation:

Extra ip route tables per gateway.

Add tables to /etc/iproute2/rt_tables. Table names and numbers can be anything as long as they are consistent later on.

echo -e "101 connection1\n102 connection2" | sudo tee -a /etc/iproute2/rt_tables

Add routes to the extra rule tables. Copy the local routes from the main table then add the default gateway specific to this connection. Replace the vars at the beginning with your relevant settings.

#!/bin/sh
DEV1=ppp0
IP1=100.0.1.1
GW1=100.0.1.254
TABLE2=connection2
DEV2=ppp1
IP2=100.0.2.1
GW2=100.0.2.254
ip route flush table $TABLE1
ip route flush table $TABLE2
ip route show table main | grep -Ev '(^default|ppp)' | while read ROUTE ; do
    ip route add table $TABLE1 $ROUTE
    ip route add table $TABLE2 $ROUTE
done
ip route add table $TABLE1 $GW1 dev $DEV1 src $IP1
ip route add table $TABLE2 $GW2 dev $DEV2 src $IP2
ip route add table $TABLE1 default via $GW1
ip route add table $TABLE2 default via $GW2

ip route output:

~# ip route show
100.0.1.254 dev ppp0  proto kernel  scope link  src 100.0.1.1
100.0.2.254 dev ppp1  proto kernel  scope link  src 100.0.2.1
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
default via 100.0.1.254 dev ppp0

~# ip route show table connection1
100.0.1.254 dev ppp0  proto kernel  scope link  src 100.0.1.1
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
default via 100.0.1.254 dev ppp0

~# ip route show table connection2
100.0.2.254 dev ppp1  proto kernel  scope link  src 100.0.2.1
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
default via 100.0.2.254 dev ppp1

Add the ip rules:

ip rule add from 100.0.1.1 lookup connection1
ip rule add from 100.0.2.1 lookup connection2
ip rule add fwmark 1 lookup connection1
ip rule add fwmark 2 lookup connection2

Add the iptables rules for SNAT:

iptables -A POSTROUTING -o ppp0 -j SNAT --to-source 100.0.1.1
iptables -A POSTROUTING -o ppp1 -j SNAT --to-source 100.0.2.1

And finally add the rules for marking the connection they should be going out on. The first PREROUTING rule is for packets we forward to be returned via the interface they were received on. The OUTPUT rule is for packets handled on this PC to be returned on the correct interface too. We only want to mark new packets and restore marks on established connections else the packets

-A PREROUTING          -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark
-A OUTPUT              -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark
-A PREROUTING -i ppp0  -m state --state NEW                 -j CONNMARK --set-mark 1
-A PREROUTING -i ppp1  -m state --state NEW                 -j CONNMARK --set-mark 2
-A PREROUTING -m connmark --mark 1                          -j MARK --set-mark 1
-A PREROUTING -m connmark --mark 2                          -j MARK --set-mark 2
-A PREROUTING -m state --state NEW -m connmark ! --mark 0   -j CONNMARK --save-mark

Selective routing:

To send all outgoing traffic on a specific table:

-A PREROUTING -i eth0 -m state --state NEW -p tcp --dport  80 -j CONNMARK --set-mark 2
-A PREROUTING -i eth0 -m state --state NEW -p tcp --dport 443 -j CONNMARK --set-mark 2

References:

http://www.clintoneast.com/articles/multihomed.php

http://linux-ip.net/html/adv-multi-internet.html

http://www.cyrius.com/debian/orion/d-link/dns-323/install.html

The install took quite a while, I selected the standard package list. Then because it gets quit hot here in Griffith I really wanted to have fan control back, so instruction here were also followed.

As I wanted to install to an 8GB Flash drive I had handy I first uploaded the netboot.img firmware, turned off the DNS-323, removed the hard drives and left only the flash drive inserted, then powered up and followed the installation instructions.

I like keeping root on the flash drive as the hard drives can be stopped all day when not in use to save temp and power.

Fans are also controlled by lm-sensors and the unit seems to run about 47degC most of the day.

I use the unit mostly for irc (screen/weechat), rsync backup target and samba media server. It handles all that pretty well but I do notice it bog a fair bit with only 64mb ram and hard drive transfer rates are not as good as I am used to, but on the whole a great little unit (now that it runs debian).

I purchased one from Dick Smiths last night plugged it into my (Ubuntu-8.10 powered) laptop and was up and running within seconds.

Signal strength and speed are fairly industry standard. Cost AU$50. I probably could have found it cheaper elseware but DSE was conveniently on my way home last night.

D-Link DWA-110

Now just to find an access point with decent signal strength.

This document will give you setup instructions for installing PostBooks from scratch. Ie, no previous install of postgresql or PostBooks on the system. You will need to read between the lines if your install does not fit that description.

Download PostBooks

Download the postbooks init and postbooks-empty (or quickstart/demo) to /tmp. You can find the latest files here. Ie:

wget -P /tmp http://downloads.sourceforge.net/postbooks/init-2.3.0beta1.sql?use_mirror=optusnet  \

http://downloads.sourceforge.net/postbooks/postbooks_empty-2.3.0beta1.backup?use_mirror=optusnet

Install PostgreSQL

sudo apt-get install postgresql postgresql-contrib

At this time I also install openssh-server mc and vim. If your more comfortable using an editor other than vim, just replace any mention of vim with your preference.

Create mfgadmin

Now to create the mfgadmin user in postgresql. The default mfgadmin password is mfgadmin. Please change this upon first connecting with the PostBooks client.

sudo -u postgres psql -U postgres -f /tmp/init-2.3.0alpha1.sql template1

Create a company

Now we create an ‘company’ database. You could create a few databases here if needed, using

sudo -u postgres createdb -h localhost -U mfgadmin mfg

Restore PostBooks to the company

sudo -u postgres pg_restore -h localhost -U mfgadmin -d mfg /tmp/postbooks_empty-2.3.0alpha1.backup -v

Allow network access to postgresql

to the following files add the indicated lines:

sudo vim /etc/postgresql/8.2/main/pg_hba.conf
host all all 172.16.80.0/24 md5

sudo vim /etc/postgresql/8.2/main/postgresql.conf
listen_addresses = '*'

And now restart postgresql

sudo /etc/init.d/postgresql-8.2 restart

Finished

You should now be able to open the PostBooks client application and connect to the ‘mfg’ database with the username and password mfgadmin/mfgadmin

If you want to create another company just go back to the Create a company step and replace mfg with your desired company database name.

I only use two devices for cursor movement. The trackpad, and a Logitech bluetooth mouse. After booting with an Ubuntu live cd, the trackpad moves at a medium-quick pace, and once I connect to the mouse, the mouse moves at a very-fast pace.

xorg.conf

My xorg.conf file has been sum what adjusted from default, but my server layout here has two pointing devices,  Mouse0 as CorePointer and Synaptics Touchpad.

Section "ServerLayout"
    Identifier     "Default Layout"
    Screen         "Default Screen" 0 0
    InputDevice    "Generic Keyboard" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Synaptics Touchpad"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
    Option         "Sensitivity" "0.3"
EndSection

Section "InputDevice"
    Identifier     "Synaptics Touchpad"
    Driver         "synaptics"
    Option         "SendCoreEvents" "true"
    Option         "Device" "/dev/psaux"
    Option         "Protocol" "auto-dev"
    Option         "HorizEdgeScroll" "0"
    Option         "SHMConfig" "On"
    Option         "MaxSpeed" "0.15"
EndSection

I cannot remember for sure which how much of the xorg.conf file I have changed in total, but the two lines in bold are the ones controlling device speed.

Synaptics Touchpad Speed

For the synaptics device there are three main lines to control pointer speed.

Option "MinSpeed" "<float>"
Option "MaxSpeed" "<float>"
Option "AccelFactor" "<float>"

The command to find your current touchpad settings is (in bold):

~/ synclient -l | grep -E '( MinSpeed| MaxSpeed| AccelFactor)'
    MinSpeed                = 0.0822368
    MaxSpeed                = 0.15
    AccelFactor             = 0.00164474

There you can see the MaxSpeed that I have changed.

You can also test touchpad speed adjustments without having to restart X by using synclient [var1=value1 [var2=value2] …]. Eg:

~/ synclient MaxSpeed=0.15

If you are following this example, also add the SHMConfig line if it is not yet present, as it adds the ‘Touchpad’ tab to the mouse preferences screen that allows you to set the touchpad to turn off while typing.

Mouse Speed

I cannot remember what the speed is relative to for the mouse section, but some quick tweeking and playing should see you straight. You will need to restart X each time you adjust the setting for it to take effect.