Notes on multi gateway routing with iptables and iproute2, suggestions and corrections gladly accepted. My notes may be incomplete or just plain wrong, I pieced them together after getting it working.

Running on Ubuntu-9.10 with two internet connections ppp0 and ppp1 both with static IP’s from two different internet providers in Australia (iiNet and Internode).

Preperation:

Extra ip route tables per gateway.

Add tables to /etc/iproute2/rt_tables. Table names and numbers can be anything as long as they are consistent later on.

echo -e "101 connection1n102 connection2" | sudo tee -a /etc/iproute2/rt_tables

Add routes to the extra rule tables. Copy the local routes from the main table then add the default gateway specific to this connection. Replace the vars at the beginning with your relevant settings.

#!/bin/sh
DEV1=ppp0
IP1=100.0.1.1
GW1=100.0.1.254
TABLE2=connection2
DEV2=ppp1
IP2=100.0.2.1
GW2=100.0.2.254
ip route flush table $TABLE1
ip route flush table $TABLE2
ip route show table main | grep -Ev '(^default|ppp)' | while read ROUTE ; do
    ip route add table $TABLE1 $ROUTE
    ip route add table $TABLE2 $ROUTE
done
ip route add table $TABLE1 $GW1 dev $DEV1 src $IP1
ip route add table $TABLE2 $GW2 dev $DEV2 src $IP2
ip route add table $TABLE1 default via $GW1
ip route add table $TABLE2 default via $GW2

ip route output:

~# ip route show
100.0.1.254 dev ppp0  proto kernel  scope link  src 100.0.1.1
100.0.2.254 dev ppp1  proto kernel  scope link  src 100.0.2.1
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
default via 100.0.1.254 dev ppp0

~# ip route show table connection1
100.0.1.254 dev ppp0  proto kernel  scope link  src 100.0.1.1
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
default via 100.0.1.254 dev ppp0

~# ip route show table connection2
100.0.2.254 dev ppp1  proto kernel  scope link  src 100.0.2.1
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
default via 100.0.2.254 dev ppp1

Add the ip rules:

ip rule add from 100.0.1.1 lookup connection1
ip rule add from 100.0.2.1 lookup connection2
ip rule add fwmark 1 lookup connection1
ip rule add fwmark 2 lookup connection2

Add the iptables rules for SNAT:

iptables -A POSTROUTING -o ppp0 -j SNAT --to-source 100.0.1.1
iptables -A POSTROUTING -o ppp1 -j SNAT --to-source 100.0.2.1

And finally add the rules for marking the connection they should be going out on. The first PREROUTING rule is for packets we forward to be returned via the interface they were received on. The OUTPUT rule is for packets handled on this PC to be returned on the correct interface too. We only want to mark new packets and restore marks on established connections else the packets

-A PREROUTING          -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark
-A OUTPUT              -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark
-A PREROUTING -i ppp0  -m state --state NEW                 -j CONNMARK --set-mark 1
-A PREROUTING -i ppp1  -m state --state NEW                 -j CONNMARK --set-mark 2
-A PREROUTING -m connmark --mark 1                          -j MARK --set-mark 1
-A PREROUTING -m connmark --mark 2                          -j MARK --set-mark 2
-A PREROUTING -m state --state NEW -m connmark ! --mark 0   -j CONNMARK --save-mark

Selective routing:

To send all outgoing traffic on a specific table:

-A PREROUTING -i eth0 -m state --state NEW -p tcp --dport  80 -j CONNMARK --set-mark 2
-A PREROUTING -i eth0 -m state --state NEW -p tcp --dport 443 -j CONNMARK --set-mark 2

References:

http://www.clintoneast.com/articles/multihomed.php

http://linux-ip.net/html/adv-multi-internet.html