It is desirable, in some circumstances on a server with multiple NIC’s to send outgoing traffic on incoming interface. In order to do so the server has to have policy routing implemented.

Let’s assume that the server is running CentOS 6 and that there are two network interfaces as follows:

  • eth0 - 10.10.10.1/24 with gateway 10.10.10.254
  • eth1 - 192.168.1.0/24 with gateway 192.168.1.254

Here is a simple example. First create routing table /etc/iproute2/rt_tables:

50	internal1
60	internal2

Names internal1 and internal2 are just random names. Now, create /etc/sysconfig/network-scripts/rule-eth0 which just defines under which circumstances routing table should be used:

iif eth0 table internal1
from 10.10.10.1 table internal1

Next, define a rule for the second network interface /etc/sysconfig/network-scripts/rule-eth1 as follows:

iif eth1 table internal2
from 192.168.1.1 table internal2

Next, define route to 10.10.10.0/24 using eth0 interface. The following lines go into /etc/sysconfig/network-scripts/route-eth0:

10.10.10.0/24 dev eth0 table internal1
default via 10.10.10.254 dev eth0 table internal1

And the same thing needs to be done for the second interface - eth1. The following lines need to be placed in /etc/sysconfig/network-scripts/route-eth1:

192.168.1.0/24 dev eth1 table internal2
default via 192.168.1.254 dev eth0 table internal2

Finally, remove GATEWAY stanza from /etc/sysconfig/network. That’s it. This will persist reboots as well.