It is desirable, in some circumstances on a server with multiple NIC’s to send outgoing traffic on incoming interface. In order to do so the server has to have policy routing implemented.

Let’s assume that the server is running CentOS 6 and that there are two network interfaces as follows:

  • eth0 - with gateway
  • eth1 - with gateway

Here is a simple example. First create routing table /etc/iproute2/rt_tables:

50	internal1
60	internal2

Names internal1 and internal2 are just random names. Now, create /etc/sysconfig/network-scripts/rule-eth0 which just defines under which circumstances routing table should be used:

iif eth0 table internal1
from table internal1

Next, define a rule for the second network interface /etc/sysconfig/network-scripts/rule-eth1 as follows:

iif eth1 table internal2
from table internal2

Next, define route to using eth0 interface. The following lines go into /etc/sysconfig/network-scripts/route-eth0: dev eth0 table internal1
default via dev eth0 table internal1

And the same thing needs to be done for the second interface - eth1. The following lines need to be placed in /etc/sysconfig/network-scripts/route-eth1: dev eth1 table internal2
default via dev eth0 table internal2

Finally, remove GATEWAY stanza from /etc/sysconfig/network. That’s it. This will persist reboots as well.