Link aggregation takes a bunch of network interfaces and creates a big pipe out of them.

Aggregation also provides redundancy. If all interfaces but one go down, the server will remain connected to the network.

Before starting make sure that:

  1. interfaces to be aggregated are of the following type: xge, e1000g, and bge
  2. interfaces to be aggregated are not plumbed
  3. they run in full duplex mode at the same speeds
  4. eeprom’s local-mac-address? variable is set to true

The following will create aggr1 interface with bge1 as one of its members:

bash-3.00# dladm create-aggr -d bge1 1

Next plumb the aggregate interface, configure an IP address on it and bring it up:

bash-3.00# ifconfig aggr1 plumb 192.168.1.5 netmask 255.255.255.0 up

At this point you can list aggregations: ```terminalbash-3.00# dladm show-aggr
key: 1 (0x0001) policy: L4 address: 0:3:ba:56:7f:ba (auto) device address speed duplex link state bge1 0:3:ba:56:7f:ba 0 Mbps unknown down standby


Now add _bge0_ as a second member of _aggr1_ aggregation interface and list aggregate interfaces:
```terminal
bash-3.00# dladm add-aggr -d bge0 1  
bash-3.00#  dladm show-aggr  
key: 1 (0x0001) policy: L4      address: 0:3:ba:56:7f:ba (auto)
           device       address                 speed           duplex  link    state
           bge1         0:3:ba:56:7f:ba   0     Mbps    unknown down    standby
           bge0         0:3:ba:56:7f:b9   1000  Mbps    full    up      attached

To keep the configuration persistent across reboots, create /etc/hostname.aggr1 with appropriate content and remove any hostname.* files pertaining to the interfaces that are now members of aggr1.

To have link aggregation working properly you need to have the switch to which server is connected to properly configured with LACP.

Another thing to consider is load balancing policy for outgoing traffic. You can load balance on layers 2,3 and 4. Load balancing policy can be changed using dladm command. Here is a quick example that will modify load balancing policy to combination of L3 and L4:

bash-3.00# dladm modify-aggr -P L3,L4 1  
bash-3.00#  dladm show-aggr -L  
key: 1 (0x0001) policy: L3,L4   address: 0:3:ba:56:7f:ba (auto)
                LACP mode: off  LACP timer: short
    device    activity timeout aggregatable sync  coll dist defaulted expired
    bge1      passive  short   yes          no    no   no   no        no
    bge0      passive  short   yes          no    no   no   no        no

And finally, command that will allow you to see utilisation of individual links within aggregation. Note the %ipkts column, I did not have LACP turned on on the switch at that time:

bash-3.00# dladm show-aggr -s  
key: 1  ipackets  rbytes      opackets   obytes          %ipkts %opkts
           Total        2723785   2287233197  1481682   710633551
           bge1 618712    115674760   870443    636559150       22.7    58.7
           bge0 2105073   2171558437  611239    74074401        77.3    41.3

More info on link aggregation is here.