Ubuntu Bonding vs PPPoE multilink (3/3)

In this post we will measure the performance on the PPPoE multilink over 2 gigabit connection.

This is the 3rd article in a series, the first article is here and the second is here.

The test setup is shown in this diagram:

In this article we will test PPPoE multilink over the two Ethernet interfaces.


Setting up PPPoE multilink in kernel mode on Ubuntu 1604 and 1804.

First we set up the server side:

You need to install the pppoe package

sudo apt install pppoe

You need to configure the server connection (/etc/ppp/pppoe-server-options):

require-pap
login
lcp-echo-interval 10
lcp-echo-failure 2
noauth
multilink

Edit (/etc/ppp/pap-sercrets) and lines for the authentication of clients like:

# client server secret acceptable_local_IP_addresses
  user   *      "password"

Then you should start the pppoe-server:

sudo pppoe-server -k -I eth2card0 -I eth2card1 -L 10.15.0.1 -l -R 10.16.0.1 -N 256 -O /etc/ppp/pppoe-server-options

After the -I you can specify the Ethernet interfaces to listen to. -L is local IP address, -R is remote address, -k is kernel mode.

On the client side:

Install the pppoe package.

Define connection parameters in /etc/ppp/peers/connection1

plugin rp-pppoe.so eth2card0
noipdefault
hide-password
lcp-echo-interval 20
lcp-echo-failure 3
noauth
persist
mtu 1492
noaccomp
default-asyncmap
multilink

Do the same for connection 2, but with a different Ethernet interface.

Add your password to /etc/ppp/pap-secrets the same way as you did on the server side.

Before connecting you have to bring the interfaces up in both sides:

sudo ip link set dev enp4s0 up
sudo ip link set dev enp5s0 up

Then connect with

pon connection1
pon connection2

To troubleshoot and check results use:

cat /var/log/syslog | grep ppp

To check the connectivity from orca:

ping 10.16.0.1

To do the tests add the 192.168.190.x ip address to the interfaces. You can do this by configuring the pppoe server as well, but for our measurements the manual method will do.

sudo ip address add 192.168.190.11/24 dev ppp0

At first time the results were not very impressive so I had to do 2 cheats:

1. Instead of using 2 Ethernet cards on a PCIe-1 dual Ethernet card, I got better results by using one from the card and the one from the main board.
2. Both on boss and NAS2 I have add the -A0 processor affinity flag for iperf3, whihc results in better troughput

Now let's do the iperf3 tests to NAS2:

Computer To(Mb/s) From(Mb/s) To(MB/s) From(MB/s)


Orca       1520     1480       190      185
Boss        911      932       113      117
Blue        913      944       114      118
Backup     1650     1510       206      189
VM Windows 1580     1480       198      185
VM Ubuntu  1550     1250       194      156

And the samba tests:

Computer  To(MB/s) From(MB/s)

Orca        135       93

Boss         83      112

Blue         75       99

Backup      139      108

VM Windows  162      111

VM Ubuntu    80      144

Analysis of the results

If we look into RAW IP connectivity:

We can see that it looks promising for the PPPoE Multilink, there is some degradation for the external 1G connections, but this is acceptable compared to the gain on directions where there is more bandwidth.

If we look at the Samba performance:

The situation changes. There is no solution where there is no significant degradation compared to a single Ethernet connection. 

So the conclusion is clear, don't spend your time in trying to set up multilink connections, simply buy the already available and affordable multigigabit equipment.

There may be one exception:

The specialty of PPPoE is that on the same Ethernet interface you can run normal IP and PPPoE parallel, so you may create a setup, where the server is accessible via multilink on one IP and via normal Ethernet on an other, and for cases where the multilink is not performing well, just use the normal IP.