Quite some time ago I discovered on a group of backup servers a bunch of very peculiar settings, regarding the compression of the ssh connection.
One line in particular in the ssh client configuration file /etc/ssh/ssh_config:
CompressionLevel 9
All the backups were using rsync over ssh. Since they will need a long time to complete, sometimes they would suffer under heavy cpu load and they could also trigger alarms during the night, I simply thought some testing was in order.
To enable the compression with scp command, the -C parameter can be used. Otherwise the compression is disabled by default.
In these backups the compression were enabled in this way:
rsync -e "ssh -C ..." ...
Not sure how it got there in the first place, but that’s not important like the test below!
To make some tests then I created this 2 files:
prova.zero, 100% compressible file, composed of zeroes
prova.random, 0% compressible file, courtesy of /dev/urandom
Then I’ve choosen one of the server and tried some file transfers:
prova.zero –> no compression –> 9.1MB/s
prova.zero –> compression 6 (default) –> 13.3MB/s
prova.zero –> compression 9 (max) –> 8.0MB/s
prova.zero –> compression 1 (min) –> 55.5MB/s
prova.random –> no compression –> 8.6MB/s
prova.random –> compression 6 (default) –> 2.9 MB/s
prova.random –> compression 9 (max) –> 2.1 MB/s
prova.random –> compression 1 (min) –> 5.7MB/s
What strikes you is:
1. 9MB/s is without doubt the connection speed
2. using compression level 9 with a 100% compressible file in this case still slow down the transfer.
My conclusions are that, regardless of the compression capability of the file, using a compression on a remote site is something to avoid firmly. If someone has still a crazy desire to use it, it is strongly suggested to set the compression at level 1.
During the test with compression 9, I also generated a high loadavg on both the remote and the backup servers.
That was for remote usage: enabling the connection compression between LAN servers I think can be considered a trivial error.
For the same reasons it is a bad idea to use the -z parameter in rsync, that I’ve seen can be easily mistaken for a good thing, but it’s just a trap.