rsync is MUCH faster than scp

Working at a hosting company that specializes in J2EE application hosting, we move EARs around a lot.  Typically I’ve just been using scp since rsync can easily get you into trouble if you’re pushing deltas you’re not super careful.  And if you’re not pushing deltas, and pushing full ears, it makes more sense to use scp rather than rsync over ssh, which just adds extra overhead.  Right?

Apparently wrong.  After doing some real testing here are some surprising numbers.  All of this was done on private Gig-E networks between beefy RHEL 5 linux servers.

318 MB Exploded EAR file:
scp – 85.8 seconds
rsync – 9 seconds

199 MB compressed tgz EAR file:
scp – 14.7 seconds
rsync – 6.3 seconds

Even doing unprimed transfers, rsync is 2-10 times faster than scp.  The moral of this story is for transfers (scripted or manual) rsync can be MUCH faster.  And of course if you can use primed syncs, rsync is MUCH faster again.


Posted

in

by

Tags:

Comments

3 responses to “rsync is MUCH faster than scp”

  1. Will Sargent Avatar

    This is because of rsync’s block transfer protocol. It’s pretty good at figuring out which parts of a file have changed and only pulling the deltas: http://tutorials.jenkov.com/rsync/network-protocol.html

    1. Devon Avatar

      Why would that help for unprimed transfers? By unprimed I mean this is the first time the file or directory has been copied over.

  2. Oommen Avatar
    Oommen

    Please share the commands (both scp and rsync) used while benchmarking. If you have used -z option in rsync, you may get a better speed. Hence you have to compare the speed with scp with -C option. Please paste the commands or do it again with comparable options.

Leave a Reply

Your email address will not be published. Required fields are marked *

PHP Code Snippets Powered By : XYZScripts.com