Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

On Linux filesystems we strongly recommend to use rsync to copy data. Here are some useful examples:

Example

Command

Note

Basic usage

rsync -Pauv $SDIR $SERV:/$DDIR/
  • Copies a source directory from the local server to the destination directory on a specified server.

  • Only newer files are copied.

  • Interrupted transmissions are continued

Copy to project directories

rsync -Pauv --no-group $SDIR $SERV:/$DDIR/
  • Everything beneath a project directory should belong to the project, hence chgrp -R $PROJECT $PROJ_DIR is executed automatically on several AWI systems.
  • A later/successive rsync into a project directory will reset (copy again) the file, if the group does not match. Use --no-group to prevent this. 

Set group/user and permissions

rsync -Pvrupog --chown=$USR:$GRP --chmod=D770,F660,Dg+s  $SDIR  $SERV:$/DDIR/
  • Sets user, group and sgid at the destination

  • please google yourself for more information on the specified options (wink)

Parallel rsync

find . -maxdepth 1 -type d -print0  | parallel -j8 -0 'rsync -Pauv {} $SERV:$/DDIR/'

cd $SDIR
ls | parallel -j8 rsync -Pauv {} $SERV:$DDIR/
ls | xargs -n1 -P8 -I% rsync -Pauv % $SERV:$DDIR/
  • Copy several (e.g. 8) directories at once.

  • Use with care, with a huge number you can easily jam the network (and you do surly (warning) not want to do that (wink))

On Windows you can install rsync via Cygwin or you can use https://en.wikipedia.org/wiki/Robocopy

...