Rsync

rsync command is used to copy and synchronization files and folders. Rsync is a single thread C application that includes a progress indicator.

Configuration files

• /etc/rsyncd.conf which includes don't compress options for some file extensions

Options

-W, --whole-file

-z, --compress compresses the file data as it is sent to the destination machine using zlib compression library

Basic commands

• rsync -Pr ORIGIN_FOLDER DESTINATION_FOLDER

-P, --partial --progress. --partial keep partially transferred files

-r --recursive

• rsync -P ORIGIN DESTINATION

-P, --partial --progress. --partial keep partially transferred files

• Copy files from remote server to local machine: rsync -chavzP --stats [email protected]:/path/to/copy /path/to/local/storage (explainshell)

-c, --checksum option can slow transfers specially for large files

-h, --human-readable

-a, --archive. Equivalent to -rlptgoD

-v, --verbose

-z, --compress

-P, --partial --progress. --partial keep partially transferred files

--stats, print statistics

-a, --archive. Equivalent to -rlptgoD

-r --recursive recurse into directoriees

-l --links copy symlinks as symlinks

-p --perms preserve permissions

-t --times preserve modification times

-g --group preserve group

-o --owner preserve owner (super-user only)

-D same as --devices --specials

rsync -chavzP --stats [email protected]:/path/to/copy /path/to/local/storage
receiving incremental file list
.../...

• rsync -Pav --ignore-existing src dst

Learn difference between: --ignore-existing and --update

• rsync -av --ignore-existing src dst

Compression

rsync support compression, but you can also compress your data with other external compression tools before using rsync. Such as bzip, gzip or xz

Compressing using xz

• xz -v -T0 file_to_compress

-v provides progress bar

-T0 use all available Threads, multi-threaded compression supported since 2014, version 5.2.0.

Activities

1. Create a remote copy of your files for backup proposes using rsync
2. Read StackOverflow questions about rsync: https://stackoverflow.com/questions/tagged/rsync?tab=Votes
3. Understand the differences between cp, scp and rsync: https://stackoverflow.com/questions/20244585/how-does-scp-differ-from-rsync
4. Learn Effect of Trailing Slash / in rsync: https://www.alibabacloud.com/blog/speeding-up-network-file-transfers-with-rsync_594337
5. Learn use of --inplace (implies also --partial) for transferring large files and its important implications: http://man7.org/linux/man-pages/man1/rsync.1.html, also evaluate -c, --checksum impact on transfering large files
6. Learn about --sparse option and conflicts with --inplace

Performance

1. Understand how single thread rsync design can affect your transfer speeds.
2. Learn about -W, --whole-file copy files whole (without delta-xfer algorithm), implications and usage cases such as not making a diff copy.

See also

• cp, dd, sftp, scp, rsync, casync, mv, fio, ln, docker cp, kubectl cp, minikube cp, multipass transfer, Copy-Item, Xcopy
• dd, mv

Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy.

Source: https://en.wikiversity.org/wiki/Linux/Basic_commands/rsync