Enable Dark Mode!
how-to-use-rsync-commands-and-the-operations-summary.jpg
By: Risvana AR

How To Use Rsync Commands & The Operations Summary

Technical Odoo 16

Using a local machine or a secure network connection, the rsync command synchronizes files from a source to a destination. It serves as a quick, adaptable, and secure alternative to the rcp command. 

A quick and incredibly versatile tool for copying files is rsync. It can copy to or from a remote rsync daemon, another host over any remote shell, or locally. It provides a vast array of controls that allow for very flexible specifications of the list of files to be copied as well as complete control over every aspect of its behavior. It is well-known for its delta-transfer method, which minimizes the amount of data transmitted over the network by only transferring the differences between the source files and the already-existing files at the destination. rsync is frequently used as a better copy command for everyday use, mirroring, and backups.

By default, the "quick check" technique used by rsync looks for files that have changed in size or last-modified time to identify those that need to be transferred. When the fast check shows that the file's data does not require updating, any modifications to the other preserved attributes (as requested by options) are made immediately on the destination file.

rsync has a few more features, including 

1. Copying links, devices, owners, groups, and permissions supported.

2. Like GNU tar, exclude and exclude-from options

3. Use any transparent remote shell, such as ssh or rsh, to ignore the same files that CVS would ignore with a CVS exclude mode.

4. does not call for super-user rights

5. File transfers are pipelined to reduce latency expenses.

6. support for rsync daemons that are anonymous or authenticated (ideal for mirroring)

Syntax:

Local use:

rsync [OPTION...] SOURCE. [DEST]

Access via a remote shell (PULL):  

rsync [OPTION.   ..] [USER@]HOST: SOURCE...... [DESTINATION]

Access via a remote shell (PUSH):  

rsync [OPTION...]SOURCE.....[USER@]HOST: DESTINATION

OPTIONS
-v, --verbose increase verbosity
-q, --quietsuppress non-error messages
-a, --archivemode; equals -rlptgoD (no -H,-A,-X)
--no-OPTIONturn off an implied OPTION (e.g., --no-D)
-r, --recursiverecurse into directories
-R, --relativeuse relative path names
-b, --backupmake backups (see --suffix & --backup-dir)
-u, --updateskip files that are newer on the receiver
--appenddata onto shorter files
--append-verify--append w/old data in file checksum
-L, --copy-linkstransform symlink into referent file/dir
--copy-unsafe-linksonly "unsafe" symlinks are transformed
--safe-linksignore symlinks that point outside the tree
-k, --copy-dirlinkstransform symlink to dir into referent dir
-K, --keep-dirlinkstreat symlinked dir on receiver as dir
-H, --hard-linkspreserve hard links
-p, --permspreserve permissions
-E, --executabilitypreserve executability
--chmod=CHMODaffect file and/or directory permissions
-X, --xattrspreserve extended attributes
-o, --ownerpreserve owner (super user only)
-g, --grouppreserve group
--devicespreserve device files (super-user only)
--specialspreserve special files
-Dsame as --devices --specials
-t, --timespreserve modification times
-O, --omit-dir-timesomit directories from --times
--superreceiver attempts super-user activities
--fake-superstore/recover privileged attrs using xattrs
-x, --one-file-systemdon't cross filesystem boundaries
-B, --block-size=SIZEforce a fixed checksum block-size
-e, --rsh=COMMANDspecify the remote shell to use
--rsync-path=PROGRAMspecify the rsync to run on remote machine
--existingskip creating new files on receiver
--delan alias for --delete-during
--deletedelete extraneous files from dest dirs
--size-onlyskip files that match in size
--modify-window=NUMcompare mod-times with reduced accuracy
-T, --temp-dir=DIRcreate temporary files in directory DIR
-y, --fuzzyfind similar file for basis if no dest file
--compare-dest=DIRalso compare received files relative to DIR
--copy-dest=DIR... and include copies of unchanged files
--link-dest=DIRhardlink to files in DIR when unchanged
-z, --compresscompress file data during the transfer
-Fsame as --filter='dir-merge /.rsync-filter'
repeated: --filter='- .rsync-filter'
--exclude=PATTERNexclude files matching PATTERN
--exclude-from=FILEread exclude patterns from FILE
--include=PATTERNdon't exclude files matching PATTERN
--include-from=FILEread include patterns from FILE
--files-from=FILEread list of source-file names from FILE
-0, --from0all *from/filter files are delimited by 0s
--address=ADDRESSbind address for outgoing socket to daemon
--port=PORTspecify double-colon alternate port number
-h, --human-readableoutput numbers in a human-readable format
--progressshow progress during transfer
-Psame as --partial --progress
-i, --itemize-changesoutput a change-summary for all updates
--read-batch=FILEread a batched update from FILE
-4, --ipv4Prefer IPv4
-6, --ipv6Prefe IPv6
--versionPrint version number
-h, --helpshow help

Both long (double-dash + word) and short (single-dash + letter) options are supported by rsync.

The complete list of choices is described below. The options are separated by commas if an option can be given in more than one way.

Some options only come in lengthy and not short versions. Even though it must also be given for the short variant if the option takes a parameter, the parameter is only listed after the long variant. You have two options when specifying a parameter: either use the format  - - option=param or omit the '=' entirely. For the parameter to survive the command-line parsing of the shell, it might be necessary to quote it in some way. Remember that your shell will substitute a leading tilde () in a file name, thus - -option=/foo will not work.

-v, --verbose

With this selection, you will receive more information throughout the transfer. By design, rsync operates invisibly. You can learn what files are being transferred and get a summary at the end with only one -v. You can find out which files are being skipped by using two -v options, and they also provide a little extra information at the conclusion. Only when debugging rsync should you use more than two -v options.

--help 

Print a brief help page outlining the rsync choices, then quit. If you use the -h option without any additional arguments, the help is displayed for backward compatibility with earlier versions of rsync.

--version

Print the rsync version number and exit.

-q, --quiet

By concealing information packets from the remote server, this option significantly reduces the quantity of information you receive throughout the transfer. When calling rsync from cron, this option is helpful.

--size-only

By using this, rsync's "quick check" algorithm -- which by default transfers files based on size change or last modification time -- is adjusted. When switching to rsync from another mirroring system that incorrectly saves timestamps, is helpful.

-a, --archive

The counterpart of this is -rlptgoD. It is a succinct method to express that you desire recursion and almost complete preservation (with -H being a notable omission). The only time the aforementioned equivalent does not apply is when the --files-from option is provided, in which case -r is not assumed.

Because it costs money to find files that are linked many times, -a does not retain hard links. Separately specifying -H is required.

-b, --backup

With this option, the previous destination files are renamed as each file is transferred or deleted. You can control where the backup file goes and what (if any) suffix is ??added using the --backup-dir and --suffix options.

-p, --perms

By selecting this option, the receiving rsync sets the destination permissions to match those of the source. For a method of changing what rsync regards as source permissions, see also the -- chmod option.

-z, --compress

With this option, rsync minimizes the amount of data being communicated over a sluggish connection by compressing the file data as it is sent to the destination system.

Because it uses the implicit information in the matching data blocks that are not explicitly provided across the connection, it frequently achieves better compression ratios than can be accomplished using a compressing remote shell or a transport. For a list of file suffixes that are by default not compressed, see the --skip-compress option.

copy file from local machine to server:

rsync -avz  /path/to/file.txt <username>@<ip/domain>:/path/to/directory/

copy file from server to the local machine

sudo rsync -avz <username>@<ip/domain>:/path/to/directory/file.txt  /path/to/directory/

Using permission key: 

copy file from local machine to server:

rsync -avz -e "ssh -i /path/to/key.pem" /path/to/file.txt  <username>@<ip/domain>:/path/to/directory/

copy file from server to the local machine

rsync -avz -e "ssh -i /path/to/key.pem" <username>@<ip/domain>:/path/to/directory/file.txt  /path/to/directory/

Let us Discuss with Some examples:

1. Copy or Sync files locally (rsync -zvh):

Let’s assume we want to copy a file from the root user’s home directory to /home/backup folder, execute the below rsync command:

rsync -zvh /home/root/file.txt /home/backup	

The parameters -z for compression, -v for verbose output, and -h for human readable output were used in the example above.

2. Copy or Sync directory locally (rsync -zavh) 

Let’s assume we want to copy or sync the root user’s home directory to /home/backup folder, execute the below rsync command with options -zavh,

rsync -zavh /home/root /home/backup

As we can see, the "root" directory and its contents are copied to the destination folder. In the aforementioned example, if the trailing/character is used after the source folder (/home/root/), the rsync command will merely replicate the contents and not construct the root directory on the destination.

3. Copy files & directories recursively locally (rsync -zrvh or rsync -zavh)

Assuming there are numerous files and directories in the root user home directory, use the rsync command below to copy the files and directories recursively. You can use the -a or -r option to do this.

rsync -zrvh /home/root /home/backup or rsync -zavh /home/root /home/backup

4.Copy or sync files and directories from the local to the remote system

Let’s suppose we want to copy the folder “/home/root/tech” from the local machine to the remote machine (IP address) under the /home

rsync -zarvh /home/root/tech root@ip<domain>:/home

5.Copy or Sync files and directories from the remote machine to the local system

Let’s suppose we want to copy files and directories from a remote machine(IP address) to our local system, in the below example I am copying the remote folder “/home/db” in my local machine under /temp folder

rsync -zarvh root@ip<domain>:/opt/db /temp

6.Specify remote shell during synchronization (rsync -e ssh)

In rsync command we can specify a remote shell like ssh during synchronization, suppose we want to use secure communication between source to destination machines via rsync, then we should use ssh as a remote shell, an example is shown below.

rsync -zarvh -e ssh  root@ip<domain>:/home/file  /temp

7.Display Synchronization progress in rsync command output

If you want to see sync or copy progress in the rsync command, use "-progress", example is shown below

rsync -avh --progress root@ip<domain>:/home/file /temp

8.Resume large file transfer after getting failed in scp

There are some cases where we started copying a large file using the SCP command in Linux admin profile, but it stopped in the middle and we are not able to copy again using scp due to its huge size and time consumption.

scp root@ip<domain>:/root/file /home/root/file/

If you copy the file it comes disconnected we can use this command

rsync -P --rsh=ssh root@1ip<domain>:/root/file /home/root/file

9.Delete files at the destination if it is not present in the source (--delete)

If you have already synced files from source to destination and deleted files from the source, you can force the rsync command to delete files at the destination using the -delete option, the example shown below.

rsync -avz --delete /home/file root@ip<domain>:/temp/file

10.Put a limit on file transfer size (--max-size)

If you don't want to transfer or copy large files using rsync, use the '--max-size={specify-size-here}' option, assuming we don't want to transfer files. Its size is more than 50

rsync -avz --max-size='500K' /home/file root@ip<domain>:/temp	

These are some methods of copying files from one machine to another. We can use these methods in our functions and other related processes.


If you need any assistance in odoo, we are online, please chat with us.



0
Comments



Leave a comment

 


whatsapp
location

Calicut

Cybrosys Technologies Pvt. Ltd.
Neospace, Kinfra Techno Park
Kakkancherry, Calicut
Kerala, India - 673635

location

Kochi

Cybrosys Technologies Pvt. Ltd.
1st Floor, Thapasya Building,
Infopark, Kakkanad,
Kochi, India - 682030.

location

Bangalore

Cybrosys Techno Solutions
The Estate, 8th Floor,
Dickenson Road,
Bangalore, India - 560042

Send Us A Message