Data backup and transport
We recommend RSYNC for data transfers. It works with both portable drives and network transfers. It has many features which make it useful for beamlne backup. Perhaps the most important is the ability to do incremental transfers. Before actually copying any files, it will compare the source with the target and only copy files which are new or altered.
Here are some examples of rsync commands:
To a portable drive:
FAT file system:
rsync -rltv --modify-window=1 /ID7B2/smith /media/disk1/chess_20170812/.
To other file systems:
rsync -av /ID7B2/smith /media/disk1/chess_20170812/.
We currently support USB drives with USB-A or USB-C connections. SSD
(solid state drives) are typically faster and more shock-resistant than
hard disk drives. Make sure you have sufficient capacity for the data
you plan to collect. Designate a drive for data transfer, and do not use
that same drive for permanent storage. We support most Windows (VFAT,
exFAT) and Linux (ext2,3,4, XFS) disk formats.
It is possible to bring a laptop, notebook or other portable computing
device to gather your data. Gigabit ethernet connections to the CHESS
network are available for this purpose. Wi-fi is more challenging, since
wi-fi service is outsourced to the Cornell campus IT services, and is
outside the CHESS firewall. In this situation, users can look into
GLOBUS as described at Data transfer for CHESS users.