If you have a job that reads the same data file many times, or makes many “random” accesses to a data file, it may be more efficient to have that data locally on a node than compete with other users to access the file server.

Each node has almost 1TB of space mounted on /tmp. This /tmp space is local to each node.

So, you could copy your data to the node and access it locally, than when your job is done, copy the results back to your home directory.

Note: If what your program does is read a file strictly sequentially just once, this copy is unlikely to help.

There is a couple of options for doing the copy.

1) Do it directly in your script..

cp /home/username/mydata.fastq /tmp
... Run your process on the data in /tmp ...
rm /tmp/mydata.fastq

(Really you would use mktemp to get a unique name to avoid clashes.)

Should be careful if you have multiple copies of your script running on a node: you could be copying the data multiple times.

2) Use Secure Copy(SCP) or Secure FTP(SFTP)

For detailed explanation refer How to use SCP and SFTP to securely transfer files

• Copy files with SCP - To copy local file filename to the directory /tmp on the remote server at 192.168.1.3.

scp filename user@192.268.1.3:/tmp/

• Copy files with SFTP - To transfer the local file /etc/filename to /tmp on the remote server.

$  sftp username@192.168.1.3
sftp> put /etc/filename /tmp/

3) Use Globus

Use Globus to transfer files with a GUI interface or to transfer very large files.

Globus is a web based file transfer application that allows resilient, unattended file transfers between two Globus endpoints. Start the transfer and Globus ensures it completes successfully and sends email when the transfer is done. Globus may be preferable to SCP or SFTP when transferring very large files because it does so unattended, in the background, with status checking and fault tolerance.

Please refer below resources to learn how to transfer files using Globus.

• How To Log In and Transfer Files with Globus
• NC State OIT Globus
• For assistance using Globus, please submit a help ticket with OIT at oit_globus@help.ncsu.edu.