copying_data_to_the_nodes
Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
copying_data_to_the_nodes [2024/02/26 10:05] – created - external edit 127.0.0.1 | copying_data_to_the_nodes [2024/10/17 13:33] (current) – yrchemut | ||
---|---|---|---|
Line 53: | Line 53: | ||
Globus is a web based file transfer application that allows resilient, unattended file transfers between two Globus endpoints. Start the transfer and Globus ensures it completes successfully and sends email when the transfer is done. Globus may be preferable to SCP or SFTP when transferring very large files because it does so unattended, in the background, with status checking and fault tolerance. | Globus is a web based file transfer application that allows resilient, unattended file transfers between two Globus endpoints. Start the transfer and Globus ensures it completes successfully and sends email when the transfer is done. Globus may be preferable to SCP or SFTP when transferring very large files because it does so unattended, in the background, with status checking and fault tolerance. | ||
+ | |||
+ | |||
+ | ==== - How to Globus in the BRC cluster ==== | ||
+ | There are two ways to use the " | ||
+ | |||
+ | ** | ||
+ | 1) Load the latest Globus module | ||
+ | ** | ||
+ | < | ||
+ | module load globuspersonal/ | ||
+ | </ | ||
+ | |||
+ | ** | ||
+ | 2) Setup the client | ||
+ | ** | ||
+ | < | ||
+ | globusconnectpersonal -setup | ||
+ | </ | ||
+ | |||
+ | The program is going to create a URL that you need to copy and paste into the browser of your personal computer.In your browser follow all the instructions to login and authenticate into your Globus account. | ||
+ | |||
+ | At some point, it will show a page with an authorization code. Copy the code from the web browser and paste it into the Linux SSH terminal window at the prompt it says 'Enter the auth code: '. | ||
+ | |||
+ | Then it will ask for a name for your new Globus Endpoint, the prompt says "Input a value for the Endpoint Name: " | ||
+ | |||
+ | You can choose any name that makes sense when referring to the BRC cluster. Recommendation is to enter the answer: BRC Cluster. | ||
+ | |||
+ | The program is going to exit and return to the Linux command line. | ||
+ | |||
+ | ** | ||
+ | 3) Using the endpoint to transfer files | ||
+ | ** | ||
+ | |||
+ | At this point you can start the client any time by doing the below | ||
+ | < | ||
+ | module load globuspersonal/ | ||
+ | globusconnectpersonal -start | ||
+ | </ | ||
+ | |||
+ | While the client is running on the BRC cluster, you can access and transfer your files from the web Globus interface by searching the endpoint in the search bar. you can type BRC cluster, or navigate until you see "Your Collections" | ||
+ | |||
+ | As we transfer large amounts of data, it would be better to keep the Globus Connect client up and running by executing the below | ||
+ | < | ||
+ | nohup globusconnectpersonal -start & | ||
+ | </ | ||
+ | |||
+ | Remember that the setup only gets done once. After that you can start the client. | ||
+ | |||
+ | |||
Please refer below resources to learn how to transfer files using Globus. | Please refer below resources to learn how to transfer files using Globus. |
copying_data_to_the_nodes.1708959933.txt.gz · Last modified: 2024/02/26 10:05 by 127.0.0.1