I recently had troubles getting a job to work. The short story is:

Download all files in a remote catalogue, over SFTP, on certain times.

I had a working solution with curl, but when the naming of the files changed (such as whitespaces) – the function broke.

lftp – the saver

After have spent a couple of hours trying to grasp lftp via the manpage, I came up with a solution:

lftp -c '
open sftp://USER:PASSWORD@remoteserver.example.com:22
mirror --verbose --use-pget-n=8 -c /remote/catalogue/ /local/catalogue/
'

And if you want to remove the source-files after download:

lftp -c '
open sftp://USER:PASSWORD@remoteserver.example.com:22
mirror --Remove-source-files --verbose --use-pget-n=8 -c /remote/catalogue/ /local/catalogue/
'

This download all files in the specified remote catalogue to the specified local one, then exits.

#linux #bash #sftp #lftp