Download all files in a remote catalogue over SFTP with lftp
Hopefully this will save some of you alot of time, energy, and save you day.
I recently had troubles getting a job to work. The short story is:
Download all files in a remote catalogue, over SFTP, on certain times.
I had a working solution with curl, but when the naming of the files changed (such as whitespaces) – the function broke.
lftp – the saver
After have spent a couple of hours trying to grasp lftp via the manpage, I came up with a solution:
lftp -c '
open sftp://USER:PASSWORD@remoteserver.example.com:22
mirror --verbose --use-pget-n=8 -c /remote/catalogue/ /local/catalogue/
'
And if you want to remove the source-files after download:
lftp -c '
open sftp://USER:PASSWORD@remoteserver.example.com:22
mirror --Remove-source-files --verbose --use-pget-n=8 -c /remote/catalogue/ /local/catalogue/
'
This download all files in the specified remote catalogue to the specified local one, then exits.