On Tue, Nov 12, 2013 at 01:02:45PM -0800, Tim Kelsch wrote:
> Hello,
> I'm using LFTP v4.4.8 on rhel6u2 to reverse mirror thousands of files from
> 5 "client" hosts to 2 different to VSFTPD servers (2.2.2) on rhel6u2. We're
> running anywhere between 10-40 LFTP clients on each client host. I'll hold
> off on giving more details here as my first question has more to do with
> something I saw in the documentation (http://lftp.yar.ru/lftp-man.html)
> that alarmed me. In the Commands section, under the Mirror command:
>
> -P, --parallel[=N] download N files in parallel
>
>
> No problem there, but reading on in the Settings section:
>
> *cmd:parallel* (number)
> Number of jobs run in parallel in non-interactive mode.
> For example, this may be useful for scripts with multiple `get'
> commands. Note that setting this to a value greater than 1
> changes conditional execution behaviour, basically makes it
> inconsistent.
These option and setting are unrelated. Conditional execution of commands
happens when commands are separated by && or ||.
> So I'm wondering if there is any possibility that the parallel
> argument to the mirror command could cause "inconsistent" behavior.
No.
> My second question is "what exactly does inconsistent behavior mean"?
> My third question is "what constitutes conditional execution"?
Consider this example:
set cmd:parallel 2
get file1; get file2 && get file3
In this case file1 and file2 will be transferred in parallel, and file3
will be transferred conditionally based on exit code of the first finished
transfer. Without cmd:parallel=2 file3 will only be transferred if file2
has been transferred successfully.
For the slowness, you can try a few settings to speed it up (try them
separately):
set ftp:sync-mode no
set ftp:use-mlsd yes
set ftp:use-stat-for-list yes
set ftp:use-mdtm no
See what setting works for you. Turn on debug in lftp, it may make it clear.
--
Alexander.
_______________________________________________
lftp mailing list
lftp@uniyar.ac.ru
http://univ.uniyar.ac.ru/mailman/listinfo/lftp
