output buffering leads to reordered lines when stdout isn't a tty

[Peter Cordes]

 Hi lftp list.

 I think I've found a problem with lftp's output buffering when stdout
isn't a tty (so stdout is block-buffered by default).  I see this with
lftp 3.7.13 compiled on AMD64 Ubuntu Jaunty, but I mostly was testing
with lftp 3.7.3 on Debian lenny, 32bit x86.
 
 I have this lftp script for seeing recent changes in a few remote
directories.

------------
#!/usr/bin/lftp -f
# sgans is an sftp bookmark with a password
open sgans &&
cls -lrt | tail -10 &&
du -s &&
cls -lrt dir1 | tail -10 &&
du -s dir1 &&
cls -lrt dir1/dir2/dir3 | tail -10 &&
du -s dir1/dir2/dir3
------------

 The du output lines make nice separators between the directory
blocks.  This all works great when lftp's stdout is a tty.  You get
output like:

..
-rwxrwxr-x  xxx yyy      3566 May  5 14:08 foo.css
42310   .
..
-rw-r--r--  xxx yyy      8129 Mar 31 23:44 dir1/index.html
34348   dir1
..
-rw-rw-r--  xxx yyy    381457 Mar 31 13:56 dir1/dir2/dir3/foo.txt
398     dir1/dir2/dir3


 However, when lftp's output is a file, the ls outputs all comes first,
and then the three du outputs.  (e.g. ./recent-ls > foo; foo contains:
..
-rwxrwxr-x  xxx yyy      3566 May  5 14:08 foo.css
..
-rw-r--r--  xxx yyy      8129 Mar 31 23:44 dir1/index.html
..
-rw-rw-r--  xxx yyy    381457 Mar 31 13:56 dir1/dir2/dir3/foo.txt
42310   .
34348   dir1
398     dir1/dir2/dir3

 Same thing with lftp -e commands > foo; tail -33 foo.  (We originally
noticed the problem because we were running the output through tail,
and not taking advantage of the ability to send output of individual
lftp commands through filters, which is all around better.  BTW,
removing the "| tail -10" part of each line makes no difference to
reordering problem)

 stdio block buffering vs. line buffering for file/pipe vs. tty is
normal, but it's a bug for the output to be different, not just coming
in chunks.  Probably ls/cls output gets buffered differently from du
output (or isn't buffered), and one kind of output doesn't flush the
other's buffer first.

 On a probably unrelated subject, I've also seen a couple segfaults
from lftp when using sftp, but that was with 3.7.3, so I'll have to
upgrade to the latest version on that machine...

 happy hacking,

-- 
#define X(x,y) x##y
Peter Cordes ;  e-mail: X(pe...@cor , des.ca)

"The gods confound the man who first found out how to distinguish the hours!
 Confound him, too, who in this place set up a sundial, to cut and hack
 my day so wretchedly into small pieces!" -- Plautus, 200 BC