On Wed, Jul 10, 2002 at 03:15:02PM +0400, Alexander V. Lukyanov wrote: > I have committed it to cvs, without commented out test code.
Normally I'd bash on this a bit more before sending it in, but since the stuff in CVS right now is pretty buggy, I'll send this now. OutputJob.cc, OutputJob.h: Update comments. Let the input copier own output_fd, so it'll delete it when it's done with it as intended (fixing the delete->close fd->output finished code path). This also means we don't have to deal with fg_data in OutputJob (since the copier owns the fd, CopyJob will do it for us.) eof is gone. Store width/tty status so we don't need output_fd after initialization. (needed in order to let the copier own it) Reversed DontFailIfBroken parameter. Disable fail_if_broken correctly; leave incorrect one in place for now (see comments.) Fixes broken pipe with zcat. Use Timer; fixes scheduler problems with updating the statusline. Clear the status line on init, since if we have a statusline and we start a filter we won't be able to clear it later (due to changed pgrp.) StatusLine.cc, StatusLine.h, OutputJob.cc, all ShowStatusLine users: When we've output to stdout recently, instead of refusing to print a statusline, instead tell the statusline to only send the next update to the title. This gives us a title status during "cat", etc. CmdExec.cc, StatusLine.cc, StatusLine.h: Don't clear the title when we're clearing the statusline due to stdout. CatJob.cc: Override ShowRunStatus to call output->ShowStatusLine. (Any job using an OutputJob should call ShowStatusLine() before it or a child displays a statusline; CopyJob, CatJob's child, was displaying a statusline without doing this.) OutputJob.cc, OutputJob.h, CatJob.cc: Don't reenable the statusbar for commands that stream data from the server, since that's annoying when the server is rate-limited per-second. The title isn't updated when there's a filter running, since we don't know if the child process is using it. We can fairly safely use it when the only filter running is zcat/bzcat, though. I'll think about this. > Ok, it is good thing to do. I've noticed that it is acually partially > applied. Could you check what is not applied and still needed? Erm. The iumacros stuff wasn't in that patch. I thought it was, so I reverted my tree, so now I don't have a copy at all. Maybe I'll redo it some time, but that particular m4 is a headache ... Separate patch: acconfig.h, m4/lftp.m4: HAVE_POLL acconfig (trivial). lftp.conf: Fix cls/hostls aliases (hostls was calling the ls alias, which was calling the cls alias; didn't notice this since I rarely use that alias.) log.cc, log.h: Add a resource (debug:quiet-used-tty) to disable the behavior of quieting debug while a subprocess is in the foreground. Useful for debugging OutputJob. Try: repeat 0s !echo hi A ^C gets eaten by the child job; you have to mash ^C to stop it. I'm not sure there's a clean way to fix this. (Repeat shouldn't stop if its command exits with an error; perhaps it should exit if the command exits because of an interrupt?) -- Glenn Maynard
Index: src/CatJob.cc =================================================================== RCS file: /home/lav/cvsroot/lftp/src/CatJob.cc,v retrieving revision 1.23 diff -u -r1.23 CatJob.cc --- src/CatJob.cc 2002/07/10 08:48:50 1.23 +++ src/CatJob.cc 2002/07/11 02:02:17 @@ -95,6 +95,8 @@ ascii=false; auto_ascii=true; + output->DontRedisplayStatusbar(); + if(!strcmp(op,"more") || !strcmp(op,"zmore") || !strcmp(op,"bzmore")) { const char *pager=getenv("PAGER"); @@ -114,3 +116,12 @@ Binary(); } } + +void CatJob::ShowRunStatus(StatusLine *s) +{ + if(cp && cp->HasStatus() && output->ShowStatusLine(s)) + { + cp->ShowRunStatus(s); + } +} + Index: src/CatJob.h =================================================================== RCS file: /home/lav/cvsroot/lftp/src/CatJob.h,v retrieving revision 1.10 diff -u -r1.10 CatJob.h --- src/CatJob.h 2002/07/10 08:48:50 1.10 +++ src/CatJob.h 2002/07/11 02:02:17 @@ -24,6 +24,7 @@ #define CATJOB_H #include "CopyJob.h" +#include "StatusLine.h" class ArgV; class OutputJob; @@ -48,6 +49,7 @@ void Ascii() { ascii=true; } void Binary() { ascii=auto_ascii=false; } + void ShowRunStatus(StatusLine *s); }; #endif /* CATJOB_H */ Index: src/CmdExec.cc =================================================================== RCS file: /home/lav/cvsroot/lftp/src/CmdExec.cc,v retrieving revision 1.91 diff -u -r1.91 CmdExec.cc --- src/CmdExec.cc 2002/07/10 13:08:04 1.91 +++ src/CmdExec.cc 2002/07/11 02:02:18 @@ -949,7 +949,7 @@ void CmdExec::pre_stdout() { if(status_line) - status_line->Clear(); + status_line->Clear(false); if(feeder_called) feeder->clear(); current->TimeoutS(1); Index: src/FileSetOutput.cc =================================================================== RCS file: /home/lav/cvsroot/lftp/src/FileSetOutput.cc,v retrieving revision 1.25 diff -u -r1.25 FileSetOutput.cc --- src/FileSetOutput.cc 2002/07/10 08:48:51 1.25 +++ src/FileSetOutput.cc 2002/07/11 02:02:18 @@ -422,7 +422,7 @@ if(fso.quiet) return; - if(!output->ShowStatusLine()) + if(!output->ShowStatusLine(s)) return; if(list_info && !list_info->Done()) Index: src/OutputJob.cc =================================================================== RCS file: /home/lav/cvsroot/lftp/src/OutputJob.cc,v retrieving revision 1.1 diff -u -r1.1 OutputJob.cc --- src/OutputJob.cc 2002/07/10 08:48:52 1.1 +++ src/OutputJob.cc 2002/07/11 02:02:18 @@ -20,15 +20,6 @@ /* Usage notes: * - * Set AllowPostpone to true if sending large amounts of data. Check the - * result of each Put and Format call to see if a write was postponed. - * If disabled, writes will always succeed. - * - * This is useful for jobs with a lot of output, like "cat". This can be - * set selectively, where convenient. For example, a job which outputs a - * line of formatted text, followed by the contents of a file, can send - * the first line with AllowPostpone off, then the file with it on. - * * Call PreFilter() to add a filter to the beginning of the chain; these * filters are initialized only once for all data. For example, * PreFilter("wc -l") @@ -37,7 +28,8 @@ /* * Implementation notes: - * Background things we can't get around: + * + * Background things we can't get around: * We must buffer (via FileCopy) output to a filter, since it might block. * * We must buffer the output from the filter to an output FileCopyPeer (ie. @@ -50,15 +42,11 @@ * * In the case where we're outputting to a URL, we set up a FileCopy from a * pipe to the URL, and then pretend we're just outputting to an FD (the - * pipe.) + * pipe.) This means in the simple case of having no filters at all, writing + * to a URL or file, we send the data an extra time through a FileCopy and a + * pipe. That's a bit inefficient, but that's "cat file1 > file2"; that's + * normally done with "get file1 -o file2", so this shouldn't happen often. * - * to it and pretend we're just outputting to a file; this simplifies things - * significantly. This means in the simple case of having no filters at - * all, writing to a URL or file, we send the data an extra time through - * a FileCopy and a pipe. That's a bit inefficient, but that's - * "cat file1 > file2"; that's normally done with "get file1 -o file2", so - * this shouldn't happen often. - * * It's very important that if the output is stdout, any filters point directly * at it, not through an extra copy: a pager, for example, will expect the output * to be a TTY. @@ -93,6 +81,18 @@ initialized=true; + if(Error()) + return; + + /* Clear the statusline, since we might change the pgrp if we create filters. */ + printf("%s", ""); /* (and avoid gcc warning) */ + + /* Some legitimate uses produce broken pipe condition (cat|head). + * We still want to produce broken pipe if we're not piping, eg + * cat > pipe. */ + if(IsFiltered()) + fail_if_broken=false; + if(filter) { /* Create the global filter: */ @@ -103,10 +103,16 @@ /* Use a FileCopy to buffer our output to the filter: */ FileCopyPeerFDStream *out = new FileCopyPeerFDStream(output_fd, FileCopyPeer::PUT); - out->DontDeleteStream(); - FileCopy *input_fc = FileCopy::New(new FileCopyPeer(FileCopyPeer::GET), out, false); + /* out now owns output_fd, and will delete it when it's finished, so + * we can't keep it around. */ + output_fd=0; + + // I don't think we need to do this; the CopyJob picks up the output_fd's + // FgData now, since we're not telling it not to delete it. + // fg_data=new FgData(output_fd->GetProcGroup(),fg); + if(!fail_if_broken) input_fc->DontFailIfBroken(); @@ -148,17 +154,17 @@ initialized=false; error=false; no_status=false; - eof=false; a0=xstrdup(_a0); - last.Set(0,0); is_stdout=false; fail_if_broken=true; output_fd=0; + is_a_tty=false; + width=-1; + statusbar_redisplay=true; } /* Local (fd) output. */ -OutputJob::OutputJob(FDStream *output_, const char *a0): - inter(1) +OutputJob::OutputJob(FDStream *output_, const char *a0) { Init(a0); @@ -167,12 +173,20 @@ if(!output_fd) output_fd=new FDStream(1,"<stdout>"); else - // some legitimate uses produce broken pipe condition (cat|head) - // TODO: once actual piping uses OutputJob, set this only when - // really doing a pipe, so cat>file can produce broken pipe + /* We don't want to produce broken pipe when we're actually + * piping, since some legitimate uses produce broken pipe, eg + * cat|head. However, that's actually handled in InitCopy(). + * User pipes aren't handled by us yet: instead of being set with + * SetFilter, they're being set up ahead of time and passed to + * us as an FDStream, so we don't really know if we're being filtered. + * + * So, until we handle pipes directly, disable broken pipe whenever + * we're being sent anywhere but stdout. */ fail_if_broken=false; is_stdout=output_fd->usesfd(1); + is_a_tty=isatty(output_fd->fd); + width=fd_width(output_fd->fd); /* We don't output status when outputting locally. */ no_status=true; @@ -187,8 +201,7 @@ } } -OutputJob::OutputJob(const char *path, const char *a0, FileAccess *fa): - inter(1) +OutputJob::OutputJob(const char *path, const char *a0, FileAccess *fa) { Init(a0); @@ -200,9 +213,6 @@ /* FIXME: This can be retryable. */ eprintf("%s: %s\n", a0, strerror(errno)); error=true; - /* This won't actually be written to, since error is set, but we must set - * it to something. */ - output_fd=new FDStream(1, "<stdout>"); return; } @@ -248,17 +258,10 @@ xfree(filter); } -void OutputJob::Reconfig(const char *r) +/* This is called to ask us "permission" to display a status line. */ +bool OutputJob::ShowStatusLine(StatusLine *s) { - if(!r || !strcmp(r,"cmd:status-interval")) - { - inter=TimeInterval((const char*)ResMgr::Query("cmd:status-interval",0)); - } -} - -bool OutputJob::ShowStatusLine() -{ - /* If our output file is gone, or isn't stdout, we don't care, */ + /* If our output file is gone, or isn't stdout, we don't care. */ if(!output || !is_stdout) return true; @@ -275,22 +278,32 @@ /* We're line buffered, so we can output a status line without stomping * on a partially output line. * - * Don't display the statusline if the we've output something within the - * last status interval, so if we're currently bursting output we won't - * flicker status for no reason. (Actually, we should be concerned about - * the last time the output peer has sent something...) */ - if(now - last < inter) - return false; + * If we've output something recently, only send the output to the title, + * to avoid flickering status for no reason. + */ + if(!update_timer.Stopped()) { + s->NextUpdateTitleOnly(); + return true; + } - last = now; + /* If we're not reenabling the status bar, and the statusbar has + * been turned off (due to output being reenabled), only send to + * the title. */ + if(!statusbar_redisplay && output->GetCopy()->WriteAllowed()) + { + s->NextUpdateTitleOnly(); + return true; + } - /* Stop the output again, so the FileCopy will clear the StatusLine - * when there's more data. */ + /* There hasn't been output in a while. Stop the output again, + * so the FileCopy will clear the StatusLine when there's more data. */ output->GetCopy()->AllowWrite(false); return true; } +/* Get our contribution to the status line, which is just the output + * status, if any. Input status is the job of the user object. */ const char *OutputJob::Status(const StatusLine *s) { if(no_status) @@ -312,9 +325,10 @@ * that we always start the input->output code path. */ Put("", 0); + /* Send an EOF to the input peer; it'll send an EOF to the output peer + * when all of its data is actually sent. */ if(InputPeer()) InputPeer()->PutEOF(); - eof=true; } /* add a filter to the beginning of the list */ @@ -338,18 +352,23 @@ filter=xstrdup(newfilter); } +/* Return the width of the output. If there's a filter, we can either + * return -1 (we might be piping through "sed", changing the width), + * or the width we know (which is sane for most pagers.) I'm not sure + * which is better. */ int OutputJob::GetWidth() const { - if(IsFiltered() || output_fd->getfd() != 1) + if(IsFiltered()) return -1; - return fd_width(1); + return width; } +/* Return true if the output is going directly to a TTY. */ bool OutputJob::IsTTY() const { - if(IsFiltered() || output_fd->getfd() != 1) + if(IsFiltered()) return false; - return isatty(1); + return is_a_tty; } /* Get the input FileCopyPeer; this is the buffer we write to. */ @@ -368,43 +387,21 @@ int OutputJob::Done() { if(Error()) - return true; - - /* We're always done if the output breaks, regardless of whether - * we treat it as an error or not. */ - if(output_fd->broken()) return true; - + if(!initialized) return false; if(input && !input->Done()) return false; if(output && !output->Done()) - return false; - if(output_fd && !output_fd->Done()) return false; - - return true; - return false; - if(eof && input && input->Done() && output && output->Done()) -// if(eof && output && output->Done()) - { - printf("xxxxxx\n"); - return true; - } - - return false; + + return true; } int OutputJob::Do() { - if(!fg_data && output_fd && output_fd->GetProcGroup()) - { - fg_data=new FgData(output_fd->GetProcGroup(),fg); - return MOVED; - } - return STALL; } @@ -418,8 +415,6 @@ error=true; if(output && input != output && output->Error() && output->Done()) error=true; - if(fail_if_broken && output_fd->broken()) - error=true; return error; } @@ -492,7 +487,7 @@ if(!InputPeer()) return; - last.SetToCurrentTime(); + update_timer.SetResource("cmd:status-interval",0); int oldpos = InputPeer()->GetPos(); InputPeer()->Put(buf, size); @@ -505,6 +500,8 @@ if(!InputPeer()) return; + update_timer.SetResource("cmd:status-interval",0); + int oldpos = InputPeer()->GetPos(); va_list v; @@ -525,11 +522,12 @@ /* If we have an input copier right now, it'll contain the top filter * (which is linked to all other filters), so send it the signal. */ if(input) - m=input->AcceptSig(sig); + input->AcceptSig(sig); /* Otherwise, the only filters we have running are in output_fd. */ - else + else if(output_fd) output_fd->Kill(sig); if(sig!=SIGCONT) AcceptSig(SIGCONT); return m; } + Index: src/OutputJob.h =================================================================== RCS file: /home/lav/cvsroot/lftp/src/OutputJob.h,v retrieving revision 1.1 diff -u -r1.1 OutputJob.h --- src/OutputJob.h 2002/07/10 08:48:52 1.1 +++ src/OutputJob.h 2002/07/11 02:02:18 @@ -24,7 +24,7 @@ #include "Job.h" #include "FileCopy.h" #include "CopyJob.h" -#include "TimeDate.h" +#include "Timer.h" class StatusBar; @@ -46,14 +46,16 @@ bool error; bool is_stdout; bool fail_if_broken; - bool eof; + bool statusbar_redisplay; + int width; + bool is_a_tty; + /* if true, we never contribute to the parent job's status * (Status() == "") */ bool no_status; - Time last; - TimeInterval inter; + Timer update_timer; void Init(const char *a0); void InitCopy(); @@ -76,7 +78,7 @@ /* Prepend a filter before the main filter: */ void PreFilter(const char *filter); - void DontFailIfBroken(bool n=false) { fail_if_broken=n; } + void DontFailIfBroken(bool y=true) { fail_if_broken=!y; } bool Error(); int Done(); @@ -87,8 +89,9 @@ void Format(const char *f,...) PRINTF_LIKE(2,3); void PutEOF(); - /* Return true if our buffers don't want any more input. (They'll always - * *accept* more input; this is optional.) */ + /* If sending large amounts of data, call this function and stop + * sending if it returns true. (This always accept more input; + * this is optional.) */ bool Full(); /* Get properties of the output: */ @@ -101,10 +104,14 @@ /* Call before showing a StatusLine on a job using this class. If it * returns false, don't display it. */ - bool ShowStatusLine(); + bool ShowStatusLine(StatusLine *s); + + /* For commands that stream output from servers, redisplaying the + * statusbar when output becomes idle can be annoying, especially + * if the line is rate-limited. */ + void DontRedisplayStatusbar() { statusbar_redisplay=false; } const char *Status(const StatusLine *s); - void Reconfig(const char *r); void Fg(); void Bg(); Index: src/StatusLine.cc =================================================================== RCS file: /home/lav/cvsroot/lftp/src/StatusLine.cc,v retrieving revision 1.20 diff -u -r1.20 StatusLine.cc --- src/StatusLine.cc 2002/05/14 11:55:23 1.20 +++ src/StatusLine.cc 2002/07/11 02:02:19 @@ -58,6 +58,7 @@ { fd=new_fd; update_delayed=false; + next_update_title_only=false; strcpy(shown,""); strcpy(def_title,""); not_term=!isatty(fd); @@ -69,7 +70,7 @@ { } -void StatusLine::Clear() +void StatusLine::Clear(bool title_also) { char newstr[sizeof(shown)]; @@ -78,7 +79,8 @@ update_delayed=false; update_timer.SetMilliSeconds(20); - WriteTitle(def_title, fd); + if(title_also) + WriteTitle(def_title, fd); } void StatusLine::DefaultTitle(const char *s) @@ -147,6 +149,12 @@ /* Don't write blank titles into the title; let Clear() do that. */ if(newstr[0]) WriteTitle(newstr, fd); + + if(next_update_title_only) + { + next_update_title_only=false; + return; + } char *end=newstr+strlen(newstr); Index: src/StatusLine.h =================================================================== RCS file: /home/lav/cvsroot/lftp/src/StatusLine.h,v retrieving revision 1.11 diff -u -r1.11 StatusLine.h --- src/StatusLine.h 2001/12/24 07:31:48 1.11 +++ src/StatusLine.h 2002/07/11 02:02:19 @@ -37,6 +37,7 @@ bool update_delayed; void update(char *); int LastWidth; + bool next_update_title_only; protected: ~StatusLine(); @@ -45,10 +46,11 @@ public: int GetWidth(); int GetWidthDelayed() const { return LastWidth; } + void NextUpdateTitleOnly() { next_update_title_only=true; } void DefaultTitle(const char *s); void Show(const char *f,...) PRINTF_LIKE(2,3); void WriteLine(const char *f,...) PRINTF_LIKE(2,3); - void Clear(); + void Clear(bool title_also=true); int getfd() const { return fd; } Index: src/echoJob.cc =================================================================== RCS file: /home/lav/cvsroot/lftp/src/echoJob.cc,v retrieving revision 1.1 diff -u -r1.1 echoJob.cc --- src/echoJob.cc 2002/07/10 08:48:53 1.1 +++ src/echoJob.cc 2002/07/11 02:02:19 @@ -70,7 +70,7 @@ /* Never call output->ShowStatusLine unless we're really going * to display something. */ const char *stat = output->Status(s); - if(*stat && output->ShowStatusLine()) + if(*stat && output->ShowStatusLine(s)) s->Show("echo: %s", stat); }
Index: lftp.conf =================================================================== RCS file: /home/lav/cvsroot/lftp/lftp.conf,v retrieving revision 1.18 diff -u -r1.18 lftp.conf --- lftp.conf 2002/07/10 07:58:51 1.18 +++ lftp.conf 2002/07/11 02:01:55 @@ -11,8 +11,8 @@ #set prompt "\[\e[1;30m\][\[\e[0;34m\]f\[\e[1m\]t\[\e[37m\]p\[\e[30m\]] \[\e[34m\]\u\[\e[0;34m\]\@\[\e[1m\]\h\[\e[1;30m\]:\[\e[1;34m\]\w\[\e[1;30m\]>\[\e[0m\] " ## Uncomment the following two lines to make switch cls and ls, making ## cls the default. -#alias ls cls -#alias hostls ls +#alias ls command cls +#alias hostls command ls ## default protocol selection #set default-protocol/ftp.* ftp Index: acconfig.h =================================================================== RCS file: /home/lav/cvsroot/lftp/acconfig.h,v retrieving revision 1.34 diff -u -r1.34 acconfig.h --- acconfig.h 2002/06/03 08:57:47 1.34 +++ acconfig.h 2002/07/11 02:01:55 @@ -26,7 +26,6 @@ #undef HAVE_H_ERRLIST_DECL #undef HAVE_H_ERRNO_DECL #undef HAVE_INET_ATON_DECL -#undef HAVE_POLL #undef HAVE_RANDOM_DECL #undef HAVE_RES_SEARCH_DECL #undef HAVE_STRCASECMP_DECL Index: m4/lftp.m4 =================================================================== RCS file: /home/lav/cvsroot/lftp/m4/lftp.m4,v retrieving revision 1.7 diff -u -r1.7 lftp.m4 --- m4/lftp.m4 2002/06/02 10:21:44 1.7 +++ m4/lftp.m4 2002/07/11 02:01:56 @@ -23,7 +23,7 @@ ]) AC_MSG_RESULT($lftp_cv_func_poll_works) if test $lftp_cv_func_poll_works = yes; then - AC_DEFINE(HAVE_POLL) + AC_DEFINE(HAVE_POLL, 1, [have poll()]) fi ]) Index: src/log.cc =================================================================== RCS file: /home/lav/cvsroot/lftp/src/log.cc,v retrieving revision 1.13 diff -u -r1.13 log.cc --- src/log.cc 2002/01/21 16:25:17 1.13 +++ src/log.cc 2002/07/11 02:01:56 @@ -28,6 +28,7 @@ #include "xstring.h" #include "log.h" #include "SMTask.h" +#include "ResMgr.h" Log *Log::global=new Log; @@ -39,8 +40,16 @@ enabled=false; level=0; tty=false; + quiet_used_tty=true; } +ResDecl res_quiet_used_tty ("debug:quiet-used-tty", "yes", ResMgr::BoolValidate,0); + +void Log::Reconfig(const char *n) +{ + quiet_used_tty = res_quiet_used_tty.QueryBool(0); +} + void Log::Write(int l,const char *s) { if(!enabled || l>level) @@ -52,7 +61,7 @@ pid_t pg=tcgetpgrp(output); if(pg==(pid_t)-1) tty=false; - else if(pg!=getpgrp()) + else if(quiet_used_tty && pg!=getpgrp()) return; } if(tty_cb && tty) Index: src/log.h =================================================================== RCS file: /home/lav/cvsroot/lftp/src/log.h,v retrieving revision 1.6 diff -u -r1.6 log.h --- src/log.h 2001/11/08 09:25:49 1.6 +++ src/log.h 2002/07/11 02:01:56 @@ -32,6 +32,7 @@ int output; bool need_close_output; bool tty; + bool quiet_used_tty; typedef void (*tty_cb_t)(); tty_cb_t tty_cb; @@ -71,6 +72,7 @@ void SetCB(tty_cb_t cb) { tty_cb=cb; } bool IsTTY() { return tty; } + void Reconfig(const char *r); void Init(); Log() { Init(); }