Hello, I made some tests with a very big file I found in my logs:
# wc messages 1298571 13998518 116929300 messages And it took indeed 40 seconds to cat! >From what I think to understand, the problem seems to be the way the screen buffer "PSCR(r, page).text" is managed. Apparently the screen and the cursor in it are always considered at the bottom of this buffer, whether this one is full, empty, or partially filled. This is why when you add something in this buffer, you will add it to the row "CURROW + SVLINES". This is absolutely not efficient as when you have to add a lot of lines in the same time in the screen buffer (as we try to do here), we begin nearly at the end of the buffer (SVLINES + CURROW so), then when we arrive just a few lines after at the real end of the buffer (SVLINES + r->TermWin.nrow), we begin to scroll the buffer one line by one line (= we rotate all data inside by a MEMMOVE!). Hence in our case, for a file of 126630 to display, we will make thousands of MEMMOVE (fortunately slightly less than the number of lines with a big --skipPages, but still a lot!), even for some lines we will never display. With a better screen text buffer management, we could do it just a few times as we would be able to move all data of many lines in one single time (so one MEMMOVE instead of a hundred! I remind that the default SVLINES value is DEFAULT_SAVELINES = 100!). For this, I think the screen limits and the cursor should be able to move in the text buffer (not stay glued always at the end as currently). This is a first solution as, just as demonstrated, it would divide the number of MEMMOVE operation (a heavy one) by 100. But there are even better solution, like transforming the buffer in some kind of FIFO, so that we never have to do any MEMMOVE ever. For this I see 2 ways: either make some kind of list system where an item point to another, but it is still not perfect (because we need to allocate/deallocate lines constantly), or simply make the buffer cyclic. I think this last solution is the better: we never have to allocate/deallocate and never have to do any MEMMOVE. We just displace the screen and the cursor in it. If this suits you, I think I can try to implement this during the weekend. Or if you prefer to do it, or even if you have a better idea or think I am wrong in this diagnostic, just tell me... Jehan P.S.: and the reason why the more you have new lines, the more it is fast is -- I think according to the code -- that there is a slight error in rxvt_process_getc. In this function, you are incrementing PVTS(r, page)->scrolled_lines each time you encounter a newline character, but also each time you go to the line because your text is too long for display. This is good. But in the same time, you have a variable called nlines, and this one is incremented only when you encounter a newline. The consequence is that when you have very very long lines but very few newline characters, this variable won't be at all significant of the real number of lines you will display on screen. And unfortunately this is this variable which will be used in rxvt_scr_add_lines for the first MEMMOVE (the only MEMMOVE which is more than one line, showing how much important it is to make fewer MEMMOVE). Gautam Iyer a écrit : > On Thu, Aug 21, 2008 at 03:14:52PM -0400, Seung Jun wrote: > >> It's absolutely fast. So the question is, can you recognize where the >> problem occurs? Any place that may get slow when it handles long >> lines? > > Ok, so maybe that's where the problem is. Can you generate some random > text file which takes a long time to cat? Send me this file. Also send > me the output of > > wc file > time cat file > fold -w76 < file > file.folded; time cat file.folded > > Once we both have the same file that produces bad results, I should > (hopefully) be able to fix the problem. > > GI > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > Materm-devel mailing list > Materm-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/materm-devel > mrxvt home page: http://materm.sourceforge.net ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Materm-devel mailing list Materm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/materm-devel mrxvt home page: http://materm.sourceforge.net
