Re: Fastest way to append line or char to a buffer
On 8/30/06, Chris Littell <[EMAIL PROTECTED]> wrote:
On 8/30/06, Nikolai Weibull <[EMAIL PROTECTED]> wrote:
> In C99 you can initialize values "out of order", yes, but you can't do
> it with ranges. Ranges are a GNU C extension. The propagation
> neither happens in any of the ANSI standards, nor in the GNU extended
> version of C. It's simple to test write the following in "a.c":
>
> #include
>
> int
> main(void)
> {
> int is[2] = { 1 };
> int i;
>
> for (i = 0; i < 2; i++)
> printf("%d\n", is[i]);
>
> return 0;
> }
>
> And then to actually test it:
>
> $ for std in c89 c99 gnu89 gnu99; do gcc -std=$std a.c && echo $std:
> && a.out; done
> c89:
> 1
> 0
> c99:
> 1
> 0
> gnu89:
> 1
> 0
> gnu99:
> 1
> 0
>
> >
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/gcc/designated-inits.html
>
> Nowhere in that section does it say that the last value is propagated.
Wow, I reread it and you are correct. I'm not sure why I held that
assumption...
Also thanks for the examples.
No worries.
nikolai
Re: Fastest way to append line or char to a buffer
On 8/30/06, Nikolai Weibull <[EMAIL PROTECTED]> wrote:
On 8/30/06, Chris Littell <[EMAIL PROTECTED]> wrote:
> On 8/30/06, Nikolai Weibull <[EMAIL PROTECTED]> wrote:
> > On 8/29/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
> > > On 29/08/06, Ilya <[EMAIL PROTECTED]> wrote:
> > > > Brad Beveridge wrote:
> > > > >static char string[2] = {0};
> > > > Should not you have "= {0, 0}" here? Second element never get
> > > > initialized but it could be accessed by ml_append_string.
> > >
> > > That might be more clear perhaps, but when you initialize an array
> > > like that in C, the last element is propagated for the whole array.
> >
> > What C compiler are you using? The last element is /not/ propagated
> > in C. The rest of the array will be initialized to zero, which is the
>
> In C89 this is true. In C99, you can initialize values out of order
> and by index range, and the final value in an array initializer is
> propogated to the rest of the values.
In C99 you can initialize values "out of order", yes, but you can't do
it with ranges. Ranges are a GNU C extension. The propagation
neither happens in any of the ANSI standards, nor in the GNU extended
version of C. It's simple to test write the following in "a.c":
#include
int
main(void)
{
int is[2] = { 1 };
int i;
for (i = 0; i < 2; i++)
printf("%d\n", is[i]);
return 0;
}
And then to actually test it:
$ for std in c89 c99 gnu89 gnu99; do gcc -std=$std a.c && echo $std:
&& a.out; done
c89:
1
0
c99:
1
0
gnu89:
1
0
gnu99:
1
0
>
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/gcc/designated-inits.html
Nowhere in that section does it say that the last value is propagated.
Wow, I reread it and you are correct. I'm not sure why I held that
assumption...
Also thanks for the examples.
nikolai
Chris
Re: Fastest way to append line or char to a buffer
On 8/30/06, Chris Littell <[EMAIL PROTECTED]> wrote:
On 8/30/06, Nikolai Weibull <[EMAIL PROTECTED]> wrote:
> On 8/29/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
> > On 29/08/06, Ilya <[EMAIL PROTECTED]> wrote:
> > > Brad Beveridge wrote:
> > > >static char string[2] = {0};
> > > Should not you have "= {0, 0}" here? Second element never get
> > > initialized but it could be accessed by ml_append_string.
> >
> > That might be more clear perhaps, but when you initialize an array
> > like that in C, the last element is propagated for the whole array.
>
> What C compiler are you using? The last element is /not/ propagated
> in C. The rest of the array will be initialized to zero, which is the
In C89 this is true. In C99, you can initialize values out of order
and by index range, and the final value in an array initializer is
propogated to the rest of the values.
In C99 you can initialize values "out of order", yes, but you can't do
it with ranges. Ranges are a GNU C extension. The propagation
neither happens in any of the ANSI standards, nor in the GNU extended
version of C. It's simple to test write the following in "a.c":
#include
int
main(void)
{
int is[2] = { 1 };
int i;
for (i = 0; i < 2; i++)
printf("%d\n", is[i]);
return 0;
}
And then to actually test it:
$ for std in c89 c99 gnu89 gnu99; do gcc -std=$std a.c && echo $std:
&& a.out; done
c89:
1
0
c99:
1
0
gnu89:
1
0
gnu99:
1
0
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/gcc/designated-inits.html
Nowhere in that section does it say that the last value is propagated.
nikolai
Re: Fastest way to append line or char to a buffer
On Wed, Aug 30, 2006 at 08:57:13AM -0400, Chris Littell wrote:
> On 8/30/06, Nikolai Weibull <[EMAIL PROTECTED]> wrote:
> >On 8/29/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
> >> On 29/08/06, Ilya <[EMAIL PROTECTED]> wrote:
> >> > Brad Beveridge wrote:
> >> > >static char string[2] = {0};
> >> > Should not you have "= {0, 0}" here? Second element never get
> >> > initialized but it could be accessed by ml_append_string.
> >>
> >> That might be more clear perhaps, but when you initialize an array
> >> like that in C, the last element is propagated for the whole array.
> >
> >What C compiler are you using? The last element is /not/ propagated
> >in C. The rest of the array will be initialized to zero, which is the
>
> In C89 this is true. In C99, you can initialize values out of order
> and by index range, and the final value in an array initializer is
> propogated to the rest of the values.
>
> http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/gcc/designated-inits.html
Quoting an example from this document:
int a[6] = { [1] = v1, v2, [4] = v4 };
is equivalent to
int a[6] = { 0, v1, v2, 0, v4, 0 };
Which behaves as Nikolai suggested and K&R explains. Any unspecified
elements are initialized to 0.
James
--
GPG Key: 1024D/61326D40 2003-09-02 James Vega <[EMAIL PROTECTED]>
signature.asc
Description: Digital signature
Re: Fastest way to append line or char to a buffer
On 8/30/06, Nikolai Weibull <[EMAIL PROTECTED]> wrote:
On 8/29/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
> On 29/08/06, Ilya <[EMAIL PROTECTED]> wrote:
> > Brad Beveridge wrote:
> > >static char string[2] = {0};
> > Should not you have "= {0, 0}" here? Second element never get
> > initialized but it could be accessed by ml_append_string.
>
> That might be more clear perhaps, but when you initialize an array
> like that in C, the last element is propagated for the whole array.
What C compiler are you using? The last element is /not/ propagated
in C. The rest of the array will be initialized to zero, which is the
In C89 this is true. In C99, you can initialize values out of order
and by index range, and the final value in an array initializer is
propogated to the rest of the values.
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/gcc/designated-inits.html
default for variables declared as being static. Actually, the
explicit initialization to zero is redundant and actually causes the
size of the resulting executable to increase. See
http://people.redhat.com/drepper/dsohowto.pdf for a better explanation
than I could give here.
nikolai
Chris
Re: Fastest way to append line or char to a buffer
On 8/29/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
On 29/08/06, Ilya <[EMAIL PROTECTED]> wrote:
> Brad Beveridge wrote:
> >static char string[2] = {0};
> Should not you have "= {0, 0}" here? Second element never get
> initialized but it could be accessed by ml_append_string.
That might be more clear perhaps, but when you initialize an array
like that in C, the last element is propagated for the whole array.
What C compiler are you using? The last element is /not/ propagated
in C. The rest of the array will be initialized to zero, which is the
default for variables declared as being static. Actually, the
explicit initialization to zero is redundant and actually causes the
size of the resulting executable to increase. See
http://people.redhat.com/drepper/dsohowto.pdf for a better explanation
than I could give here.
nikolai
Re: Fastest way to append line or char to a buffer
On 29/08/06, Ilya <[EMAIL PROTECTED]> wrote:
Brad Beveridge wrote:
> On 26/08/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
>
>>
>> Cheers
>> Brad
>
> Hi, sorry for all the trouble :) I'm having redraw issues, where
> running this code to append chars is causing strange problems.
> The scenario is this (always using the code below for char output):
> 1) Write a char to a buffer (line1)
> 2) Write a newline (line 2)
> 3) Move the cursor to the bottom of the buffer (line2 empty)
> 4) Write another Char (appears in line2)
> 5) Press Ctrl-L
> 6) Notice that line1 has vanished.
>
> moving the cursor to the new blank line at the bottom of the line is
> critical here. I'm sure that I'm doing something wrong, but I really
> can't figure it out.
> As a side note, it feels very clumsy to have to manually call "changed
> line" functions when calling ml_* funcs - is there some notes I can
> read about why this is?
> Anyhow, here is the code I'm using to put chars in a buffer - I really
> appreciate any thoughts, this feels so close to working, but I've
> spend several hours trying to get it right. Very frustrating :)
>
> Cheers
> Brad
>
> static void vim_append_char (buf_T target_buf, char c)
> {
>static char string[2] = {0};
Should not you have "= {0, 0}" here? Second element never get
initialized but it could be accessed by ml_append_string.
That might be more clear perhaps, but when you initialize an array
like that in C, the last element is propagated for the whole array.
Cheers
Brad
Re: Fastest way to append line or char to a buffer
Brad Beveridge wrote:
On 26/08/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
Cheers
Brad
Hi, sorry for all the trouble :) I'm having redraw issues, where
running this code to append chars is causing strange problems.
The scenario is this (always using the code below for char output):
1) Write a char to a buffer (line1)
2) Write a newline (line 2)
3) Move the cursor to the bottom of the buffer (line2 empty)
4) Write another Char (appears in line2)
5) Press Ctrl-L
6) Notice that line1 has vanished.
moving the cursor to the new blank line at the bottom of the line is
critical here. I'm sure that I'm doing something wrong, but I really
can't figure it out.
As a side note, it feels very clumsy to have to manually call "changed
line" functions when calling ml_* funcs - is there some notes I can
read about why this is?
Anyhow, here is the code I'm using to put chars in a buffer - I really
appreciate any thoughts, this feels so close to working, but I've
spend several hours trying to get it right. Very frustrating :)
Cheers
Brad
static void vim_append_char (buf_T target_buf, char c)
{
static char string[2] = {0};
Should not you have "= {0, 0}" here? Second element never get
initialized but it could be accessed by ml_append_string.
buf_T *savebuf = curbuf;
curbuf = target_buf;
int start_line = curbuf->b_ml.ml_line_count;
int lines_changed = 0;
string[0] = c;
if (string[0] == '\n')
{
ml_append (start_line, "", 0, FALSE);
lines_changed = 0;
}
else
ml_append_string (start_line, string, -1);
changed_lines (start_line, 0, start_line, lines_changed);
/* restore and return */
curbuf = savebuf;
}
Re: Fastest way to append line or char to a buffer
On 28/08/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote: Brad Beveridge wrote: > Hi, sorry for all the trouble :) I'm having redraw issues, where > running this code to append chars is causing strange problems. > The scenario is this (always using the code below for char output): > 1) Write a char to a buffer (line1) > 2) Write a newline (line 2) > 3) Move the cursor to the bottom of the buffer (line2 empty) > 4) Write another Char (appears in line2) > 5) Press Ctrl-L > 6) Notice that line1 has vanished. > > moving the cursor to the new blank line at the bottom of the line is > critical here. I'm sure that I'm doing something wrong, but I really > can't figure it out. > As a side note, it feels very clumsy to have to manually call "changed > line" functions when calling ml_* funcs - is there some notes I can > read about why this is? The screen updating code doesn't want to redraw everything all the time. Thus you must somehow specify what text got changed/inserted/deleted. If you don't do that you might indeed see redraw problems that go away when using CTRL-L. Actually, it is just the opposite - the screen looks fine UNTIL I press CTRL-L :) You are doing low level stuff here, thus you must take care of all these things yourself. I guess that is fair enough. Do you save text for undo somewhere? That should be done for every change that is being made, otherwise undo won't work. For right now I'm not concerned about undo. Probably later I'll add undo at the per-line level. Cheers Brad
Re: Fastest way to append line or char to a buffer
Brad Beveridge wrote:
> Hi, sorry for all the trouble :) I'm having redraw issues, where
> running this code to append chars is causing strange problems.
> The scenario is this (always using the code below for char output):
> 1) Write a char to a buffer (line1)
> 2) Write a newline (line 2)
> 3) Move the cursor to the bottom of the buffer (line2 empty)
> 4) Write another Char (appears in line2)
> 5) Press Ctrl-L
> 6) Notice that line1 has vanished.
>
> moving the cursor to the new blank line at the bottom of the line is
> critical here. I'm sure that I'm doing something wrong, but I really
> can't figure it out.
> As a side note, it feels very clumsy to have to manually call "changed
> line" functions when calling ml_* funcs - is there some notes I can
> read about why this is?
The screen updating code doesn't want to redraw everything all the time.
Thus you must somehow specify what text got changed/inserted/deleted.
If you don't do that you might indeed see redraw problems that go away
when using CTRL-L.
You are doing low level stuff here, thus you must take care of all these
things yourself.
> Anyhow, here is the code I'm using to put chars in a buffer - I really
> appreciate any thoughts, this feels so close to working, but I've
> spend several hours trying to get it right. Very frustrating :)
>
> Cheers
> Brad
>
> static void vim_append_char (buf_T target_buf, char c)
> {
> static char string[2] = {0};
> buf_T *savebuf = curbuf;
> curbuf = target_buf;
> int start_line = curbuf->b_ml.ml_line_count;
> int lines_changed = 0;
>
> string[0] = c;
> if (string[0] == '\n')
> {
> ml_append (start_line, "", 0, FALSE);
> lines_changed = 0;
> }
> else
> ml_append_string (start_line, string, -1);
> changed_lines (start_line, 0, start_line, lines_changed);
>
> /* restore and return */
> curbuf = savebuf;
> }
Do you save text for undo somewhere? That should be done for every
change that is being made, otherwise undo won't work.
--
hundred-and-one symptoms of being an internet addict:
250. You've given up the search for the "perfect woman" and instead,
sit in front of the PC until you're just too tired to care.
/// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\download, build and distribute -- http://www.A-A-P.org///
\\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: Fastest way to append line or char to a buffer
On 26/08/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
Cheers
Brad
Hi, sorry for all the trouble :) I'm having redraw issues, where
running this code to append chars is causing strange problems.
The scenario is this (always using the code below for char output):
1) Write a char to a buffer (line1)
2) Write a newline (line 2)
3) Move the cursor to the bottom of the buffer (line2 empty)
4) Write another Char (appears in line2)
5) Press Ctrl-L
6) Notice that line1 has vanished.
moving the cursor to the new blank line at the bottom of the line is
critical here. I'm sure that I'm doing something wrong, but I really
can't figure it out.
As a side note, it feels very clumsy to have to manually call "changed
line" functions when calling ml_* funcs - is there some notes I can
read about why this is?
Anyhow, here is the code I'm using to put chars in a buffer - I really
appreciate any thoughts, this feels so close to working, but I've
spend several hours trying to get it right. Very frustrating :)
Cheers
Brad
static void vim_append_char (buf_T target_buf, char c)
{
static char string[2] = {0};
buf_T *savebuf = curbuf;
curbuf = target_buf;
int start_line = curbuf->b_ml.ml_line_count;
int lines_changed = 0;
string[0] = c;
if (string[0] == '\n')
{
ml_append (start_line, "", 0, FALSE);
lines_changed = 0;
}
else
ml_append_string (start_line, string, -1);
changed_lines (start_line, 0, start_line, lines_changed);
/* restore and return */
curbuf = savebuf;
}
Re: Fastest way to append line or char to a buffer
On 22/08/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote: I understand. It would be a bit more efficient if the new, empty line that you add when a newline is encountered already has room for a certain number of characters. Ah, I see. True, but since I don't know the code base that well I didn't want to touch too much code all over the place. > > Also, for appending to a different line than before you copy the line > > twice. You can avoid that by using vim_strnsave() instead of > > vim_strsave() and reserving space for new characters right away. > > Yes, this is a slight inefficiency. The main reason that I did this > was that I wanted all the code to do with re-allocation in the same > place, ie the next if block. Looking at the code again now, the > solution is probably to not copy in the "cache miss" code, where we > flush - just do > curbuf->b_ml.ml_line_ptr = ml_get(lnum); I suppose that should work (didn't look at it carefully). I might have a look later if this code can be used in certain situations. That should make some operations work faster, e.g. pasting a large chunk of text into a terminal emulator. I think it will work from the quick glance over it. I can test it more throughly tonight. I'd love it if I made an actually useful contrib to Vim - I love this editor, thanks for writing it! Brad
Re: Fastest way to append line or char to a buffer
Brad Beveridge wrote:
> > I wonder how you create a new line. Using the old mechanism you mostly
> > end up reallocating the line at the next insert. Allocating a bit of
> > room to anticipate further appends would be more efficient.
> I'm sorry, I don't understand quite what you mean with that comment.
> Do you mean "if you only used ml_append_string, how do new lines ever
> get created?" If you do mean that, then the answer is: new lines need
> to be created by an upper level function. I have an upper level
> "append_char" function that is more or less:
> void append_char (buf, char) {
> static uchar string[2] = {0};
> curbuf = buf;
> if (char == '\n')
> ml_append // create a new line
> else
> {
> string[0] = char;
> ml_append_string (string)
> ...
> The reason that I didn't do new line creation in append_string is that
> you would need to check all incoming strings for newlines, the upper
> level layers can better do this check because they have more
> information about the incoming string - they could know it will never
> contain newlines.
I understand. It would be a bit more efficient if the new, empty line
that you add when a newline is encountered already has room for a
certain number of characters.
> > Also, for appending to a different line than before you copy the line
> > twice. You can avoid that by using vim_strnsave() instead of
> > vim_strsave() and reserving space for new characters right away.
>
> Yes, this is a slight inefficiency. The main reason that I did this
> was that I wanted all the code to do with re-allocation in the same
> place, ie the next if block. Looking at the code again now, the
> solution is probably to not copy in the "cache miss" code, where we
> flush - just do
> curbuf->b_ml.ml_line_ptr = ml_get(lnum);
I suppose that should work (didn't look at it carefully).
I might have a look later if this code can be used in certain
situations. That should make some operations work faster, e.g. pasting
a large chunk of text into a terminal emulator.
--
Some of the well know MS-Windows errors:
EMULTI Multitasking attempted, system confused
EKEYBOARD Keyboard locked, try getting out of this one!
EXPLAIN Unexplained error, please tell us what happened
EFUTURE Reserved for our future mistakes
/// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\download, build and distribute -- http://www.A-A-P.org///
\\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: Fastest way to append line or char to a buffer
On 22/08/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote:
Brad Beveridge wrote:
Thanks for making this patch.
No problem - it happened to be my particular itch :)
I wonder how you create a new line. Using the old mechanism you mostly
end up reallocating the line at the next insert. Allocating a bit of
room to anticipate further appends would be more efficient.
I'm sorry, I don't understand quite what you mean with that comment.
Do you mean "if you only used ml_append_string, how do new lines ever
get created?" If you do mean that, then the answer is: new lines need
to be created by an upper level function. I have an upper level
"append_char" function that is more or less:
void append_char (buf, char) {
static uchar string[2] = {0};
curbuf = buf;
if (char == '\n')
ml_append // create a new line
else
{
string[0] = char;
ml_append_string (string)
...
The reason that I didn't do new line creation in append_string is that
you would need to check all incoming strings for newlines, the upper
level layers can better do this check because they have more
information about the incoming string - they could know it will never
contain newlines.
Also, for appending to a different line than before you copy the line
twice. You can avoid that by using vim_strnsave() instead of
vim_strsave() and reserving space for new characters right away.
Yes, this is a slight inefficiency. The main reason that I did this
was that I wanted all the code to do with re-allocation in the same
place, ie the next if block. Looking at the code again now, the
solution is probably to not copy in the "cache miss" code, where we
flush - just do
curbuf->b_ml.ml_line_ptr = ml_get(lnum);
Cheers
Brad
Re: Fastest way to append line or char to a buffer
Brad Beveridge wrote: > The last code I posted was a little broken. Here is a patch that > appears (after a reasonable amount of testing) to work quite well for > very quickly appending chars to a vim line. > As I said, scripting language authors that use Vim output buffers as > STDOUT style streams could use this to good advantage. Thanks for making this patch. I wonder how you create a new line. Using the old mechanism you mostly end up reallocating the line at the next insert. Allocating a bit of room to anticipate further appends would be more efficient. Also, for appending to a different line than before you copy the line twice. You can avoid that by using vim_strnsave() instead of vim_strsave() and reserving space for new characters right away. -- hundred-and-one symptoms of being an internet addict: 192. Your boss asks you to "go fer" coffee and you come up with 235 FTP sites. /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: Fastest way to append line or char to a buffer
The last code I posted was a little broken. Here is a patch that appears (after a reasonable amount of testing) to work quite well for very quickly appending chars to a vim line. As I said, scripting language authors that use Vim output buffers as STDOUT style streams could use this to good advantage. Cheers Brad append-char.patch.gz Description: GNU Zip compressed data
Re: Fastest way to append line or char to a buffer
On 16/08/06, Brad Beveridge <[EMAIL PROTECTED]> wrote:
On 16/08/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote:
>
> Brad -
>
> > As a side note, if I wanted to get really fast char append, how does
> > this method sound:
> > 1) In a structure (not sure if a mem_line or bufT struct), have three
> > new entries
> > uint current_line_num;
> > uchar *current_line_buffer;
> > uint current_line_alloced;
> > 2) current_line_buffer points to the string data for the current line,
> > it starts off at a sane size (say 80, or screen_width), and doubles in
> > alloced size when it is too full.
> > 3) ml_append_char (or whatever I call it), will check the target line
> > number against current_line_number, if the match then
> > - append the char, possibly reallocing the current_line_buffer if we
> > have run out of space
> > ELSE, the edit line number and our "cached line" don't match
> > - trim the size of the current_line_buffer to match the STRLEN of the line
> > - alloc a new oversized line buffer and copy the new edit line into it
> > - append the char as above
>
> Vim caches the last changed line, you could do something with that.
> Start looking at ml_flush_line(), you can find the rest from there.
>
> > Would this kind of optimisation be more widely useful?
>
> Perhaps. Changing the same line several times is not unusual. It
> mostly depends on how much code would need to be changed compared to
> how much we would gain.
>
> - Bram
>
Thanks for the tips, I'll test all this out and see what I can come up
with. I also accidentally posted just to Bram, so I've included the
list in this reply.
Thanks
Brad
I ended up writing a string append function, that has sped up my
output code by a couple of orders of magnitude :) Here it is, does it
look sane?
Cheers
Brad
/*
* Append string to line lnum, with buffering, in current buffer.
*
* Always makes a copy of the incoming string
*
* Check: The caller of this function should probably also call
* changed_lines(), unless update_screen(NOT_VALID) is used.
*
* return FAIL for failure, OK otherwise
*/
int
ml_append_string(lnum, string, slen)
linenr_Tlnum;
char_u *string;
int slen;
{
int append_len = 0;
int line_len = 0;
if (slen == -1)
append_len = STRLEN(string);
else
append_len = slen;
if (string == NULL) /* just checking... */
return FAIL;
if (lnum == 0)
ml_append (lnum, string, slen, 0);
/* When starting up, we might still need to create the memfile */
if (curbuf->b_ml.ml_mfp == NULL && open_buffer(FALSE, NULL) == FAIL)
return FAIL;
#ifdef FEAT_NETBEANS_INTG
/*
#error I am 99% sure this is broken
if (usingNetbeans)
{
netbeans_removed(curbuf, lnum, 0, (long)STRLEN(ml_get(lnum)));
netbeans_inserted(curbuf, lnum, 0, string, (int)STRLEN(string));
}
*/
#endif
if (curbuf->b_ml.ml_line_lnum != lnum) /* other line buffered */
{
ml_flush_line(curbuf); /* flush it, this frees
ml_line_ptr */
/* Take a pointer to the existing line, will get re-alloced right
* away below because it is not big enough */
curbuf->b_ml.ml_line_ptr = vim_strsave(ml_get(lnum));
curbuf->b_ml.ml_line_len = STRLEN(curbuf->b_ml.ml_line_ptr);
curbuf->b_ml.ml_line_lnum = lnum;
}
/* By here,
* curbuf->b_ml.ml_line_ptr - is filled with the correct existing line
* curbuf->b_ml.ml_line_len - is the alloced size of the line
*/
line_len = STRLEN(curbuf->b_ml.ml_line_ptr);
if (curbuf->b_ml.ml_line_len < line_len + append_len + 1)
{
int new_len = (curbuf->b_ml.ml_line_len + line_len + 1) * 2;
char_u *new_line;
if (new_len < 80) new_len = 80;
new_line = vim_strnsave(curbuf->b_ml.ml_line_ptr, new_len);
vim_free(curbuf->b_ml.ml_line_ptr);
curbuf->b_ml.ml_line_ptr = new_line;
curbuf->b_ml.ml_line_len = new_len;
}
/* by here:
* ml_line_ptr is allocated to fit the current line + the append
* ml_line_len is the total alloced size of the line
*/
STRCPY (&curbuf->b_ml.ml_line_ptr[line_len], string); /* Append
our string */
curbuf->b_ml.ml_line_ptr[line_len + append_len] = NUL; /* End the line */
curbuf->b_ml.ml_flags = (curbuf->b_ml.ml_flags | ML_LINE_DIRTY) & ~ML_EMPTY;
return OK;
}
Re: Fastest way to append line or char to a buffer
On 16/08/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote: Brad - > As a side note, if I wanted to get really fast char append, how does > this method sound: > 1) In a structure (not sure if a mem_line or bufT struct), have three > new entries > uint current_line_num; > uchar *current_line_buffer; > uint current_line_alloced; > 2) current_line_buffer points to the string data for the current line, > it starts off at a sane size (say 80, or screen_width), and doubles in > alloced size when it is too full. > 3) ml_append_char (or whatever I call it), will check the target line > number against current_line_number, if the match then > - append the char, possibly reallocing the current_line_buffer if we > have run out of space > ELSE, the edit line number and our "cached line" don't match > - trim the size of the current_line_buffer to match the STRLEN of the line > - alloc a new oversized line buffer and copy the new edit line into it > - append the char as above Vim caches the last changed line, you could do something with that. Start looking at ml_flush_line(), you can find the rest from there. > Would this kind of optimisation be more widely useful? Perhaps. Changing the same line several times is not unusual. It mostly depends on how much code would need to be changed compared to how much we would gain. - Bram Thanks for the tips, I'll test all this out and see what I can come up with. I also accidentally posted just to Bram, so I've included the list in this reply. Thanks Brad
Re: Fastest way to append line or char to a buffer
Nikolai Weibull wrote: > On 8/16/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote: > > > There are only a few operations that profit from a "gap" strategy. > > Overall it gets much more complicated and inefficient. You would have > > to move the gap around for every change made. > > Not quite true. You'd have to move it around for every > /length-altering/ change made (but non-length-altering changes work > well without a gap as well). Just about any change will potentially change the length. Keep in mind that changing a character may change the byte count for multi-byte characters. > And considering that insertion is the most common operation and that > it's one of the operations that would benefit, it's a trade-off that > may or may not benefit buffer modification as a whole. Insertion is just one of the many operations. And it's one that can be put in a function used from many places, while other changes are much more specific and require separate implementation. > Still, I suppose the most common scenario for the most common > operation is insertion at the end of a line, which I assume there /is/ > support for in Vim already, i.e., allocating some additional space for > every loaded line so that one doesn't have to realloc() for every > insertion. Not really. I don't think it makes sense to optimize for one of the very many operations. Keeping the code simple and reliable is much more important. > Remember: Emacs uses the gap strategy for the whole buffer, and I > haven't seen this to be particularly inefficient. The big problem is > the continuity of the buffer in memory, which isn't much of a problem > on a per-line basis. I have never seen a reliable, real-world comparison of performance for various strategies. Only ones with so many assumptions that the result becomes meaningless. You quickly get to the situation that the outcome mainly depends on the type of edits you test with. The mechanism Vim uses is probably too complex for scientific research anyway. Optimizing for file loading is one thing that could still be done in Vim and would probably improve performance. Currently every line has to be copied after it has been read. I know it's possible to avoid that, but quite a bit of work (esp. for testing, this code has to be very reliable). Since I don't hear remarks that loading a file is slow it's not high on the priority list. -- hundred-and-one symptoms of being an internet addict: 161. You get up before the sun rises to check your e-mail, and you find yourself in the very same chair long after the sun has set. /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: Fastest way to append line or char to a buffer
On 8/16/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote: There are only a few operations that profit from a "gap" strategy. Overall it gets much more complicated and inefficient. You would have to move the gap around for every change made. Not quite true. You'd have to move it around for every /length-altering/ change made (but non-length-altering changes work well without a gap as well). And considering that insertion is the most common operation and that it's one of the operations that would benefit, it's a trade-off that may or may not benefit buffer modification as a whole. Still, I suppose the most common scenario for the most common operation is insertion at the end of a line, which I assume there /is/ support for in Vim already, i.e., allocating some additional space for every loaded line so that one doesn't have to realloc() for every insertion. Remember: Emacs uses the gap strategy for the whole buffer, and I haven't seen this to be particularly inefficient. The big problem is the continuity of the buffer in memory, which isn't much of a problem on a per-line basis. nikolai
Re: Fastest way to append line or char to a buffer
Nikolai Weibull wrote: > On 8/16/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote: > > Vim internally works linewise. Appending a character means taking the > > current line and replacing it with one that has the character appended. > > If you keep adding characters to the same line you could do some > > optimization to avoid having to allocate/free lines all the time. > > Bram, have you ever given any thought to using a gap > buffering-strategy for lines? This would make many operations on long > lines (a recurring issue discussed on this list) faster. It will > complicate the buffer handling a lot, though. There are only a few operations that profit from a "gap" strategy. Overall it gets much more complicated and inefficient. You would have to move the gap around for every change made. -- hundred-and-one symptoms of being an internet addict: 155. You forget to eat because you're too busy surfing the net. /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: Fastest way to append line or char to a buffer
On 8/16/06, Bram Moolenaar <[EMAIL PROTECTED]> wrote: Vim internally works linewise. Appending a character means taking the current line and replacing it with one that has the character appended. If you keep adding characters to the same line you could do some optimization to avoid having to allocate/free lines all the time. Bram, have you ever given any thought to using a gap buffering-strategy for lines? This would make many operations on long lines (a recurring issue discussed on this list) faster. It will complicate the buffer handling a lot, though. nikolai
Re: Fastest way to append line or char to a buffer
Brad Beveridge wrote: > I'm working with a Vim that has been patched to support ECL Lisp > (http://wiki.alu.org/Vim_ECL). Part of our project involves > outputting potentially large amounts of data to a buffer in a > streaming fashion, ie always adding at the end. > I think that I have worked out a reasonable way to append lines[1], > but I can't see any obvious way to quickly append single chars to a > buffer. > Are there any functions like ml_append I can look to for this support? > How should I go about appending single chars very quickly? Vim internally works linewise. Appending a character means taking the current line and replacing it with one that has the character appended. If you keep adding characters to the same line you could do some optimization to avoid having to allocate/free lines all the time. -- hundred-and-one symptoms of being an internet addict: 151. You find yourself engaged to someone you've never actually met, except through e-mail. /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
