On Wednesday, July 3, 2013 4:06:32 PM UTC+2, Christian Wellenbrock wrote:
> 2013/7/3 Bram Moolenaar <[email protected]>
>
>
>
>
> Christian Wellenbrock wrote:
>
>
>
> > I added the MB_CHAR2LEN macro and used that instead of mb_char2len.
>
> >
>
> > Does the mb_char2bytes call need a has_mbyte check as well?
>
> >
>
> > Are there any other issues that need to resolved to bring this patch
>
> > forward?
>
>
>
> I finally had time to look at this patch. It turns out only one use of
>
> "fillchar" is handled. There are many others, they all need to be
>
> changed.
>
>
>
> Yes I also realized that shortly after. I already extended my patch to
> support multibyte fillchars in all those instances. I will test them all and
> upload the improved patch soon.
In order to test the changes I made, I started out with a patch that helped me
to find and verify 'statusline' options that show the nine different uses of
the 'fillchar'. In `1_replace_fillchars.patch` I replaced them all with the
different hardcoded placeholder characters 'a' to 'i'. Here are nine vim
invocations for each of them:
(I'm using ○ (digraph 0m) as multibyte fillchar in all invocations and 〇
(digraph 0_) als a double-width character)
[a] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S%2.2(X〇%)E"
[b] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S%-1(%)E"
[c] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S%1(%)E"
[d] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S%4PE"
[e] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S%{'A\ B'}E"
[f] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S%-4PE"
[g] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S234567890〇%<1"
[h] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S〇123456789E"
[i] vim -u NONE -c "set ls=2 fcs=stl:○ stl=S%=E"
The binary resulting from patch 1 shows these statuslines respectively:
(in a terminal that is exactly 12 characters wide)
[a] S<aE○○○○○○○○
[b] SbE○○○○○○○○○
[c] ScE○○○○○○○○○
[d] SdAllE○○○○○○
[e] SAeBE○○○○○○○
[f] SAllfE○○○○○○
[g] S234567890>
[h] <123456789Eh
[i] SiiiiiiiiiiE
Note that (almost) every statusline contains the corresponding hardcoded
'fillchar' placeholder. Except the 'g' is missing for invocation [g]. I believe
the reason for that is a bug that I fixed in `2_fix_fillchar_g.patch`. The new
binary shows the expected result:
[a] S<aE○○○○○○○○
[b] SbE○○○○○○○○○
[c] ScE○○○○○○○○○
[d] SdAllE○○○○○○
[e] SAeBE○○○○○○○
[f] SAllfE○○○○○○
[g] S234567890g>
[h] <123456789Eh
[i] SiiiiiiiiiiE
Now in `3_use_multibyte.patch` I finally added support for multibyte support
for all these fillchars:
[a] S<○E○○○○○○○○
[b] S○E○○○○○○○○○
[c] S○E○○○○○○○○○
[d] S○AllE○○○○○○
[e] SA○BE○○○○○○○
[f] SAll○E○○○○○○
[g] S234567890○>
[h] <123456789E○
[i] S○○○○○○○○○○E
Every former fillchar placeholder is now replaced by the real multibyte
fillchar.
Bonus: As I recognized a pattern that repeated itself several times, I
extracted a macro `MB_CHAR2BYTES` to simplify matters in `4_add_macro.patch`.
I'm not sure about this name regarding the advancing side effect, though. This
could obviously be renamed. Another possibility would be to remove the side
effect and add `b += MB_CHAR2LEN(fillchar);` on the caller sites.
I'm also not quite sure if the case [h] is fully tested by my invocation. There
is some code moving the subsequent items in memory, but I wasn't able to find a
'statusline', where the fillchar was somewhere in the middle of the statusline.
So I didn't really understand the intention of that logic and I might broke
that part. If someone knows a 'statusline' option where my implementation
breaks expected behaviour, I would be happy to fix it.
If there are any other issues, please let me know. Thank you!
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
diff -r 58686d48772e src/buffer.c
--- a/src/buffer.c Wed Jul 03 22:28:36 2013 +0200
+++ b/src/buffer.c Fri Jul 05 00:36:30 2013 +0200
@@ -3699,7 +3699,7 @@
#ifdef FEAT_MBYTE
/* Fill up space left over by half a double-wide char. */
while (++l < item[groupitem[groupdepth]].minwid)
- *p++ = fillchar;
+ *p++ = 'a'; // TODO: use fillchar
#endif
/* correct the start of the items for the truncation */
@@ -3719,7 +3719,7 @@
/* fill by appending characters */
n = 0 - n;
while (l++ < n && p + 1 < out + outlen)
- *p++ = fillchar;
+ *p++ = 'b'; // TODO: use fillchar
}
else
{
@@ -3732,7 +3732,7 @@
for (n = groupitem[groupdepth] + 1; n < curitem; n++)
item[n].start += l;
for ( ; l > 0; l--)
- *t++ = fillchar;
+ *t++ = 'c'; // TODO: use fillchar
}
}
continue;
@@ -4103,7 +4103,7 @@
if (l + 1 == minwid && fillchar == '-' && VIM_ISDIGIT(*t))
*p++ = ' ';
else
- *p++ = fillchar;
+ *p++ = 'd'; // TODO: use fillchar
}
minwid = 0;
}
@@ -4116,10 +4116,10 @@
* digit follows. */
if (fillable && p[-1] == ' '
&& (!VIM_ISDIGIT(*t) || fillchar != '-'))
- p[-1] = fillchar;
+ p[-1] = 'e'; // TODO: use fillchar
}
for (; l < minwid && p + 1 < out + outlen; l++)
- *p++ = fillchar;
+ *p++ = 'f'; // TODO: use fillchar
}
else if (num >= 0)
{
@@ -4223,7 +4223,7 @@
}
/* Fill up for half a double-wide character. */
while (++width < maxwidth)
- *s++ = fillchar;
+ *s++ = 'g'; // TODO: use fillchar
}
else
#endif
@@ -4258,7 +4258,7 @@
while (++width < maxwidth)
{
s = s + STRLEN(s);
- *s++ = fillchar;
+ *s++ = 'h'; // TODO: use fillchar
*s = NUL;
}
@@ -4284,7 +4284,7 @@
p = item[l].start + maxwidth - width;
STRMOVE(p, item[l].start);
for (s = item[l].start; s < p; s++)
- *s = fillchar;
+ *s = 'i'; // TODO: use fillchar
for (l++; l < itemcnt; l++)
item[l].start += maxwidth - width;
width = maxwidth;
diff -r 58686d48772e src/buffer.c
--- a/src/buffer.c Wed Jul 03 22:28:36 2013 +0200
+++ b/src/buffer.c Fri Jul 05 00:41:39 2013 +0200
@@ -3699,7 +3699,7 @@
#ifdef FEAT_MBYTE
/* Fill up space left over by half a double-wide char. */
while (++l < item[groupitem[groupdepth]].minwid)
- *p++ = fillchar;
+ *p++ = 'a'; // TODO: use fillchar
#endif
/* correct the start of the items for the truncation */
@@ -3719,7 +3719,7 @@
/* fill by appending characters */
n = 0 - n;
while (l++ < n && p + 1 < out + outlen)
- *p++ = fillchar;
+ *p++ = 'b'; // TODO: use fillchar
}
else
{
@@ -3732,7 +3732,7 @@
for (n = groupitem[groupdepth] + 1; n < curitem; n++)
item[n].start += l;
for ( ; l > 0; l--)
- *t++ = fillchar;
+ *t++ = 'c'; // TODO: use fillchar
}
}
continue;
@@ -4103,7 +4103,7 @@
if (l + 1 == minwid && fillchar == '-' && VIM_ISDIGIT(*t))
*p++ = ' ';
else
- *p++ = fillchar;
+ *p++ = 'd'; // TODO: use fillchar
}
minwid = 0;
}
@@ -4116,10 +4116,10 @@
* digit follows. */
if (fillable && p[-1] == ' '
&& (!VIM_ISDIGIT(*t) || fillchar != '-'))
- p[-1] = fillchar;
+ p[-1] = 'e'; // TODO: use fillchar
}
for (; l < minwid && p + 1 < out + outlen; l++)
- *p++ = fillchar;
+ *p++ = 'f'; // TODO: use fillchar
}
else if (num >= 0)
{
@@ -4214,16 +4214,14 @@
{
s = out;
width = 0;
- for (;;)
+ while (width + ptr2cells(s) < maxwidth)
{
width += ptr2cells(s);
- if (width >= maxwidth)
- break;
s += (*mb_ptr2len)(s);
}
/* Fill up for half a double-wide character. */
while (++width < maxwidth)
- *s++ = fillchar;
+ *s++ = 'g'; // TODO: use fillchar
}
else
#endif
@@ -4258,7 +4256,7 @@
while (++width < maxwidth)
{
s = s + STRLEN(s);
- *s++ = fillchar;
+ *s++ = 'h'; // TODO: use fillchar
*s = NUL;
}
@@ -4284,7 +4282,7 @@
p = item[l].start + maxwidth - width;
STRMOVE(p, item[l].start);
for (s = item[l].start; s < p; s++)
- *s = fillchar;
+ *s = 'i'; // TODO: use fillchar
for (l++; l < itemcnt; l++)
item[l].start += maxwidth - width;
width = maxwidth;
diff -r 58686d48772e src/buffer.c
--- a/src/buffer.c Wed Jul 03 22:28:36 2013 +0200
+++ b/src/buffer.c Fri Jul 05 23:13:37 2013 +0200
@@ -3570,11 +3570,6 @@
if (fillchar == 0)
fillchar = ' ';
-#ifdef FEAT_MBYTE
- /* Can't handle a multi-byte fill character yet. */
- else if (mb_char2len(fillchar) > 1)
- fillchar = '-';
-#endif
/* Get line & check if empty (cursorpos will show "0-1"). Note that
* p will become invalid when getting another buffer line. */
@@ -3699,7 +3694,7 @@
#ifdef FEAT_MBYTE
/* Fill up space left over by half a double-wide char. */
while (++l < item[groupitem[groupdepth]].minwid)
- *p++ = fillchar;
+ p += (*mb_char2bytes)(fillchar, p);
#endif
/* correct the start of the items for the truncation */
@@ -3719,20 +3714,28 @@
/* fill by appending characters */
n = 0 - n;
while (l++ < n && p + 1 < out + outlen)
+#ifdef FEAT_MBYTE
+ p += (*mb_char2bytes)(fillchar, p);
+#else
*p++ = fillchar;
+#endif
}
else
{
/* fill by inserting characters */
- mch_memmove(t + n - l, t, (size_t)(p - t));
- l = n - l;
+ l = (n - l) * MB_CHAR2LEN(fillchar);
+ mch_memmove(t + l, t, (size_t)(p - t));
if (p + l >= out + outlen)
l = (long)((out + outlen) - p - 1);
p += l;
for (n = groupitem[groupdepth] + 1; n < curitem; n++)
item[n].start += l;
for ( ; l > 0; l--)
+#ifdef FEAT_MBYTE
+ t += (*mb_char2bytes)(fillchar, t);
+#else
*t++ = fillchar;
+#endif
}
}
continue;
@@ -4103,23 +4106,36 @@
if (l + 1 == minwid && fillchar == '-' && VIM_ISDIGIT(*t))
*p++ = ' ';
else
+#ifdef FEAT_MBYTE
+ p += (*mb_char2bytes)(fillchar, p);
+#else
*p++ = fillchar;
+#endif
}
minwid = 0;
}
else
minwid *= -1;
- while (*t && p + 1 < out + outlen)
+ for (; *t && p + 1 < out + outlen; t++)
{
- *p++ = *t++;
/* Change a space by fillchar, unless fillchar is '-' and a
* digit follows. */
- if (fillable && p[-1] == ' '
- && (!VIM_ISDIGIT(*t) || fillchar != '-'))
- p[-1] = fillchar;
+ if (fillable && *t == ' '
+ && (!VIM_ISDIGIT(*(t+1)) || fillchar != '-'))
+#ifdef FEAT_MBYTE
+ p += (*mb_char2bytes)(fillchar, p);
+#else
+ *p++ = fillchar;
+#endif
+ else
+ *p++ = *t;
}
for (; l < minwid && p + 1 < out + outlen; l++)
+#ifdef FEAT_MBYTE
+ p += (*mb_char2bytes)(fillchar, p);
+#else
*p++ = fillchar;
+#endif
}
else if (num >= 0)
{
@@ -4214,16 +4230,14 @@
{
s = out;
width = 0;
- for (;;)
+ while (width + ptr2cells(s) < maxwidth)
{
width += ptr2cells(s);
- if (width >= maxwidth)
- break;
s += (*mb_ptr2len)(s);
}
/* Fill up for half a double-wide character. */
while (++width < maxwidth)
- *s++ = fillchar;
+ s += (*mb_char2bytes)(fillchar, s);
}
else
#endif
@@ -4258,7 +4272,11 @@
while (++width < maxwidth)
{
s = s + STRLEN(s);
+#ifdef FEAT_MBYTE
+ s += (*mb_char2bytes)(fillchar, s);
+#else
*s++ = fillchar;
+#endif
*s = NUL;
}
@@ -4281,12 +4299,17 @@
break;
if (l < itemcnt)
{
- p = item[l].start + maxwidth - width;
+ int middlelength = (maxwidth - width) * MB_CHAR2LEN(fillchar);
+ p = item[l].start + middlelength;
STRMOVE(p, item[l].start);
- for (s = item[l].start; s < p; s++)
- *s = fillchar;
+ for (s = item[l].start; s < p;)
+#ifdef FEAT_MBYTE
+ s += (*mb_char2bytes)(fillchar, s);
+#else
+ *s++ = fillchar;
+#endif
for (l++; l < itemcnt; l++)
- item[l].start += maxwidth - width;
+ item[l].start += middlelength;
width = maxwidth;
}
}
diff -r 58686d48772e src/buffer.c
--- a/src/buffer.c Wed Jul 03 22:28:36 2013 +0200
+++ b/src/buffer.c Fri Jul 05 23:54:36 2013 +0200
@@ -3570,11 +3570,6 @@
if (fillchar == 0)
fillchar = ' ';
-#ifdef FEAT_MBYTE
- /* Can't handle a multi-byte fill character yet. */
- else if (mb_char2len(fillchar) > 1)
- fillchar = '-';
-#endif
/* Get line & check if empty (cursorpos will show "0-1"). Note that
* p will become invalid when getting another buffer line. */
@@ -3699,7 +3694,7 @@
#ifdef FEAT_MBYTE
/* Fill up space left over by half a double-wide char. */
while (++l < item[groupitem[groupdepth]].minwid)
- *p++ = fillchar;
+ MB_CHAR2BYTES(fillchar, p);
#endif
/* correct the start of the items for the truncation */
@@ -3719,20 +3714,20 @@
/* fill by appending characters */
n = 0 - n;
while (l++ < n && p + 1 < out + outlen)
- *p++ = fillchar;
+ MB_CHAR2BYTES(fillchar, p);
}
else
{
/* fill by inserting characters */
- mch_memmove(t + n - l, t, (size_t)(p - t));
- l = n - l;
+ l = (n - l) * MB_CHAR2LEN(fillchar);
+ mch_memmove(t + l, t, (size_t)(p - t));
if (p + l >= out + outlen)
l = (long)((out + outlen) - p - 1);
p += l;
for (n = groupitem[groupdepth] + 1; n < curitem; n++)
item[n].start += l;
for ( ; l > 0; l--)
- *t++ = fillchar;
+ MB_CHAR2BYTES(fillchar, t);
}
}
continue;
@@ -4103,23 +4098,24 @@
if (l + 1 == minwid && fillchar == '-' && VIM_ISDIGIT(*t))
*p++ = ' ';
else
- *p++ = fillchar;
+ MB_CHAR2BYTES(fillchar, p);
}
minwid = 0;
}
else
minwid *= -1;
- while (*t && p + 1 < out + outlen)
+ for (; *t && p + 1 < out + outlen; t++)
{
- *p++ = *t++;
/* Change a space by fillchar, unless fillchar is '-' and a
* digit follows. */
- if (fillable && p[-1] == ' '
- && (!VIM_ISDIGIT(*t) || fillchar != '-'))
- p[-1] = fillchar;
+ if (fillable && *t == ' '
+ && (!VIM_ISDIGIT(*(t+1)) || fillchar != '-'))
+ MB_CHAR2BYTES(fillchar, p);
+ else
+ *p++ = *t;
}
for (; l < minwid && p + 1 < out + outlen; l++)
- *p++ = fillchar;
+ MB_CHAR2BYTES(fillchar, p);
}
else if (num >= 0)
{
@@ -4214,16 +4210,14 @@
{
s = out;
width = 0;
- for (;;)
+ while (width + ptr2cells(s) < maxwidth)
{
width += ptr2cells(s);
- if (width >= maxwidth)
- break;
s += (*mb_ptr2len)(s);
}
/* Fill up for half a double-wide character. */
while (++width < maxwidth)
- *s++ = fillchar;
+ MB_CHAR2BYTES(fillchar, s);
}
else
#endif
@@ -4258,7 +4252,7 @@
while (++width < maxwidth)
{
s = s + STRLEN(s);
- *s++ = fillchar;
+ MB_CHAR2BYTES(fillchar, s);
*s = NUL;
}
@@ -4281,12 +4275,13 @@
break;
if (l < itemcnt)
{
- p = item[l].start + maxwidth - width;
+ int middlelength = (maxwidth - width) * MB_CHAR2LEN(fillchar);
+ p = item[l].start + middlelength;
STRMOVE(p, item[l].start);
- for (s = item[l].start; s < p; s++)
- *s = fillchar;
+ for (s = item[l].start; s < p;)
+ MB_CHAR2BYTES(fillchar, s);
for (l++; l < itemcnt; l++)
- item[l].start += maxwidth - width;
+ item[l].start += middlelength;
width = maxwidth;
}
}
diff -r 58686d48772e src/macros.h
--- a/src/macros.h Wed Jul 03 22:28:36 2013 +0200
+++ b/src/macros.h Fri Jul 05 23:54:36 2013 +0200
@@ -269,16 +269,18 @@
/* get length of multi-byte char, not including composing chars */
# define mb_cptr2len(p) (enc_utf8 ? utf_ptr2len(p) : (*mb_ptr2len)(p))
-# define MB_COPY_CHAR(f, t) if (has_mbyte) mb_copy_char(&f, &t); else *t++ = *f++
-# define MB_CHARLEN(p) (has_mbyte ? mb_charlen(p) : (int)STRLEN(p))
-# define MB_CHAR2LEN(c) (has_mbyte ? mb_char2len(c) : 1)
-# define PTR2CHAR(p) (has_mbyte ? mb_ptr2char(p) : (int)*(p))
+# define MB_COPY_CHAR(f, t) if (has_mbyte) mb_copy_char(&f, &t); else *t++ = *f++
+# define MB_CHAR2BYTES(c, b) if (has_mbyte) b += (*mb_char2bytes)(c, b); else *b++ = c
+# define MB_CHARLEN(p) (has_mbyte ? mb_charlen(p) : (int)STRLEN(p))
+# define MB_CHAR2LEN(c) (has_mbyte ? mb_char2len(c) : 1)
+# define PTR2CHAR(p) (has_mbyte ? mb_ptr2char(p) : (int)*(p))
#else
# define MB_PTR2LEN(p) 1
# define mb_ptr_adv(p) ++p
# define mb_cptr_adv(p) ++p
# define mb_ptr_back(s, p) --p
# define MB_COPY_CHAR(f, t) *t++ = *f++
+# define MB_CHAR2BYTES(c, b) *b++ = c
# define MB_CHARLEN(p) STRLEN(p)
# define MB_CHAR2LEN(c) 1
# define PTR2CHAR(p) ((int)*(p))
