Re: CVS commit: src/external/bsd/mdocml/include

[Martin Husemann]

On Sun, Mar 18, 2018 at 01:26:12PM +, Christos Zoulas wrote:
> I would just leave it broken until the perseant branch is merged :-)
> You can always fix it, buy changing the locale in the environment to one
> that works, instead of gutting it for all locales.

IIUC we have to weigh:

 - the number of documents in base or pkgsrc that benefit from unicode
   support (my guess: close to zero)
 - the number of users of non-unicode based locales where the current
   version is just broken (a regression from previous relesases), which
   is pretty huge

The version after the change should mostly work exactly the same for all
common documents even when the user uses a UTF-8 locale setting.

Or am I missing something?

Martin

Re: CVS commit: src/external/bsd/mdocml/include

[Christos Zoulas]

In article <20180318.153625.846954900.nakay...@netbsd.org>,
Takeshi Nakayama   wrote:
 chris...@astron.com (Christos Zoulas) wrote
>
>> In article <20180317235857.ga5...@homeworld.netbsd.org>,
>>   wrote:
>> >Hi,
>> >
>> >I'm not sure what the changes affect. Does this mean UTF-8 won't work
>> >either? that's sounds like a big price to pay.
>> >Also, the problem doesn't sound specific to mdocml.
>> 
>> Well, yes it is a bit drastic. Perhaps it can be handled with an
>> environment variable instead (to disable for locales that don't work).
>
>I thought about the following change, but it is too ugly.

I would just leave it broken until the perseant branch is merged :-)
You can always fix it, buy changing the locale in the environment to one
that works, instead of gutting it for all locales.

christos

Re: CVS commit: src/external/bsd/mdocml/include

[Takeshi Nakayama]

>>> chris...@astron.com (Christos Zoulas) wrote

> In article <20180317235857.ga5...@homeworld.netbsd.org>,
>   wrote:
> >Hi,
> >
> >I'm not sure what the changes affect. Does this mean UTF-8 won't work
> >either? that's sounds like a big price to pay.
> >Also, the problem doesn't sound specific to mdocml.
> 
> Well, yes it is a bit drastic. Perhaps it can be handled with an
> environment variable instead (to disable for locales that don't work).

I thought about the following change, but it is too ugly.

-- Takeshi Nakayama


--- external/bsd/mdocml/dist/term_ascii.c.orig  2017-03-21 19:13:13.0 
+0900
+++ external/bsd/mdocml/dist/term_ascii.c   2018-03-18 15:30:41.0 
+0900
@@ -22,6 +22,7 @@
 #include 
 #if HAVE_WCHAR
 #include 
+#include 
 #endif
 #include 
 #include 
@@ -99,7 +100,7 @@
v = TERMENC_LOCALE == enc ?
setlocale(LC_CTYPE, "") :
setlocale(LC_CTYPE, UTF8_LOCALE);
-   if (NULL != v && MB_CUR_MAX > 1) {
+   if (NULL != v && MB_CUR_MAX > 1 && NULL != strstr(v, ".UTF-8")) 
{
p->enc = enc;
p->advance = locale_advance;
p->endline = locale_endline;

Re: CVS commit: src/external/bsd/mdocml/include

[Takeshi Nakayama]

>>> m...@netbsd.org wrote

> Hi,
> 
> I'm not sure what the changes affect. Does this mean UTF-8 won't work
> either? that's sounds like a big price to pay.
> Also, the problem doesn't sound specific to mdocml.

No, this is specific to mdocml.  mdocml assums wchar_t's internal
representation UCS-4 (__STDC_ISO_10646__).

See line 31 of:
 http://mandoc.bsd.lv/cgi-bin/cvsweb/configure.local.example?annotate=1.33

But our locale's wchar_t is not always UCS-4, we use a
locale-dependent encoding for now (eg. ja_JP.eucJP).

If branch perseant-stdc-iso10646 is merged, I think that the
situation will change.

> Is the problem: netbsd mishandles some locales it claims to support?

No.

-- Takeshi Nakayama