Re: CVS commit: src/lib/libc/locale

2021-02-15 Thread Joerg Sonnenberger
On Mon, Feb 15, 2021 at 04:37:15PM +0100, Thomas Klausner wrote:
> Hi!
> 
> Thanks for the man pages.
> 
> There is none for uselocale(3), which is heavily referenced from
> these, nor do I find a prototype in /usr/include...
> so how does one use these? :)

We don't support uselocale. You use this functions by passing the
locale_t to the *_l functions directly.

Joerg


Re: CVS commit: src/lib/libc/locale

2021-02-15 Thread Thomas Klausner
Hi!

Thanks for the man pages.

There is none for uselocale(3), which is heavily referenced from
these, nor do I find a prototype in /usr/include...
so how does one use these? :)
 Thomas

On Mon, Feb 15, 2021 at 09:35:04AM -0500, Christos Zoulas wrote:
> Module Name:  src
> Committed By: christos
> Date: Mon Feb 15 14:35:04 UTC 2021
> 
> Modified Files:
>   src/lib/libc/locale: Makefile.inc
> Added Files:
>   src/lib/libc/locale: duplocale.3 freelocale.3 newlocale.3
> 
> Log Message:
> Add missing man pages (from FreeBSD)
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.64 -r1.65 src/lib/libc/locale/Makefile.inc
> cvs rdiff -u -r0 -r1.1 src/lib/libc/locale/duplocale.3 \
> src/lib/libc/locale/freelocale.3 src/lib/libc/locale/newlocale.3
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
> 

> Modified files:
> 
> Index: src/lib/libc/locale/Makefile.inc
> diff -u src/lib/libc/locale/Makefile.inc:1.64 
> src/lib/libc/locale/Makefile.inc:1.65
> --- src/lib/libc/locale/Makefile.inc:1.64 Sun Aug 18 16:03:48 2013
> +++ src/lib/libc/locale/Makefile.inc  Mon Feb 15 09:35:04 2021
> @@ -1,5 +1,5 @@
>  #from: @(#)Makefile.inc  5.1 (Berkeley) 2/18/91
> -#$NetBSD: Makefile.inc,v 1.64 2013/08/18 20:03:48 joerg Exp $
> +#$NetBSD: Makefile.inc,v 1.65 2021/02/15 14:35:04 christos Exp $
>  
>  # locale sources
>  .PATH: ${ARCHDIR}/locale ${.CURDIR}/locale
> @@ -21,7 +21,7 @@ CPPFLAGS.runetable.c+=  -I${LIBCDIR}/cit
>  CPPFLAGS.multibyte_c90.c+=   -I${LIBCDIR}/citrus
>  CPPFLAGS.multibyte_amd1.c+=  -I${LIBCDIR}/citrus
>  
> -MAN+=setlocale.3 nl_langinfo.3
> +MAN+=duplocale.3 freelocale.3 newlocale.3 setlocale.3 nl_langinfo.3 
>  
>  MAN+=mbtowc.3 mbstowcs.3 wctomb.3 wcstombs.3 mblen.3 \
>  
> 
> Added files:
> 
> Index: src/lib/libc/locale/duplocale.3
> diff -u /dev/null src/lib/libc/locale/duplocale.3:1.1
> --- /dev/null Mon Feb 15 09:35:04 2021
> +++ src/lib/libc/locale/duplocale.3   Mon Feb 15 09:35:04 2021
> @@ -0,0 +1,80 @@
> +.\" $NetBSD: duplocale.3,v 1.1 2021/02/15 14:35:04 christos Exp $
> +.\" Copyright (c) 2011 The FreeBSD Foundation
> +.\" All rights reserved.
> +.\"
> +.\" This documentation was written by David Chisnall under sponsorship from
> +.\" the FreeBSD Foundation.
> +.\"
> +.\" Redistribution and use in source and binary forms, with or without
> +.\" modification, are permitted provided that the following conditions
> +.\" are met:
> +.\" 1. Redistributions of source code must retain the above copyright
> +.\"notice, this list of conditions and the following disclaimer.
> +.\" 2. Redistributions in binary form must reproduce the above copyright
> +.\"notice, this list of conditions and the following disclaimer in the
> +.\"documentation and/or other materials provided with the distribution.
> +.\"
> +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
> +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 
> PURPOSE
> +.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
> +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 
> CONSEQUENTIAL
> +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
> +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 
> STRICT
> +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> +.\" SUCH DAMAGE.
> +.\"
> +.\" $FreeBSD: head/lib/libc/locale/duplocale.3 281925 2015-04-24 10:17:55Z 
> theraven $
> +.\"
> +.Dd September 17, 2011
> +.Dt DUPLOCALE 3
> +.Os
> +.Sh NAME
> +.Nm duplocale
> +.Nd duplicate an locale
> +.Sh LIBRARY
> +.Lb libc
> +.Sh SYNOPSIS
> +.In locale.h
> +.Ft locale_t
> +.Fn duplocale "locale_t locale"
> +.Sh DESCRIPTION
> +Duplicates an existing
> +.Fa locale_t
> +returning a new
> +.Fa locale_t
> +that refers to the same locale values but has an independent internal state.
> +Various functions, such as
> +.Xr mblen 3
> +require a persistent state.
> +These functions formerly used static variables and calls to them from 
> multiple
> +threads had undefined behavior.
> +They now use fields in the
> +.Fa locale_t
> +associated with the current thread by
> +.Xr uselocale 3 .
> +These calls are therefore only thread safe on threads with a unique 
> per-thread
> +locale.
> +The locale returned by this call must be freed with
> +.Xr freelocale 3 .
> +.Sh SEE ALSO
> +.Xr freelocale 3 ,
> +.Xr localeconv 3 ,
> +.Xr newlocale 3 ,
> +.\" .Xr querylocale 3 ,
> +.Xr uselocale 3 ,
> +.\" .Xr xlocale 3
> +.Sh STANDARDS
> +This function conforms to
> +.St -p1003.1-2008 .
> +.Sh BUGS
> +Ideally,
> +.Xr uselocale 3
> +should make a 

Re: CVS commit: src/lib/libc/locale

2013-04-22 Thread Joerg Sonnenberger
On Mon, Apr 22, 2013 at 01:45:52AM +, YAMAMOTO Takashi wrote:
  On Thu, Apr 18, 2013 at 06:58:42AM +, YAMAMOTO Takashi wrote:
   On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote:
On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
so that the struct _locale __C_locale in libc is much more wasteful.

I should add that it is an internal detail and the way the composed C
locale is stored can and likely will change later. So the way it is
essentially a copy of (old) global locale is just a way to be 
minimally
intrusive.

Joerg
   
   i care the API.
   
   if you really want it be in libc, how about having libc provide a
   locale_t get_static_c_locale(); style API rather than using NULL?
   it's better because 1) less code in *_l, 2) autoconf-like can detect
   the extension easily, and 3) a portable application can trivially
   have a fallback implementation using newlocale+pthread_once.
   
   It adds more cost on the caller side. So far, there are three mechanisms
   available (especially for libraries):
   
   (1) Adhoc access to internal locale state. This is used with glibc.
   (2) Explicit newlocale().
   (3) Implicit access via 0 argument to *_l.
   
   The first one is clearly a hack and not acceptable. Portable code can
   always conditionally use (2), but it requires additional setup and
   storage cost. (3) is used by Apple (which is where a large part of the
   *_l interface originates from) and FreeBSD. It is orthogonal to (2) and
   certainly easier to use. Exposing it via a special library call is also
   possible and effectively a way to implement (2) by a static wrapper.
   It still adds more cost to every caller and this is a classic case where
   there are typically much more callers.
   
   Joerg
  
  (3) adds small costs to every calls of *_l, even when the extension
  is not used.  i sounds worse than a one-time cost of (2) to me.
  
  (2) still needs to load the address (instead of a constant), so it isn't
  free either. Given that this is very popular as functionality and at
  least on modern CPUs often implementable as conditional-move, it sounds
  like a much better trade off.
 
 very popular?  i'm not aware of a single software which uses this extension.
 can you provide examples?

The very popular specifically applies to the need for accessing the C
locale. Most bigger libraries need it one way or another. I am currently
talking with Apple about providing a visible macro (LC_C_LOCALE) to
address the testability issue.

Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-21 Thread YAMAMOTO Takashi
 On Thu, Apr 18, 2013 at 06:58:42AM +, YAMAMOTO Takashi wrote:
  On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote:
   On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
   so that the struct _locale __C_locale in libc is much more wasteful.
   
   I should add that it is an internal detail and the way the composed C
   locale is stored can and likely will change later. So the way it is
   essentially a copy of (old) global locale is just a way to be minimally
   intrusive.
   
   Joerg
  
  i care the API.
  
  if you really want it be in libc, how about having libc provide a
  locale_t get_static_c_locale(); style API rather than using NULL?
  it's better because 1) less code in *_l, 2) autoconf-like can detect
  the extension easily, and 3) a portable application can trivially
  have a fallback implementation using newlocale+pthread_once.
  
  It adds more cost on the caller side. So far, there are three mechanisms
  available (especially for libraries):
  
  (1) Adhoc access to internal locale state. This is used with glibc.
  (2) Explicit newlocale().
  (3) Implicit access via 0 argument to *_l.
  
  The first one is clearly a hack and not acceptable. Portable code can
  always conditionally use (2), but it requires additional setup and
  storage cost. (3) is used by Apple (which is where a large part of the
  *_l interface originates from) and FreeBSD. It is orthogonal to (2) and
  certainly easier to use. Exposing it via a special library call is also
  possible and effectively a way to implement (2) by a static wrapper.
  It still adds more cost to every caller and this is a classic case where
  there are typically much more callers.
  
  Joerg
 
 (3) adds small costs to every calls of *_l, even when the extension
 is not used.  i sounds worse than a one-time cost of (2) to me.
 
 (2) still needs to load the address (instead of a constant), so it isn't
 free either. Given that this is very popular as functionality and at
 least on modern CPUs often implementable as conditional-move, it sounds
 like a much better trade off.

very popular?  i'm not aware of a single software which uses this extension.
can you provide examples?

YAMAMOTO Takashi

 
 Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-18 Thread YAMAMOTO Takashi
 On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote:
  On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
  so that the struct _locale __C_locale in libc is much more wasteful.
  
  I should add that it is an internal detail and the way the composed C
  locale is stored can and likely will change later. So the way it is
  essentially a copy of (old) global locale is just a way to be minimally
  intrusive.
  
  Joerg
 
 i care the API.
 
 if you really want it be in libc, how about having libc provide a
 locale_t get_static_c_locale(); style API rather than using NULL?
 it's better because 1) less code in *_l, 2) autoconf-like can detect
 the extension easily, and 3) a portable application can trivially
 have a fallback implementation using newlocale+pthread_once.
 
 It adds more cost on the caller side. So far, there are three mechanisms
 available (especially for libraries):
 
 (1) Adhoc access to internal locale state. This is used with glibc.
 (2) Explicit newlocale().
 (3) Implicit access via 0 argument to *_l.
 
 The first one is clearly a hack and not acceptable. Portable code can
 always conditionally use (2), but it requires additional setup and
 storage cost. (3) is used by Apple (which is where a large part of the
 *_l interface originates from) and FreeBSD. It is orthogonal to (2) and
 certainly easier to use. Exposing it via a special library call is also
 possible and effectively a way to implement (2) by a static wrapper.
 It still adds more cost to every caller and this is a classic case where
 there are typically much more callers.
 
 Joerg

(3) adds small costs to every calls of *_l, even when the extension
is not used.  i sounds worse than a one-time cost of (2) to me.

YAMAMOTO Takashi


Re: CVS commit: src/lib/libc/locale

2013-04-18 Thread Joerg Sonnenberger
On Thu, Apr 18, 2013 at 06:58:42AM +, YAMAMOTO Takashi wrote:
  On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote:
   On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
   so that the struct _locale __C_locale in libc is much more wasteful.
   
   I should add that it is an internal detail and the way the composed C
   locale is stored can and likely will change later. So the way it is
   essentially a copy of (old) global locale is just a way to be minimally
   intrusive.
   
   Joerg
  
  i care the API.
  
  if you really want it be in libc, how about having libc provide a
  locale_t get_static_c_locale(); style API rather than using NULL?
  it's better because 1) less code in *_l, 2) autoconf-like can detect
  the extension easily, and 3) a portable application can trivially
  have a fallback implementation using newlocale+pthread_once.
  
  It adds more cost on the caller side. So far, there are three mechanisms
  available (especially for libraries):
  
  (1) Adhoc access to internal locale state. This is used with glibc.
  (2) Explicit newlocale().
  (3) Implicit access via 0 argument to *_l.
  
  The first one is clearly a hack and not acceptable. Portable code can
  always conditionally use (2), but it requires additional setup and
  storage cost. (3) is used by Apple (which is where a large part of the
  *_l interface originates from) and FreeBSD. It is orthogonal to (2) and
  certainly easier to use. Exposing it via a special library call is also
  possible and effectively a way to implement (2) by a static wrapper.
  It still adds more cost to every caller and this is a classic case where
  there are typically much more callers.
  
  Joerg
 
 (3) adds small costs to every calls of *_l, even when the extension
 is not used.  i sounds worse than a one-time cost of (2) to me.

(2) still needs to load the address (instead of a constant), so it isn't
free either. Given that this is very popular as functionality and at
least on modern CPUs often implementable as conditional-move, it sounds
like a much better trade off.

Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-18 Thread Richard Hansen
On 2013-04-16 00:55, YAMAMOTO Takashi wrote:
 On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote:
 POSIX2008 spec say, *_l func with invalid locale handle may EINVAL.
 NULL or (locale_t)0 is invalid locale handle.

Note that POSIX Issue 7 TC1 was recently published [1], and there are
some relevant locale fixes.  Specifically, [2] removes the may fail
EINVAL error from *_l(), and adds the following paragraph:

The behavior is undefined if the locale argument to *_l() is the
special locale object LC_GLOBAL_LOCALE or is not a valid locale
object handle.

-Richard

[1] http://article.gmane.org/gmane.comp.standards.posix.austin.general/7004
[2] http://austingroupbugs.net/view.php?id=283


Re: CVS commit: src/lib/libc/locale

2013-04-17 Thread Joerg Sonnenberger
On Tue, Apr 16, 2013 at 11:34:52PM +, YAMAMOTO Takashi wrote:
  On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
  so that the struct _locale __C_locale in libc is much more wasteful.
  
  I should add that it is an internal detail and the way the composed C
  locale is stored can and likely will change later. So the way it is
  essentially a copy of (old) global locale is just a way to be minimally
  intrusive.
  
  Joerg
 
 i care the API.
 
 if you really want it be in libc, how about having libc provide a
 locale_t get_static_c_locale(); style API rather than using NULL?
 it's better because 1) less code in *_l, 2) autoconf-like can detect
 the extension easily, and 3) a portable application can trivially
 have a fallback implementation using newlocale+pthread_once.

It adds more cost on the caller side. So far, there are three mechanisms
available (especially for libraries):

(1) Adhoc access to internal locale state. This is used with glibc.
(2) Explicit newlocale().
(3) Implicit access via 0 argument to *_l.

The first one is clearly a hack and not acceptable. Portable code can
always conditionally use (2), but it requires additional setup and
storage cost. (3) is used by Apple (which is where a large part of the
*_l interface originates from) and FreeBSD. It is orthogonal to (2) and
certainly easier to use. Exposing it via a special library call is also
possible and effectively a way to implement (2) by a static wrapper.
It still adds more cost to every caller and this is a classic case where
there are typically much more callers.

Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-16 Thread Joerg Sonnenberger
On Tue, Apr 16, 2013 at 04:55:01AM +, YAMAMOTO Takashi wrote:
 hi,
 
  On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote:
  POSIX2008 spec say, *_l func with invalid locale handle may EINVAL.
  NULL or (locale_t)0 is invalid locale handle.
  why are you think fallback to C locale?
  
  That's what Apple and FreeBSD provide and which is actually quite
  useful. Wanting access to the C locale is the most common case for many
  locale issues. Making that easy sounds like a good idea in general.
 
 is there consumers of this extension in the field?

Yes.

 an application can have a global c_locale = newlocale(C)
 if it wants an easy access to the C locale.

...which means keeping a copy around in various libraries etc.
It is possible, but wastes memory.

Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-16 Thread Joerg Sonnenberger
On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
 before implementing such non-portable *broken* stuff.

At the very least it is a conforming extension.

  ...which means keeping a copy around in various libraries etc.
  It is possible, but wastes memory.
 
 most of application in the world doesn't require multi-locale stuff.
 so that the struct _locale __C_locale in libc is much more wasteful.

Most applications can/should enable the user locales. That means that as
soon as they also have to deal with accessing data from non-internalized
sources, they have to deal with both the user and the C locale. The very
same reasoning applies to libraries.

Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-16 Thread Joerg Sonnenberger
On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
  That's what Apple and FreeBSD provide and which is actually quite
  useful.
 
 useful? don't say that.
 
 I have heard same word from who spreads broken iconv(3) usage,
 (such as non-portable //TRANSLIT features that annoying pkgsrc peoples).
 http://mail-index.netbsd.org/tech-userlevel/2006/04/03/.html

//TRANSLIT is quite a different deal because it is a workaround for the
bad default behavior of GNU iconv...

Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-16 Thread YAMAMOTO Takashi
 On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
 so that the struct _locale __C_locale in libc is much more wasteful.
 
 I should add that it is an internal detail and the way the composed C
 locale is stored can and likely will change later. So the way it is
 essentially a copy of (old) global locale is just a way to be minimally
 intrusive.
 
 Joerg

i care the API.

if you really want it be in libc, how about having libc provide a
locale_t get_static_c_locale(); style API rather than using NULL?
it's better because 1) less code in *_l, 2) autoconf-like can detect
the extension easily, and 3) a portable application can trivially
have a fallback implementation using newlocale+pthread_once.

YAMAMOTO Takashi


Re: CVS commit: src/lib/libc/locale

2013-04-15 Thread Takehiko NOZAKI
I can' t understand why this change is needed.

POSIX2008 spec say, *_l func with invalid locale handle may EINVAL.
NULL or (locale_t)0 is invalid locale handle.
why are you think fallback to C locale?

http://pubs.opengroup.org/onlinepubs/9699919799/functions/isalpha.html
http://pubs.opengroup.org/onlinepubs/9699919799/functions/nl_langinfo.html
http://pubs.opengroup.org/onlinepubs/9699919799/functions/newlocale.html

glibc2(*_l function's origin) does more simpler, they don't check NULL
and cause Segmentation Fault.
I think glibc2 is reasonable, because POSIX2008  does change for the worse.
as far as is*_l functions, return value is non-zero(=matches class) or
zero(=not matches class) only.
we can't know the error from it.


2013/4/15 Joerg Sonnenberger jo...@netbsd.org:
 Module Name:src
 Committed By:   joerg
 Date:   Sun Apr 14 23:44:54 UTC 2013

 Modified Files:
 src/lib/libc/locale: Makefile.inc setlocale_local.h
 Added Files:
 src/lib/libc/locale: c_locale.c

 Log Message:
 Provide a const copy of global_locale for libc-internal use.
 This will be used by *_l when a NULL pointer is given.


 To generate a diff of this commit:
 cvs rdiff -u -r1.60 -r1.61 src/lib/libc/locale/Makefile.inc
 cvs rdiff -u -r0 -r1.1 src/lib/libc/locale/c_locale.c
 cvs rdiff -u -r1.10 -r1.11 src/lib/libc/locale/setlocale_local.h

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.


--
Takehiko NOZAKItakehiko.noz...@gmail.com


Re: CVS commit: src/lib/libc/locale

2013-04-15 Thread Joerg Sonnenberger
On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote:
 POSIX2008 spec say, *_l func with invalid locale handle may EINVAL.
 NULL or (locale_t)0 is invalid locale handle.
 why are you think fallback to C locale?

That's what Apple and FreeBSD provide and which is actually quite
useful. Wanting access to the C locale is the most common case for many
locale issues. Making that easy sounds like a good idea in general.

Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-15 Thread YAMAMOTO Takashi
hi,

 On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote:
 POSIX2008 spec say, *_l func with invalid locale handle may EINVAL.
 NULL or (locale_t)0 is invalid locale handle.
 why are you think fallback to C locale?
 
 That's what Apple and FreeBSD provide and which is actually quite
 useful. Wanting access to the C locale is the most common case for many
 locale issues. Making that easy sounds like a good idea in general.

is there consumers of this extension in the field?

an application can have a global c_locale = newlocale(C)
if it wants an easy access to the C locale.

YAMAMOTO Takashi

 
 Joerg


Re: CVS commit: src/lib/libc/locale

2010-06-06 Thread Izumi Tsutsui
 I was just wondering why this change was made? It seems to be a backwards
 step to me?

If we need exact-width interger we should use uintNN_t
but I don't think exact width is required in locale.

As other guys say we can't assume uint8_t == unsigned char implicitly:
http://cvsweb.NetBSD.org/bsdweb.cgi/src/sys/arch/pdp10/include/Attic/int_types.h?rev=1.4
(though pdp10 has been removed)
---
Izumi Tsutsui


Re: CVS commit: src/lib/libc/locale

2010-06-06 Thread Alan Barrett
On Sun, 06 Jun 2010, Izumi Tsutsui wrote:
 If we need exact-width interger we should use uintNN_t
 but I don't think exact width is required in locale.

The [u]int_leastN_t types may be useful in this sort of code.  For
example, [u]int_least32_t is required to exist, and on a 36-bit machine
it would presumably be a 36-bit type; whereas [u]int32_t would not exist
on a 36-bit machine (because C99 requires the [u]intN_t types to have
exact sizes with no padding bits).

--apb (Alan Barrett)


Re: CVS commit: src/lib/libc/locale

2010-06-06 Thread Takehiko NOZAKI
hi, all

 I was just wondering why this change was made? It seems to be a backwards
 step to me?

my intension is quite paranoia.

changing uint8_t - unsigned char / int16_t - short is because
_ctype_ / _tolower_tab_ / _toupper_tab_ in ctype.h (now splitted
sys/ctype_bits.h)
declared as unsigned char / short.

ctype.h(rev1.29) -- http://tinyurl.com/365ycfq

extern const unsigned char  *_ctype_;
extern const short  *_tolower_tab_;
extern const short  *_toupper_tab_;


but ctypeio.c read them from LC_CTYPE as uint8_t / int16_t and network endian
and directly cast to unsigned char/short.

ctypeio.c(rev1.11) -- http://tinyurl.com/22my4t8

if (fread((void *)ptr, sizeof(uint8_t), len, fp) != len)
...
if (fread((void *)ptr, sizeof(int16_t), len, fp) != len)
...
if (fread((void *)ptr, sizeof(int16_t), len, fp) != len)
...
#if BYTE_ORDER == LITTLE_ENDIAN
for (i = 1; i = len; i++) {
new_toupper[i] = ntohs(new_toupper[i]);
new_tolower[i] = ntohs(new_tolower[i]);
}
#endif
...
data-ctype_tab = (const unsigned char *)new_ctype;
data-toupper_tab = (const short *)new_toupper;
data-tolower_tab = (const short *)new_tolower;

if the case, someone port to the machine that sizeof(unsigned char) !=
sizeof(uint8_t)
or sizeof(short) != sizeof(int16_t), is*/to* function doesn't work correctlly.
# such as NetBSD/pdp10(9bit char), or cray1(64bit char)


 If you ever port to a 36-bit machine, you are going to have to lie and
 define int32_t as 'int' anyway

intN_t is not portable type, these are optimal:

ISO/IEC9899:1999 7.18.1.1

3 These types are optional. However, if an implementation provides
integer types with widths of 8, 16, 32, or 64 bits, no padding bits,
and (for the signed types) that have a two’s complement representation,
it shall define the corresponding typedef names.

NetBSD/pdp10 seems that providing them by using gcc extension
__attribute__((size N)).


 The [u]int_leastN_t types may be useful in this sort of code.

exactly, but it require C99 stdint.h(or sys/types.h).
ctype.h should keep C90 namespace, i'm not willing to include it.


P.S.

i'm now attempting increase _ctype_ bits 8 - 16 before branching netbsd-6.
because these ancient code(delived from 4BSD) have two bugs:

1. _B bit desn't means isblank(3) but isprint(3)  !isgraph(3).
   so we have to provide real isblank bit, but there is no bit space 
left.

2. isprint(3) doesn't recognize some japanese half-width character
   and vietnamese's phonogram character, we have to provide
phonogram bit too.

i'll post patch to tech-userlevel@, this weekend or next week.


very truly yours.
--
Takehiko NOZAKI tnoz...@netbsd.org


Re: CVS commit: src/lib/libc/locale

2010-06-05 Thread Alistair Crooks
Hi Nozaki-san,

On Wed, Jun 02, 2010 at 04:04:52PM +, Takehiko NOZAKI wrote:
 Module Name:  src
 Committed By: tnozaki
 Date: Wed Jun  2 16:04:52 UTC 2010
 
 Modified Files:
   src/lib/libc/locale: bsdctype.c
 
 Log Message:
 uint8_t - unsigned char, int16_t - short.

As ever, thank you for looking after our locale code.

I was just wondering why this change was made? It seems to be a backwards
step to me?

Thanks,
Alistair


Re: CVS commit: src/lib/libc/locale

2010-06-05 Thread David Holland
On Sat, Jun 05, 2010 at 11:53:32PM +0200, Alistair Crooks wrote:
   Module Name:   src
   Committed By:  tnozaki
   Date:  Wed Jun  2 16:04:52 UTC 2010
   
   Modified Files:
  src/lib/libc/locale: bsdctype.c
   
   Log Message:
   uint8_t - unsigned char, int16_t - short.
  
  As ever, thank you for looking after our locale code.
  
  I was just wondering why this change was made? It seems to be a backwards
  step to me?

I looked at it and it appears to me that it's a change to use
non-sized types (that are guaranteed to be large enough) in the
in-memory structures. If we ever do a port to a 36-bit machine or
whatever it's probably desirable.

-- 
David A. Holland
dholl...@netbsd.org


Re: CVS commit: src/lib/libc/locale

2010-06-05 Thread M. Warner Losh
In message: 20100606041254.gb19...@netbsd.org
David Holland dholland-sourcechan...@netbsd.org writes:
: On Sat, Jun 05, 2010 at 11:53:32PM +0200, Alistair Crooks wrote:
:Module Name: src
:Committed By:tnozaki
:Date:Wed Jun  2 16:04:52 UTC 2010
:
:Modified Files:
: src/lib/libc/locale: bsdctype.c
:
:Log Message:
:uint8_t - unsigned char, int16_t - short.
:   
:   As ever, thank you for looking after our locale code.
:   
:   I was just wondering why this change was made? It seems to be a backwards
:   step to me?
: 
: I looked at it and it appears to me that it's a change to use
: non-sized types (that are guaranteed to be large enough) in the
: in-memory structures. If we ever do a port to a 36-bit machine or
: whatever it's probably desirable.

If you ever port to a 36-bit machine, you are going to have to lie and
define int32_t as 'int' anyway

Warner


Re: CVS commit: src/lib/libc/locale

2010-06-05 Thread David Holland
On Sat, Jun 05, 2010 at 11:28:31PM -0600, M. Warner Losh wrote:
  : I looked at it and it appears to me that it's a change to use
  : non-sized types (that are guaranteed to be large enough) in the
  : in-memory structures. If we ever do a port to a 36-bit machine or
  : whatever it's probably desirable.
  
  If you ever port to a 36-bit machine, you are going to have to lie and
  define int32_t as 'int' anyway

Well, maybe... it would be sort of interesting to try, actually.

-- 
David A. Holland
dholl...@netbsd.org