Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 1:56 PM, Andrey Chernov wrote:
> On 15.11.2015 15:46, Baptiste Daroussin wrote:
>> On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote:
>>> On 15.11.2015 10:09, John Marino wrote:
>>> ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
>>> with that). Lots of ports (even at configure stage!) have checks for
>>> them. Since we generate locales from CLDR now, it will be no cost to
>>> bring all 8859-1 back to not violate POLA and not fix every failing port.
>>>
>> Exp-run have been made and no ports were failing with the removed locales.
> 
> There is soft-fail, configure just marks that locales are not supported
> and use "C". Sorry I don't remember port names where I saw it right now
> and don't have a time to search for them right now too. Soft-fails (like
> in tcl with nl_langinfo) are almost impossible to detect excepting
> specific situation happens or source code inspection. Do we ever need
> them when there is no harm to keep 8859-1 locales?
> 

Well, there is "harm".  The -1/-15 confusion happens a lot.

Is the plan to keep every locale ever created for ever and ever?  Never
do any kind of kind of cleanup or reorg?

This is not really any kind of problem.  Update the regression test and
nobody will even notice after that in all liklihood.

John
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[sthaug]

> > There is soft-fail, configure just marks that locales are not supported
> > and use "C". Sorry I don't remember port names where I saw it right now
> > and don't have a time to search for them right now too. Soft-fails (like
> > in tcl with nl_langinfo) are almost impossible to detect excepting
> > specific situation happens or source code inspection. Do we ever need
> > them when there is no harm to keep 8859-1 locales?
> 
> Is it ok if I readd those locales as aliases on 8859-15?

Why on earth would you want to do that when we can have the real thing?

I still see no good reason for dropping 8859-1.

Steinar Haug, Nethelp consulting, sth...@nethelp.no
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 16:14, John Marino wrote:
>> It is user confusion and his responsibility. It not leads to wrong
>> program build f.e. Moreover, you can't protect users who set 8859-1 that
>> way, they do not convert to 8859-15 as you assume but start to complain
>> everywhere that FreeBSD is not working instead.
> 
> Invalid.  "locale -a" shows what locales are available.
> The confusion is not with one user.  It's with one user that produces
> document in one encoding and a second user that choses the wrong one
> (usually -1).  -15 was designed to replace -1.

No end-user use 'locale -a' to check locales. An end-user keep some
8859-1 version is set many years ago, and "FreeBSD not working" problem
I tell about is not different text document encoding, but program
failure due to inability to set his 8859-1 locale.

In any case it is related to the user behavior an various views on it.
They can be different and I see no point to insist on my particular view
at all. But... Programs configure soft-fails shows real danger here.

> OpenBSD removed ISO8859* completely.

OpenBSD was never good in locales in any case for all that years and
contributes nothing in that area (f.e. Citrus was made by NetBSD). Now
they try to simplify their life of supporting them, but since for us now
all locales are autogenerated from CLDR I see no room for simplification
in that way. Our "cleaned" locales (against to POLA) can be restored by
autogeneration with minimal efforts, and they even took very little disk
space.

> Also invalid.  Locales are not standardized with regard to encoding, so
> maintaining a museum of locales is specific to FreeBSD.  Linux calls
> them differently.

Most of our (old) locales have the same names as linux ones.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 2:10 PM, Andrey Chernov wrote:
>> Well, there is "harm".  The -1/-15 confusion happens a lot.
> 
> It is user confusion and his responsibility. It not leads to wrong
> program build f.e. Moreover, you can't protect users who set 8859-1 that
> way, they do not convert to 8859-15 as you assume but start to complain
> everywhere that FreeBSD is not working instead.

Invalid.  "locale -a" shows what locales are available.
The confusion is not with one user.  It's with one user that produces
document in one encoding and a second user that choses the wrong one
(usually -1).  -15 was designed to replace -1.

OpenBSD removed ISO8859* completely.


> 
>> Is the plan to keep every locale ever created for ever and ever?  Never
>> do any kind of kind of cleanup or reorg?
> 
> It will be nice to do it that way. FreeBSD have a little part of world
> locales, which indirectly assumes that they are really used.

Also invalid.  Locales are not standardized with regard to encoding, so
maintaining a museum of locales is specific to FreeBSD.  Linux calls
them differently.

John
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 16:54, John Marino wrote:
> We are talking about people that install FreeBSD 11 as a release.  If
> it's an upgrade, it's documented in UPDATING (it will be) and anybody on
> -CURRENT is taking responsibility for knowing what they are doing.  *IF*
> this is an obstacle, it's either trivial or that user shouldn't be using
> -CURRENT to begin with.

As I already say, I don't want to insist on any particular point of view
in such area as human behavior. I just say that it is POLA violation
(even while it is upgrade) and we can let users decide by themselves,
what it best for them (without me at least).

>> I tell about is not different text document encoding, but program
>> failure due to inability to set his 8859-1 locale.
> 
> Please provide an example of such a program (in ports).

See gettext-0.19.6/gettext-tools/configure, part started with
# Test for the usual locale name.
I don't have time to dig more such code now, but I remember I saw more
for sure.

>> In any case it is related to the user behavior an various views on it.
>> They can be different and I see no point to insist on my particular view
>> at all. But... Programs configure soft-fails shows real danger here.
> 
> and the impact of this is ... ?

Various. It depends on for what reason program sense locale.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 3:39 PM, Andrey Chernov wrote:
> As I already say, I don't want to insist on any particular point of view
> in such area as human behavior. I just say that it is POLA violation
> (even while it is upgrade) and we can let users decide by themselves,
> what it best for them (without me at least).

I am starting to think "POLA" as an acronym is subject to abuse.  By
this definition, *any* change would "astonish" a user (picturing the
most incompetent user impossible too).

POLA is meant for unreasonable and unexplained changes.  I don't think
tidying up locales for the first time in a decade is unreasonable or
unexplained.

Let's not dilute "POLA".  It's pretty good but you can apply it to anything.

>>> I tell about is not different text document encoding, but program
>>> failure due to inability to set his 8859-1 locale.
>>
>> Please provide an example of such a program (in ports).
> 
> See gettext-0.19.6/gettext-tools/configure, part started with
> # Test for the usual locale name.
> I don't have time to dig more such code now, but I remember I saw more
> for sure.
> 
>>> In any case it is related to the user behavior an various views on it.
>>> They can be different and I see no point to insist on my particular view
>>> at all. But... Programs configure soft-fails shows real danger here.
>>
>> and the impact of this is ... ?
> 
> Various. It depends on for what reason program sense locale.

If the ports framework is not overriding locale to "C" during the build
then that's probably something that should be introduced.  Ports should
not be producing different results based on the configuration of the
building machine at the time.

Right?  It's also good practice because C/POSIX never results in illegal
sequence errors while locales such as UTF-8 easily can.

John
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 18:38, Andrey Chernov wrote:
>> For the traditional checks, It's ironic, DragonFly uses short locales
>> like "fr_FR" which link to the approprioprate ISO8859 or UTF-8 locale.
>> Bapt removed them to avoid a bike shed and if he had not done that, this
>> gettext configure would not be an issue as "fr_FR" is the very first check.
>
> But I dislike short locales due to their uncertainty

And gettext-0.19.6/gettext-tools/configure is good example of that
uncertainty. If DragonFly short locale fr_FR links to 8859-1, configure
test does what it is intended, but if it links to UTF-8 it does not.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 2:04 PM, Andrey Chernov wrote:
> On 15.11.2015 16:00, Baptiste Daroussin wrote:
>> On Sun, Nov 15, 2015 at 03:56:26PM +0300, Andrey Chernov wrote:
>>> On 15.11.2015 15:46, Baptiste Daroussin wrote:
 On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote:
> On 15.11.2015 10:09, John Marino wrote:
> ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
> with that). Lots of ports (even at configure stage!) have checks for
> them. Since we generate locales from CLDR now, it will be no cost to
> bring all 8859-1 back to not violate POLA and not fix every failing port.
>
 Exp-run have been made and no ports were failing with the removed locales.
>>>
>>> There is soft-fail, configure just marks that locales are not supported
>>> and use "C". Sorry I don't remember port names where I saw it right now
>>> and don't have a time to search for them right now too. Soft-fails (like
>>> in tcl with nl_langinfo) are almost impossible to detect excepting
>>> specific situation happens or source code inspection. Do we ever need
>>> them when there is no harm to keep 8859-1 locales?
>>
>> Is it ok if I readd those locales as aliases on 8859-15?
> 
> It is hacking solution leads to wrong collating order and character
> classes. It is better to generate true 8859-1 just in the same way you
> already do for 8859-15.
> 
> BTW, I can't check right now, but in case 8859-5 is removed too, it is
> better to restore it, it was used in Suns as their standard Russian
> encoding.
> 

DragonFly:

muscles# locale -a | grep 8859-5
uk_UA.ISO8859-5
be_BY.ISO8859-5
ru_RU.ISO8859-5
sr_Cyrl_RS.ISO8859-5

I agree that if -1 is brought back, it needs to be changed at the
charset.xml level.  At that point FreeBSD and DragonFly will diverge.  I
also agree using ports as justification for keeping -1 in western europe
is invalid (as in, it's not causing problems in ports)


John
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 17:47, John Marino wrote:
>>> Please provide an example of such a program (in ports).
>>
>> See gettext-0.19.6/gettext-tools/configure, part started with
>> # Test for the usual locale name.
>> I don't have time to dig more such code now, but I remember I saw more
>> for sure.

> If the ports framework is not overriding locale to "C" during the build
> then that's probably something that should be introduced.  Ports should
> not be producing different results based on the configuration of the
> building machine at the time.

This is not about resetting locale to "C" during the build. Please
really look at "configure" specific part mentioned above, it is you who
ask at least one specific example and then it is you who ignore the
answer takes my time to find it.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 18:01, Baptiste Daroussin wrote:
> I have restored the 8859-1

Thanx!
BTW, speaking with John I find at least one configure script from ports
which depends on 8859-1 presence.
See gettext-0.19.6/gettext-tools/configure, part started with
# Test for the usual locale name.

-- 
http://ache.vniz.net/



signature.asc
Description: OpenPGP digital signature

Re: Need help fixing failing locale tests

[Pedro Giffuni]

FWIW;

While I personally don’t use it, Latin is the official language for the
Holy See [1]. I think it is still taught in schools in Italy for cultural
reasons and because it’s supposed to make easier to learn other
“romance” languages.

It shouldn't hurt to keep it but I have no strong opinion.

Pedro.

[1] https://en.wikipedia.org/wiki/Holy_See
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Miroslav Lachman]

John Marino wrote on 11/15/2015 15:47:

On 11/15/2015 3:39 PM, Andrey Chernov wrote:

As I already say, I don't want to insist on any particular point of view
in such area as human behavior. I just say that it is POLA violation
(even while it is upgrade) and we can let users decide by themselves,
what it best for them (without me at least).


I am starting to think "POLA" as an acronym is subject to abuse.  By
this definition, *any* change would "astonish" a user (picturing the
most incompetent user impossible too).

POLA is meant for unreasonable and unexplained changes.  I don't think
tidying up locales for the first time in a decade is unreasonable or
unexplained.

Let's not dilute "POLA".  It's pretty good but you can apply it to anything.


I agree. Everytime FreeBSD is changing something in the base, there are 
voices with "POLA". Removing Perl from base, removing BIND from base... 
these were more significant changes than changing something in locales.


Miroslav Lachman

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Michael Mitchell]

Documentation on any existing regressions and how they are expected to
fail would be a good thing… Change is necessary, but so is modification of
a regression to continue to pass (or fail) as expected. Demonstration of the
unit testing process can also be very beneficial.

: mdm

> On Nov 15, 2015, at 8:10 AM, Miroslav Lachman <000.f...@quip.cz> wrote:
> 
>>> can



smime.p7s
Description: S/MIME cryptographic signature

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 2:36 PM, Andrey Chernov wrote:
> On 15.11.2015 16:14, John Marino wrote:
>>> It is user confusion and his responsibility. It not leads to wrong
>>> program build f.e. Moreover, you can't protect users who set 8859-1 that
>>> way, they do not convert to 8859-15 as you assume but start to complain
>>> everywhere that FreeBSD is not working instead.
>>
>> Invalid.  "locale -a" shows what locales are available.
>> The confusion is not with one user.  It's with one user that produces
>> document in one encoding and a second user that choses the wrong one
>> (usually -1).  -15 was designed to replace -1.
> 
> No end-user use 'locale -a' to check locales. 

To quote you, that means it "user confusion and responsibility".
Fine.  He can "ls -d /usr/share/locales".  Otherwise, how does said user
set locales for the FIRST time.

We are talking about people that install FreeBSD 11 as a release.  If
it's an upgrade, it's documented in UPDATING (it will be) and anybody on
-CURRENT is taking responsibility for knowing what they are doing.  *IF*
this is an obstacle, it's either trivial or that user shouldn't be using
-CURRENT to begin with.


> An end-user keep some
> 8859-1 version is set many years ago, and "FreeBSD not working" problem
> I tell about is not different text document encoding, but program
> failure due to inability to set his 8859-1 locale.

Please provide an example of such a program (in ports).


> In any case it is related to the user behavior an various views on it.
> They can be different and I see no point to insist on my particular view
> at all. But... Programs configure soft-fails shows real danger here.

and the impact of this is ... ?


> 
>> OpenBSD removed ISO8859* completely.
> 
> OpenBSD was never good in locales in any case for all that years and
> contributes nothing in that area (f.e. Citrus was made by NetBSD). Now
> they try to simplify their life of supporting them, but since for us now
> all locales are autogenerated from CLDR I see no room for simplification
> in that way. Our "cleaned" locales (against to POLA) can be restored by
> autogeneration with minimal efforts, and they even took very little disk
> space.

It's not a technical issue.  -1 (and some -15) weren't removed for
technical issues, but to keep order.

>> Also invalid.  Locales are not standardized with regard to encoding, so
>> maintaining a museum of locales is specific to FreeBSD.  Linux calls
>> them differently.
> 
> Most of our (old) locales have the same names as linux ones.

Actually, none of the ISO* encodings.  Linux will accept "ISO8859-1"
because it's hyphen-insensitive, it it calls it "ISO-8859-1"

Maybe obsolete ones like cp866, GB2312 ... but there are more different
than the same.

John
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Baptiste Daroussin]

On Sun, Nov 15, 2015 at 04:04:33PM +0300, Andrey Chernov wrote:
> On 15.11.2015 16:00, Baptiste Daroussin wrote:
> > On Sun, Nov 15, 2015 at 03:56:26PM +0300, Andrey Chernov wrote:
> >> On 15.11.2015 15:46, Baptiste Daroussin wrote:
> >>> On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote:
>  On 15.11.2015 10:09, John Marino wrote:
>  ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
>  with that). Lots of ports (even at configure stage!) have checks for
>  them. Since we generate locales from CLDR now, it will be no cost to
>  bring all 8859-1 back to not violate POLA and not fix every failing port.
> 
> >>> Exp-run have been made and no ports were failing with the removed locales.
> >>
> >> There is soft-fail, configure just marks that locales are not supported
> >> and use "C". Sorry I don't remember port names where I saw it right now
> >> and don't have a time to search for them right now too. Soft-fails (like
> >> in tcl with nl_langinfo) are almost impossible to detect excepting
> >> specific situation happens or source code inspection. Do we ever need
> >> them when there is no harm to keep 8859-1 locales?
> > 
> > Is it ok if I readd those locales as aliases on 8859-15?
> 
> It is hacking solution leads to wrong collating order and character
> classes. It is better to generate true 8859-1 just in the same way you
> already do for 8859-15.
> 
> BTW, I can't check right now, but in case 8859-5 is removed too, it is
> better to restore it, it was used in Suns as their standard Russian
> encoding.
> 
I have restored the 8859-1

The 8859-5 were never removed:
be_BY.ISO8859-5
ru_RU.ISO8859-5
sr_Cyrl_RS.ISO8859-5
uk_UA.ISO8859-5

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 18:17, John Marino wrote:
 See gettext-0.19.6/gettext-tools/configure, part started with
 # Test for the usual locale name.
 I don't have time to dig more such code now, but I remember I saw more
 for sure.

> Of course it's the same topic.
> If the configure of a port is affected by the locale, that's a big
> problem.  How is this not the same topic?

This is environment problem, not presence or absence of some locales in
the system (f.e. 8859-1 ones) we talked about. And can be easily fixed
in the environment, unlike configure's silent script fallbacks, which
can be fixed by recompiling only.

About environment problem, we already have LANG=C somewhere in the
/usr/ports/Mk/* (probably not in every place) and many ports set LANG=C
or LC_ALL by themselves.

> But I did look at gettext, which is configuring package based on what's
> on the system (not environment).  For the unicode checks, there is
> obviously no issue.
> 
> For the traditional checks, It's ironic, DragonFly uses short locales
> like "fr_FR" which link to the approprioprate ISO8859 or UTF-8 locale.
> Bapt removed them to avoid a bike shed and if he had not done that, this
> gettext configure would not be an issue as "fr_FR" is the very first check.

All parts are optional excepting language itself, so fr_FR is not short
enough, but fr is. From POSIX:

language[_territory][.codeset][@modifier]

But I dislike short locales due to their uncertainty and remember some
programs and shell scripts that attempts to parse locale name directly
by itself for reasons I don't remember now.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[NGie Cooper]

> On Nov 15, 2015, at 03:30, John Marino  wrote:

…

> by they way, this was meant to be "removed from DragonFly".  I don't
> know if "make upgrade" is set right for FreeBSD.  (it appears not)

make upgrade doesn’t exist on FreeBSD:

$ (cd /usr/src/svn/; make upgrade)
make: don't know how to make upgrade. Stop

make: stopped in /usr/src/svn
$

> It looks like the deleted locales were not removed.
> They should be.

Hence my previous reply…

> I think somebody needs to add these locales to be reaped by make upgrade.

ObsoleteFiles.inc is missing old locale files/directories.

Cheers,
-NGie
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[sthaug]

> >> There's also a common
> >> problem that users view -15 documents with -1 accidently.  So there was
> >> a conscience decision to have either ISO8859-1 or ISO8859-15 but not
> >> both.  For western Europe this means the ISO8859-1 versions were dropped.
> 
> It is pure user problem choosing its own locale (self footshooting), it
> not leads to program build failures (like configure checks) or tests
> failures etc. BTW, linux keeps 8859-1 locales.

I see no good reason whatsoever to drop ISO8859-1.

Steinar Haug, Nethelp consulting, sth...@nethelp.no
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

libXO-ification - Why - and is it a symptom of deeper issues?

[Dan Partelly]

Hi all,

I was looking at the new facility of dumping JSON,XML from many utils in base 
and after some funny minutes, I couldn't stop ask myself “ Ok, this is funny , 
but why ? “ And I couldn't find a real answer. Ill outline what I think:


1. Undoubtedly, it makes base code slightly harder to understand and maintain. 
2. I have seen the idea that this makes the information dumped by utilities in 
the base easily accessible programatically. OK, maybe it does , but
it doesn't fit with the current paradigm of "tool | filter | tool” at all. 
There are no tools able to accept JSON and filter it in any meaningful way, and 
I
dont see too many ppl changing their code to read JSON instead of text.  I 
don't even see the base tools changing. This output may be useful in corner 
cases only.
3. The integration of libxo IMO only points at a much deeper issue IMO. It is 
only an expression of the need of a mechanism aimed at binary code reuse. But 
it does not solve the problem, it only adds yet another possibility in a world 
where too much choices already result in too much splits and incompatible APIs. 
4. This whole effort would have been IMO much better served  by porting the 
bulk of ifconfig(8) , route(8) and wpaclient(8) to a library API, much like the 
libs for geom, zfs , etc , ready for reuse of 3rd party code. Eventually 
writing network control daemons in time over it , much like solaris does.

5. A port of partial OS config data to UCL …. would induce yet induce another 
orthogonality violation. What makes UCL better than the bestiary of ad hoc 
databases already existing in BSDs ? Programatic readability, yes. but it does 
not add any real much needed functionality such as *transactional databases* 
for system tools.   Why not research a proper solution - easily accessible by 
other programs ,orthogonal , transactional, and ACL protected   solution  which 
can be used all over the place , from OS boot, to ABI management, service 
management, network management, user management.  I hope this day will come, a 
day when I will not have to edit a single config file manually, yet I would 
have access to all the config and system state  easy with wrapper APIs. In the 
light of this point, why go with UCL ? It is not orthogonal, it is not 
transnational, and editing the config files directly would result in the same 
old human errors which bite as all from time to time.

5. It is my opinion that Solaris addressed some of those issue. Solaris FMRI 
and SMF are lightyears ahead of the very tired models we keep using on BSDs. 
Why not build on the insight offered by those (or even on the insight offered 
by Windows :P) , then inventing more adhoc solutions and ad-hoc databases, 
which do not address the real issues we have , like binary code reuse, service 
management issues,  lack of a system wide published -subscriber bus ( not kdbus 
:P ) fault detection and reaction, fault reporting, all much needed parts of a 
modern OS. 

And now thee questions

1. Why lib XO ? Why burden the OS for some corner cases where it may be useful ?

2. Was there any real talk on how to bring FreeBSD up to speed regarding those 
issues ?  A period of research on what exists, on what can be done , and ensure 
important things are not showed in background and replaced with yet another 
ad-hoc config database which lacks modern features ?
From where I am standing, this could be a project spawning multiple years , but 
it would be well worth it, and in my opinion it would be also worthy of 
the freeSBD foundation sponsorship for several years in a row. The features I 
touched upon became very important parts of oder OSes, and rightly so. 

Note:

this message is serious and it is not intended to start flame wars, religious 
crusades, or offend anyone. 
 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 4:46 AM, NGie Cooper wrote:
> 
>> On Nov 14, 2015, at 19:28, NGie Cooper  wrote:
> 
> …
> 
>> Why were these locales removed?
>>
>>  58 OLD_FILES+=usr/share/locale/la_LN.ISO8859-1/LC_COLLATE
>>  59 OLD_FILES+=usr/share/locale/la_LN.ISO8859-1/LC_CTYPE
>>  60 OLD_FILES+=usr/share/locale/la_LN.ISO8859-1/LC_TIME
>>  61 OLD_DIRS+=usr/share/locale/la_LN.ISO8859-1
>>  62 OLD_FILES+=usr/share/locale/la_LN.ISO8859-13/LC_COLLATE
>>  63 OLD_FILES+=usr/share/locale/la_LN.ISO8859-13/LC_CTYPE
>>  64 OLD_DIRS+=usr/share/locale/la_LN.ISO8859-13
> 
> la_LN.ISO8859-1 is the old Latin locale, which is no longer installed. 
> Copying over locale files from a stable/10 host works, but I’m confused as to 
> why a bunch of locales weren’t ported over in their non-UTF-8 forms.
> Thanks,
> 

We (DragonFly) didn't just update locales.  We took the opportunity to
do spring cleaning.  We didn't want to be as drastic as OpenBSD which
removed all encodings except for C/POSIX and UTF, but we did remove
several locales intentionally.

In the case of ISO8859-1:
All ISO8859-* is basically obsolete.
In western Europe, if somebody wants ISO-8859, they want ISO8859-15, not
ISO8859-1.  They are similar, but the former is tailored for western
europe with "Euro" currency and 9 other symbols.  It comes at the
expense of removing 10 characters from ISO8859-1.  There's also a common
problem that users view -15 documents with -1 accidently.  So there was
a conscience decision to have either ISO8859-1 or ISO8859-15 but not
both.  For western Europe this means the ISO8859-1 versions were dropped.

ISO8859-15:
In the case of USA and other non-European countries, they keep -1 and
dropped -15.

currency based:
In the case of countries where the currency symbols is not part of
ISO8859, it was dropped.  E.g. Costa Rica uses the Colon which is only
in UTF-8, so there's no ISO8859-* encoding at all for CR.

Latin:
Who speaks Latin today?
This was mainly an alias for 7-bit ascii.  We originally dropped that,
but later moved it to US-ASCII (which was just a symlink to Latin before)

Bapt liked DF approach well enough that he adopted it.  Even Edwin was
first in desiring to clean up locales.  The major update was a perfect time.

Bottom line:
The testsuite needs to be updated.
e.g. use de_DE.ISO8859-15 intead of de_DE.ISO8859-1
For latin, replace with US-ASCII equivalent.


John
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Craig Rodrigues]

On Sat, Nov 14, 2015 at 11:47 PM, John Marino 
wrote:

>
> With 4.) they have been removed from FreeBSD when "Make upgrade" is run
> after rebuilding.  Does FreeBSD have a cleanup like that, and did you
> run it?  If not, then maybe that needs to be updated.
>
>
I did not run "make upgrade".
I did this:
cd /usr/src
svn up

I then ran the 11 steps listed in /usr/src/Makefile:

https://github.com/freebsd/freebsd/blob/master/Makefile#L73

It looks like after running those steps, my system is in an inconsistent
state.  I see that the following symlinks point to nonexistent files.

/usr/share/locale/nl_BE.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/nl_BE.ISO8859-1/LC_COLLATE ->
../la_LN.ISO8859-1/LC_COLLATE
/usr/share/locale/en_CA.ISO8859-15/LC_CTYPE -> ../la_LN.ISO8859-15/LC_CTYPE
/usr/share/locale/en_CA.ISO8859-15/LC_COLLATE ->
../la_LN.ISO8859-15/LC_COLLATE
/usr/share/locale/en_US.ISO8859-15/LC_COLLATE ->
../la_LN.ISO8859-15/LC_COLLATE
/usr/share/locale/en_US.ISO8859-15/LC_CTYPE -> ../la_LN.ISO8859-15/LC_CTYPE
/usr/share/locale/ca_FR.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/nb_NO.ISO8859-1/LC_MONETARY ->
../no_NO.ISO8859-1/LC_MONETARY
/usr/share/locale/nb_NO.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/nb_NO.ISO8859-1/LC_COLLATE ->
../no_NO.ISO8859-1/LC_COLLATE
/usr/share/locale/nb_NO.ISO8859-1/LC_MESSAGES ->
../no_NO.ISO8859-1/LC_MESSAGES
/usr/share/locale/nb_NO.ISO8859-1/LC_NUMERIC ->
../no_NO.ISO8859-1/LC_NUMERIC
/usr/share/locale/sr_YU.ISO8859-2/LC_COLLATE ->
../la_LN.ISO8859-2/LC_COLLATE
/usr/share/locale/sr_YU.ISO8859-2/LC_CTYPE -> ../la_LN.ISO8859-2/LC_CTYPE
/usr/share/locale/is_IS.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/zh_TW.Big5/zh_Hant_TW.Big5 -> zh_Hant_TW.Big5
/usr/share/locale/af_ZA.ISO8859-15/LC_CTYPE -> ../la_LN.ISO8859-15/LC_CTYPE
/usr/share/locale/af_ZA.ISO8859-15/LC_COLLATE ->
../la_LN.ISO8859-15/LC_COLLATE
/usr/share/locale/en_AU.ISO8859-15/LC_COLLATE ->
../la_LN.ISO8859-15/LC_COLLATE
/usr/share/locale/en_AU.ISO8859-15/LC_CTYPE -> ../la_LN.ISO8859-15/LC_CTYPE
/usr/share/locale/fr_CH.ISO8859-1/LC_COLLATE ->
../la_LN.ISO8859-1/LC_COLLATE
/usr/share/locale/fr_CH.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/zh_CN.UTF-8/LC_COLLATE -> ../la_LN.US-ASCII/LC_COLLATE
/usr/share/locale/zh_CN.UTF-8/zh_Hans_CN.UTF-8 -> zh_Hans_CN.UTF-8
/usr/share/locale/zh_CN.UTF-8/LC_CTYPE -> ../UTF-8/LC_CTYPE
/usr/share/locale/sv_SE.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/it_IT.ISO8859-1/LC_COLLATE ->
../la_LN.ISO8859-1/LC_COLLATE
/usr/share/locale/it_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/it_CH.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/it_CH.ISO8859-1/LC_COLLATE ->
../la_LN.ISO8859-1/LC_COLLATE
/usr/share/locale/nn_NO.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/nn_NO.ISO8859-1/LC_COLLATE ->
../no_NO.ISO8859-1/LC_COLLATE
/usr/share/locale/nn_NO.ISO8859-1/LC_MONETARY ->
../no_NO.ISO8859-1/LC_MONETARY
/usr/share/locale/nn_NO.ISO8859-1/LC_NUMERIC ->
../no_NO.ISO8859-1/LC_NUMERIC
/usr/share/locale/nn_NO.ISO8859-1/LC_MESSAGES ->
../no_NO.ISO8859-1/LC_MESSAGES
/usr/share/locale/nl_NL.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/nl_NL.ISO8859-1/LC_COLLATE ->
../la_LN.ISO8859-1/LC_COLLATE
/usr/share/locale/zh_TW.UTF-8/LC_COLLATE -> ../la_LN.US-ASCII/LC_COLLATE
/usr/share/locale/zh_TW.UTF-8/zh_Hant_TW.UTF-8 -> zh_Hant_TW.UTF-8
/usr/share/locale/zh_TW.UTF-8/LC_CTYPE -> ../UTF-8/LC_CTYPE
/usr/share/locale/de_AT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/en_GB.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/en_GB.ISO8859-1/LC_COLLATE ->
../la_LN.ISO8859-1/LC_COLLATE
/usr/share/locale/ca_ES.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/zh_CN.GBK/zh_Hans_CN.GBK -> zh_Hans_CN.GBK
/usr/share/locale/zh_CN.GBK/LC_COLLATE -> ../la_LN.US-ASCII/LC_COLLATE
/usr/share/locale/de_DE.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/fi_FI.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
/usr/share/locale/fi_FI.ISO8859-1/LC_COLLATE ->
../la_LN.ISO8859-1/LC_COLLATE
/usr/share/locale/zh_CN.eucCN/zh_Hans_CN.eucCN -> zh_Hans_CN.eucCN
/usr/share/locale/zh_CN.eucCN/LC_COLLATE -> ../la_LN.US-ASCII/LC_COLLATE
/usr/share/locale/zh_HK.UTF-8/LC_COLLATE -> ../la_LN.US-ASCII/LC_COLLATE
/usr/share/locale/zh_HK.UTF-8/LC_CTYPE -> ../UTF-8/LC_CTYPE
/usr/share/locale/sr_YU.UTF-8/LC_COLLATE -> ../la_LN.US-ASCII/LC_COLLATE
/usr/share/locale/sr_YU.UTF-8/LC_CTYPE -> ../UTF-8/LC_CTYPE
/usr/share/locale/zh_CN.GB18030/zh_Hans_CN.GB18030 -> zh_Hans_CN.GB18030
/usr/share/locale/zh_CN.GB18030/LC_COLLATE -> ../la_LN.US-ASCII/LC_COLLATE
/usr/share/locale/fr_CA.ISO8859-15/LC_CTYPE -> ../la_LN.ISO8859-15/LC_CTYPE

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 8:24 AM, NGie Cooper wrote:
>> On Nov 14, 2015, at 23:09, John Marino  wrote:
>> Bapt liked DF approach well enough that he adopted it.  Even Edwin was
>> first in desiring to clean up locales.  The major update was a perfect time.
>>
>> Bottom line:
>> The testsuite needs to be updated.
>> e.g. use de_DE.ISO8859-15 intead of de_DE.ISO8859-1
>> For latin, replace with US-ASCII equivalent.
> 
>   I wish this had been clearly communicated here: 
> https://svnweb.freebsd.org/changeset/base/290494 … instead, there are some 
> POLA violations:
>   1. No RelNotes.
>   2. No summary of locales that have been removed.
>   3. No UPDATING entry for people to migrate from a locale to another.
>   4. Deprecated locales (as you described it above) are still present on 
> my system after running `make delete-old` (assuming I have my acronyms right, 
> for example, ca_ES, it_CH, etc should be -1, not -15 based on your claim 
> above).
>   I will update the testcases to use locales if possible, but more work 
> needs to be done finishing off this project and documenting for end-users 
> what has changed.

With 4.) they have been removed from FreeBSD when "Make upgrade" is run
after rebuilding.  Does FreeBSD have a cleanup like that, and did you
run it?  If not, then maybe that needs to be updated.

with 3.) That's a bit like saying changing the color of shoe laces needs
instructions for people to learn how to adjust.  Run "locale -a" to see
what's available on the system.  This has always been true.  Attempting
to switch to a locale that doesn't exist results in staying in "C"
locale, which is quite obvious.

with 2.) Could be combined with 1.) but in any case, there really
shouldn't be POLA on anything that has no standard.  Should you announce
every time you add a locale too, in release notes?
Really "locale -a" should tell the whole story.

1.) A paragraph on removed locales is fine.  Those removed have been
removed because they are a) likely unused and b) if used, they are used
by mistake.  in this case you hit c) where the testsuite probably chose
the "wrong" locale in the first place.  Note that GCC did this too with
their libstdc++ testsuite and that was fixed this weekend (a huge patch
to GCC 6 upstream).

John
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 12:23 PM, Craig Rodrigues wrote:
> 
> 
> On Sat, Nov 14, 2015 at 11:47 PM, John Marino  > wrote:
> 
> 
> With 4.) they have been removed from FreeBSD when "Make upgrade" is run
> after rebuilding.  Does FreeBSD have a cleanup like that, and did you
> run it?  If not, then maybe that needs to be updated.

by they way, this was meant to be "removed from DragonFly".  I don't
know if "make upgrade" is set right for FreeBSD.  (it appears not)

> 
> I did not run "make upgrade".
> I did this:
> cd /usr/src
> svn up
> 
> I then ran the 11 steps listed in /usr/src/Makefile:
> 
> https://github.com/freebsd/freebsd/blob/master/Makefile#L73
> 
> It looks like after running those steps, my system is in an inconsistent
> state.  I see that the following symlinks point to nonexistent files.

[snip]

It looks like the deleted locales were not removed.
They should be.


> What is the fix for this?

I think somebody needs to add these locales to be reaped by make upgrade.


> It looks like there is some Makefile logic which is wrong.


John


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 15:24, Andrey Chernov wrote:
> On 15.11.2015 10:09, John Marino wrote:
>> We (DragonFly) didn't just update locales.  We took the opportunity to
>> do spring cleaning.  We didn't want to be as drastic as OpenBSD which
>> removed all encodings except for C/POSIX and UTF, but we did remove
>> several locales intentionally.
>>
>> In the case of ISO8859-1:
>> All ISO8859-* is basically obsolete.
>> In western Europe, if somebody wants ISO-8859, they want ISO8859-15, not
>> ISO8859-1.  They are similar, but the former is tailored for western
>> europe with "Euro" currency and 9 other symbols.  It comes at the
>> expense of removing 10 characters from ISO8859-1. 

Forget to reply on that part:

>> There's also a common
>> problem that users view -15 documents with -1 accidently.  So there was
>> a conscience decision to have either ISO8859-1 or ISO8859-15 but not
>> both.  For western Europe this means the ISO8859-1 versions were dropped.

It is pure user problem choosing its own locale (self footshooting), it
not leads to program build failures (like configure checks) or tests
failures etc. BTW, linux keeps 8859-1 locales.

> ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
> with that). Lots of ports (even at configure stage!) have checks for
> them. Since we generate locales from CLDR now, it will be no cost to
> bring all 8859-1 back to not violate POLA and not fix every failing port.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 15:46, Baptiste Daroussin wrote:
> On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote:
>> On 15.11.2015 10:09, John Marino wrote:
>> ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
>> with that). Lots of ports (even at configure stage!) have checks for
>> them. Since we generate locales from CLDR now, it will be no cost to
>> bring all 8859-1 back to not violate POLA and not fix every failing port.
>>
> Exp-run have been made and no ports were failing with the removed locales.

There is soft-fail, configure just marks that locales are not supported
and use "C". Sorry I don't remember port names where I saw it right now
and don't have a time to search for them right now too. Soft-fails (like
in tcl with nl_langinfo) are almost impossible to detect excepting
specific situation happens or source code inspection. Do we ever need
them when there is no harm to keep 8859-1 locales?

-- 
http://ache.vniz.net/



signature.asc
Description: OpenPGP digital signature

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 16:00, Baptiste Daroussin wrote:
> On Sun, Nov 15, 2015 at 03:56:26PM +0300, Andrey Chernov wrote:
>> On 15.11.2015 15:46, Baptiste Daroussin wrote:
>>> On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote:
 On 15.11.2015 10:09, John Marino wrote:
 ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
 with that). Lots of ports (even at configure stage!) have checks for
 them. Since we generate locales from CLDR now, it will be no cost to
 bring all 8859-1 back to not violate POLA and not fix every failing port.

>>> Exp-run have been made and no ports were failing with the removed locales.
>>
>> There is soft-fail, configure just marks that locales are not supported
>> and use "C". Sorry I don't remember port names where I saw it right now
>> and don't have a time to search for them right now too. Soft-fails (like
>> in tcl with nl_langinfo) are almost impossible to detect excepting
>> specific situation happens or source code inspection. Do we ever need
>> them when there is no harm to keep 8859-1 locales?
> 
> Is it ok if I readd those locales as aliases on 8859-15?

It is hacking solution leads to wrong collating order and character
classes. It is better to generate true 8859-1 just in the same way you
already do for 8859-15.

BTW, I can't check right now, but in case 8859-5 is removed too, it is
better to restore it, it was used in Suns as their standard Russian
encoding.

-- 
http://ache.vniz.net/



signature.asc
Description: OpenPGP digital signature

Re: Need help fixing failing locale tests

[Andrey Chernov]

On 15.11.2015 10:09, John Marino wrote:
> We (DragonFly) didn't just update locales.  We took the opportunity to
> do spring cleaning.  We didn't want to be as drastic as OpenBSD which
> removed all encodings except for C/POSIX and UTF, but we did remove
> several locales intentionally.
> 
> In the case of ISO8859-1:
> All ISO8859-* is basically obsolete.
> In western Europe, if somebody wants ISO-8859, they want ISO8859-15, not
> ISO8859-1.  They are similar, but the former is tailored for western
> europe with "Euro" currency and 9 other symbols.  It comes at the
> expense of removing 10 characters from ISO8859-1.  There's also a common
> problem that users view -15 documents with -1 accidently.  So there was
> a conscience decision to have either ISO8859-1 or ISO8859-15 but not
> both.  For western Europe this means the ISO8859-1 versions were dropped.

ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
with that). Lots of ports (even at configure stage!) have checks for
them. Since we generate locales from CLDR now, it will be no cost to
bring all 8859-1 back to not violate POLA and not fix every failing port.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Baptiste Daroussin]

On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote:
> On 15.11.2015 10:09, John Marino wrote:
> > We (DragonFly) didn't just update locales.  We took the opportunity to
> > do spring cleaning.  We didn't want to be as drastic as OpenBSD which
> > removed all encodings except for C/POSIX and UTF, but we did remove
> > several locales intentionally.
> > 
> > In the case of ISO8859-1:
> > All ISO8859-* is basically obsolete.
> > In western Europe, if somebody wants ISO-8859, they want ISO8859-15, not
> > ISO8859-1.  They are similar, but the former is tailored for western
> > europe with "Euro" currency and 9 other symbols.  It comes at the
> > expense of removing 10 characters from ISO8859-1.  There's also a common
> > problem that users view -15 documents with -1 accidently.  So there was
> > a conscience decision to have either ISO8859-1 or ISO8859-15 but not
> > both.  For western Europe this means the ISO8859-1 versions were dropped.
> 
> ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
> with that). Lots of ports (even at configure stage!) have checks for
> them. Since we generate locales from CLDR now, it will be no cost to
> bring all 8859-1 back to not violate POLA and not fix every failing port.
> 
Exp-run have been made and no ports were failing with the removed locales.

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: Need help fixing failing locale tests

[Baptiste Daroussin]

On Sun, Nov 15, 2015 at 03:56:26PM +0300, Andrey Chernov wrote:
> On 15.11.2015 15:46, Baptiste Daroussin wrote:
> > On Sun, Nov 15, 2015 at 03:24:19PM +0300, Andrey Chernov wrote:
> >> On 15.11.2015 10:09, John Marino wrote:
> >> ISO8859-1 locales are legacy even if obsoleted in modern world (I agree
> >> with that). Lots of ports (even at configure stage!) have checks for
> >> them. Since we generate locales from CLDR now, it will be no cost to
> >> bring all 8859-1 back to not violate POLA and not fix every failing port.
> >>
> > Exp-run have been made and no ports were failing with the removed locales.
> 
> There is soft-fail, configure just marks that locales are not supported
> and use "C". Sorry I don't remember port names where I saw it right now
> and don't have a time to search for them right now too. Soft-fails (like
> in tcl with nl_langinfo) are almost impossible to detect excepting
> specific situation happens or source code inspection. Do we ever need
> them when there is no harm to keep 8859-1 locales?

Is it ok if I readd those locales as aliases on 8859-15?

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: Need help fixing failing locale tests

[Andrey Chernov]

> Well, there is "harm".  The -1/-15 confusion happens a lot.

It is user confusion and his responsibility. It not leads to wrong
program build f.e. Moreover, you can't protect users who set 8859-1 that
way, they do not convert to 8859-15 as you assume but start to complain
everywhere that FreeBSD is not working instead.

> Is the plan to keep every locale ever created for ever and ever?  Never
> do any kind of kind of cleanup or reorg?

It will be nice to do it that way. FreeBSD have a little part of world
locales, which indirectly assumes that they are really used.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Yonas Yanfa]

Hi Dan,

Sounds like you'd be interested in NextBSD:

https://en.wikipedia.org/wiki/NextBSD

Cheers,
Yonas


On 11/15/2015 07:54, Dan Partelly wrote:

Hi all,

I was looking at the new facility of dumping JSON,XML from many utils in base 
and after some funny minutes, I couldn't stop ask myself “ Ok, this is funny , 
but why ? “ And I couldn't find a real answer. Ill outline what I think:


1. Undoubtedly, it makes base code slightly harder to understand and maintain.
2. I have seen the idea that this makes the information dumped by utilities in 
the base easily accessible programatically. OK, maybe it does , but
it doesn't fit with the current paradigm of "tool | filter | tool” at all. 
There are no tools able to accept JSON and filter it in any meaningful way, and I
dont see too many ppl changing their code to read JSON instead of text.  I 
don't even see the base tools changing. This output may be useful in corner 
cases only.
3. The integration of libxo IMO only points at a much deeper issue IMO. It is 
only an expression of the need of a mechanism aimed at binary code reuse. But 
it does not solve the problem, it only adds yet another possibility in a world 
where too much choices already result in too much splits and incompatible APIs.
4. This whole effort would have been IMO much better served  by porting the 
bulk of ifconfig(8) , route(8) and wpaclient(8) to a library API, much like the 
libs for geom, zfs , etc , ready for reuse of 3rd party code. Eventually 
writing network control daemons in time over it , much like solaris does.

5. A port of partial OS config data to UCL …. would induce yet induce another 
orthogonality violation. What makes UCL better than the bestiary of ad hoc 
databases already existing in BSDs ? Programatic readability, yes. but it does 
not add any real much needed functionality such as *transactional databases* 
for system tools.   Why not research a proper solution - easily accessible by 
other programs ,orthogonal , transactional, and ACL protected   solution  which 
can be used all over the place , from OS boot, to ABI management, service 
management, network management, user management.  I hope this day will come, a 
day when I will not have to edit a single config file manually, yet I would 
have access to all the config and system state  easy with wrapper APIs. In the 
light of this point, why go with UCL ? It is not orthogonal, it is not 
transnational, and editing the config files directly would result in the same 
old human errors which bite as all from time to time.

5. It is my opinion that Solaris addressed some of those issue. Solaris FMRI 
and SMF are lightyears ahead of the very tired models we keep using on BSDs. 
Why not build on the insight offered by those (or even on the insight offered 
by Windows :P) , then inventing more adhoc solutions and ad-hoc databases, 
which do not address the real issues we have , like binary code reuse, service 
management issues,  lack of a system wide published -subscriber bus ( not kdbus 
:P ) fault detection and reaction, fault reporting, all much needed parts of a 
modern OS.

And now thee questions

1. Why lib XO ? Why burden the OS for some corner cases where it may be useful ?

2. Was there any real talk on how to bring FreeBSD up to speed regarding those 
issues ?  A period of research on what exists, on what can be done , and ensure 
important things are not showed in background and replaced with yet another 
ad-hoc config database which lacks modern features ?
 From where I am standing, this could be a project spawning multiple years , 
but it would be well worth it, and in my opinion it would be also worthy of
the freeSBD foundation sponsorship for several years in a row. The features I 
touched upon became very important parts of oder OSes, and rightly so.

Note:

this message is serious and it is not intended to start flame wars, religious 
crusades, or offend anyone.
  
___

freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


--

Yonas Yanfa
In Love With Open Source
Drupal  :: GitHub 
 :: Mozilla 
 :: iPhone 


fizk.net | yo...@fizk.net

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

FreeBSD_HEAD - Build #3530 - Failure

[jenkins-admin]

FreeBSD_HEAD - Build #3530 - Failure:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD/3530/
Full change log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD/3530/changes
Full build log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD/3530/console

Change summaries:

290857 by trasz:
Speed up rctl operation with large rulesets, by holding the lock
during iteration instead of relocking it for each traversed rule.

Reviewed by:mjg@
MFC after:  1 month
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D4110

290856 by bapt:
also skip the definition of ':fopen_regular' to avoid the build to fail due to
unused variables defined by ATF macros

290855 by mav:
Increase reset assertion time from 10 to 100us.

On my own tests I see no effect from this change, but I also can't
reproduce the reported problem in general.

PR: 127391
PR: 204554
Submitted by:   s...@iranger.com
MFC after:  2 weeks

290851 by ngie:
Change WARNS to 2 across the board with all the libc testcases

This effectively "reverts" r290846

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

290850 by ngie:
Cast xdr_void to xdrproc_t to mute -Wincompatible-pointer-types warnings from
clang

This pattern is used in other areas of lib/libc/rpc

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

290849 by ngie:
Fix -Wmissing-braces warnings by adding braces around all the
testcase inputs

MFC after: 1 week
X-MFC with: r290572
Sponsored by: EMC / Isilon Storage Division

290848 by ngie:
Fix -Wunused warnings

MFC after: 1 week
X-MFC with: r290572
Sponsored by: EMC / Isilon Storage Division

290847 by ngie:
Fix -Wunused warnings with variables used unlit code by adding appropriate 
#ifdef
guards around the variables

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

290846 by ngie:
Bump WARNS to 2

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

290845 by ngie:
Remove unused variables; sort by alignment where needed

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

290844 by ngie:
Polish up iswctype_test

- Split up the testcases into C locale and ja_JP.eucJP testcases.
- Avoid a segfault in the event that setlocale fails, similar to r290843
- Replace `sizeof(x) / sizeof(*x)` pattern with `nitems(x)`

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

290843 by ngie:
Polish up the tests a bit more after projects/collation was merged to head

Provide more meaningful diagnostic messages if LC_CTYPE can't be set properly
instead of segfaulting, because setlocale returns NULL and strcmp(NULL, b) will
always segfault

Split up the testcases so one failing (in this case en_US.ISO8859-15) won't
cause the rest of the testcases to be skipped

Remove some unused variables

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

290842 by ngie:
Fix the Indian numbering system (hi_IN.ISCII-DEV) tests

Submitted by: ache
X-MFC with: r290494 (if that ever happens)
Sponsored by: EMC / Isilon Storage Division

290840 by ngie:
Setup the symlink to /sys to mirror one's current source, e.g. if my source
tree was /usr/src/svn, /sys would point to usr/src/svn

This fixes the assumption that the source tree will always exist at
${DESTDIR}/usr/src

MFC after: 1 week
PR: 76362
Reported by: Scot Hetzel 
Sponsored by: EMC / Isilon Storage Division



The end of the build log:

[...truncated 137526 lines...]
--- protoent_test ---
echo '#! /usr/libexec/atf-sh' > protoent_test.tmp
cat /builds/FreeBSD_HEAD/contrib/netbsd-tests/lib/libc/net/t_protoent.sh 
>>protoent_test.tmp
chmod +x protoent_test.tmp
mv protoent_test.tmp protoent_test
--- servent_test ---
echo '#! /usr/libexec/atf-sh' > servent_test.tmp
cat /builds/FreeBSD_HEAD/contrib/netbsd-tests/lib/libc/net/t_servent.sh 
>>servent_test.tmp
chmod +x servent_test.tmp
mv servent_test.tmp servent_test
--- Kyuafile.auto ---
===> lib/libc/tests/regex (all)
--- h_regex ---
(cd /builds/FreeBSD_HEAD/lib/libc/tests/regex &&  DEPENDFILE=.depend.h_regex  
NO_SUBDIR=1 make -f /builds/FreeBSD_HEAD/lib/libc/tests/regex/Makefile 
_RECURSING_PROGS=  PROG=h_regex )
--- main.o ---
cc  -O2 -pipe   -I/builds/FreeBSD_HEAD/contrib/netbsd-tests/lib/libc/regex 
-I/builds/FreeBSD_HEAD/lib/libc/regex -MD -MP -MF.depend.h_regex.main.o 
-MTmain.o -std=gnu99 -fstack-protector-strong -Wsystem-headers -Werror -Wall 
-Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-empty-body 
-Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare 
-Wno-unused-value -Wno-parentheses-equality -Wno-unused-function 
-Wno-enum-conversion -Wno-unused-local-typedef -Wno-switch -Wno-switch-enum 
-Wno-knr-promoted-parameter -Qunused-arguments -c 
/builds/FreeBSD_HEAD/contrib/netbsd-tests/lib/libc/regex/main.c -o main.o
--- all_subdir_gnu ---
--- frame-base.o ---
cc   -O2 -pipe   

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[David Chisnall]

On 15 Nov 2015, at 19:12, Simon J. Gerraty  wrote:
> 
> Garrett Cooper  wrote:
>> We lack a [dtd/json] spec for tools, so programming for xo'ification
>> doesn't seems like the best idea in the world to me from a end-user
>> sysadmin/developer perspective.
> 
> A dtd etc is good for sure, and we (Juniper) do have them, as well as
> ui-police to help ensure things go smoothly - even that doesn't catch
> everything.

At the GSoC Mentor Summit, someone from GitHub pointed me at a tool[1] for 
generating schemas from JSON (which can then be refined manually) and can be 
used for validating that the schema is still respected in newer versions.  I 
hope to integrate something like this into our regression test suite before 11.

For the people complaining that there are no command-line tools for filtering 
JSON in shell scripts, I’d suggest that they look at JQ[2].  I’m not sure if 
it’s worth bringing it into base, but being able to install it easily 
(textproc/jq, no dependencies) from ports suddenly makes a load of stuff 
trivial with shell scripts that were either incredibly hard or impossible prior 
to libxo.

David

[1] https://github.com/perenecabuto/json_schema_generator
[2] https://stedolan.github.io/jq/

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Dan Partelly]

> I would welcome competing ideas/solutions, but someone would have to actually 
> build them, not just

> rattle off some ideas on the mailing list.

Am I missing the point of a mailing list ?  it is a place to present and 
exchange ideas, ask why some things are the way they are , and get criticism. 
Rattling ideas is generally where it all starts. You cant realistically expect 
one to start a pervasive project like transactioanl databases for an OS 
configuration without not one mailing list discussion, but a lot of 
consultation. 


> There are tools available to filter json/ucl output
yes they are. vim is one . sed is another. awk is the third …  But a pipe needs 
both ends to talk the same language. I can generate json as output, i can 
filter it  , but what tool in FreeBSD will accept it as input ? None.  A

> and my forthcoming uclcmd

What uclcmd does / will do ?  

> UCL is a good solution to having a universal config file, and is better
> than the bespoke config files each utility has


I agree with you, if you manage somehow to port every ad-hoc database FreeBSD 
has in base to ucl, (including all the bestiary we have in /etc ) and offer 
tools to parse them in the rc init system to fetch the settings. I dont expect 
this to be done in one release , but do you tend towards this in your projects 
? One config format to rule them all is good. Yet another config format in 
selected places in base is evil.

>  A transactional database
> might be better (for some uses, likely less so for some people), but I
> don't hear anyone volunteering to do that work.

IIRC Solaris uses sqlite. Might be a decent solution. ( a lot of the ad hoc 
unix databases are relational databases in disguise, with unix tools acting as 
operators)  And if you abstract a config interface API over UCL, someone could 
benefit  from it by plugging in a transactional backend one day. All you would 
have to do is not directly plug in UCL, but work on a orthogonal abstract API. 
You could even  model it after UCL API.
Please consider it. 


> I would welcome competing ideas/solutions, but someone would have to actually 
> build them, not just

> 
> lib-izing all of the utilities instead of using libxo is a better
> solution. It would likely be gratefully accepted if someone were to do
> it, but most likely, no one will.
> 
> libxo exists now, and most applications can be converted very quickly
> (although care does need to be taken, and it sometimes has not been).
> 
> There are tools available to filter json/ucl output, namely textproc/jq
> and my forthcoming uclcmd
> 
> One of the major other consumers of the json/xml output of libxo, is web
> control panels. This is why Juniper is doing the work, and I can think
> of a list of other appliance vendors who would love to replace fragile
> per-application parsers with a json parser to extract data from various
> command line tools.
> 
> UCL is a good solution to having a universal config file, and is better
> than the bespoke config files each utility has. A transactional database
> might be better (for some uses, likely less so for some people), but I
> don't hear anyone volunteering to do that work.
> 
> So, libxo and libucl may not be the very best solutions, but they are
> the ones that are moving forward. I would welcome competing
> ideas/solutions, but someone would have to actually build them, not just
> rattle off some ideas on the mailing list.
> 
> -- 
> Allan Jude
> 

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Build failed in Jenkins: Build-UFS-image #2744

[jenkins-admin]

See 

--
[...truncated 3362 lines...]

 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 

===> lib/libc/tests (install)
install -o root  -g wheel -m 444  Kyuafile.auto  

===> lib/libc/tests/tls_dso (install)
install -C -o root -g wheel -m 444   libh_tls_dynamic.a 

install -C -o root -g wheel -m 444   libh_tls_dynamic_p.a 

install -s -o root -g wheel -m 444 libh_tls_dynamic.so.1 

install -l s libh_tls_dynamic.so.1 

===> lib/libc/tests/c063 (install)
install -o root  -g wheel -m 444  Kyuafile.auto  

(cd /builds/FreeBSD_HEAD/lib/libc/tests/c063 &&  
DEPENDFILE=.depend.faccessat_test  NO_SUBDIR=1 make -f 
/builds/FreeBSD_HEAD/lib/libc/tests/c063/Makefile _RECURSING_PROGS=  
PROG=faccessat_test  install)
install -s -o root -g wheel 

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Dan Partelly]

HI Simon,


Thanks for the write-up . One question:


>> The ability to get machine parsable output from OS components is a big part 
>> of the success of Junos CLI, netconf etc.

Once you get machine parsable  output, and feed it to your GUIs , WEB, other 
tools, and modify it, how do you feed it back
to your underlying OS ? 


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Dan Partelly]

> It's all fine and good making technical decisions based on drawings
> and handwaving and philosophizing, but at some point someone has to do
> the code.

HI Adrian,

. What I eluded too is not a small project. It is something that would need 
proper discussion and agreement, since it would be pervasive and touch 
critical parts of the OS, such as the init system, system config databases , 
and add proper services management facility. It would also benefit 
from a new form of  kernel IPC. It would need consensus from FreeBSD board or 
whatever to have any chance of even starting up. Nobody in his 
sane mind would start it otherwise. Most likely he would work in vain,
 
And when consensus that something HAS to be done will exist, and from empty 
discussion you would have a implementation plan, when maybe the FreeBSD 
foundation would get involved and sponsor such a important project to see it 
done to the end.


And there are efforts today to go down the path I mentioned, NextBSD is the 
incarnation of such an effort. And while they offer code and they do make 
progress I do not seeing anyone in FreeBSD beeing too eager to commit that code 
:P (Im not saying that you should adapt launchd and add another comapt layer 
for FreeBSD for mach ). I for one like what Solaris does. What  Im saying that 
such work would never be possible directly in FreeBSD, because lack of 
consensus that anything serious should be done, apart from patching on sides.

 I am saying that gathering consensus that something has to be done must exist 
before any code is written . Else you wont get much.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Adrian Chadd]

On 15 November 2015 at 09:10, Dan Partelly  wrote:
> Meaning, is that simple to push things in head , if somone does the work, 
> even with with no proper review of the problem at hand , and the proposed 
> solutions ?

Nope and yup. The juniper folk had a solution to a problem multiple
people had requested work on, and their proposal was by far the
furthest along code and use wise.

It's all fine and good making technical decisions based on drawings
and handwaving and philosophizing, but at some point someone has to do
the code. Juniper's libxo was the furthest along in implementation and
production.


-a
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Garrett Cooper]

> On Nov 15, 2015, at 09:51, Andrey Chernov  wrote:
> 
>> On 15.11.2015 20:37, Adrian Chadd wrote:
>>> On 15 November 2015 at 09:10, Dan Partelly  wrote:
>>> Meaning, is that simple to push things in head , if somone does the work, 
>>> even with with no proper review of the problem at hand , and the proposed 
>>> solutions ?
>> 
>> Nope and yup. The juniper folk had a solution to a problem multiple
>> people had requested work on, and their proposal was by far the
>> furthest along code and use wise.
>> 
>> It's all fine and good making technical decisions based on drawings
>> and handwaving and philosophizing, but at some point someone has to do
>> the code. Juniper's libxo was the furthest along in implementation and
>> production.
> 
> It seems it is the only and final argument for libXO existence. I
> remember 2 or 3 discussions against libXO spontaneously happens in the
> FreeBSD lists, all ended with that, approximately: "we already have the
> code and you have just speculations". Alternative and more architecture
> clean ideas, like making standalone template-oriented parser probably
> based on liXO, are never seriously considered, because nobody will code
> it, not for other reasons.

We lack a [dtd/json] spec for tools, so programming for xo'ification doesn't 
seems like the best idea in the world to me from a end-user sysadmin/developer 
perspective.

I could just as easily use standard tools like awk, grep, sed, and more 
advanced languages like perl or Python to parse output, and assuming output 
doesn't get a major rewrite, I'd just go with that method that's worked pretty 
well for me over the last 10 years of my career..

Cheers,
-NGie
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Simon J. Gerraty]

Hi Dan

> Meaning, is that simple to push things in head , if somone does the
> work, even with with no proper review of the problem at hand , and the
> proposed solutions ?

Not sure what sort of review you are looking for.
But I can speak to some of the history behind this.

FreeBSD holds regular "vendor summit" meetings where folk using FreeBSD
in products can get together - sort of a swap meet (I have this, I need
this ...).

At one such meeting at BSDCan 2011, I metioned that we (Juniper) had
tweaked many of the startard tools to output XML, and asked if anyone
else was interested in the facility.
I was frankly surprised at the number of hands raised.
There was clearly demand from a segment of the FreeBSD community.

The ability to get machine parsable output from OS components is a big
part of the success of Junos CLI, netconf etc.

It took a few years to get some time from phil@ to design a solution
that we could consider upstreaming - libxo was the result.
BTW this is a "problem space" that phil has been deeply involved in for
over 15 years, so yes I think we can say the problem has been studied.

That demand I mentioned? also resulted in a GSoC project to do the same
thing - though it was much like approach that Juniper had done over a
decade ago that we had considered unsuitable for upstreaming.
But it rather clearly demonstrates that there was demand beyond the whim
of a small group of folk or just one company pushing  something unwanted
into the project.
The number of developers who have jumped in to XO'ify apps also speaks
to that.

The original proposal was for XML only (that's what we'd used and found
useful), but others wanted JSON as well.
Libxo can also output very rich HTML - I forget which HTML or JSON, is
used but this allows some seriously slick UI's to be implemented using a
modern web browser.

Hope that helps
--sjg

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[Simon J. Gerraty]

Craig Rodrigues  wrote:
> I don't know much about locales, so don't know what to do.

I find LANG=C LC_ALL=C solves most locale induced issues.
I suspect the tests in question assume the above anyway.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Garrett Cooper]

> On Nov 15, 2015, at 10:05, Allan Jude  wrote:
> 
>> On 2015-11-15 07:54, Dan Partelly wrote:
>> 
>> Hi all,
>> 
>> I was looking at the new facility of dumping JSON,XML from many utils in 
>> base and after some funny minutes, I couldn't stop ask myself “ Ok, this is 
>> funny , but why ? “ And I couldn't find a real answer. Ill outline what I 
>> think:
>> 
>> 
>> 1. Undoubtedly, it makes base code slightly harder to understand and 
>> maintain.
> 
> I am not sure that libxo actually makes the code any harder to
> understand and maintain. It might actually make it slightly better.
> 
> replacing:
> 
> printf("%s %s %d\n", foo, bar, number);
> 
> with:
> 
> xo_emit("{:foo/%s} {:bar/%s} {:number/%d}", foo, bar, number);
> 
> it not really hurting much.

That's by and large true, but there are other APIs that need to be called on 
exit (xo_finish?) and in other scenarios where flushing, etc is needed. If you 
don't do that, you don't get the output you expect (which broke uptime/w 
several months ago..).

Also, typos with the meta language into the xo_emit calls have bit is more than 
once ;(.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Simon J. Gerraty]

Garrett Cooper  wrote:
> We lack a [dtd/json] spec for tools, so programming for xo'ification
> doesn't seems like the best idea in the world to me from a end-user
> sysadmin/developer perspective.

A dtd etc is good for sure, and we (Juniper) do have them, as well as
ui-police to help ensure things go smoothly - even that doesn't catch
everything.
But all that is a layer above something like libxo - which is just the
mechanism.
All that is fundamentally required to obtain a reasonable result is
consistent use of verbs and nouns.

> I could just as easily use standard tools like awk, grep, sed, and
> more advanced languages like perl or Python to parse output, and

If you are parsing plain text output - that is all the data you have
and it is a small subset of what the app knows.

XML and HTML allow the app to provide lots of ancilliary/meta data
that is invaluable in doing clever things with the data.
Eg. by simply marking something as hostname/ipaddr the ui can hook in
pulldown menus to let you do things with that info.
I don't know if we've released anything I can point you at easily
but 5 minutes would suffice to make a believer of you ;-)

BTW libifying apps is a nice thing too, but does not eliminate the need
to do all the exact same UI work - just changes where you do it, since
the libraries themselves would need to be XO'ified to be useful.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Build failed in Jenkins: Build-UFS-image #2746

[jenkins-admin]

See 

--
[...truncated 3362 lines...]

 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 

===> lib/libc/tests (install)
install -o root  -g wheel -m 444  Kyuafile.auto  

===> lib/libc/tests/tls_dso (install)
install -C -o root -g wheel -m 444   libh_tls_dynamic.a 

install -C -o root -g wheel -m 444   libh_tls_dynamic_p.a 

install -s -o root -g wheel -m 444 libh_tls_dynamic.so.1 

install -l s libh_tls_dynamic.so.1 

===> lib/libc/tests/c063 (install)
install -o root  -g wheel -m 444  Kyuafile.auto  

(cd /builds/FreeBSD_HEAD/lib/libc/tests/c063 &&  
DEPENDFILE=.depend.faccessat_test  NO_SUBDIR=1 make -f 
/builds/FreeBSD_HEAD/lib/libc/tests/c063/Makefile _RECURSING_PROGS=  
PROG=faccessat_test  install)
install -s -o root -g wheel 

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Don Lewis]

On 15 Nov, Allan Jude wrote:
> On 2015-11-15 13:06, Garrett Cooper wrote:
>> 
>>> On Nov 15, 2015, at 09:51, Andrey Chernov  wrote:
>>>
 On 15.11.2015 20:37, Adrian Chadd wrote:
> On 15 November 2015 at 09:10, Dan Partelly  wrote:
> Meaning, is that simple to push things in head , if somone does the work, 
> even with with no proper review of the problem at hand , and the proposed 
> solutions ?

 Nope and yup. The juniper folk had a solution to a problem multiple
 people had requested work on, and their proposal was by far the
 furthest along code and use wise.

 It's all fine and good making technical decisions based on drawings
 and handwaving and philosophizing, but at some point someone has to do
 the code. Juniper's libxo was the furthest along in implementation and
 production.
>>>
>>> It seems it is the only and final argument for libXO existence. I
>>> remember 2 or 3 discussions against libXO spontaneously happens in the
>>> FreeBSD lists, all ended with that, approximately: "we already have the
>>> code and you have just speculations". Alternative and more architecture
>>> clean ideas, like making standalone template-oriented parser probably
>>> based on liXO, are never seriously considered, because nobody will code
>>> it, not for other reasons.
>> 
>> We lack a [dtd/json] spec for tools, so programming for xo'ification doesn't 
>> seems like the best idea in the world to me from a end-user 
>> sysadmin/developer perspective.
>> 
>> I could just as easily use standard tools like awk, grep, sed, and more 
>> advanced languages like perl or Python to parse output, and assuming output 
>> doesn't get a major rewrite, I'd just go with that method that's worked 
>> pretty well for me over the last 10 years of my career..
>> 
>> Cheers,
>> -NGie
>> ___
>> freebsd-current@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>> 
> 
> The big difference is, a json parser isn't going to blow up if a new
> field gets added in the middle, and your awk/grep/sed script probably will.

Or more likely, if some value gets large causing adjacent columns run
together.  Or even if you the utility is used to display something where
some of the data contains whitespace.

How is an awk script going to cope with this:

# grep spacey /etc/passwd
spacey user:*:::Update Builder:/tmp/spacey:/bin/csh
# su 'spacey user'
% cd ~
% touch 'a file'
% ls -l
total 1
-rw-r--r--  1 spacey user    0 Nov 15 15:24 a file

vs a json parser:

% ls --libxo json -l
{"__version": "1", "file-information": {"directory": [{"total-blocks":1, 
"entry": [{"name":"a 
file","mode":"-rw-r--r--","mode_octal":644,"links":1,"user":"spacey 
user","group":"","type":"regular","size":0,"modify-time":1447629885}]}]}
}

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Wake on LAN broken (probably between r290542 - r290606)?

[Marius Strobl]

On Sat, Nov 14, 2015 at 09:56:36AM -0800, David Wolfskill wrote:
> On Wed, Nov 11, 2015 at 06:33:37AM -0800, David Wolfskill wrote:
> > ...
> > But a quick perusal of
> >  doesn't show
> > anything especially like a "smoking gun" -- to me, anyway.
> > 
> > Can anyone else confirm or refute my observations?  Or suggest a
> > hint?  I'll try narrowing it down myself, but I need to do it during
> > times I'm at home (so I can manually power the machine back up when
> > it fails to respond to WoL), so it may be a few days before I can
> > accomplish much that way.
> > 
> 
> r290565 still works; r290566 fails -- in my case.  r290566 changed some
> re(4) behavior, and the NIC on my affected machine is an re(4):
> 
> re0@pci0:3:0:0: class=0x02 card=0x05b71028 chip=0x816810ec rev=0x0c
> hdr=0x00
> vendor = 'Realtek Semiconductor Co., Ltd.'
> device = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet
> Controller'
> class  = network
> subclass   = ethernet
> 
> from "pciconf -lv" while running:
> 
> D freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1904  
> r290565M/290565:1100089: Sat Nov 14 09:44:33 PST 2015 
> r...@freebeast.catwhisker.org:/common/S3/obj/usr/src/sys/GENERIC  amd64
> 
> I've placed a copy of a verbose dmes.boot in
> .
> 
> I'm happy to test suggested changes.
> 

*sigh* Okay, could you please test whether the attached patch restores
WOL capability for you?

Marius

Index: if_re.c
===
--- if_re.c	(revision 290566)
+++ if_re.c	(working copy)
@@ -3851,6 +3852,11 @@ re_setwol(struct rl_softc *sc)
 			CSR_READ_1(sc, RL_GPIO) & ~0x01);
 	}
 	if ((ifp->if_capenable & IFCAP_WOL) != 0) {
+		if ((sc->rl_flags & RL_FLAG_8168G_PLUS) != 0) {
+			/* Disable RXDV gate. */
+			CSR_WRITE_4(sc, RL_MISC, CSR_READ_4(sc, RL_MISC) &
+			~0x0008);
+		}
 		re_set_rxmode(sc);
 		if ((sc->rl_flags & RL_FLAG_WOL_MANLINK) != 0)
 			re_set_linkspeed(sc);
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Wake on LAN broken (probably between r290542 - r290606)?

[David Wolfskill]

On Sun, Nov 15, 2015 at 10:57:06PM +0100, Marius Strobl wrote:
> ...
> > I've placed a copy of a verbose dmes.boot in
> > .
> > 
> > I'm happy to test suggested changes.
> > 
> 
> *sigh* Okay, could you please test whether the attached patch restores
> WOL capability for you?
> 

Aye; that worked for me.

I used the slice where I had tested r290566 and r290565 (most recently,
the latter), used "svn update -r290566" to update sources to r290566
first, then used "svn patch" to apply the aptch, then re-built world &
kernel & re-installed kernel & world, then rebooted, then "shutdown -p
now" to power off.

Once serial console reported:

...
re0: link state changed to DOWN
re0: link state changed to UP
acpi0: Powering system off


I used /usr/local/bin/wol on the machine that is supposed to wake the
build machine up, and it powered right up as it did before.

Thanks!

Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


signature.asc
Description: PGP signature

Re: make installworld failing with locales due to broken symlinks

[Craig Rodrigues]

On Sun, Nov 15, 2015 at 5:57 PM, NGie Cooper  wrote:

> $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
> ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
>


I built the latest world, and then did the following:

rm -fr /tmp/testdir
mkdir /tmp/testdir

make installworld DESTDIR=/tmp/testdir

cd /tmp/testdir/usr/share/locale
for f in $(find . -type l) ; do [ ! -e $f ] && echo "$f does not exist";
done

./zh_CN.GB2312/LC_NUMERIC does not exist
./zh_CN.GB2312/LC_MONETARY does not exist
./zh_CN.GB2312/LC_COLLATE does not exist
./zh_CN.GB2312/LC_CTYPE does not exist
./zh_CN.GB2312/LC_MESSAGES does not exist
./zh_CN.GB2312/LC_TIME does not exist
./zh_CN.eucCN/LC_MESSAGES does not exist
./zh_CN.eucCN/LC_NUMERIC does not exist
./zh_CN.eucCN/LC_TIME does not exist
./zh_CN.eucCN/LC_CTYPE does not exist
./zh_CN.eucCN/LC_MONETARY does not exist
./zh_CN.eucCN/LC_COLLATE does not exist
./zh_HK.UTF-8/LC_NUMERIC does not exist
./zh_HK.UTF-8/LC_MONETARY does not exist
./zh_HK.UTF-8/LC_COLLATE does not exist
./zh_HK.UTF-8/LC_MESSAGES does not exist
./zh_HK.UTF-8/LC_TIME does not exist
./zh_HK.UTF-8/LC_CTYPE does not exist
./zh_CN.GBK/LC_CTYPE does not exist
./zh_CN.GBK/LC_NUMERIC does not exist
./zh_CN.GBK/LC_MESSAGES does not exist
./zh_CN.GBK/LC_COLLATE does not exist
./zh_CN.GBK/LC_MONETARY does not exist
./zh_CN.GBK/LC_TIME does not exist
./zh_CN.GB18030/LC_COLLATE does not exist
./zh_CN.GB18030/LC_MESSAGES does not exist
./zh_CN.GB18030/LC_CTYPE does not exist
./zh_CN.GB18030/LC_TIME does not exist
./zh_CN.GB18030/LC_NUMERIC does not exist
./zh_CN.GB18030/LC_MONETARY does not exist
./zh_TW.UTF-8/LC_MESSAGES does not exist
./zh_TW.UTF-8/LC_NUMERIC does not exist
./zh_TW.UTF-8/LC_MONETARY does not exist
./zh_TW.UTF-8/LC_TIME does not exist
./zh_TW.UTF-8/LC_CTYPE does not exist
./zh_TW.UTF-8/LC_COLLATE does not exist
./zh_HK.Big5HKSCS/LC_MONETARY does not exist
./zh_HK.Big5HKSCS/LC_NUMERIC does not exist
./zh_HK.Big5HKSCS/LC_CTYPE does not exist
./zh_HK.Big5HKSCS/LC_TIME does not exist
./zh_HK.Big5HKSCS/LC_COLLATE does not exist
./zh_HK.Big5HKSCS/LC_MESSAGES does not exist
./zh_TW.Big5/LC_MONETARY does not exist
./zh_TW.Big5/LC_CTYPE does not exist
./zh_TW.Big5/LC_TIME does not exist
./zh_TW.Big5/LC_NUMERIC does not exist
./zh_TW.Big5/LC_MESSAGES does not exist
./zh_TW.Big5/LC_COLLATE does not exist

--
Craig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make installworld failing with locales due to broken symlinks

[Andrey Chernov]

On 16.11.2015 4:57, NGie Cooper wrote:
> make: stopped in /usr/src/git
> $ ls -l /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
> lrwxr-xr-x  1 root  wheel  27 Nov  1 16:24 
> /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
> $ readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
> /usr/share/locale/la_LN.ISO8859-1
> $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
> ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory

As I remember, we already hit this type of problem before with old
locales. "ln -sf" and "rm -rf" should be used everywhere on install
target to avoid that.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make installworld failing with locales due to broken symlinks

[Craig Rodrigues]

On Sun, Nov 15, 2015 at 6:27 PM, Craig Rodrigues 
wrote:

> On Sun, Nov 15, 2015 at 5:57 PM, NGie Cooper 
> wrote:
>
>> $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
>> ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
>>
>
>
> I built the latest world, and then did the following:
>
> rm -fr /tmp/testdir
> mkdir /tmp/testdir
>
> make installworld DESTDIR=/tmp/testdir
>
> cd /tmp/testdir/usr/share/locale
> for f in $(find . -type l) ; do [ ! -e $f ] && echo "$f does not exist";
> done
>
> ./zh_CN.GB2312/LC_NUMERIC does not exist
> ./zh_CN.GB2312/LC_MONETARY does not exist
> ./zh_CN.GB2312/LC_COLLATE does not exist
> ./zh_CN.GB2312/LC_CTYPE does not exist
> ./zh_CN.GB2312/LC_MESSAGES does not exist
> ./zh_CN.GB2312/LC_TIME does not exist
> ./zh_CN.eucCN/LC_MESSAGES does not exist
> ./zh_CN.eucCN/LC_NUMERIC does not exist
> ./zh_CN.eucCN/LC_TIME does not exist
> ./zh_CN.eucCN/LC_CTYPE does not exist
> ./zh_CN.eucCN/LC_MONETARY does not exist
> ./zh_CN.eucCN/LC_COLLATE does not exist
> ./zh_HK.UTF-8/LC_NUMERIC does not exist
> ./zh_HK.UTF-8/LC_MONETARY does not exist
> ./zh_HK.UTF-8/LC_COLLATE does not exist
> ./zh_HK.UTF-8/LC_MESSAGES does not exist
> ./zh_HK.UTF-8/LC_TIME does not exist
> ./zh_HK.UTF-8/LC_CTYPE does not exist
> ./zh_CN.GBK/LC_CTYPE does not exist
> ./zh_CN.GBK/LC_NUMERIC does not exist
> ./zh_CN.GBK/LC_MESSAGES does not exist
> ./zh_CN.GBK/LC_COLLATE does not exist
> ./zh_CN.GBK/LC_MONETARY does not exist
> ./zh_CN.GBK/LC_TIME does not exist
> ./zh_CN.GB18030/LC_COLLATE does not exist
> ./zh_CN.GB18030/LC_MESSAGES does not exist
> ./zh_CN.GB18030/LC_CTYPE does not exist
> ./zh_CN.GB18030/LC_TIME does not exist
> ./zh_CN.GB18030/LC_NUMERIC does not exist
> ./zh_CN.GB18030/LC_MONETARY does not exist
> ./zh_TW.UTF-8/LC_MESSAGES does not exist
> ./zh_TW.UTF-8/LC_NUMERIC does not exist
> ./zh_TW.UTF-8/LC_MONETARY does not exist
> ./zh_TW.UTF-8/LC_TIME does not exist
> ./zh_TW.UTF-8/LC_CTYPE does not exist
> ./zh_TW.UTF-8/LC_COLLATE does not exist
> ./zh_HK.Big5HKSCS/LC_MONETARY does not exist
> ./zh_HK.Big5HKSCS/LC_NUMERIC does not exist
> ./zh_HK.Big5HKSCS/LC_CTYPE does not exist
> ./zh_HK.Big5HKSCS/LC_TIME does not exist
> ./zh_HK.Big5HKSCS/LC_COLLATE does not exist
> ./zh_HK.Big5HKSCS/LC_MESSAGES does not exist
> ./zh_TW.Big5/LC_MONETARY does not exist
> ./zh_TW.Big5/LC_CTYPE does not exist
> ./zh_TW.Big5/LC_TIME does not exist
> ./zh_TW.Big5/LC_NUMERIC does not exist
> ./zh_TW.Big5/LC_MESSAGES does not exist
> ./zh_TW.Big5/LC_COLLATE does not exist
>
> --
> Craig
>
>


This fixed it for me:

Index: Makefile
===
--- Makefile(revision 290902)
+++ Makefile(working copy)
@@ -15,7 +15,7 @@

 .for from to in ${ALIASES}
 .for f in ${LC_FILES}
-SYMLINKS+= ${from}/${f} ${LOCALEDIR}/${to}/${f}
+SYMLINKS+= ../${from}/${f} ${LOCALEDIR}/${to}/${f}
 .endfor
 .endfor

--
Craig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make installworld failing with locales due to broken symlinks

[NGie Cooper]

> On Nov 15, 2015, at 19:08, Craig Rodrigues  wrote:
> 
> On Sun, Nov 15, 2015 at 6:27 PM, Craig Rodrigues 
> wrote:
> 
>> On Sun, Nov 15, 2015 at 5:57 PM, NGie Cooper 
>> wrote:
>> 
>>> $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
>>> ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
>>> 
>> 
>> 
>> I built the latest world, and then did the following:
>> 
>> rm -fr /tmp/testdir
>> mkdir /tmp/testdir
>> 
>> make installworld DESTDIR=/tmp/testdir
>> 
>> cd /tmp/testdir/usr/share/locale
>> for f in $(find . -type l) ; do [ ! -e $f ] && echo "$f does not exist";
>> done
>> 
>> ./zh_CN.GB2312/LC_NUMERIC does not exist
>> ./zh_CN.GB2312/LC_MONETARY does not exist
>> ./zh_CN.GB2312/LC_COLLATE does not exist
>> ./zh_CN.GB2312/LC_CTYPE does not exist
>> ./zh_CN.GB2312/LC_MESSAGES does not exist
>> ./zh_CN.GB2312/LC_TIME does not exist
>> ./zh_CN.eucCN/LC_MESSAGES does not exist
>> ./zh_CN.eucCN/LC_NUMERIC does not exist
>> ./zh_CN.eucCN/LC_TIME does not exist
>> ./zh_CN.eucCN/LC_CTYPE does not exist
>> ./zh_CN.eucCN/LC_MONETARY does not exist
>> ./zh_CN.eucCN/LC_COLLATE does not exist
>> ./zh_HK.UTF-8/LC_NUMERIC does not exist
>> ./zh_HK.UTF-8/LC_MONETARY does not exist
>> ./zh_HK.UTF-8/LC_COLLATE does not exist
>> ./zh_HK.UTF-8/LC_MESSAGES does not exist
>> ./zh_HK.UTF-8/LC_TIME does not exist
>> ./zh_HK.UTF-8/LC_CTYPE does not exist
>> ./zh_CN.GBK/LC_CTYPE does not exist
>> ./zh_CN.GBK/LC_NUMERIC does not exist
>> ./zh_CN.GBK/LC_MESSAGES does not exist
>> ./zh_CN.GBK/LC_COLLATE does not exist
>> ./zh_CN.GBK/LC_MONETARY does not exist
>> ./zh_CN.GBK/LC_TIME does not exist
>> ./zh_CN.GB18030/LC_COLLATE does not exist
>> ./zh_CN.GB18030/LC_MESSAGES does not exist
>> ./zh_CN.GB18030/LC_CTYPE does not exist
>> ./zh_CN.GB18030/LC_TIME does not exist
>> ./zh_CN.GB18030/LC_NUMERIC does not exist
>> ./zh_CN.GB18030/LC_MONETARY does not exist
>> ./zh_TW.UTF-8/LC_MESSAGES does not exist
>> ./zh_TW.UTF-8/LC_NUMERIC does not exist
>> ./zh_TW.UTF-8/LC_MONETARY does not exist
>> ./zh_TW.UTF-8/LC_TIME does not exist
>> ./zh_TW.UTF-8/LC_CTYPE does not exist
>> ./zh_TW.UTF-8/LC_COLLATE does not exist
>> ./zh_HK.Big5HKSCS/LC_MONETARY does not exist
>> ./zh_HK.Big5HKSCS/LC_NUMERIC does not exist
>> ./zh_HK.Big5HKSCS/LC_CTYPE does not exist
>> ./zh_HK.Big5HKSCS/LC_TIME does not exist
>> ./zh_HK.Big5HKSCS/LC_COLLATE does not exist
>> ./zh_HK.Big5HKSCS/LC_MESSAGES does not exist
>> ./zh_TW.Big5/LC_MONETARY does not exist
>> ./zh_TW.Big5/LC_CTYPE does not exist
>> ./zh_TW.Big5/LC_TIME does not exist
>> ./zh_TW.Big5/LC_NUMERIC does not exist
>> ./zh_TW.Big5/LC_MESSAGES does not exist
>> ./zh_TW.Big5/LC_COLLATE does not exist
>> 
>> --
>> Craig
>> 
>> 
> 
> 
> This fixed it for me:
> 
> Index: Makefile
> ===
> --- Makefile(revision 290902)
> +++ Makefile(working copy)
> @@ -15,7 +15,7 @@
> 
> .for from to in ${ALIASES}
> .for f in ${LC_FILES}
> -SYMLINKS+= ${from}/${f} ${LOCALEDIR}/${to}/${f}
> +SYMLINKS+= ../${from}/${f} ${LOCALEDIR}/${to}/${f}
> .endfor
> .endfor

LGTM
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make installworld failing with locales due to broken symlinks

[Craig Rodrigues]

On Sun, Nov 15, 2015 at 5:57 PM, NGie Cooper  wrote:

> $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
> ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
>


I built the latest world, and then did the following:

rm -fr /tmp/testdir
mkdir /tmp/testdir

make installworld DESTDIR=/tmp/testdir

cd /tmp/testdir/usr/share/locale
for f in $(find . -type l) ; do [ ! -e $f ] && echo "$f does not exist";
done

./zh_CN.GB2312/LC_NUMERIC does not exist
./zh_CN.GB2312/LC_MONETARY does not exist
./zh_CN.GB2312/LC_COLLATE does not exist
./zh_CN.GB2312/LC_CTYPE does not exist
./zh_CN.GB2312/LC_MESSAGES does not exist
./zh_CN.GB2312/LC_TIME does not exist
./zh_CN.eucCN/LC_MESSAGES does not exist
./zh_CN.eucCN/LC_NUMERIC does not exist
./zh_CN.eucCN/LC_TIME does not exist
./zh_CN.eucCN/LC_CTYPE does not exist
./zh_CN.eucCN/LC_MONETARY does not exist
./zh_CN.eucCN/LC_COLLATE does not exist
./zh_HK.UTF-8/LC_NUMERIC does not exist
./zh_HK.UTF-8/LC_MONETARY does not exist
./zh_HK.UTF-8/LC_COLLATE does not exist
./zh_HK.UTF-8/LC_MESSAGES does not exist
./zh_HK.UTF-8/LC_TIME does not exist
./zh_HK.UTF-8/LC_CTYPE does not exist
./zh_CN.GBK/LC_CTYPE does not exist
./zh_CN.GBK/LC_NUMERIC does not exist
./zh_CN.GBK/LC_MESSAGES does not exist
./zh_CN.GBK/LC_COLLATE does not exist
./zh_CN.GBK/LC_MONETARY does not exist
./zh_CN.GBK/LC_TIME does not exist
./zh_CN.GB18030/LC_COLLATE does not exist
./zh_CN.GB18030/LC_MESSAGES does not exist
./zh_CN.GB18030/LC_CTYPE does not exist
./zh_CN.GB18030/LC_TIME does not exist
./zh_CN.GB18030/LC_NUMERIC does not exist
./zh_CN.GB18030/LC_MONETARY does not exist
./zh_TW.UTF-8/LC_MESSAGES does not exist
./zh_TW.UTF-8/LC_NUMERIC does not exist
./zh_TW.UTF-8/LC_MONETARY does not exist
./zh_TW.UTF-8/LC_TIME does not exist
./zh_TW.UTF-8/LC_CTYPE does not exist
./zh_TW.UTF-8/LC_COLLATE does not exist
./zh_HK.Big5HKSCS/LC_MONETARY does not exist
./zh_HK.Big5HKSCS/LC_NUMERIC does not exist
./zh_HK.Big5HKSCS/LC_CTYPE does not exist
./zh_HK.Big5HKSCS/LC_TIME does not exist
./zh_HK.Big5HKSCS/LC_COLLATE does not exist
./zh_HK.Big5HKSCS/LC_MESSAGES does not exist
./zh_TW.Big5/LC_MONETARY does not exist
./zh_TW.Big5/LC_CTYPE does not exist
./zh_TW.Big5/LC_TIME does not exist
./zh_TW.Big5/LC_NUMERIC does not exist
./zh_TW.Big5/LC_MESSAGES does not exist
./zh_TW.Big5/LC_COLLATE does not exist

--
Craig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make installworld failing with locales due to broken symlinks

[Andrey Chernov]

On 16.11.2015 5:37, Andrey Chernov wrote:
> On 16.11.2015 4:57, NGie Cooper wrote:
>> make: stopped in /usr/src/git
>> $ ls -l /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
>> lrwxr-xr-x  1 root  wheel  27 Nov  1 16:24 
>> /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
>> $ readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
>> /usr/share/locale/la_LN.ISO8859-1
>> $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
>> ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
> 
> As I remember, we already hit this type of problem before with old
> locales. "ln -sf" and "rm -rf" should be used everywhere on install
> target to avoid that.
> 
Oops, auto-typo. "rm -f" instead, since all directories are created by
mtree.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

make installworld failing with locales due to broken symlinks

[NGie Cooper]

Hi,
I run into this error when running `make installworld` with a world 
installed prior and during the projects/collation merge to head — reason is 
that the target for the symlink doesn’t exist. This might be fallout from 
recent build changes, or a side effect of the broken symlinks…
Thanks,
-NGie

install: //usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE: No such file or directory
*** Error code 71

Stop.
make[5]: stopped in /usr/src/git/share/ctypedef
*** Error code 1

Stop.
make[4]: stopped in /usr/src/git/share
*** Error code 1

Stop.
make[3]: stopped in /usr/src/git
*** Error code 1

Stop.
make[2]: stopped in /usr/src/git
*** Error code 1

Stop.
make[1]: stopped in /usr/src/git
*** Error code 1

Stop.
make: stopped in /usr/src/git
$ ls -l /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
lrwxr-xr-x  1 root  wheel  27 Nov  1 16:24 
/usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
$ readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
/usr/share/locale/la_LN.ISO8859-1
$ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

FreeBSD_HEAD-tests - Build #1695 - Still Unstable

[jenkins-admin]

FreeBSD_HEAD-tests - Build #1695 - Still Unstable:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1695/
Full change log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1695/changes
Full build log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD-tests/1695/console

Change summaries:

No changes


The failed test cases:

6 tests failed.
FAILED:  lib.libc.locale.c16rtomb_test.c16rtomb_iso_8859_15_test

Error Message:
setlocale(LC_CTYPE, "en_US.ISO8859-15") failed; errno=22

FAILED:  lib.libc.locale.io_test.good_big5_getwc

Error Message:
/builds/FreeBSD_HEAD/contrib/netbsd-tests/lib/libc/locale/t_io.c:155: getwc(fp) 
!= 0xcf40

FAILED:  lib.libc.locale.io_test.good_big5_swprintf

Error Message:
/builds/FreeBSD_HEAD/contrib/netbsd-tests/lib/libc/locale/t_io.c:117: 
swprintf(obuf, sizeof(obuf), L"%ls\n", ibuf) != 2

FAILED:  lib.libc.locale.io_test.good_big5_wprintf

Error Message:
/builds/FreeBSD_HEAD/contrib/netbsd-tests/lib/libc/locale/t_io.c:102: 
wprintf(L"%ls\n", ibuf) != 2

FAILED:  lib.libc.locale.mbrtoc16_test.mbrtoc16_iso_8859_15_test

Error Message:
setlocale(LC_CTYPE, "en_US.ISO8859-15") failed; errno=22

FAILED:  lib.libc.locale.mbtowc_test.mbtowc

Error Message:
Test case was expecting a failure but none were raised

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make installworld failing with locales due to broken symlinks

[Craig Rodrigues]

On Sun, Nov 15, 2015 at 5:57 PM, NGie Cooper  wrote:

>
> install: //usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE: No such file or
> directory
> *** Error code 71
>
> $ ls -l /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
> lrwxr-xr-x  1 root  wheel  27 Nov  1 16:24
> /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
> $ readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
> /usr/share/locale/la_LN.ISO8859-1
> $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
> ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
>


Hi,

I ran into the same error when I tried to upgrade a system that I did an
installworld on
2 days ago.

--
Craig
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

CURRENT installworld fails: LC_CTYPE: No such file or directory

[O. Hartmann]

The recent CURRENT starts dropping "make installworld" since two days ago, now 
at
revision 290923, with the error shown below.

Regards,

oh


[...]
===> share/ctypedef (install)
install -o root  -g wheel -m 444
be_BY.CP1131.LC_CTYPE  /usr/share/locale/be_BY.CP1131/LC_CTYPE install -o root  
-g wheel
-m 444  ca_IT.ISO8859-1.LC_CTYPE  /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
install: /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE: No such file or directory 
*** Error
code 71


pgp9Yrfg5trvP.pgp
Description: OpenPGP digital signature

Re: make installworld failing with locales due to broken symlinks

[O. Hartmann]

Am Sun, 15 Nov 2015 19:59:54 -0800
Craig Rodrigues  schrieb:

> On Sun, Nov 15, 2015 at 5:57 PM, NGie Cooper  wrote:
> 
> >
> > install: //usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE: No such file or
> > directory
> > *** Error code 71
> >
> > $ ls -l /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
> > lrwxr-xr-x  1 root  wheel  27 Nov  1 16:24
> > /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
> > $ readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
> > /usr/share/locale/la_LN.ISO8859-1
> > $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
> > ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
> >  
> 
> 
> Hi,
> 
> I ran into the same error when I tried to upgrade a system that I did an
> installworld on
> 2 days ago.
> 
> --
> Craig
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

A "me, too" from here, I'm with CURRENT at revision 290924. Still failing to 
"make
installworld".

Kind regards,

oh


pgpg9nTuBNPfe.pgp
Description: OpenPGP digital signature

Re: make installworld failing with locales due to broken symlinks

[O. Hartmann]

Am Mon, 16 Nov 2015 08:14:36 +0100
"O. Hartmann"  schrieb:

> Am Sun, 15 Nov 2015 19:59:54 -0800
> Craig Rodrigues  schrieb:
> 
> > On Sun, Nov 15, 2015 at 5:57 PM, NGie Cooper  wrote:
> >   
> > >
> > > install: //usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE: No such file or
> > > directory
> > > *** Error code 71
> > >
> > > $ ls -l /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
> > > lrwxr-xr-x  1 root  wheel  27 Nov  1 16:24
> > > /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE -> ../la_LN.ISO8859-1/LC_CTYPE
> > > $ readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE
> > > /usr/share/locale/la_LN.ISO8859-1
> > > $ ls `readlink -f /usr/share/locale/ca_IT.ISO8859-1/LC_CTYPE`
> > > ls: /usr/share/locale/la_LN.ISO8859-1: No such file or directory
> > >
> > 
> > 
> > Hi,
> > 
> > I ran into the same error when I tried to upgrade a system that I did an
> > installworld on
> > 2 days ago.
> > 
> > --
> > Craig
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"  
> 
> A "me, too" from here, I'm with CURRENT at revision 290924. Still failing to 
> "make
> installworld".
> 
> Kind regards,
> 
> oh

All right, missed the part "rm -rf". I deleted the file/link in question and 
the world
gets installed as usual.

oh


pgp1B_0Tt35yY.pgp
Description: OpenPGP digital signature

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Adrian Chadd]

Hi,

The reason is simple - someone offered to do the work and push it through.

This isn't a commercial thing where we get to make project decisions
and allocate resources - the juniper folk came up with a solution that
they're using and willing to push forward in freebsd. It's been an
interesting experiment and maybe it shouldn't have been done in -HEAD,
but it isn't all that pervasive and we've gotten good feedback from it
all.


-adrian
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 4:38 PM, Andrey Chernov wrote:
> On 15.11.2015 18:17, John Marino wrote:
>>
>> For the traditional checks, It's ironic, DragonFly uses short locales
>> like "fr_FR" which link to the approprioprate ISO8859 or UTF-8 locale.
>> Bapt removed them to avoid a bike shed and if he had not done that, this
>> gettext configure would not be an issue as "fr_FR" is the very first check.
> 
> All parts are optional excepting language itself, so fr_FR is not short
> enough, but fr is. From POSIX:
> 
> language[_territory][.codeset][@modifier]

"fr" (or "french") by itself isn't very useful, so IMO
language_territory is the minimum.

> But I dislike short locales due to their uncertainty and remember some
> programs and shell scripts that attempts to parse locale name directly
> by itself for reasons I don't remember now.

Absolutely true about the uncertainty.  It's just subjective.  We did
what we thought made sense but bapt recognized an impeding bikeshed and
just did away with it.

For some reason gettext-tools only cares about french and japanese (thus
easily patched).  Here's what it looks like on current DragonFly:

http://muscles.dragonflybsd.org/bulk/bleeding-edge-potential/latest-per-pkg/gettext-tools-0.19.6.log

Note the identified traditional french is "fr_FR" which maps to
fr_FR.ISO8859-15 on DragonFly.

It's a moot point now I guess.  ISO8859-1 has returned to FreeBSD for
western europe (probably forever).

John




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Dan Partelly]

Meaning, is that simple to push things in head , if somone does the work, even 
with with no proper review of the problem at hand , and the proposed solutions 
? 


> On 15 Nov 2015, at 19:05, Adrian Chadd  wrote:
> 
> Hi,
> 
> The reason is simple - someone offered to do the work and push it through.
> 
> This isn't a commercial thing where we get to make project decisions
> and allocate resources - the juniper folk came up with a solution that
> they're using and willing to push forward in freebsd. It's been an
> interesting experiment and maybe it shouldn't have been done in -HEAD,
> but it isn't all that pervasive and we've gotten good feedback from it
> all.
> 
> 
> -adrian
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Need help fixing failing locale tests

[John Marino]

On 11/15/2015 3:59 PM, Andrey Chernov wrote:
> On 15.11.2015 17:47, John Marino wrote:
 Please provide an example of such a program (in ports).
>>>
>>> See gettext-0.19.6/gettext-tools/configure, part started with
>>> # Test for the usual locale name.
>>> I don't have time to dig more such code now, but I remember I saw more
>>> for sure.
> 
>> If the ports framework is not overriding locale to "C" during the build
>> then that's probably something that should be introduced.  Ports should
>> not be producing different results based on the configuration of the
>> building machine at the time.
> 
> This is not about resetting locale to "C" during the build. Please
> really look at "configure" specific part mentioned above, it is you who
> ask at least one specific example and then it is you who ignore the
> answer takes my time to find it.

Hmmm?
Of course it's the same topic.
If the configure of a port is affected by the locale, that's a big
problem.  How is this not the same topic?

But I did look at gettext, which is configuring package based on what's
on the system (not environment).  For the unicode checks, there is
obviously no issue.

For the traditional checks, It's ironic, DragonFly uses short locales
like "fr_FR" which link to the approprioprate ISO8859 or UTF-8 locale.
Bapt removed them to avoid a bike shed and if he had not done that, this
gettext configure would not be an issue as "fr_FR" is the very first check.

John

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Allan Jude]

On 2015-11-15 13:06, Garrett Cooper wrote:
> 
>> On Nov 15, 2015, at 09:51, Andrey Chernov  wrote:
>>
>>> On 15.11.2015 20:37, Adrian Chadd wrote:
 On 15 November 2015 at 09:10, Dan Partelly  wrote:
 Meaning, is that simple to push things in head , if somone does the work, 
 even with with no proper review of the problem at hand , and the proposed 
 solutions ?
>>>
>>> Nope and yup. The juniper folk had a solution to a problem multiple
>>> people had requested work on, and their proposal was by far the
>>> furthest along code and use wise.
>>>
>>> It's all fine and good making technical decisions based on drawings
>>> and handwaving and philosophizing, but at some point someone has to do
>>> the code. Juniper's libxo was the furthest along in implementation and
>>> production.
>>
>> It seems it is the only and final argument for libXO existence. I
>> remember 2 or 3 discussions against libXO spontaneously happens in the
>> FreeBSD lists, all ended with that, approximately: "we already have the
>> code and you have just speculations". Alternative and more architecture
>> clean ideas, like making standalone template-oriented parser probably
>> based on liXO, are never seriously considered, because nobody will code
>> it, not for other reasons.
> 
> We lack a [dtd/json] spec for tools, so programming for xo'ification doesn't 
> seems like the best idea in the world to me from a end-user 
> sysadmin/developer perspective.
> 
> I could just as easily use standard tools like awk, grep, sed, and more 
> advanced languages like perl or Python to parse output, and assuming output 
> doesn't get a major rewrite, I'd just go with that method that's worked 
> pretty well for me over the last 10 years of my career..
> 
> Cheers,
> -NGie
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> 

The big difference is, a json parser isn't going to blow up if a new
field gets added in the middle, and your awk/grep/sed script probably will.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Garrett Cooper]

> On Nov 15, 2015, at 10:09, Allan Jude  wrote:

...

> The big difference is, a json parser isn't going to blow up if a new
> field gets added in the middle, and your awk/grep/sed script probably will.

That's a plus to those formats, yes, but if someone changes the field name 
(which can happen today, on a whim, and would go unnoticed for a while because 
no tests/spec), you'll run into a KeyError in Python or an equivalent error 
message in your language of choice.

Thanks,
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Allan Jude]

On 2015-11-15 13:16, Allan Jude wrote:
> On 2015-11-15 13:14, Garrett Cooper wrote:
>>
>>> On Nov 15, 2015, at 10:09, Allan Jude  wrote:
>>
>> ...
>>
>>> The big difference is, a json parser isn't going to blow up if a new
>>> field gets added in the middle, and your awk/grep/sed script probably will.
>>
>> That's a plus to those formats, yes, but if someone changes the field name 
>> (which can happen today, on a whim, and would go unnoticed for a while 
>> because no tests/spec), you'll run into a KeyError in Python or an 
>> equivalent error message in your language of choice.
>>
>> Thanks,
>>
> 
> But, if the same change was made to a pure text output, then our 'column
> 3' would suddenly be username instead of ip address, and your program
> would not work at intended either. so at least json would give you a
> warning.
> 
> So, the need for a spec on the output is not specific to encoded
> outputs, changing the output in text is just as disruptful.
> 

Also, libxo now supports the versioning of output, to make it possible
for your json parser to detect when a change to the schema has been made.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[NGie Cooper]

> On Nov 15, 2015, at 10:14, Allan Jude  wrote:

…

> You can setup an atexit() call to call xo_finish automatically when the
> program exits. The original changes to uptime had a few other issues,
> which I fixed.

Programmers are lazy. Telling someone “you need to setup an atexit handler that 
calls xo_finish” will be met with teeth gnashing and complaints (I’ve had to 
deal with a bunch of people who complain that FreeBSD is not Linux at $work; I 
don’t have any interest in fighting developers over stuff like this).

> Yes, but, a typo in any change is likely to cause a problem. This is not
> a problem unique to libxo.

Yes, but there’s a big difference between a typo in !libxo with the meta 
language, and a typo in libxo. Typo in !libxo: typos visible, unless it was 
something silly like mis-spelling \n (which doesn’t seem to happen much); typo 
in libxo: xo_emit call is invalid (no output)/segfaults.

Fortunately libxo seems to support __printflike :).

Thanks,
-NGie
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[NGie Cooper]

> On Nov 15, 2015, at 10:23, Allan Jude  wrote:

…

> Also, libxo now supports the versioning of output, to make it possible
> for your json parser to detect when a change to the schema has been made.

This is the ideal scenario, yes, if there was some design around the 
libxo’ification (even man page documentation would have been nice, but that 
wasn’t done :/). However, as I’ve said many times before: no spec, no tests -> 
not useful to endorse to others because things can change at any given point in 
time and I don’t want to be the support hub for all things libxo at $work.

Thanks,
-NGie
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Build failed in Jenkins: Build-UFS-image #2743

[jenkins-admin]

See 

--
[...truncated 3362 lines...]

 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 


 -> 

===> lib/libc/tests (install)
install -o root  -g wheel -m 444  Kyuafile.auto  

===> lib/libc/tests/tls_dso (install)
install -C -o root -g wheel -m 444   libh_tls_dynamic.a 

install -C -o root -g wheel -m 444   libh_tls_dynamic_p.a 

install -s -o root -g wheel -m 444 libh_tls_dynamic.so.1 

install -l s libh_tls_dynamic.so.1 

===> lib/libc/tests/c063 (install)
install -o root  -g wheel -m 444  Kyuafile.auto  

(cd /builds/FreeBSD_HEAD/lib/libc/tests/c063 &&  
DEPENDFILE=.depend.faccessat_test  NO_SUBDIR=1 make -f 
/builds/FreeBSD_HEAD/lib/libc/tests/c063/Makefile _RECURSING_PROGS=  
PROG=faccessat_test  install)
install -s -o root -g wheel 

FreeBSD_HEAD - Build #3531 - Fixed

[jenkins-admin]

FreeBSD_HEAD - Build #3531 - Fixed:

Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD/3531/
Full change log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD/3531/changes
Full build log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD/3531/console

Change summaries:

290866 by bapt:
Regenerate locales after readding ISO8859-1 for locales that have ISO8859-15

Requested by:   arche

290865 by bapt:
Generate in the FreeBSD keyword when generating the Makefiles

290864 by bapt:
Add ISO8859-1 everywhere ISO8859-15 exists

290863 by bapt:
Allow to generate the locale when the source directory is not /usr/src

290862 by bapt:
Simplify a bit the aliases generation

290861 by trasz:
Doh, commit in a wrong directory.  Fix r290857.

MFC after:  1 month
Sponsored by:   The FreeBSD Foundation

290860 by bapt:
Remove unused variables to fix building world

290859 by bapt:
Rework locale-links to not make symlinks on directories but symlinks on files

The goal here is to make the upgrade seamless for users
Add aliases for zh_HK
Remove bad symlinks created by previous bad upgrade procedure.
Complete ObsoleteFiles.inc with more locales that have been removed

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Dan Partelly]

I know about NexBSD, but, as I said in my original message, I am interested on 
what happen in FreeBSD proper, and what issues 
XOification  tries to  solve )and if any proper process was given to solving 
the identified issues , instead of just adding XO because 
it could be added.

> On 15 Nov 2015, at 15:10, Yonas Yanfa  wrote:
> 
> Hi Dan,
> 
> Sounds like you'd be interested in NextBSD:
> 
> https://en.wikipedia.org/wiki/NextBSD
> 
> Cheers,
> Yonas
> 
> 
> 
> 
> Yonas Yanfa
> In Love With Open Source
> Drupal  :: GitHub  :: 
> Mozilla  :: 
> iPhone 
> fizk.net | yo...@fizk.net
> 
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Andrey Chernov]

On 15.11.2015 20:37, Adrian Chadd wrote:
> On 15 November 2015 at 09:10, Dan Partelly  wrote:
>> Meaning, is that simple to push things in head , if somone does the work, 
>> even with with no proper review of the problem at hand , and the proposed 
>> solutions ?
> 
> Nope and yup. The juniper folk had a solution to a problem multiple
> people had requested work on, and their proposal was by far the
> furthest along code and use wise.
> 
> It's all fine and good making technical decisions based on drawings
> and handwaving and philosophizing, but at some point someone has to do
> the code. Juniper's libxo was the furthest along in implementation and
> production.

It seems it is the only and final argument for libXO existence. I
remember 2 or 3 discussions against libXO spontaneously happens in the
FreeBSD lists, all ended with that, approximately: "we already have the
code and you have just speculations". Alternative and more architecture
clean ideas, like making standalone template-oriented parser probably
based on liXO, are never seriously considered, because nobody will code
it, not for other reasons.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Allan Jude]

On 2015-11-15 07:54, Dan Partelly wrote:
> 
> Hi all,
> 
> I was looking at the new facility of dumping JSON,XML from many utils in base 
> and after some funny minutes, I couldn't stop ask myself “ Ok, this is funny 
> , but why ? “ And I couldn't find a real answer. Ill outline what I think:
> 
> 
> 1. Undoubtedly, it makes base code slightly harder to understand and 
> maintain. 
> 2. I have seen the idea that this makes the information dumped by utilities 
> in the base easily accessible programatically. OK, maybe it does , but
> it doesn't fit with the current paradigm of "tool | filter | tool” at all. 
> There are no tools able to accept JSON and filter it in any meaningful way, 
> and I
> dont see too many ppl changing their code to read JSON instead of text.  I 
> don't even see the base tools changing. This output may be useful in corner 
> cases only.
> 3. The integration of libxo IMO only points at a much deeper issue IMO. It is 
> only an expression of the need of a mechanism aimed at binary code reuse. But 
> it does not solve the problem, it only adds yet another possibility in a 
> world where too much choices already result in too much splits and 
> incompatible APIs. 
> 4. This whole effort would have been IMO much better served  by porting the 
> bulk of ifconfig(8) , route(8) and wpaclient(8) to a library API, much like 
> the libs for geom, zfs , etc , ready for reuse of 3rd party code. Eventually 
> writing network control daemons in time over it , much like solaris does.
> 
> 5. A port of partial OS config data to UCL …. would induce yet induce another 
> orthogonality violation. What makes UCL better than the bestiary of ad hoc 
> databases already existing in BSDs ? Programatic readability, yes. but it 
> does not add any real much needed functionality such as *transactional 
> databases* for system tools.   Why not research a proper solution - easily 
> accessible by other programs ,orthogonal , transactional, and ACL protected   
> solution  which can be used all over the place , from OS boot, to ABI 
> management, service management, network management, user management.  I hope 
> this day will come, a day when I will not have to edit a single config file 
> manually, yet I would have access to all the config and system state  easy 
> with wrapper APIs. In the light of this point, why go with UCL ? It is not 
> orthogonal, it is not transnational, and editing the config files directly 
> would result in the same old human errors which bite as all from time to time.
> 
> 5. It is my opinion that Solaris addressed some of those issue. Solaris FMRI 
> and SMF are lightyears ahead of the very tired models we keep using on BSDs. 
> Why not build on the insight offered by those (or even on the insight offered 
> by Windows :P) , then inventing more adhoc solutions and ad-hoc databases, 
> which do not address the real issues we have , like binary code reuse, 
> service management issues,  lack of a system wide published -subscriber bus ( 
> not kdbus :P ) fault detection and reaction, fault reporting, all much needed 
> parts of a modern OS. 
> 
> And now thee questions
> 
> 1. Why lib XO ? Why burden the OS for some corner cases where it may be 
> useful ?
> 
> 2. Was there any real talk on how to bring FreeBSD up to speed regarding 
> those issues ?  A period of research on what exists, on what can be done , 
> and ensure important things are not showed in background and replaced with 
> yet another ad-hoc config database which lacks modern features ?
> From where I am standing, this could be a project spawning multiple years , 
> but it would be well worth it, and in my opinion it would be also worthy of 
> the freeSBD foundation sponsorship for several years in a row. The features I 
> touched upon became very important parts of oder OSes, and rightly so. 
> 
> Note:
> 
> this message is serious and it is not intended to start flame wars, religious 
> crusades, or offend anyone. 
>  
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> 

lib-izing all of the utilities instead of using libxo is a better
solution. It would likely be gratefully accepted if someone were to do
it, but most likely, no one will.

libxo exists now, and most applications can be converted very quickly
(although care does need to be taken, and it sometimes has not been).

There are tools available to filter json/ucl output, namely textproc/jq
and my forthcoming uclcmd

One of the major other consumers of the json/xml output of libxo, is web
control panels. This is why Juniper is doing the work, and I can think
of a list of other appliance vendors who would love to replace fragile
per-application parsers with a json parser to extract data from various
command line tools.

UCL is a good solution to having a universal config file, and 

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Allan Jude]

On 2015-11-15 07:54, Dan Partelly wrote:
> 
> Hi all,
> 
> I was looking at the new facility of dumping JSON,XML from many utils in base 
> and after some funny minutes, I couldn't stop ask myself “ Ok, this is funny 
> , but why ? “ And I couldn't find a real answer. Ill outline what I think:
> 
> 
> 1. Undoubtedly, it makes base code slightly harder to understand and 
> maintain. 

I am not sure that libxo actually makes the code any harder to
understand and maintain. It might actually make it slightly better.

replacing:

printf("%s %s %d\n", foo, bar, number);

with:

xo_emit("{:foo/%s} {:bar/%s} {:number/%d}", foo, bar, number);

it not really hurting much.

> 2. I have seen the idea that this makes the information dumped by utilities 
> in the base easily accessible programatically. OK, maybe it does , but
> it doesn't fit with the current paradigm of "tool | filter | tool” at all. 
> There are no tools able to accept JSON and filter it in any meaningful way, 
> and I
> dont see too many ppl changing their code to read JSON instead of text.  I 
> don't even see the base tools changing. This output may be useful in corner 
> cases only.
> 3. The integration of libxo IMO only points at a much deeper issue IMO. It is 
> only an expression of the need of a mechanism aimed at binary code reuse. But 
> it does not solve the problem, it only adds yet another possibility in a 
> world where too much choices already result in too much splits and 
> incompatible APIs. 
> 4. This whole effort would have been IMO much better served  by porting the 
> bulk of ifconfig(8) , route(8) and wpaclient(8) to a library API, much like 
> the libs for geom, zfs , etc , ready for reuse of 3rd party code. Eventually 
> writing network control daemons in time over it , much like solaris does.
> 
> 5. A port of partial OS config data to UCL …. would induce yet induce another 
> orthogonality violation. What makes UCL better than the bestiary of ad hoc 
> databases already existing in BSDs ? Programatic readability, yes. but it 
> does not add any real much needed functionality such as *transactional 
> databases* for system tools.   Why not research a proper solution - easily 
> accessible by other programs ,orthogonal , transactional, and ACL protected   
> solution  which can be used all over the place , from OS boot, to ABI 
> management, service management, network management, user management.  I hope 
> this day will come, a day when I will not have to edit a single config file 
> manually, yet I would have access to all the config and system state  easy 
> with wrapper APIs. In the light of this point, why go with UCL ? It is not 
> orthogonal, it is not transnational, and editing the config files directly 
> would result in the same old human errors which bite as all from time to time.
> 
> 5. It is my opinion that Solaris addressed some of those issue. Solaris FMRI 
> and SMF are lightyears ahead of the very tired models we keep using on BSDs. 
> Why not build on the insight offered by those (or even on the insight offered 
> by Windows :P) , then inventing more adhoc solutions and ad-hoc databases, 
> which do not address the real issues we have , like binary code reuse, 
> service management issues,  lack of a system wide published -subscriber bus ( 
> not kdbus :P ) fault detection and reaction, fault reporting, all much needed 
> parts of a modern OS. 
> 
> And now thee questions
> 
> 1. Why lib XO ? Why burden the OS for some corner cases where it may be 
> useful ?
> 
> 2. Was there any real talk on how to bring FreeBSD up to speed regarding 
> those issues ?  A period of research on what exists, on what can be done , 
> and ensure important things are not showed in background and replaced with 
> yet another ad-hoc config database which lacks modern features ?
> From where I am standing, this could be a project spawning multiple years , 
> but it would be well worth it, and in my opinion it would be also worthy of 
> the freeSBD foundation sponsorship for several years in a row. The features I 
> touched upon became very important parts of oder OSes, and rightly so. 
> 
> Note:
> 
> this message is serious and it is not intended to start flame wars, religious 
> crusades, or offend anyone. 
>  
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> 


-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Adrian Chadd]

On 15 November 2015 at 09:51, Andrey Chernov  wrote:
> On 15.11.2015 20:37, Adrian Chadd wrote:
>> On 15 November 2015 at 09:10, Dan Partelly  wrote:
>>> Meaning, is that simple to push things in head , if somone does the work, 
>>> even with with no proper review of the problem at hand , and the proposed 
>>> solutions ?
>>
>> Nope and yup. The juniper folk had a solution to a problem multiple
>> people had requested work on, and their proposal was by far the
>> furthest along code and use wise.
>>
>> It's all fine and good making technical decisions based on drawings
>> and handwaving and philosophizing, but at some point someone has to do
>> the code. Juniper's libxo was the furthest along in implementation and
>> production.
>
> It seems it is the only and final argument for libXO existence. I
> remember 2 or 3 discussions against libXO spontaneously happens in the
> FreeBSD lists, all ended with that, approximately: "we already have the
> code and you have just speculations". Alternative and more architecture
> clean ideas, like making standalone template-oriented parser probably
> based on liXO, are never seriously considered, because nobody will code
> it, not for other reasons.

Right. Technical progress is made when people do some work. You can do
all the planning and designing you want, but computers available today
run on code, not on hopes and dreams. :)

I'd love to see the libxo stuff morph into a split between
Allan/others idea of "libify things" and "stuff that handles output
serialisation". Someone just has to do the design and do the code.

So, who wants to code up the template driven, library-for-access,
library-for-output-serialisation pieces? That's what we can commit. :)



-adrian
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Allan Jude]

On 2015-11-15 13:14, Garrett Cooper wrote:
> 
>> On Nov 15, 2015, at 10:09, Allan Jude  wrote:
> 
> ...
> 
>> The big difference is, a json parser isn't going to blow up if a new
>> field gets added in the middle, and your awk/grep/sed script probably will.
> 
> That's a plus to those formats, yes, but if someone changes the field name 
> (which can happen today, on a whim, and would go unnoticed for a while 
> because no tests/spec), you'll run into a KeyError in Python or an equivalent 
> error message in your language of choice.
> 
> Thanks,
> 

But, if the same change was made to a pure text output, then our 'column
3' would suddenly be username instead of ip address, and your program
would not work at intended either. so at least json would give you a
warning.

So, the need for a spec on the output is not specific to encoded
outputs, changing the output in text is just as disruptful.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: libXO-ification - Why - and is it a symptom of deeper issues?

[Allan Jude]

On 2015-11-15 13:10, Garrett Cooper wrote:
> 
>> On Nov 15, 2015, at 10:05, Allan Jude  wrote:
>>
>>> On 2015-11-15 07:54, Dan Partelly wrote:
>>>
>>> Hi all,
>>>
>>> I was looking at the new facility of dumping JSON,XML from many utils in 
>>> base and after some funny minutes, I couldn't stop ask myself “ Ok, this is 
>>> funny , but why ? “ And I couldn't find a real answer. Ill outline what I 
>>> think:
>>>
>>>
>>> 1. Undoubtedly, it makes base code slightly harder to understand and 
>>> maintain.
>>
>> I am not sure that libxo actually makes the code any harder to
>> understand and maintain. It might actually make it slightly better.
>>
>> replacing:
>>
>> printf("%s %s %d\n", foo, bar, number);
>>
>> with:
>>
>> xo_emit("{:foo/%s} {:bar/%s} {:number/%d}", foo, bar, number);
>>
>> it not really hurting much.
> 
> That's by and large true, but there are other APIs that need to be called on 
> exit (xo_finish?) and in other scenarios where flushing, etc is needed. If 
> you don't do that, you don't get the output you expect (which broke uptime/w 
> several months ago..).

You can setup an atexit() call to call xo_finish automatically when the
program exits. The original changes to uptime had a few other issues,
which I fixed.

> 
> Also, typos with the meta language into the xo_emit calls have bit is more 
> than once ;(.
> 

Yes, but, a typo in any change is likely to cause a problem. This is not
a problem unique to libxo.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature