Re: git tools for building in base?

2020-11-25 Thread Baptiste Daroussin
On Tue, Nov 24, 2020 at 09:59:15PM -0700, Warner Losh wrote:
> On Tue, Nov 24, 2020 at 2:19 PM tech-lists  wrote:
> 
> > As subject - what will there be in base to interact with the new git repo?
> > I mean, right now, for svn there is svnlite. What for git?
> >
> 
> 'pkg add git' is your choice now.

pkg install not pkg add
> 
> 
> > Shouldn't it be in base before the move to git?
> >
> 
> We will have got (from OpenBSD: Game Of Trees) in the future. It isn't
> quite there yet, however, so it's not in base. When we migrated from CVS to
> Subversion, we didn't grow svnlite in the base for many months after the
> conversion. We hope to have it finished in time for 13.0.
> 
> Warner
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


signature.asc
Description: PGP signature


Re: Deprecating ftpd in the FreeBSD base system?

2020-09-17 Thread Baptiste Daroussin
On Thu, Sep 17, 2020 at 07:04:41AM -0700, Cy Schubert wrote:
> In message  om>
> , Ed Maste writes:
> > FTP is (becoming?) a legacy protocol, and I think it may be time to
> > remove the ftp server from the FreeBSD base system - with the recent
> > security advisory for ftpd serving as a reminder.
> >
> > I've proposed adding a deprecation notice to the man page in
> > https://reviews.freebsd.org/D26447 to start this off. There are a
> > number of ftp servers in ports, and if we're going to remove the base
> > system one we can create a port for it first, as well.
> >
> > Any comments or concerns, please follow up in the code review or in email 
> > her
> > e.
> 
> We should also deprecate the FTP client.
> 
> I've been advocating removing FTP (and HTTP) from libfetch as well. People 
> should be using HTTPS only. (libfetch could support a plugin that might be 
> supplied by a port should someone be inclined to write one.)
> 
That that and we can throw away half of the ports tree ;)

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Move from termcap(5) to terminfo(5)

2020-05-07 Thread Baptiste Daroussin
On Thu, May 07, 2020 at 03:22:20PM +0200, Mateusz Piotrowski wrote:
> Hi,
> 
> On 5/7/20 2:41 PM, Baptiste Daroussin wrote:
> > Hello everyone,
> > 
> > I can't find any proper rationale in our history (maybe I missed it) which
> > explains why our ncurses is stuck on using termcap(5) instead of terminfo(5)
> > Except an argument in the Makefile that builds ncurses:
> > "Used instead of the hideous read_termcap.c abomination."
> > 
> > Which I do not find really useful.
> > 
> > I would like to make the move from termcap to terminfo which would give us 
> > the
> > bonus to be able to track terminfo sources from upstream aka ncurses and to
> > add and use tic(1).
> That's great! I've been bitten by termcap not being well understood by the
> open source community many times. I am supporting the move to terminfo
> wholeheartedly.
> > If you have some knowledge you want to share: "be careful about this or 
> > that" I
> > would be more than happy to collect it, to make sure the transition is as 
> > smooth
> > as possible.
> 
> What comes to my mind is that we should probably pay special attention to
> terminal ports, like x11/sterm. We might need to tell those ports to install
> their terminfo files. I don't remember the details though.
> 
All their terminfo files are already in recent ncurses, so we will have nothing
to do with them.

Best regards,
Bapt


signature.asc
Description: PGP signature


Move from termcap(5) to terminfo(5)

2020-05-07 Thread Baptiste Daroussin
Hello everyone,

I can't find any proper rationale in our history (maybe I missed it) which
explains why our ncurses is stuck on using termcap(5) instead of terminfo(5)
Except an argument in the Makefile that builds ncurses:
"Used instead of the hideous read_termcap.c abomination."

Which I do not find really useful.

I would like to make the move from termcap to terminfo which would give us the
bonus to be able to track terminfo sources from upstream aka ncurses and to
add and use tic(1).

Given the very few people that are actually maintaining the termcap database. I
don't think there is a good rationale at keeping our own format (as far as I
know everyone moved to terminfo(5)) and parser.

Without any strong arguments against it I will start working on that move by
next week.

If you have some knowledge you want to share: "be careful about this or that" I
would be more than happy to collect it, to make sure the transition is as smooth
as possible.

Best regards,
Bapt
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: sysutils/screen-ncurses port

2020-05-05 Thread Baptiste Daroussin
On Tue, May 05, 2020 at 03:24:15PM +0300, Slawa Olhovchenkov wrote:
> On Thu, Apr 30, 2020 at 03:04:49PM +0200, Baptiste Daroussin wrote:
> 
> > > > This way ports with random termcap info to add would be able to do it 
> > > > witho=
> > > > ut
> > > > the requirement to wait for a commit in base and a MFC.
> > > 
> > > This is probably outside of my scope at the moment but, yes, agreed.
> > > 
> > I will then.
> > I added that to my TODO
> 
> Can you also see at termcap/ncurses related code?
> Latest commit in base do unusable sh/tcsh  w/ emacs
> tramp mode and inside screen.

I am not aware of sh/tcsh being unsable at all. I heard about an emacs thingy
but not being an emacs guy myself I haven't reproduced.

I can have a look but I would need a more specific feedback with examples of
failures.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: sysutils/screen-ncurses port

2020-05-04 Thread Baptiste Daroussin
On Mon, May 04, 2020 at 06:35:03AM -0700, Cy Schubert wrote:
> In message <20200504072624.wlyd73pehq25t...@ivaldir.net>, Baptiste 
> Daroussin wr
> ites:
> > 
> >
> > --ma2vde2ykv3k7k6b
> > Content-Type: text/plain; charset=us-ascii
> > Content-Disposition: inline
> > Content-Transfer-Encoding: quoted-printable
> >
> > On Sun, May 03, 2020 at 01:10:58PM -0700, Cy Schubert wrote:
> > > In message <20200430130449.cwsf3x42o6w67...@ivaldir.net>, Baptiste=20
> > > Daroussin wr
> > > ites:
> > > >=20
> > > >
> > > > --mvhxgm4zl62unzlf
> > > > Content-Type: text/plain; charset=3Dus-ascii
> > > > Content-Disposition: inline
> > > > Content-Transfer-Encoding: quoted-printable
> > > >
> > > > On Thu, Apr 30, 2020 at 05:56:54AM -0700, Cy Schubert wrote:
> > > > > In message <20200430075337.3wdzglshhorcd...@ivaldir.net>, Baptiste=3D=
> > 20
> > > > > Daroussin wr
> > > > > ites:
> > > > > >=3D20
> > > > > >
> > > > > > --vwrr5drfobpkyvop
> > > > > > Content-Type: text/plain; charset=3D3Dus-ascii
> > > > > > Content-Disposition: inline
> > > > > > Content-Transfer-Encoding: quoted-printable
> > > > > >
> > > > > > On Wed, Apr 29, 2020 at 11:41:46AM -0700, Cy Schubert wrote:
> > > > > > > Would people be open to the idea of a sysutils/screen-ncurses por=
> > t th=3D
> > > > at=3D3D
> > > > > > =3D3D20
> > > > > > > depends on devel/ncurses instead of ncureses in base? The reason =
> > for =3D
> > > > this=3D3D
> > > > > > =3D3D20
> > > > > > > is there are screen.* terminfo entries in devel/ncurses that don'=
> > t ex=3D
> > > > ist =3D3D
> > > > > > in=3D3D20
> > > > > > > termcap(5). People who want that extra functionality would be adv=
> > ised=3D
> > > >  to=3D3D
> > > > > > =3D3D20
> > > > > > > install the alternative pkg or build the sysutils/screen port wit=
> > h th=3D
> > > > e=3D3D20
> > > > > > > appropriate option.
> > > > > > >=3D3D20
> > > > > > > Or, simply change the default from whatever ncurses is available =
> > to a=3D
> > > > lway=3D3D
> > > > > > s=3D3D20
> > > > > > > install devel/ncurses. People could always select one of the othe=
> > r op=3D
> > > > tion=3D3D
> > > > > > s.=3D3D20
> > > > > > > Personally, I'm not enamoured with this approach.
> > > > > >
> > > > > > I think it is a terrible idea, and we should fix the initial proble=
> > m in=3D
> > > > stea=3D3D
> > > > > > d of
> > > > > > workarounding it.
> > > > > >
> > > > > > 1/ why those are not in our termcap(5) ? they should be added if th=
> > ey a=3D
> > > > re
> > > > > > missing. and MFC asap (prior 11.4 and 12.2 would be nice)
> > > > >=3D20
> > > > > I came to this conclusion last night after sending this email thread =
> > oud=3D
> > > > =3D20
> > > > > and will test it some time today.
> > > > >=3D20
> > > > > >
> > > > > > 2/ we should allow our base ncurses to get informations from newer =
> > term=3D
> > > > cap(=3D3D
> > > > > > 5) if
> > > > > > needed.
> > > > > > So far the default TERMCAP is
> > > > > > ${HOME}/.termcap{,.db}:/etc/termcap{,.db}:/usr/share/misc/termcap{,=
> > =2Edb}
> > > > > >
> > > > > > First the user can be advise to point configure the $home/.termcap =
> > this=3D
> > > >  is =3D3D
> > > > > > for
> > > > > > quick now.
> > > >
> > > > that is in your scope via a pkg-message :D
> > > >
> > > > > >
> > > > > > Second for later futur proof mechanism we could modify our termcap =
> > read=3D
> > > > er (=3D3D
> > > > > > we
> > > > > > use our own, not the one in provided by ncurses). to be able to fet=
> > ch t=3D
> > > > ermc=3D3D
> > > > > > ap
> > > &

Re: sysutils/screen-ncurses port

2020-05-04 Thread Baptiste Daroussin
On Sun, May 03, 2020 at 01:10:58PM -0700, Cy Schubert wrote:
> In message <20200430130449.cwsf3x42o6w67...@ivaldir.net>, Baptiste 
> Daroussin wr
> ites:
> > 
> >
> > --mvhxgm4zl62unzlf
> > Content-Type: text/plain; charset=us-ascii
> > Content-Disposition: inline
> > Content-Transfer-Encoding: quoted-printable
> >
> > On Thu, Apr 30, 2020 at 05:56:54AM -0700, Cy Schubert wrote:
> > > In message <20200430075337.3wdzglshhorcd...@ivaldir.net>, Baptiste=20
> > > Daroussin wr
> > > ites:
> > > >=20
> > > >
> > > > --vwrr5drfobpkyvop
> > > > Content-Type: text/plain; charset=3Dus-ascii
> > > > Content-Disposition: inline
> > > > Content-Transfer-Encoding: quoted-printable
> > > >
> > > > On Wed, Apr 29, 2020 at 11:41:46AM -0700, Cy Schubert wrote:
> > > > > Would people be open to the idea of a sysutils/screen-ncurses port th=
> > at=3D
> > > > =3D20
> > > > > depends on devel/ncurses instead of ncureses in base? The reason for =
> > this=3D
> > > > =3D20
> > > > > is there are screen.* terminfo entries in devel/ncurses that don't ex=
> > ist =3D
> > > > in=3D20
> > > > > termcap(5). People who want that extra functionality would be advised=
> >  to=3D
> > > > =3D20
> > > > > install the alternative pkg or build the sysutils/screen port with th=
> > e=3D20
> > > > > appropriate option.
> > > > >=3D20
> > > > > Or, simply change the default from whatever ncurses is available to a=
> > lway=3D
> > > > s=3D20
> > > > > install devel/ncurses. People could always select one of the other op=
> > tion=3D
> > > > s.=3D20
> > > > > Personally, I'm not enamoured with this approach.
> > > >
> > > > I think it is a terrible idea, and we should fix the initial problem in=
> > stea=3D
> > > > d of
> > > > workarounding it.
> > > >
> > > > 1/ why those are not in our termcap(5) ? they should be added if they a=
> > re
> > > > missing. and MFC asap (prior 11.4 and 12.2 would be nice)
> > >=20
> > > I came to this conclusion last night after sending this email thread oud=
> > =20
> > > and will test it some time today.
> > >=20
> > > >
> > > > 2/ we should allow our base ncurses to get informations from newer term=
> > cap(=3D
> > > > 5) if
> > > > needed.
> > > > So far the default TERMCAP is
> > > > ${HOME}/.termcap{,.db}:/etc/termcap{,.db}:/usr/share/misc/termcap{,.db}
> > > >
> > > > First the user can be advise to point configure the $home/.termcap this=
> >  is =3D
> > > > for
> > > > quick now.
> >
> > that is in your scope via a pkg-message :D
> >
> > > >
> > > > Second for later futur proof mechanism we could modify our termcap read=
> > er (=3D
> > > > we
> > > > use our own, not the one in provided by ncurses). to be able to fetch t=
> > ermc=3D
> > > > ap
> > > > capabilities from /usr/local/share/misc/termcap/*.conf for example
> > > >
> > > > This way ports with random termcap info to add would be able to do it w=
> > itho=3D
> > > > ut
> > > > the requirement to wait for a commit in base and a MFC.
> > >=20
> > > This is probably outside of my scope at the moment but, yes, agreed.
> > >=20
> > I will then.
> > I added that to my TODO
> 
> There's already a utility in devel/ncurses called infotocap (and its 
> corresponding captoinfo) that already does this. Both are links to tic. Our 
> ncurses import includes tic. Looks like all that's needed is add it to 
> buildworld.
> 
> I can look at it later tonight. Seems like a quick win.
> 
That is not the point, tic won't work here except if create your own version or
use infotocap. Tic is for terminfo databases while we are still using the 
termcap for historical reason

Having both ncurses from ports and ncurses from base installed at the same time
can open a special can of worm so imho that is not really something we want to
look forward.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: sysutils/screen-ncurses port

2020-04-30 Thread Baptiste Daroussin
On Thu, Apr 30, 2020 at 05:56:54AM -0700, Cy Schubert wrote:
> In message <20200430075337.3wdzglshhorcd...@ivaldir.net>, Baptiste 
> Daroussin wr
> ites:
> > 
> >
> > --vwrr5drfobpkyvop
> > Content-Type: text/plain; charset=us-ascii
> > Content-Disposition: inline
> > Content-Transfer-Encoding: quoted-printable
> >
> > On Wed, Apr 29, 2020 at 11:41:46AM -0700, Cy Schubert wrote:
> > > Would people be open to the idea of a sysutils/screen-ncurses port that=
> > =20
> > > depends on devel/ncurses instead of ncureses in base? The reason for this=
> > =20
> > > is there are screen.* terminfo entries in devel/ncurses that don't exist =
> > in=20
> > > termcap(5). People who want that extra functionality would be advised to=
> > =20
> > > install the alternative pkg or build the sysutils/screen port with the=20
> > > appropriate option.
> > >=20
> > > Or, simply change the default from whatever ncurses is available to alway=
> > s=20
> > > install devel/ncurses. People could always select one of the other option=
> > s.=20
> > > Personally, I'm not enamoured with this approach.
> >
> > I think it is a terrible idea, and we should fix the initial problem instea=
> > d of
> > workarounding it.
> >
> > 1/ why those are not in our termcap(5) ? they should be added if they are
> > missing. and MFC asap (prior 11.4 and 12.2 would be nice)
> 
> I came to this conclusion last night after sending this email thread oud 
> and will test it some time today.
> 
> >
> > 2/ we should allow our base ncurses to get informations from newer termcap(=
> > 5) if
> > needed.
> > So far the default TERMCAP is
> > ${HOME}/.termcap{,.db}:/etc/termcap{,.db}:/usr/share/misc/termcap{,.db}
> >
> > First the user can be advise to point configure the $home/.termcap this is =
> > for
> > quick now.

that is in your scope via a pkg-message :D

> >
> > Second for later futur proof mechanism we could modify our termcap reader (=
> > we
> > use our own, not the one in provided by ncurses). to be able to fetch termc=
> > ap
> > capabilities from /usr/local/share/misc/termcap/*.conf for example
> >
> > This way ports with random termcap info to add would be able to do it witho=
> > ut
> > the requirement to wait for a commit in base and a MFC.
> 
> This is probably outside of my scope at the moment but, yes, agreed.
> 
I will then.
I added that to my TODO

Bestr regards,
Bapt


signature.asc
Description: PGP signature


Re: sysutils/screen-ncurses port

2020-04-30 Thread Baptiste Daroussin
On Wed, Apr 29, 2020 at 11:41:46AM -0700, Cy Schubert wrote:
> Would people be open to the idea of a sysutils/screen-ncurses port that 
> depends on devel/ncurses instead of ncureses in base? The reason for this 
> is there are screen.* terminfo entries in devel/ncurses that don't exist in 
> termcap(5). People who want that extra functionality would be advised to 
> install the alternative pkg or build the sysutils/screen port with the 
> appropriate option.
> 
> Or, simply change the default from whatever ncurses is available to always 
> install devel/ncurses. People could always select one of the other options. 
> Personally, I'm not enamoured with this approach.

I think it is a terrible idea, and we should fix the initial problem instead of
workarounding it.

1/ why those are not in our termcap(5) ? they should be added if they are
missing. and MFC asap (prior 11.4 and 12.2 would be nice)

2/ we should allow our base ncurses to get informations from newer termcap(5) if
needed.
So far the default TERMCAP is
${HOME}/.termcap{,.db}:/etc/termcap{,.db}:/usr/share/misc/termcap{,.db}

First the user can be advise to point configure the $home/.termcap this is for
quick now.

Second for later futur proof mechanism we could modify our termcap reader (we
use our own, not the one in provided by ncurses). to be able to fetch termcap
capabilities from /usr/local/share/misc/termcap/*.conf for example

This way ports with random termcap info to add would be able to do it without
the requirement to wait for a commit in base and a MFC.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: r358062(ncurses) breaks installed ports, howto check?

2020-02-25 Thread Baptiste Daroussin
On Tue, Feb 25, 2020 at 04:19:56AM -0500, Thomas Dickey wrote:
> On Mon, Feb 24, 2020 at 08:28:03PM -0500, Thomas Dickey wrote:
> > On Mon, Feb 24, 2020 at 06:35:16PM -0500, Thomas Dickey wrote:
> > > On Mon, Feb 24, 2020 at 06:25:30PM -0500, Thomas Dickey wrote:
> > > > On Tue, Feb 25, 2020 at 04:37:11AM +0900, Yasuhiro KIMURA wrote:
> > > > > From: "O. Hartmann" 
> > > > > Subject: r358062(ncurses) breaks installed ports, howto check?
> > > > > Date: Mon, 24 Feb 2020 20:19:59 +0100
> > > > > 
> > > > > > After r358062, many installed ports do not work anymore on several 
> > > > > > running systems (CURRENT).
> > > > > > /usr/src/UPDATING states one should reinstall all ncurses depending 
> > > > > > ports, but no hint is
> > > > > > given! Can someone mitigate this lack of information? Is there a 
> > > > > > simple way to check what
> > > > > > ports installed on a system rely on ncurses provided by the system?
> > > > > 
> > > > > Check thread starting with following message.
> > > > > 
> > > > > https://lists.freebsd.org/pipermail/freebsd-ports/2020-February/117710.html
> > > > 
> > > > That's a start, but it gives an overly-broad approach, saying that
> > > > anything linked to the ncurses library has to be recompiled.
> > > > 
> > > > The ABI change is just to the (upper-level) curses interface.
> > > > Most of the programs you'll have in ports use the (lower-level) termcap
> > > > or terminfo interfaces.
> > > > 
> > > > For example gettext uses terminfo (not curses).
> > > > 
> > > > Curses applications use initscr or newterm (nm helps).
> > > > I have a script which uses nm to tell me which interface is used.
> > > > 
> > > > Actually, in my own ports, I don't see any which would be affected,
> > > > since all of the curses applications are the utilities for ncurses
> > > > (or for my testing of ncurses).
> > > > 
> > > > Here's an example of what it tells me
> > > > (n5==ncurses5, tc=termcap, ti=terminfo):
> > > > 
> > > > ti  bison
> > > > n5*+ti  captoinfo
> > > > n5*+ti  captoinfo6
> > > > n5*+ti  clear
> > > > n5*+ti  clear6
> > > > n5+tc   ded
> > > > n5+ti   dialog4ports
> > > 
> > > actually this one isn't one of mine (needs to be recompiled)
> > > 
> > > But for the rest - recompiling would be a waste of time.
> > 
> > ...that's just looking at /usr/local/bin.  I see Millard's list
> > includes /usr/local/lib.  I have some of those:
> > 
> > c3+tc   libXvMCr600.so
> > tc  libedit.so
> > tc  libedit.so.0
> > ti  libgettextsrc.so
> > tc  libreadline.so
> > tc  libslang.so
> > ti  libtextstyle.so
> > c3+tc   libvulkan_radeon.so
> > 
> > that is, mesa-dri uses curses, but libedit and libreadline do not.
> > 
> > I have llvm80, but that doesn't live in either of /usr/local/bin
> > or /usr/local/lib.  It's in its own directory (with a script in
> > the former pointing there).  It uses curses (and is not a quick
> > recompile).
> 
> now... I discussed this briefly with the developer on the 20th of January.
> 
> ncurses can be configured/built to use the old binary interface (ABI 5),
> but he ran into some problem doing that, and decided to use ABI 6.
> 
> At the time, I had supposed that cost would be contained by the system
> builders, not considering the impact on users rebuilding ports.
> 
> Depending on where FreeBSD is along that path, it might be worth
> revisiting, to see exactly why the build with ABI 5 did not work.
> 
I think we should anyway move on on ABI 6, which is why I didn't insist on ABI 5
when I could not find the root cause of it not working. In the futur we will
update ncurses on more regular basis, and ABI 6 is what everyone else is using
so probably what is the most tested.

if I understand correctly upstream build system, staying on ABI 5 disable many
features that are available in ABI 6.

For those interested my current plan with ncurses is the following for freebsd
13.0:
- split terminfo from ncurses (like the upstream build system) so a futur change
  in high level ABI will impact less consumers
- only keep the widechar version of ncurses in base
- add the pkgconf file in base to ease the ports tree.

Again for people who don't want to rebuild everything compat12x package has been
created and can be used to have all binaries working.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: [CFT][patch] mandoc: don't segfault on empty tbl(1) continuation blocks

2019-07-17 Thread Baptiste Daroussin
On Wed, Jul 17, 2019 at 01:39:42PM +0300, Eygene Ryabinkin wrote:
> Baptiste, good day.
> 
> Wed, Jul 17, 2019 at 09:12:02AM +0200, Baptiste Daroussin wrote:
> > On Tue, Jul 16, 2019 at 10:31:24PM +0300, Eygene Ryabinkin wrote:
> > > Attached is the patch that makes built-in tbl(1) processor in
> > > mandoc to avoid dumping core when it renders the table with empty
> > > "T{ T}" block and horizontally-ruled table.
> >
> > Thank you for the patch! Have it been discussed with upstream? I
> > kind of remind something like this being reported to upstream, but I
> > haven't checked the status.
> 
> Was fixed:
>   https://mandoc.bsd.lv/cgi-bin/cvsweb/tbl_term.c.diff?r1=1.69=1.70
>   
> https://github.com/openbsd/src/commit/5f6e3232931ab08da9c8121d568c8207c0c4662c#diff-bc5842dc5d7897de7bdac08f74804c57
> A bit differently: people just check for dpn->string being NULL.
> 
> And there is another one NULL pointer fix,
>   https://mandoc.bsd.lv/cgi-bin/cvsweb/tbl_term.c.diff?r1=1.70=1.71
>   
> https://github.com/openbsd/src/commit/7514a273fe4561e94f1277f4ee5991c9af9cba2e#diff-bc5842dc5d7897de7bdac08f74804c57
> Can't trigger it with upstream's testcase,
>   
> https://mandoc.bsd.lv/cgi-bin/cvsweb/regress/tbl/layout/shortlines.in?rev=1.1=text/x-cvsweb-markup
>   
> https://raw.githubusercontent.com/openbsd/src/7514a273fe4561e94f1277f4ee5991c9af9cba2e/regress/usr.bin/mandoc/tbl/layout/shortlines.in
> since current FreeBSD's mandoc lacks this modification,
>   https://mandoc.bsd.lv/cgi-bin/cvsweb/tbl_term.c.diff?r1=1.68=1.69
>   
> https://github.com/openbsd/src/commit/b3e6a3251dfa92e66aa539518119564bd1945cc0#diff-bc5842dc5d7897de7bdac08f74804c57
> but I believe that 'cpp' still can be NULL and will try to see
> if it is triggerable.
> 
> So, the patch that corresponds to the upstream change is attached.
> 
> Nothing was released after 1.14.5 (yet).  What will be the route?
> Will you
>  - wait for the new release;
>- if yes, will you incorporate the testing part?
>  - if no, I think you will use the closer-to-upstream patch?
> 
> Thanks.

Thank you for the patch and the test case, with mandoc, usually I try to be as
close as upstream as possible (targetting 100% ;).

So my approach in such case is to move to a snapshot of their cvs tree (as soon
as it has the fix incorporated).

As for the test case, the best would be that this test ends up incorporated in
the upstream testsuite (note that I need to plug it into our test framework
one day) I added the tech mailing list of mandoc in CC to give a chance to Ingo
to step on this.

I will be off for a week starting tonight, but I will update to the latest
snapshot of mandoc once back.o
We can still integrate some test case of our own as well, and I will be happy to
integrate yours if not integrated in the upstream testsuite.

Best regards,
Bapt

> -- 
> Eygene Ryabinkin,,,^..^,,,
> [ Life's unfair - but root password helps!   | codelabs.ru ]
> [ 82FE 06BC D497 C0DE 49EC  4FF0 16AF 9EAE 8152 ECFB | freebsd.org ]

> mandoc: fix built-in tbl(1) processing of empty continuation blocks
> 
> Empty "T{ T}" (continuation) blocks produce NULL-valued string
> for their data block: getdata() allocates structure with string
> set to NULL and tbl_cdata() will just return when it sees
> the end ("T}") of the block without any further manipulations
> with dat->string.
> 
> This is completely legal; moreover, tbl.h specifies that for
> 'struct tbl_dat' the 'string' member is NULL when entry type
> is not TBL_DATA_DATA.  This is not so all the time, but one
> shouldn't rely on this.
> 
> The segfault in question was plain NULL pointer dereference
> triggered from tbl_term.c::tbl_hrule().
> 
> Added check for dpn->string not being NULL to be in sync
> with upstream,
>   https://mandoc.bsd.lv/cgi-bin/cvsweb/tbl_term.c.diff?r1=1.69=1.70
> Also added regression test to find such problems in the future.
> 
> The real-world case when manpage was provoking core dump
> is notmuch-config.1 for mail/notmuch port: it is auto-generated
> from reStructuredText, so has empty blocks at the places where
> it would be enough just to specify the empty value.
> 
> Index: usr.bin/mandoc/Makefile
> ===
> --- usr.bin/mandoc/Makefile   (revision 349971)
> +++ usr.bin/mandoc/Makefile   (working copy)
> @@ -101,4 +101,7 @@
>  CFLAGS.gcc+= -Wno-format
>  LIBADD=  openbsd z
>  
> +HAS_TESTS=
> +SUBDIR.${MK_TESTS}+= tests
> +
>  .include 
> Index: usr.bin/mandoc/tests/Makefile
> ===
> --- usr.bin/mandoc/te

Re: [CFT][patch] mandoc: don't segfault on empty tbl(1) continuation blocks

2019-07-17 Thread Baptiste Daroussin
On Tue, Jul 16, 2019 at 10:31:24PM +0300, Eygene Ryabinkin wrote:
> Good day.
> 
> Attached is the patch that makes built-in tbl(1) processor in mandoc
> to avoid dumping core when it renders the table with empty "T{ T}"
> block and horizontally-ruled table.
> 
> The simplest way to reproduce the issue is to either
>  - run 'man notmuch-config' with mail/notmuch installed;
>  - run 'mandoc tests/empty-table-cdata.1' against the attached
>test-only manpage.
> 
> With the patch applied, one can utilize 'make check': regression
> test was added.  Perhaps an invocation of
> {{{
> mtree -deU -f /usr/src/etc/mtree/BSD.tests.dist -p /usr/tests
> }}}
> will be needed to run 'make check' without remaking/installing
> the world.
> 
> The patch is for the fresh -CURRENT.  Be interested in any results
> of its application and usage.
> 
> Thanks!
> 
> P.S.: please, CC me: I am not subscribed to the list.
> -- 
> Eygene Ryabinkin,,,^..^,,,
> [ Life's unfair - but root password helps!   | codelabs.ru ]
> [ 82FE 06BC D497 C0DE 49EC  4FF0 16AF 9EAE 8152 ECFB | freebsd.org ]

Hello,

Thank you for the patch! Have it been discussed with upstream? I kind of remind
something like this being reported to upstream, but I haven't checked the
status.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: pkg: Cannot open /dev/null:No such file or directory

2019-06-12 Thread Baptiste Daroussin
On Tue, Jun 11, 2019 at 11:52:31AM -0700, Johannes Lundberg wrote:
> Hi
> 
> This is probably overkill but I've attached a diff that show my patches
> to image.sh. It's just a hack so far to make it do what I want and not
> meant as general purpose. Use the changes you need for your application.
> 
> Changes (only meant to work for building usb images on amd64 and i386)
> 
> - mount devfs so pkg will work properly
> - add "install packages from official repo" option so you won't have to
> build your own packages for each jail
> - downloaded packages from official repo is cached locally to avoid
> excessive downloads
> - hack for i386 so packages are installed properly
> - increase swap space to allow for kernel core dumps (for usb image)
> - execute post-install script "overlay.sh" in the jail if provided in
> the overlay folder
> 
> Bapt: Maybe you'll find some of these changes useful? :)
> 
The first I already committed days ago in poudriere, just I should have added it
as a patch to the poudriere version in ports, I will do it asap.

As for other improvements, sure they sounds like useful, would be happy to
review them if you make pull requests out of them!

Thanks a lot,
Bapt


signature.asc
Description: PGP signature


Re: pkg: Cannot open /dev/null:No such file or directory

2019-06-03 Thread Baptiste Daroussin
On Tue, Jun 04, 2019 at 07:32:16AM +0200, O. Hartmann wrote:
> Hello List,
> 
> lately I ran into a serious problem installing packages in a nanoBSD
> environment, in which the package repository server is "remotely" on site. The
> issue as documented below occurs on both 12-STABLE r348529 and CURRENT r348600
> and must have been introduced shortly, since the last known good installation
> with the environment of ours was on 21st May 2019.
> 
> As far as I know,, the package installation is performed via "chroot'ed"
> environment and somehow /dev/null is out of a sudden not accessible anymore
> while pkg tries to delegate some output to /dev/null.
> 
> What happened here?
> 
> Kind regards and thanks in advance,
> 
> oh
> 
> [...]
> All repositories are up to date.
> The following 10 package(s) will be affected (of 0 checked):
> 
> New packages to be INSTALLED:
> python3: 3_3 [zeit4]
> sudo: 1.8.27_1 [zeit4]
> devcpu-data: 1.22 [zeit4]
> python36: 3.6.8_2 [zeit4]
> readline: 8.0.0 [zeit4]
> indexinfo: 0.3.1 [zeit4]
> libffi: 3.2.1_3 [zeit4]
> gettext-runtime: 0.19.8.1_2 [zeit4]
> openldap-sasl-client: 2.4.47 [zeit4]
> cyrus-sasl: 2.1.27 [zeit4]
> 
> Number of packages to be installed: 10
> 
What is new is that pkg is using /dev/null as input when running script? this is
new since pkg 1.11 . Somehow this does not seems to be avaalaible in your
environement.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: What is evdev and autoloading?

2019-02-18 Thread Baptiste Daroussin
On Mon, Feb 18, 2019 at 07:12:24AM -0800, Rodney W. Grimes wrote:
> > On 2/18/19 12:06 PM, Stefan Blachmann wrote:
> > > On 2/18/19, Vladimir Kondratyev  wrote:
> > >> On 2019-02-17 21:03, Steve Kargl wrote:
> > >>> Anyone have insight into what evdev is?
> > >> evdev.ko is a small in-kernel library that makes all your input events
> > >> like keyboard presses libinput-compatible.
> > > 
> > > And libinput was created by the Freedesktop Wayland team to create
> > > pressure on OS people to make their systems Wayland-compatible.
> > > 
> > >>> I do not need nor what these modules loaded.
> > >> I think removing "option EVDEV_SUPPORT" from your kernel config should
> > >> disable most of evdev.ko dependencies
> > > 
> > > Shouldn't the EVDEV_SUPPORT default be off on FreeBSD anyway, as well
> > > as libinput not be part of the standard packages?
> > > 
> > > The Freedesktop Wayland team consists of people with the Kay Sievers
> > > mentality, which made Linus Torvalds ban his contributions. They do
> > > not care about the bugs they introduce, forcing others to clean up the
> > > mess they create.
> > > 
> > > I'd be glad if FreeBSD would keep clean of following that Wayland fad...
> > 
> > EVDEV_SUPPORT was enabled in GENERIC on 13 and 12-stable to improve 
> > input device handling in X and Wayland.  Not having it means that a lot 
> > of input devices stop working, or work much worse.
> > 
> > We in the FreeBSD Graphics Team are working very hard to improve the 
> > FreeBSD Desktop experience, since it is an avenue to recruit new users, 
> > and make current users use FreeBSD more.
> 
> Sadly your execution on that seems to be missing the mark,
> telling people they have to go get a port now to get drm working
> because it could not be maintained in base, and then telling them,
> oh, you need this new code in base so that it is so much easier
> to use graphical stuff this way.
> 
> These seem to be conflicting stories.
> 
You are missing the point, one does not evolve as fast as the other, meaning
one can be maintained within usual freebsd lifecycle, the other cannot or it
becomes very painful.

Bapt


signature.asc
Description: PGP signature


Re: WITH_CTF breaks CD loader: "File too big"

2018-12-02 Thread Baptiste Daroussin
On Sun, Dec 02, 2018 at 06:08:34PM +0300, Yuri Pankov wrote:
> Hi,
> 
> Building disc1.iso using `make release` and having WITH_CTF set in
> src.conf leads to "File too big" displayed when booting the image.
> 
> Would it make sense to build loader and related parts without CTF
> unconditionally as it doesn't look useful there?
> 

Fully agree with you

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: C.UTF-8 as default locale

2018-11-04 Thread Baptiste Daroussin
On Sun, Nov 04, 2018 at 09:35:48AM +0300, Yuri Pankov wrote:
> Hi,
> 
> Looking through https://wiki.freebsd.org/DevSummit/201810, I noticed the
> following entry:
> 
> - Make C.UTF-8 the default locale (conrad, dteske(installer))
> 
> As this sort of change is better done early, I have put togetger a
> simple change introducing C.UTF-8 locale using the same common LC_CTYPE
> map as other UTF-8 locales (and it's now the source of symlinks instead
> of en_US one), and having all other components use C locale:
> 
> https://reviews.freebsd.org/D17833
> 
> With that in place, next step is likely to be updating 'default' entry
> in /etc/login.conf to include 'charset=UTF-8' and 'lang=C.UTF-8', and
> changes to installer are likely to be not needed?
> 


Hey I never thought it could be done in such an easy way! don't know why,
I was always looking at it in a complex way, thus never started it.

Thank you very much, this looks fine to me! and that is imho a great addition

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: how to enforce one version of python

2018-09-11 Thread Baptiste Daroussin
On Tue, Sep 11, 2018 at 03:28:15PM +0100, tech-lists wrote:
> Hi,
> 
> There are a number of ports that seem to have their own preferential flavour
> of python, and some for example want to install python27 and python36 in the
> same place, and it's a pain when using portupgrade or similar tools.
> 
> I have this in my /etc/make.conf:
> 
> DEFAULT_VERSIONS+= python=2.7
> 
> Is this incorrect? I assume it must be, as for example devel/pylint
> (pylint-py27-1.9.2) wants to upgrade to pylint-py36-2.1.1
> 
That is because portupgrade is not flavor aware (or badly if it has been patched
to support flavors)

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: IGNORE_OSVERSION=yes -- can't install pkg

2018-05-07 Thread Baptiste Daroussin
On Sat, May 05, 2018 at 10:47:36AM -0600, Ian Lepore wrote:
> On Sat, 2018-05-05 at 08:26 -0700, Chris H wrote:
> > On Fri, 04 May 2018 22:57:52 -0700  said
> > 
> > > 
> > > I just setup a jail from a 12-CURRENT I built awhile ago. It has no
> > > ports
> > > tree. So I'm attempting
> > > to install svnlite. issuing pkg search svnlite returns
> > > The package management tool is not yet installed on your system.
> > > Do you want to fetch and install it now? [y/N]: y
> > > Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/
> > > latest,
> > > please wait...
> > > Verifying signature with trusted certificate
> > > pkg.freebsd.org.2013102301...
> > > done
> > > [12current.localhost] Installing pkg-1.10.5...
> > > Newer FreeBSD version for package pkg:
> > > To ignore this error set IGNORE_OSVERSION=yes
> > > - package: 1200062
> > > - running kernel: 1200054
> > > Allow missmatch now?[Y/n]:
> > > 
> > > Umm, what? Should I ignore this error? If so, why is there an error
> > > at all?
> > > I answered no. Guess I won't be able to use pkg(8) on this jail(8).
> > > :-(
> > > 
> > > --Chris
> > OK the only reference[1] I can find regarding this, indicates that
> > answering "Y"
> > to Allow missmatch now? resulted in an ABI mismatch that caused
> > pkg(8) to be
> > unusable.
> > This is on an older version of 12, so I don't have anything that
> > might have
> > appeared in UPDATING. I really need this jail to resolve accumulating
> > pr(1)'s
> > on ports(7) I maintain.
> > 
> > Thank you.
> 
> The difference between 1200062 and 1200054 isn't going to affect
> anything except modules which are intimate with kernel internals, such
> as video drivers or virtualbox type stuff.
> 
> IMO, this new version checking done by pkg(8) is just bad Bad BAD. The
> only control you get is a knob that tells you to ignore any version
> mismatch. There appears to be no option to get the historical worked-
> really-well behavior of ignoring mismatches of the minor version for
> people who track -current.
> 

Except you devs are looking at it with a -CURRENT usage in mind.

Most of our users are running releases.

And you end up with en issue when let's say FreeBSD 10.0 is EOLed then the
packages are now built on 10.1, if people continue running 10.0 because for
instance they missed the notice about the 10.0 being EOL, they end up installing
packages that may be broken: new libc symbols for example, new syscalls etc.

This check was one of the number 1 request over the last 3 years...
For all people running -CURRENT they can add IGNORE_OSVERSION=yes.

More over, I received so many false bug report because actually developpers were
reporting "pkg is broken!!!" because they run pkg upgrade on a current system
that was 6+ month old or running pkg upgrade just after an ABI change that I
consider this warning worth it.

The only thing I would accept considering here is an advice on how to make the
tests more smooth for -CURRENT users. I consider an IGNORE_OSVERSION to be good
enough.

I might change in next versions of pkg the runtime OSVERSION detection reading
/bin/ls binary to be replaced by uname(1) to make it more friendly with
incremental rebuild.

Bapt


signature.asc
Description: PGP signature


Re: [self base packages] pkg: packages for wrong OS version: FreeBSD:12:amd64

2018-01-18 Thread Baptiste Daroussin
On Thu, Jan 18, 2018 at 11:32:26AM -0500, Ryan Stone wrote:
> On Wed, Jan 10, 2018 at 1:53 PM, Baptiste Daroussin <b...@freebsd.org> wrote:
> > One has to specify pkg -o OSVERSION=1200055 to allow packages built on 
> > 1200055
> > to install on 1200054.
> 
> This workaround doesn't appear to work for pkg bootstrap:
> 
> # pkg -o OSVERSION=1200055 bootstrap
> The package management tool is not yet installed on your system.
> Do you want to fetch and install it now? [y/N]: y
> Bootstrapping pkg from
> pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest, please wait...
> Verifying signature with trusted certificate pkg.freebsd.org.2013102301... 
> done
> Installing pkg-1.10.4...
> pkg-static: Newer FreeBSD version for package pkg:
> - package: 1200055
> - running kernel: 1200054
> 
> Failed to install the following 1 package(s): /tmp//pkg.txz.ngJJEM

The bootstrap does not know about OSVERSION, but setting OSVERSION in your env
should fix it

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: [self base packages] pkg: packages for wrong OS version: FreeBSD:12:amd64

2018-01-11 Thread Baptiste Daroussin
On Thu, Jan 11, 2018 at 08:57:34AM -0700, Ian Lepore wrote:
> On Wed, 2018-01-10 at 19:53 +0100, Baptiste Daroussin wrote:
> > On Wed, Jan 10, 2018 at 09:29:04PM +0300, Boris Samorodov wrote:
> > > 
> > > Hi All,
> > > 
> > > I use self built base packages. Seems that I have a problem with pkg.
> > > Today I've got this:
> > > ===
> > > % sudo pkg update -f
> > > Updating FreeBSD-base repository catalogue...
> > > Fetching meta.txz: 100%268 B   0.3kB/s00:01
> > > Fetching packagesite.txz: 100%   29 KiB  29.4kB/s00:01
> > > Processing entries:   0%
> > > pkg: Newer FreeBSD version for package FreeBSD-libulog:
> > > - package: 1200055
> > > - running kernel: 1200054
> > > pkg: repository FreeBSD-base contains packages for wrong OS version:
> > > FreeBSD:12:amd64
> > > Processing entries: 100%
> > > Unable to update repository FreeBSD-base
> > > [...]
> > > 
> > > % uname -aKU
> > > FreeBSD latt.bsnet 12.0-CURRENT FreeBSD 12.0-CURRENT #2 r327719: Tue Jan
> > >  9 14:32:13 MSK 2018
> > > bsam@builder.bsnet:/usr/obj/usr/src/amd64.amd64/sys/PKG64X  amd64
> > > 1200054 1200054
> > > 
> > > % pkg -v
> > > 1.10.4
> > > 
> > hum
> > 
> > pkg now has a mechanism to protect from installing too new package (aka 
> > packages
> > built on a more recent system than the system you are running on.
> > 
> > While this is great for ports this is a bit painful for upgrading base 
> > packages
> > when building on current
> > 
> > One has to specify pkg -o OSVERSION=1200055 to allow packages built on 
> > 1200055
> > to install on 1200054.
> > 
> > I need to figure out a mechanism to make this simpler to handle to upgrade 
> > of
> > base system while keeping this safety belt for users.
> > 
> > Any idea is welcome
> > 
> > Best regards,
> > Bapt
> 
> Is this new mechanism disabled if installing into something other than
> the live system, such as when using -c or -r (and maybe -j)?
> 

The new mechanism grab the information from the files in the userland, meaning,
-c, -r and -j will discover the OSVERSION of the target binaries

Bapt


signature.asc
Description: PGP signature


Re: [self base packages] pkg: packages for wrong OS version: FreeBSD:12:amd64

2018-01-10 Thread Baptiste Daroussin
On Wed, Jan 10, 2018 at 09:29:04PM +0300, Boris Samorodov wrote:
> Hi All,
> 
> I use self built base packages. Seems that I have a problem with pkg.
> Today I've got this:
> ===
> % sudo pkg update -f
> Updating FreeBSD-base repository catalogue...
> Fetching meta.txz: 100%268 B   0.3kB/s00:01
> Fetching packagesite.txz: 100%   29 KiB  29.4kB/s00:01
> Processing entries:   0%
> pkg: Newer FreeBSD version for package FreeBSD-libulog:
> - package: 1200055
> - running kernel: 1200054
> pkg: repository FreeBSD-base contains packages for wrong OS version:
> FreeBSD:12:amd64
> Processing entries: 100%
> Unable to update repository FreeBSD-base
> [...]
> 
> % uname -aKU
> FreeBSD latt.bsnet 12.0-CURRENT FreeBSD 12.0-CURRENT #2 r327719: Tue Jan
>  9 14:32:13 MSK 2018
> bsam@builder.bsnet:/usr/obj/usr/src/amd64.amd64/sys/PKG64X  amd64
> 1200054 1200054
> 
> % pkg -v
> 1.10.4
> 
hum

pkg now has a mechanism to protect from installing too new package (aka packages
built on a more recent system than the system you are running on.

While this is great for ports this is a bit painful for upgrading base packages
when building on current

One has to specify pkg -o OSVERSION=1200055 to allow packages built on 1200055
to install on 1200054.

I need to figure out a mechanism to make this simpler to handle to upgrade of
base system while keeping this safety belt for users.

Any idea is welcome

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: what's changed with newsyslog?

2017-12-18 Thread Baptiste Daroussin
On Sun, Dec 17, 2017 at 04:41:29PM -0500, Michael Butler wrote:
> On 12/17/17 16:38, Ben Woods wrote:
> > On Mon, 18 Dec 2017 at 3:47 am, Michael Butler
> > > wrote:
> > 
> > In the past week or so I've been getting warnings like this ..
> > 
> > bzip2: Can't open input file /var/log/snmpd.log.0: No such file or
> > directory.
> > newsyslog: `/usr/bin/bzip2 -f /var/log/snmpd.log.0
> > /var/log/fwlw_clean_log.0' terminated with a non-zero status (1)
> 
>  [ .. snip .. ]
> 
> > 
> > 
> > Hints?
> > 
> >         imb
> > 
> > Could this be related to bapt’s recent change 11 days ago to allow
> > newsyslog to use different style of compression commands?
> > https://svnweb.freebsd.org/base?view=revision=326617
> 
> Yes - I reverted to r326616 and the issue no longer appears,
> 
>   imb
> 

Should be fixed in r326930, sorry about that

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: RFC: Removing hpt* drivers from GENERIC

2017-10-25 Thread Baptiste Daroussin
On Thu, Oct 26, 2017 at 03:12:57AM +0100, Andrew Turner wrote:
> 
> > On 25 Oct 2017, at 22:43, Colin Percival  wrote:
> > 
> > Hi developers,
> > 
> > I'd like to remove the hpt* drivers from GENERIC.  These are the drivers
> > for the HighPoint storage hardware -- SATA (hptnr) and RAID (hpt27xx, 
> > hptiop,
> > hptmv, hptrr).
> > 
> > My reason for wanting to remove them is that the hpt27xx and hptnr drivers
> > spend ~150 ms in their DEVICE_PROBE routines every time the system boots.
> > Since they are roughly 1000x slower than the median driver, this is clearly
> > excessive; unfortunately the time is being spent inside a binary blob, so
> > there is no apparent way to fix the drivers.  (The other three drives from
> > the same vendor -- hptiop, hptmv, and hptrr -- don't exhibit this particular
> > bug, but I don't see any strong argument in favour of not removing them 
> > along
> > with the two problem drivers.)
> > 
> > All of these are available via kernel modules, so the impact upon users
> > should be minimal.  Obviously I would not plan on MFCing this change.
> > 
> > Any objections?
GOGOGO
> 
> Why are we building these binary blobs into the kernel? We don’t have the 
> source for these so it’s more difficult to audit them for security issues. If 
> the user wishes to load them as modules they are fine to do that, however I 
> don’t think we shouldn’t be linking any of these blobs in a GENERIC kernel.

I totally agree here!

Bapt


signature.asc
Description: PGP signature


Re: kqueue(2) changes

2017-06-15 Thread Baptiste Daroussin
On Fri, Jun 02, 2017 at 10:45:16AM +0300, Konstantin Belousov wrote:
> I implemented an option to specify absolute time for kqueue(2) timers,
> and did required type changes to support larger values in struct kevent.
> Please see https://reviews.freebsd.org/D11025 for the patch, including
> man page update, and for some more detailed explanation.
> 
> Please review.

For me that looks good. As one of those requesting for that change (actually
proxying the request from glib people for a appel's kqueue64 compatible
interface). I find this approach more clever.

As far as I remember it perfectly fits glib folks requirements (unfortunatly I
could not reach out any of them to review yet)

Here is a mail describing their requirement at the time:
https://lists.freebsd.org/pipermail/freebsd-hackers/2014-March/044451.html

In any case it opens the doors to be able to use kevent based timers with a
nanonsecond precision useful, so looks good to me.

Thank you,

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: The futur of the roff toolchain

2017-05-25 Thread Baptiste Daroussin
On Thu, May 25, 2017 at 01:12:51PM +0100, tj wrote:
> On Thu, May 25, 2017 at 02:07:00PM +0200, Baptiste Daroussin wrote:
> > On Thu, May 25, 2017 at 11:45:19AM +, Glen Barber wrote:
> > > On Sun, May 21, 2017 at 02:57:33PM +0200, Baptiste Daroussin wrote:
> > > > No the problem left is documentations available in share/doc.
> > > > 
> > > > I would like to push them elsewhere. Those documents are mostly useful 
> > > > for
> > > > historical reason (hence we want to keep them) but not really for daily 
> > > > use of
> > > > modern FreeBSD.
> > > > Another issue with those documentation, they are installed as 
> > > > text/ascii version
> > > > in base, which makes most of them not really readable (as the documents 
> > > > has not
> > > > be written for a ascii/text target but more for a PDF/html view - using 
> > > > pic(1)
> > > > for example)
> > > > 
> > > > A plan was to push as sources in the svn doc repository and continue to 
> > > > build
> > > > them. This approach also have an issue: over the time roff evolved a 
> > > > bit and
> > > > while working on heirloom doctools import I had to fix a bunch of 
> > > > markup to make
> > > > the rendering of those documents clean (also meaning almost noone 
> > > > should read
> > > > them considering some were not really readable).
> > > > 
> > > > What I want to propose now, it to render them as PDF (html?) once and 
> > > > push them
> > > > somewhere (to be defined) as static document on our documentation 
> > > > website.
> > > > Please doceng@ provide me a location where to push them.
> > > > 
> > > 
> > > Unless anyone on doceng@ objects within the next three days, I will
> > > create a new directory for the PDFs under doc/head/en_US.ISO8859-1/.
> > > 
> > 
> > Thank you,
> > 
> > For the record I have pushed the generated PDF:
> > https://people.freebsd.org/~bapt/pdfdocs/
> > 
> > The have been built with a modern groff and I forced the embedded fonts for 
> > all
> > fonts used.
> > 
> 
> Multicolumn doesn't render correct in firefox for this:
> 
> https://people.freebsd.org/~bapt/pdfdocs/papers/timecounter.pdf
> 
> but is fine for this:
> 
> https://people.freebsd.org/~bapt/pdfdocs/papers/devfs.pdf
> 
> - [tj]

Fixed, and reuploaded. because there should be some caching on people.f.o you
cannot yet see the fixed version but I have fixed it. It is supposed to be a
monocolumn as far as I understand it.

Bapt


signature.asc
Description: PGP signature


Re: The futur of the roff toolchain

2017-05-25 Thread Baptiste Daroussin
On Thu, May 25, 2017 at 01:12:51PM +0100, tj wrote:
> On Thu, May 25, 2017 at 02:07:00PM +0200, Baptiste Daroussin wrote:
> > On Thu, May 25, 2017 at 11:45:19AM +, Glen Barber wrote:
> > > On Sun, May 21, 2017 at 02:57:33PM +0200, Baptiste Daroussin wrote:
> > > > No the problem left is documentations available in share/doc.
> > > > 
> > > > I would like to push them elsewhere. Those documents are mostly useful 
> > > > for
> > > > historical reason (hence we want to keep them) but not really for daily 
> > > > use of
> > > > modern FreeBSD.
> > > > Another issue with those documentation, they are installed as 
> > > > text/ascii version
> > > > in base, which makes most of them not really readable (as the documents 
> > > > has not
> > > > be written for a ascii/text target but more for a PDF/html view - using 
> > > > pic(1)
> > > > for example)
> > > > 
> > > > A plan was to push as sources in the svn doc repository and continue to 
> > > > build
> > > > them. This approach also have an issue: over the time roff evolved a 
> > > > bit and
> > > > while working on heirloom doctools import I had to fix a bunch of 
> > > > markup to make
> > > > the rendering of those documents clean (also meaning almost noone 
> > > > should read
> > > > them considering some were not really readable).
> > > > 
> > > > What I want to propose now, it to render them as PDF (html?) once and 
> > > > push them
> > > > somewhere (to be defined) as static document on our documentation 
> > > > website.
> > > > Please doceng@ provide me a location where to push them.
> > > > 
> > > 
> > > Unless anyone on doceng@ objects within the next three days, I will
> > > create a new directory for the PDFs under doc/head/en_US.ISO8859-1/.
> > > 
> > 
> > Thank you,
> > 
> > For the record I have pushed the generated PDF:
> > https://people.freebsd.org/~bapt/pdfdocs/
> > 
> > The have been built with a modern groff and I forced the embedded fonts for 
> > all
> > fonts used.
> > 
> 
> Multicolumn doesn't render correct in firefox for this:
> 
> https://people.freebsd.org/~bapt/pdfdocs/papers/timecounter.pdf
> 
> but is fine for this:
> 
> https://people.freebsd.org/~bapt/pdfdocs/papers/devfs.pdf
> 
Good catch thanks! I will see what I can do

Bapt


signature.asc
Description: PGP signature


Re: The futur of the roff toolchain

2017-05-25 Thread Baptiste Daroussin
On Thu, May 25, 2017 at 11:45:19AM +, Glen Barber wrote:
> On Sun, May 21, 2017 at 02:57:33PM +0200, Baptiste Daroussin wrote:
> > No the problem left is documentations available in share/doc.
> > 
> > I would like to push them elsewhere. Those documents are mostly useful for
> > historical reason (hence we want to keep them) but not really for daily use 
> > of
> > modern FreeBSD.
> > Another issue with those documentation, they are installed as text/ascii 
> > version
> > in base, which makes most of them not really readable (as the documents has 
> > not
> > be written for a ascii/text target but more for a PDF/html view - using 
> > pic(1)
> > for example)
> > 
> > A plan was to push as sources in the svn doc repository and continue to 
> > build
> > them. This approach also have an issue: over the time roff evolved a bit and
> > while working on heirloom doctools import I had to fix a bunch of markup to 
> > make
> > the rendering of those documents clean (also meaning almost noone should 
> > read
> > them considering some were not really readable).
> > 
> > What I want to propose now, it to render them as PDF (html?) once and push 
> > them
> > somewhere (to be defined) as static document on our documentation website.
> > Please doceng@ provide me a location where to push them.
> > 
> 
> Unless anyone on doceng@ objects within the next three days, I will
> create a new directory for the PDFs under doc/head/en_US.ISO8859-1/.
> 

Thank you,

For the record I have pushed the generated PDF:
https://people.freebsd.org/~bapt/pdfdocs/

The have been built with a modern groff and I forced the embedded fonts for all
fonts used.

Best regards,
Bapt


signature.asc
Description: PGP signature


The futur of the roff toolchain

2017-05-21 Thread Baptiste Daroussin
Hi all,

I have been working for a while to try to import a modern roff toolchain into
base.

I didn't like the initial approach that consisted in simply removing all roff
toolchain in base.

Recap of the situation in base:
* We have GNU roff version 1.19.2 in base (latest GPLv2 version). Lots of bug
  fixes has been made upstream in newer version (GPLv3) in particular regarding
  unicode but not only. (and we cannot update it anymore)
* GNU roff is now only used to generate the documentation in share/doc and as a
  fallback for manpages which mandoc does not support.

On the manpages front:
* No manpages in base are not supported by mandoc except groff manpages
  themselves
* man(1) can fallback on ports version of groff if installed (for ports not
  providing manpages not compatible with mandoc)

Alternatives to GNU roff:
* Heirloom doctools (which I tried to import) licensed both CDDL/BSD (in C)
* neartoff http://litcave.rudi.ir/ BSD licensed (in C)

I went the road of using heirloom doctools it is 90% compatible with GNU roff,
good enough for all our base roff based documents.

After getting down that road for a while, including lots of patches sent
upstream (thanks them for being so reactive and integrating them quickly as well
as fixing the issues I wasn't able to fix myself quickly).

The problem is there are lot of corner small corner cases where heirloom is
different from GNU roff and hard to make it compatible. While this is corner
cases, it breaks document generation for some large documents people are
writing. Those users could use (and actually would benefit a lot from it) GNU
roff from the ports tree, but have to be careful about the path of the tool they
call to ensure only calling the one from GNU roff and not the one (with the same
name) from heirloom doctools.

Concerning neatroff it is barely compatible with GNU roff, so not an option
(last I tested at least).

I would like to change this approach and get back to the initial approach taken
by others before I jumped in and I would like just entirely remove the roff
toolchain from base and let people rely on GNU roff from ports.

man(1) is already asking the user to install groff from ports if the manpage
cannot be read with mandoc.

No the problem left is documentations available in share/doc.

I would like to push them elsewhere. Those documents are mostly useful for
historical reason (hence we want to keep them) but not really for daily use of
modern FreeBSD.
Another issue with those documentation, they are installed as text/ascii version
in base, which makes most of them not really readable (as the documents has not
be written for a ascii/text target but more for a PDF/html view - using pic(1)
for example)

A plan was to push as sources in the svn doc repository and continue to build
them. This approach also have an issue: over the time roff evolved a bit and
while working on heirloom doctools import I had to fix a bunch of markup to make
the rendering of those documents clean (also meaning almost noone should read
them considering some were not really readable).

What I want to propose now, it to render them as PDF (html?) once and push them
somewhere (to be defined) as static document on our documentation website.
Please doceng@ provide me a location where to push them.

And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal of groff.
I also want to remove most of roff related tools (the one provided by toolchains
available in ports) for which we kept a BSD version (not really maintained in
base):
namely:
- checknr
- vgrind
- colcrt

Only keeping:
- col (useful in other places than roff)
- soelim (also used for manpages and we have a clean BSD licensed version which
  is also now parts of mandoc)

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: make concurrency kit a module

2017-05-17 Thread Baptiste Daroussin
On Wed, May 17, 2017 at 06:04:09PM -0700, Adrian Chadd wrote:
> https://reviews.freebsd.org/D10778
> 

Except there are plans to use it elsewhere. Many areas may be improved using it.

Having it as a module would mean some devs might refrain from using it because
there is no waranty for it to be there

Areas like VFS and network stack could have a good benefice from using it.

Out of curiousity what size is saved?

Bapt


signature.asc
Description: PGP signature


Re: NFSv2 boot & OLD_NFSV2

2017-03-21 Thread Baptiste Daroussin
On Tue, Mar 21, 2017 at 09:58:21AM +0200, Daniel Braniss wrote:
> 
> > On 20 Mar 2017, at 23:55, Toomas Soome <tso...@me.com> wrote:
> > 
> >> 
> >> On 20. märts 2017, at 23:53, Rick Macklem <rmack...@uoguelph.ca> wrote:
> >> 
> >> Baptiste Daroussin wrote:
> >>> On Mon, Mar 20, 2017 at 08:22:12PM +0200, Toomas Soome wrote:
> >>>> Hi!
> >>>> 
> >>>> The current boot code is building NFSv3, with preprocessor conditional 
> >>>> OLD_NFSV2. Should NFSv2 code still be kept around or can we burn it?
> >>>> 
> >>>> rgds,
> >>>> toomas
> >>> 
> >>> I vote burn
> >>> 
> >>> Bapt
> >> I would be happy to see NFSv2 go away. However, depending on how people 
> >> configure
> >> their diskless root fs, they do end up using NFSv2 for their root fs.
> >> 
> >> Does booting over NFSv3 affect this?
> >> 
> >> I think the answer is no for a FreeBSD server (since the NFSv2 File Handle 
> >> is the same as
> >> the NFSv3 one, except padded with 0 bytes to 32bytes long).
> >> However, there might be non-FreeBSD NFS servers where the NFSv2 file 
> >> handle is different
> >> than the NFSv3 one and for that case, the user would need NFSv2 boot code 
> >> (or
> >> reconfigure their root fs to use NFSv3).
> >> 
> >> To be honest, I suspect few realize that they are using NFSv2 for their 
> >> root fs.
> >> (They'd see it in a packet trace or via "nfsstat -m", but otherwise they 
> >> probably
> >> think they are using NFSv3 for their root fs.)
> >> 
> >> rick
> > 
> > if they do not suspect, they most likely use v3 - due to simple fact that 
> > you have to rebuild loader to use NFSv2 - it is compile time option.
> > 
> 
> old systems, 8.x, still use/boot v2, and so do old linux.
> NetApp has discontinued support for v2, so we had to move this machines to 
> use FreeBSD server and the day was
> saved. So, till these machines get upgraded/discontinued we have a problem. 
> There are several solutions
> to this issue, but as long as it's a matter of getting rid for the sake of 
> it, I would vote to keep it a while longer.
> 
> danny
> 
> 
Given you are speaking of 8.x I suppose you are using the loader that comes with
it, meaning you are safe if we remove it from the loader in 12.0 (note as said
by Toomas that is it is already off by default in the 12.0 loader) am I missing
something?

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: NFSv2 boot & OLD_NFSV2

2017-03-20 Thread Baptiste Daroussin
On Mon, Mar 20, 2017 at 08:22:12PM +0200, Toomas Soome wrote:
> Hi!
> 
> The current boot code is building NFSv3, with preprocessor conditional 
> OLD_NFSV2. Should NFSv2 code still be kept around or can we burn it?
> 
> rgds,
> toomas

I vote burn

Bapt


signature.asc
Description: PGP signature


Re: MANPATH not handled correctly

2017-03-10 Thread Baptiste Daroussin
On Sun, Jan 08, 2017 at 11:26:33AM -0800, Steve Kargl wrote:
> MANPATH is not handled correctly.  According to the documentation
> in apropos(1) and whatis(1):
> 
>   MANPATH   The standard search path used by man(1) may be changed by
> specifying a path in the MANPATH environment variable.  Invalid
> paths, or paths without manual databases, are ignored.
> Overridden by -M.  If MANPATH begins with a colon, it is
> appended to the default list; if it ends with a colon, it is
> prepended to the default list; or if it contains two adjacent
> colons, the standard search path is inserted between the
> colons.  If none of these conditions are met, it overrides the
> standard search path.
> 
> I have a manpage named mkpic in $HOME/man/man1.  I also have the FreeBSD
> installed manpages, e.g., /usr/share/man/man1/cat.1.gz.  If I have
> 'setenv MANPATH :$HOME/man' in my .cshrc file, then the following occurs:  
> 
> % setenv | grep MANPATH
> MANPATH=:/home/kargl/man
> % apropos mkpic
> (Warning: MANPATH environment variable set)
> mkpic(1) - construct a contour image in MIFF image format
> % apropos cat
> (Warning: MANPATH environment variable set)
> matrix(3) - Array and matrix allocation for FFT library
> 
> So, the above description of MANPATH is incorrect as :/home/kargl/man
> should have been appended to the default MANPATH.
> 
> Interestingly, manpath(1) seems to described what actually happens
> (long lines wrapped):
> 
> % unsetenv MANPATH
> % manpath
> /home/kargl/man:/usr/local/man:/usr/share/man:/usr/share/openssl/man:\
>/usr/local/lib/perl5/site_perl/man:/usr/local/lib/perl5/5.20/perl/man:\
>/usr/local/share/xpdf/man
> % setenv MANPATH :$HOME/sman
> % manpath
> (Warning: MANPATH environment variable set)
> :/home/kargl/man
> 
> The expected result according apropos(1) and whatis(1) for last command is
> 
> % manpath
> /home/kargl/man:/usr/local/man:/usr/share/man:/usr/share/openssl/man:\
>/usr/local/lib/perl5/site_perl/man:/usr/local/lib/perl5/5.20/perl/man:\
>/usr/local/share/xpdf/man:/home/kargl/man

Sorry it took time, but it is now fixed. the issue was we have kept our man(1)
when importing mandoc and I have not noticed that mandoc had that feature.

I implemented it in our man(1)

While here I removed the painful warning

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Problem with pkg

2017-02-27 Thread Baptiste Daroussin
On Mon, Feb 27, 2017 at 03:37:55PM +0100, Domagoj Stolfa wrote:
> Having the same issues, even after numerous attempts to manually fix it.
> 

This is a problem on the freebsd side a fix has been made, new repository should
be out soon

Only 12-CURRENT is inpacted right now
11.0-RELEASE amd64 was also inpacted but has been worked around

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Problem with x.org

2017-02-13 Thread Baptiste Daroussin
On Mon, Feb 13, 2017 at 12:40:41PM +, Filippo Moretti wrote:
> root@sting:~ # uname -a
> FreeBSD sting 12.0-CURRENT FreeBSD 12.0-CURRENT #5 r313678M: Sun Feb 12 
> 18:37:07 CET 2017 root@sting:/usr/obj/usr/src/sys/STING_VT  i386After 
> upgrading to the latest x.org server I cannot start X as a normal user but 
> only as a root.I enclose the error in Xorg.0.log.oldFilippops I checked 
> permission of /usr/local/bin/Xorg and are correct

How did you upgrade to newer Xorg stack ?
portmaster? portupgrade?

if yes please rebuild again, those 2 tools are known in this case to pick wrong
decision in the rebuild order that result in this segfault, also depending on
user reports, they might miss some necessary rebuild (don't know why)

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: recent change to vim defaults?

2017-01-18 Thread Baptiste Daroussin
On Wed, Jan 18, 2017 at 10:37:32AM -0700, Adam Weinberger wrote:
> > On 18 Jan, 2017, at 10:35, Julian Elischer <jul...@freebsd.org> wrote:
> > 
> > On 17/01/2017 1:23 AM, Adam Weinberger wrote:
> >>> On 16 Jan, 2017, at 9:25, Baptiste Daroussin <b...@freebsd.org> wrote:
> >>> 
> >>> On Mon, Jan 16, 2017 at 12:03:08AM +0800, Julian Elischer wrote:
> >>>> I noticed that suddenly vim is grabbing mouse movements, which makes life
> >>>> really hard.
> >>>> 
> >>>> Was there a specific revision that brought in this change, and can it be
> >>>> removed?
> >>> This change appeared in one of the last patchset of vim 7.4 and was one 
> >>> of the
> >>> "features" of the vim 8.0 release.
> >>> 
> >>> I do agree this is just totally painful :(
> >>> 
> >>> Best regards,
> >>> Bapt
> >> One of the things that I inherited with the Vim port was the DEFAULT_VIMRC 
> >> option (which installs /usr/ports/editors/vim/files/vimrc), and I haven't 
> >> touched it.
> >> 
> >> I have moused disabled in all my boxes so I have no idea about bad mouse 
> >> behaviour in Vim. If there is a bad default that is causing grief, let's 
> >> just fix it in that default vimrc.
> >> 
> >> I'm not really understanding what the unexpected behaviour is so I can't 
> >> make an intelligent recommendation myself, but I'll go with whatever you 
> >> folks suggest.
> >> 
> >> # Adam
> > I'm in iterm on my mac.
> > I ssh to a freebsd machine
> > I use vim on a file.
> > I used to be able to use the mouse on my mac to copy a few lines into the 
> > cut buffer.. slide-shift-click etc..
> > now suddenly if I try highlight some code in vim to copy it vim drags stuff 
> > around, scrolls up and down, deletes stuff and generally makes a mess.
> > if click, instead of starting a copy zone it grabs some of the text.
> > 
> > basically it makes hte mouse useless.
> > I can;t copy and paste from a file I'm ediitng. I end up having to exit vim 
> > and do it in vi.
> 
> There have been a number of recommendations in this thread for you, Julian, 
> including "set mouse=a" and "set mouse=v". Test some of them out and let me 
> know what works for you.

set mouse=
(with nothing) brings back the original behaviour

Bapt


signature.asc
Description: PGP signature


Re: recent change to vim defaults?

2017-01-16 Thread Baptiste Daroussin
On Mon, Jan 16, 2017 at 07:46:41PM +0300, Slawa Olhovchenkov wrote:
> On Mon, Jan 16, 2017 at 05:25:26PM +0100, Baptiste Daroussin wrote:
> 
> > On Mon, Jan 16, 2017 at 12:03:08AM +0800, Julian Elischer wrote:
> > > I noticed that suddenly vim is grabbing mouse movements, which makes life
> > > really hard.
> > > 
> > > Was there a specific revision that brought in this change, and can it be
> > > removed?
> > 
> > This change appeared in one of the last patchset of vim 7.4 and was one of 
> > the
> > "features" of the vim 8.0 release.
> > 
> > I do agree this is just totally painful :(
> 
> Wat about edit 8-bit files in utf-8 locale?
> Still corrupted files?

What are you speaking about I never had this issue with vim

We had this issue with vi in base which is now worked arounded so it just fails
so save instead of corrupting (which is still bad but a bit better :))

Bapt


signature.asc
Description: PGP signature


Re: recent change to vim defaults?

2017-01-16 Thread Baptiste Daroussin
On Mon, Jan 16, 2017 at 12:03:08AM +0800, Julian Elischer wrote:
> I noticed that suddenly vim is grabbing mouse movements, which makes life
> really hard.
> 
> Was there a specific revision that brought in this change, and can it be
> removed?

This change appeared in one of the last patchset of vim 7.4 and was one of the
"features" of the vim 8.0 release.

I do agree this is just totally painful :(

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: MANPATH not handled correctly

2017-01-10 Thread Baptiste Daroussin
On Sun, Jan 08, 2017 at 11:26:33AM -0800, Steve Kargl wrote:
> MANPATH is not handled correctly.  According to the documentation
> in apropos(1) and whatis(1):
> 
>   MANPATH   The standard search path used by man(1) may be changed by
> specifying a path in the MANPATH environment variable.  Invalid
> paths, or paths without manual databases, are ignored.
> Overridden by -M.  If MANPATH begins with a colon, it is
> appended to the default list; if it ends with a colon, it is
> prepended to the default list; or if it contains two adjacent
> colons, the standard search path is inserted between the
> colons.  If none of these conditions are met, it overrides the
> standard search path.
> 
> I have a manpage named mkpic in $HOME/man/man1.  I also have the FreeBSD
> installed manpages, e.g., /usr/share/man/man1/cat.1.gz.  If I have
> 'setenv MANPATH :$HOME/man' in my .cshrc file, then the following occurs:  
> 
> % setenv | grep MANPATH
> MANPATH=:/home/kargl/man
> % apropos mkpic
> (Warning: MANPATH environment variable set)
> mkpic(1) - construct a contour image in MIFF image format
> % apropos cat
> (Warning: MANPATH environment variable set)
> matrix(3) - Array and matrix allocation for FFT library
> 
> So, the above description of MANPATH is incorrect as :/home/kargl/man
> should have been appended to the default MANPATH.
> 
> Interestingly, manpath(1) seems to described what actually happens
> (long lines wrapped):
> 
> % unsetenv MANPATH
> % manpath
> /home/kargl/man:/usr/local/man:/usr/share/man:/usr/share/openssl/man:\
>/usr/local/lib/perl5/site_perl/man:/usr/local/lib/perl5/5.20/perl/man:\
>/usr/local/share/xpdf/man
> % setenv MANPATH :$HOME/sman
> % manpath
> (Warning: MANPATH environment variable set)
> :/home/kargl/man
> 
> The expected result according apropos(1) and whatis(1) for last command is
> 
> % manpath
> /home/kargl/man:/usr/local/man:/usr/share/man:/usr/share/openssl/man:\
>/usr/local/lib/perl5/site_perl/man:/usr/local/lib/perl5/5.20/perl/man:\
>/usr/local/share/xpdf/man:/home/kargl/man
> 
> Instead of (un)fixing the documentation for apropos(1) and whatis(1), it
> would be preferable to fix manpath to match the description in those
> manpages.  In addition, the Warning should be removed or at least an
> option should be available to suppress the (useless/annoying) Warning.
> This would restore man(1), apropros(1), and whatis(1) to its historical
> behavior prior to svn revision 213317.
> 
> If the documentation for apropos(1) and whatis(1) is unfixed, then manpath(1)
> should have HISTORY and BUGS sections.  The BUGS section should explicitly
> not that MANPATH is no longer a changeable environmental variable by a
> user without incurring the Warning.
> 

Sounds like a bug, I will have a look as soon as I can

Thanks for reporting,
Best regards,
Bapt


signature.asc
Description: PGP signature


Re: FreeBSD_HEAD_amd64_gcc - Build #1736 - Still Failing

2016-12-18 Thread Baptiste Daroussin
On Mon, Dec 19, 2016 at 12:01:45AM +0800, Li-Wen Hsu wrote:
> On Sun, Dec 18, 2016 at 15:39:50 +0100, Baptiste Daroussin wrote:
> > On Sun, Dec 18, 2016 at 03:18:47PM +0100, Dimitry Andric wrote:
> > > On 18 Dec 2016, at 12:48, jenkins-ad...@freebsd.org wrote:
> > > > 
> > > > FreeBSD_HEAD_amd64_gcc - Build #1736 - Still Failing:
> > > > 
> > > > Build information: 
> > > > https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1736/
> > > ...
> > > > --- atf-check.full ---
> > > > /usr/local/bin/x86_64-portbld-freebsd10.1-g++ -isystem 
> > > > /builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/include/c++/v1
> > > >  -std=c++11 -nostdinc++ -isystem 
> > > > /builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/include
> > > >  
> > > > -L/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> > > >  
> > > > -B/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> > > >  
> > > > --sysroot=/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp
> > > >  -B/usr/local/x86_64-freebsd/bin/ -O2 -pipe -DHAVE_CONFIG_H 
> > > > -I/builds/FreeBSD_HEAD_amd64_gcc/contrib/atf -DATF_SHELL='"/bin/sh"' -g 
> > > > -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W 
> > > > -Wno-unused-parameter -Wpointer-arith -Wno-uninitialized -Wformat=2 
> > > > -Wno-format-extra-args -Wno-error=address -Wno-error=array-bounds 
> > > > -Wno-error=attributes -Wno-error=bool-compare -Wno-error=cast-align 
> > > > -Wno-error=clobbered -Wno-error=enum-compare -Wno-error=extra 
> > > > -Wno-error=inline -Wno-error=logical-not-parentheses 
> > > > -Wno-error=strict-aliasing -Wno-error=uninitialized 
> > > > -Wno-error=unused-but-set-variable -Wno-error=unused-function 
> > > > -Wno-error=unused-value -Wno-error=strict-overflow 
> > > > -Wno-error=misleading-indentation -Wno-error=nonnull-compare 
> > > > -Wno-error=shift-negative-value -Wno-error=tautological-compare 
> > > > -Wno-error=unused-const-variable  -o atf-check.full  atf-check.o 
> > > > -lprivateatf-c++ -lprivateatf-c
> > > > --- all_subdir_lib ---
> > > > --- mulosi4.po ---
> > > > /usr/local/bin/x86_64-portbld-freebsd10.1-gcc -isystem 
> > > > /builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/include
> > > >  
> > > > -L/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> > > >  
> > > > -B/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> > > >  
> > > > --sysroot=/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp
> > > >  -B/usr/local/x86_64-freebsd/bin/ -pg  -O2 -pipe   -fpic 
> > > > -fvisibility=hidden -DVISIBILITY_HIDDEN 
> > > > -I/builds/FreeBSD_HEAD_amd64_gcc/contrib/libcxxrt -MD  
> > > > -MF.depend.mulosi4.po -MTmulosi4.po -std=gnu99 -fstack-protector-strong 
> > > > -Wsystem-headers -Wall -Wno-format-y2k -Wno-uninitialized 
> > > > -Wno-pointer-sign -Wno-error=address -Wno-error=array-bounds 
> > > > -Wno-error=attributes -Wno-error=bool-compare -Wno-error=cast-align 
> > > > -Wno-error=clobbered -Wno-error=enum-compare -Wno-error=extra 
> > > > -Wno-error=inline -Wno-error=logical-not-parentheses 
> > > > -Wno-error=strict-aliasing -Wno-error=uninitialized 
> > > > -Wno-error=unused-but-set-variable -Wno-error=unused-function 
> > > > -Wno-error=unused-value -Wno-error=strict-overflow 
> > > > -Wno-error=misleading-indentation -Wno-error=nonnull-compare 
> > > > -Wno-error=shift-negative-value -Wno-error=tautological-compare 
> > > > -Wno-error=unused-const-variable -c 
> > > > /builds/FreeBSD_HEAD_amd64_gcc/contrib/compiler-rt/lib/builtins/mulosi4.c
> > > >  -o mulosi4.po
> > > > --- all_subdir_bin ---
> > > > --- dd.1.gz ---
> > > > gzip -cn /builds/FreeBSD_HEAD_amd64_gcc/bin/dd/dd.1 > dd.1.gz
> > > > --- dd.debug ---
> > > > /usr/local/x86_64-freebsd/bin/objcopy --only-keep-debug dd.full dd.debug
> > > > --- dd ---
> > > > /usr/local/x86_64-freebsd/bin/objcopy --strip-debug 
> > > > --add-gnu-debuglink=dd.debug  dd.full dd
> > > > --- all_subdir_bin/df ---
> > > > ===> bin/df (all)
> > > > --- all_subdir_libexec ---
> > > > /usr/local/bin/x86_64-freebsd-ld: atf-check.o: relocation R_X86_64_32 
> > > > against symbol `_ZTIN3atf12system_errorE' can not be used when making a 
> > > > shared object; recompile with -fPIC
> > > > /usr/local/bin/x86_64-freebsd-ld: final link failed: Nonrepresentable 
> > > > section on output
> > > > collect2: error: ld returned 1 exit status
> > > > *** [atf-check.full] Error code 1
> > > 
> > > Baptiste, is this the problem again with -lstdc++ being linked in 
> > > accidentally again?
> > 
> > Should not be, is the object directory always cleaned before building? by
> > cleaned I mean rm -rf
> 

There was another mistake in amd64-gcc which I have just fixed and would make
updating to 6.2.0 wait another round of package build

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: FreeBSD_HEAD_amd64_gcc - Build #1736 - Still Failing

2016-12-18 Thread Baptiste Daroussin
On Sun, Dec 18, 2016 at 03:18:47PM +0100, Dimitry Andric wrote:
> On 18 Dec 2016, at 12:48, jenkins-ad...@freebsd.org wrote:
> > 
> > FreeBSD_HEAD_amd64_gcc - Build #1736 - Still Failing:
> > 
> > Build information: 
> > https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1736/
> ...
> > --- atf-check.full ---
> > /usr/local/bin/x86_64-portbld-freebsd10.1-g++ -isystem 
> > /builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/include/c++/v1
> >  -std=c++11 -nostdinc++ -isystem 
> > /builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/include
> >  
> > -L/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> >  
> > -B/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> >  
> > --sysroot=/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp
> >  -B/usr/local/x86_64-freebsd/bin/ -O2 -pipe -DHAVE_CONFIG_H 
> > -I/builds/FreeBSD_HEAD_amd64_gcc/contrib/atf -DATF_SHELL='"/bin/sh"' -g 
> > -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W 
> > -Wno-unused-parameter -Wpointer-arith -Wno-uninitialized -Wformat=2 
> > -Wno-format-extra-args -Wno-error=address -Wno-error=array-bounds 
> > -Wno-error=attributes -Wno-error=bool-compare -Wno-error=cast-align 
> > -Wno-error=clobbered -Wno-error=enum-compare -Wno-error=extra 
> > -Wno-error=inline -Wno-error=logical-not-parentheses 
> > -Wno-error=strict-aliasing -Wno-error=uninitialized 
> > -Wno-error=unused-but-set-variable -Wno-error=unused-function 
> > -Wno-error=unused-value -Wno-error=strict-overflow 
> > -Wno-error=misleading-indentation -Wno-error=nonnull-compare 
> > -Wno-error=shift-negative-value -Wno-error=tautological-compare 
> > -Wno-error=unused-const-variable  -o atf-check.full  atf-check.o 
> > -lprivateatf-c++ -lprivateatf-c
> > --- all_subdir_lib ---
> > --- mulosi4.po ---
> > /usr/local/bin/x86_64-portbld-freebsd10.1-gcc -isystem 
> > /builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/include
> >  
> > -L/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> >  
> > -B/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> >  
> > --sysroot=/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp
> >  -B/usr/local/x86_64-freebsd/bin/ -pg  -O2 -pipe   -fpic 
> > -fvisibility=hidden -DVISIBILITY_HIDDEN 
> > -I/builds/FreeBSD_HEAD_amd64_gcc/contrib/libcxxrt -MD  
> > -MF.depend.mulosi4.po -MTmulosi4.po -std=gnu99 -fstack-protector-strong 
> > -Wsystem-headers -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign 
> > -Wno-error=address -Wno-error=array-bounds -Wno-error=attributes 
> > -Wno-error=bool-compare -Wno-error=cast-align -Wno-error=clobbered 
> > -Wno-error=enum-compare -Wno-error=extra -Wno-error=inline 
> > -Wno-error=logical-not-parentheses -Wno-error=strict-aliasing 
> > -Wno-error=uninitialized -Wno-error=unused-but-set-variable 
> > -Wno-error=unused-function -Wno-error=unused-value 
> > -Wno-error=strict-overflow -Wno-error=misleading-indentation 
> > -Wno-error=nonnull-compare -Wno-error=shift-negative-value 
> > -Wno-error=tautological-compare -Wno-error=unused-const-variable -c 
> > /builds/FreeBSD_HEAD_amd64_gcc/contrib/compiler-rt/lib/builtins/mulosi4.c 
> > -o mulosi4.po
> > --- all_subdir_bin ---
> > --- dd.1.gz ---
> > gzip -cn /builds/FreeBSD_HEAD_amd64_gcc/bin/dd/dd.1 > dd.1.gz
> > --- dd.debug ---
> > /usr/local/x86_64-freebsd/bin/objcopy --only-keep-debug dd.full dd.debug
> > --- dd ---
> > /usr/local/x86_64-freebsd/bin/objcopy --strip-debug 
> > --add-gnu-debuglink=dd.debug  dd.full dd
> > --- all_subdir_bin/df ---
> > ===> bin/df (all)
> > --- all_subdir_libexec ---
> > /usr/local/bin/x86_64-freebsd-ld: atf-check.o: relocation R_X86_64_32 
> > against symbol `_ZTIN3atf12system_errorE' can not be used when making a 
> > shared object; recompile with -fPIC
> > /usr/local/bin/x86_64-freebsd-ld: final link failed: Nonrepresentable 
> > section on output
> > collect2: error: ld returned 1 exit status
> > *** [atf-check.full] Error code 1
> 
> Baptiste, is this the problem again with -lstdc++ being linked in 
> accidentally again?

Should not be, is the object directory always cleaned before building? by
cleaned I mean rm -rf

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: FreeBSD_HEAD_amd64_gcc - Build #1733 - Still Failing

2016-12-17 Thread Baptiste Daroussin
On Sat, Dec 17, 2016 at 06:29:09PM +0100, Dimitry Andric wrote:
> On 15 Dec 2016, at 18:56, Li-Wen Hsu  wrote:
> > 
> > On Thu, Dec 15, 2016 at 12:42:00 +0100, Dimitry Andric wrote:
> >> On 15 Dec 2016, at 12:03, jenkins-ad...@freebsd.org wrote:
> >>> 
> >>> FreeBSD_HEAD_amd64_gcc - Build #1733 - Still Failing:
> >>> 
> >>> Build information: 
> >>> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1733/
> >>> Full change log: 
> >>> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1733/changes
> >>> Full build log: 
> >>> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1733/console
> >> ...
> >>> building shared library libprivatedevdctl.so.0
> >>> /usr/local/bin/x86_64-portbld-freebsd10.1-g++ -isystem 
> >>> /builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/include/c++/v1
> >>>  -std=c++11 -nostdinc++ -isystem 
> >>> /builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/include
> >>>  
> >>> -L/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> >>>  
> >>> -B/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp/usr/lib
> >>>  
> >>> --sysroot=/builds/FreeBSD_HEAD_amd64_gcc/obj/builds/FreeBSD_HEAD_amd64_gcc/tmp
> >>>  -B/usr/local/x86_64-freebsd/bin/ -fstack-protector-strong -shared -Wl,-x 
> >>> -Wl,--fatal-warnings -Wl,--warn-shared-textrel  -o 
> >>> libprivatedevdctl.so.0.full -Wl,-soname,libprivatedevdctl.so.0  
> >>> `NM='/usr/local/x86_64-freebsd/bin/nm' NMFLAGS='' lorder consumer.pico 
> >>> event.pico event_factory.pico exception.pico guid.pico |  tsort -q`
> >>> /usr/local/bin/x86_64-freebsd-ld: cannot find -lstdc++
> >> 
> >> Dear Jenkins admins, can you please install a more recent version of
> >> amd64-xtoolchain-gcc on the slave?  This is required to avoid the above
> >> error about not being able to find libstdc++.
> > 
> > Also from the console log:
> > 
> > + sudo pkg install -y devel/amd64-xtoolchain-gcc
> > Updating FreeBSD repository catalogue...
> > FreeBSD repository is up-to-date.
> > All repositories are up-to-date.
> > Checking integrity... done (0 conflicting)
> > The most recent version of packages are already installed
> > 
> > The build script install/update the latest devel/amd64-xtoolchain-gcc
> > package.  I think the problem is we did not upgrade the dependencies.  I
> > cannot think of a elegant way to upgrade all amd64-xtoolchain-gcc's
> > dependencies, so I added a line to build script to upgrade everything
> > before build.
> > 
> > Let see if this works.
> 
> See https://jenkins.freebsd.org/job/FreeBSD_HEAD_amd64_gcc/1735/, it
> doesn't seem to be able to find the correct package:
> 
> + sudo pkg install -y devel/amd64-xtoolchain-gcc
> Updating FreeBSD repository catalogue...
> FreeBSD repository is up-to-date.
> All repositories are up-to-date.
> pkg: No packages available to install matching 'devel/amd64-xtoolchain-gcc' 
> have been found in the repositories
> 
yes it was broken, I have fixed it since, so would be in the next update of the
reposirory

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: CURRENT: pkg broken on r309320

2016-11-30 Thread Baptiste Daroussin
On Wed, Nov 30, 2016 at 12:10:31PM +, Matthew Seaman wrote:
> On 2016/11/30 10:14, Baptiste Daroussin wrote:
> > revert r309314 the issue is there:
> > 
> > The ports tree defines PKG_CMD as pkg register but now bsd.own.mk 
> > predefines it
> > to simply pkg.
> > 
> > There is a collision there.
> > 
> 
> Pointy hat to me.  I'll revert that commit.
> 
>   Matthew
> 

I have proposed https://reviews.freebsd.org/D8677 instead which make sense and
does not require your to modify bsd.own.mk

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: CURRENT: pkg broken on r309320

2016-11-30 Thread Baptiste Daroussin
On Wed, Nov 30, 2016 at 10:22:28AM +0100, O. Hartmann wrote:
> Running CURRENT, r309320 and try to update ports. After the port has been 
> build, pkg
> fails to install the port quitting with:
> 
> pkg: illegal option -- i
> DBG(1)[67562]> pkg initialized
> pkg: unknown command: /usr/ports/www/libwww/work/stage
> 
> For more information on available commands and options see 'pkg help'.
> *** Error code 64
> 
> Stop.
> make: stopped in /usr/ports/www/libwww
> 
> Seems there has been introduced a bug since rr309298
> 
> Kind regards,
> Oliver

revert r309314 the issue is there:

The ports tree defines PKG_CMD as pkg register but now bsd.own.mk predefines it
to simply pkg.

There is a collision there.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: r308432: Capsicumized `basename` make zsh prompt broken

2016-11-27 Thread Baptiste Daroussin
On Mon, Nov 28, 2016 at 02:05:49AM -0500, Allan Jude wrote:
> On 2016-11-27 23:55, Conrad Meyer wrote:
> > Hi Iblis,
> > 
> > I see no such problem running 'basename $HOME' in a normal shell 
> > environment:
> > 
> >> $ basename $HOME
> >> cmeyer
> > 
> > I suppose in your use, perhaps stdin is already closed?  I think this
> > is a limitation of caph_limit_stdio() in general.
> > 
> > Can you try instead:
> > 
> > function set_prompt {
> > prompt="$(basename $HOME < /dev/null) >"
> > }
> > 
> > And see if it resolves the issue?
> > 
> > Thanks,
> > Conrad
> > 
> > On Sun, Nov 27, 2016 at 8:33 PM, iblis  wrote:
> >> Hi,
> >> Here is a minimal config of zsh prompt invoking `basename`:
> >> ```
> >> └─[iblis@abeing]% cat /home/ib-test/.zshenv
> >>
> >> function set_prompt {
> >> prompt="$(basename $HOME) >"
> >> }
> >>
> >> function zle-line-init zle-keymap-select {
> >> set_prompt
> >> zle reset-prompt
> >> }
> >>
> >> zle -N zle-line-init
> >> zle -N zle-keymap-select
> >>
> >> set_prompt
> >> ```
> >>
> >> and launching zsh will get something like this:
> >>
> >> ```
> >> └─[iblis@abeing]% sudo su ib-test
> >>
> >> ib-test >basename: capsicum: Bad file descriptor
> >>>
> >>> basename: capsicum: Bad file descriptor
> >>>
> >> ```
> >>
> >>
> >> To be honest, I have no idea about what casper/caspicum is. I just changed
> >> the `basename.c` and zsh work again.
> >>
> >> Index: basename.c
> >> ===
> >> --- basename.c (revision 309213)
> >> +++ basename.c (working copy)
> >> @@ -65,7 +65,7 @@
> >>
> >> setlocale(LC_ALL, "");
> >>
> >> - if (caph_limit_stdio() < 0 || (cap_enter() < 0 && errno != ENOSYS))
> >> + if (cap_enter() < 0 && errno != ENOSYS)
> >> err(1, "capsicum");
> >>
> >> aflag = 0;
> >>
> >>
> >> Any idea?
> >>
> >> --
> >> Iblis Lin
> >> ___
> >> freebsd-current@freebsd.org mailing list
> >> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> >> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> > 
> 
> IIRC, bapt@ specifically mentioned this case in the review for
> caph_limit_stdio() or one of the reviews that lead to the creation of
> the helpers.

I mention this is the review of the cap_helpers themselves. I still think
caph_limit_stdio should grow a flag for testing that. I figured out the issue
based on one of the conversion to capsicum by Conrad on I don't remember which
tool.

Bapt


signature.asc
Description: PGP signature


Re: installworld fails on missing tzsetup when WITHOUT_DIALOG is set

2016-10-23 Thread Baptiste Daroussin
On Sun, Oct 23, 2016 at 11:41:23AM +0300, Guy Yur wrote:
> On Sat, Oct 22, 2016 at 7:23 PM, Baptiste Daroussin <b...@freebsd.org> wrote:
> > On Sat, Oct 22, 2016 at 06:51:28PM +0300, Guy Yur wrote:
> >> Hi,
> >> ...
> >
> > My proposal is a bit different: build tzsetup without dialog support :)
> >
> > https://reviews.freebsd.org/D8325
> >
> > Best regards,
> > Bapt
> 
> Thanks.

FYI it is in

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: installworld fails on missing tzsetup when WITHOUT_DIALOG is set

2016-10-22 Thread Baptiste Daroussin
On Sat, Oct 22, 2016 at 06:51:28PM +0300, Guy Yur wrote:
> Hi,
> 
> installworld fails on missing tzsetup when src.conf has WITHOUT_DIALOG=
> and delete-old was previously run to remove tzsetup from the system.
> 
> mkdir -p /tmp/install.8gNIwAFV
> progs=$(for prog in [ awk cap_mkdb cat chflags chmod chown cmp cp
> date echo egrep find grep id install   ln make mkdir mtree mv pwd_mkdb
>  rm sed services_mkdb sh strip sysctl test true uname wc zic tzsetup
> makewhatis; do  if progpath=`which $prog`; then  echo $progpath;  else
>  echo "Required tool $prog not found in PATH." >&2;  exit 1;  fi;
> done);  libs=$(ldd -f "%o %p\n" -f "%o %p\n" $progs 2>/dev/null | sort
> -u |  while read line; do  set -- $line;  if [ "$2 $3" != "not found"
> ]; then  echo $2;  else  echo "Required library $1 not found." >&2;
> exit 1;  fi;  done);  cp $libs $progs /tmp/install.8gNIwAFV
> Required tool tzsetup not found in PATH.
> *** Error code 1
> 
> tzsetup is used in share/zoneinfo/Makefile when ${DESTDIR}/var/db/zoneinfo
> exists and some other conditions.
> 
> In my case, I don't have /var/db/zoneinfo since I manually created a symlink
> from /usr/share/zoneinfo/... to /etc/localtime instead of using tzsetup.
> 
> A possible fix is to add a WITHOUT_TZSETUP knob and not use
> tzsetup when the knob is enabled.
> 
> https://github.com/guyyur/freebsd-src_patches/blob/master/without_tzsetup_knob.patch
> (patch doesn't include regenerating src.conf.5)
> 
> Thanks,
> Guy
> ___

My proposal is a bit different: build tzsetup without dialog support :)

https://reviews.freebsd.org/D8325

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: [RFC] remove GNU rcs from FreeBSD 12

2016-09-11 Thread Baptiste Daroussin
On Sun, Sep 11, 2016 at 10:17:26AM -0600, Warner Losh wrote:
> On Sun, Sep 11, 2016 at 7:38 AM, Baptiste Daroussin <b...@freebsd.org> wrote:
> > hi,
> >
> > For long we are planning to remove GNU rcs from base, after a failed attempt
> > before FreeBSD 10.0. Let see where we are to be able to remove it from 
> > FreeBSD
> > 12.
> >
> > GNU rcs is a GPLv2 software with newer version being GPLv3 preventing any
> > updates/fixes.
> >
> > From previous discussions there were issues that has been raised in previous
> > attempts:
> > - ident(1) is still useful given we still have Keywords in our sources. It 
> > has
> >   been replaced by a BSD Licensed version (enhanced to improve compatibility
> >   with Subversion Keyword) for FreeBSD 11. So that tool will remain in base
> >   after removal of GNU rcs.
> 
> So no affect.
> 
> > - etc-update uses merge(1) from GNU rcs, this has been changed in head to 
> > use
> >   diff3 instead.
> 
> Also no effect. Is our diff3 still the gpl'd one, or has bsd-diff
> finally grown that functionality?

Not yet bsd diff and bsd diff3 are very far from mature, however the sdiff(1)
part has landed as I have finished it. I still have plans for other diff
components.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: [RFC] remove GNU rcs from FreeBSD 12

2016-09-11 Thread Baptiste Daroussin
On Sun, Sep 11, 2016 at 03:38:04PM +0200, Baptiste Daroussin wrote:
> hi,
> 
> For long we are planning to remove GNU rcs from base, after a failed attempt
> before FreeBSD 10.0. Let see where we are to be able to remove it from FreeBSD
> 12.
> 
> GNU rcs is a GPLv2 software with newer version being GPLv3 preventing any
> updates/fixes.
> 
> From previous discussions there were issues that has been raised in previous
> attempts:
> - ident(1) is still useful given we still have Keywords in our sources. It has
>   been replaced by a BSD Licensed version (enhanced to improve compatibility
>   with Subversion Keyword) for FreeBSD 11. So that tool will remain in base
>   after removal of GNU rcs.
> - etc-update uses merge(1) from GNU rcs, this has been changed in head to use
>   diff3 instead.
> - rc.subr allows to use rcs for the backup file functionality. This
>   functionality is off by default as such I plan to make a warning if rcs is 
> not
>   installed and recommand to install rcs from base (or if noone claim using 
> the
>   feature I will just remove the functionality and only keep the default
>   behaviour aka keep one backup copy).
> - people uses rcs to handle configuration files in /etc for example. for those
>   multiple compatible alternatives are available in ports:
>   * rcs57: a copy of the latest version of GNU rcs in base before removal
> (GPLv2)
>   * rcs: latest GNU rcs version (GPLv3)
> 
> I haven't gone the direction of importing OpenRCS (BSD licensed version from
> OpenBSD) as it needs way more work to be 100% compatible with latest version 
> of
> GNU rcs.
> 
> How to proceed:
> - First turn off GNU rcs by default for a couple of month.
> - Totally remove GNU rcs if no blockers has been raised.
> 
> Best regards,
> Bapt

I forgot to say freebsd-update uses merge(1) and which I will replaced with
diff3(1).

Best regards,
Bapt


signature.asc
Description: PGP signature


[RFC] remove GNU rcs from FreeBSD 12

2016-09-11 Thread Baptiste Daroussin
hi,

For long we are planning to remove GNU rcs from base, after a failed attempt
before FreeBSD 10.0. Let see where we are to be able to remove it from FreeBSD
12.

GNU rcs is a GPLv2 software with newer version being GPLv3 preventing any
updates/fixes.

From previous discussions there were issues that has been raised in previous
attempts:
- ident(1) is still useful given we still have Keywords in our sources. It has
  been replaced by a BSD Licensed version (enhanced to improve compatibility
  with Subversion Keyword) for FreeBSD 11. So that tool will remain in base
  after removal of GNU rcs.
- etc-update uses merge(1) from GNU rcs, this has been changed in head to use
  diff3 instead.
- rc.subr allows to use rcs for the backup file functionality. This
  functionality is off by default as such I plan to make a warning if rcs is not
  installed and recommand to install rcs from base (or if noone claim using the
  feature I will just remove the functionality and only keep the default
  behaviour aka keep one backup copy).
- people uses rcs to handle configuration files in /etc for example. for those
  multiple compatible alternatives are available in ports:
  * rcs57: a copy of the latest version of GNU rcs in base before removal
(GPLv2)
  * rcs: latest GNU rcs version (GPLv3)

I haven't gone the direction of importing OpenRCS (BSD licensed version from
OpenBSD) as it needs way more work to be 100% compatible with latest version of
GNU rcs.

How to proceed:
- First turn off GNU rcs by default for a couple of month.
- Totally remove GNU rcs if no blockers has been raised.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: date(1) default format changed between 10.3 and 11.0-BETA3

2016-08-06 Thread Baptiste Daroussin
On Sat, Aug 06, 2016 at 02:15:36PM +1000, Greg 'groggy' Lehey wrote:
> On Friday,  5 August 2016 at 18:56:33 +0300, Andrey A. Chernov wrote:
> > On 05.08.2016 18:44, Mark Martinec wrote:
> >> On 2016-08-05 17:23, Andrey Chernov wrote:
> >>> On 05.08.2016 17:47, Mark Martinec wrote:
>  [Bug 211598]
>    date(1) default format in en_EN locale breaks compatibility with 10.3
>  and violates POSIX
> 
>  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211598
> >>>
> >>> It breaks compatibility but not violates POSIX. POSIX care of only its
> >>> own POSIX (or C) locale.
> >>
> >> POSIX does say that the default format should be the same
> >> as with "+%a %b %e %H:%M:%S %Z %Y".
> >> It also says that %a and %b are locale's abbreviated names.
> >
> > It is true for _POSIX_ locale only, as I already say. en_US.* is not
> > POSIX or C locale.
> 
> It still violates POLA.
> 
I really do not think that it violates POLA fiven that the behaviour you are
expecting is still available in the default configurtion that is still POSIX.

Set LC_TIME to C and then you are back on your behaviour (and this is the
default when you install FreeBSD).

locales should be seen as tzdata for exemple, they are a moving target
complicated to handle for every locale we do support: 78 for 11.0-RELEASE and
193 if we do count the encoding variants. locales are updated very often (for
exemple cldr unicode make a new release of the data every 8 month or so)

No locales defines the same date format and that was already the case before the
change we did

Now if people have strong arguments for a specific locale I'm inclined to add
some hacks in our tool that generates our locales to make sure we fix the
upstream data (http://cldr.unicode.org) we already committed some and I'm
planning to report upstream (cldr) all the issues we have faced to improve.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: IWM(7260), no connect

2016-07-28 Thread Baptiste Daroussin
On Thu, Jul 28, 2016 at 06:10:26PM -0500, Larry Rosenman wrote:
> negative.  no reponse (except for this one) at all.
> :(
> 
He did reply:

https://lists.freebsd.org/pipermail/freebsd-wireless/2016-July/006897.html

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Weekday missing in date(1) output

2016-07-23 Thread Baptiste Daroussin
On Mon, Jul 04, 2016 at 01:45:11PM +0200, Trond Endrestøl wrote:
> On Mon, 4 Jul 2016 08:34+0200, Baptiste Daroussin wrote:
> 
> > On Sun, Jul 03, 2016 at 10:17:54PM +0200, Thomas Eberhardt wrote:
> > > % uname -a
> > > FreeBSD clarence.ocp.lan 11.0-ALPHA6 FreeBSD 11.0-ALPHA6 #0 r302324: Sun 
> > > Jul  3 21:27:41 CEST 2016 
> > > tho...@clarence.ocp.lan:/usr/obj/usr/src/sys/GENERIC-NODEBUG  amd64
> > > % env LANG=en_US.UTF-8 date
> > > Sunday, July  3, 2016 at 10:14:21 PM CEST
> > > % env LANG=de_DE.UTF-8 date
> > >  3. Juli 2016 um 22:14:22 CEST
> > > 
> > > stable10 gives:
> > > % env LANG=de_DE.UTF-8 date
> > > So  3 Jul 2016 19:34:18 CEST
> > > 
> > > Is this intentional?
> > > 
> > I have readded the weekday in most locales, I will recheck all of them to 
> > ensure
> > the weekday is readded everywhere it was expected
> 
> nb_NO.UTF-8 and nb_NO.ISO8859-1 are also affected.
> 
Sorry it took long, but done: r303219

I will merge it on 11 tomorrow

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: UTF-8 by default?

2016-07-20 Thread Baptiste Daroussin
On Wed, Jul 20, 2016 at 10:36:13PM +0200, Herbert J. Skuhra wrote:
> Baptiste Daroussin skrev:
> > 
> > On Wed, Jul 20, 2016 at 10:47:45AM -0230, Jonathan Anderson wrote:
> >> On 20 Jul 2016, at 9:13, Tim Čas wrote:
> >> 
> >> > So, without further ado:
> >> > 1) What are the reasons that UTF-8 isn't the default yet?
> >> > 2) Would it be possible to make this the default in 11.0? What about
> >> > 12.0?
> >> > 3) Assuming an effort is started towards making UTF-8 the default,
> >> > what changes would be required?
> >> 
> >> At least according to one of my students (who makes more extensive use of
> >> i18n than I do), enabling UTF-8 by default is pretty straightforward:
> >> 
> >> https://github.com/musec/freebsd/wiki/Common-setup#utf-8-support
> > 
> > the LC_COLLATE=C is not needed anymore with freebsd 11+
> 
> Weird, when I set LC_ALL or LC_COLLATE to nb_NO.UTF-8 I can no longer
> run ls in $HOME. It hangs until I kill it. Same happens with
> da_DK.UTF-8; no problem with sv_SE.UTF-8 or en_GB.UTF-8.
> 
> I am running FreeBSD 11.0-BETA1 i386 (r302984) and I think someone
> else reported a similiar issue earlier.

This has been fixed in stable/11 just a few minutes ago

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: UTF-8 by default?

2016-07-20 Thread Baptiste Daroussin
On Wed, Jul 20, 2016 at 09:22:23PM +0200, Tim Čas wrote:
> On 20 July 2016 at 20:33, Don Lewis  wrote:
> > wc(1) has problems with its multibyte support pointed out by Coverity
> > as I recall.
> 
> Not sure how critical that issue is (e.g. byte counts [`-c`], line
> counts [`-l`], and such should still work as intended; whether word
> counts work or not depends on whether we should count Unicode
> whitespace as, well, whitespace). I do wonder if everyone agrees that
> an effort should be made towards UTF-8 default, though?
> 
> I'm willing to contribute some of my time to fixing these bugs, but I
> don't think I can fix *all* of this by myself. I guess wc(1) is as
> good a start as any, but I'd first like to talk to whoever is the
> maintainer for that bit of code, as I've never done any work in base
> before (only in ports).

good I would recommand to have a look at work done in OpenBSD in that regards,
since about a year Ingo Schwarze is atting UTF-8 support to all the tool in
their base system, including wc(1) not sure how theirs differs from our.

If working on that do not hesitate to push the changes you do propose in
phabricator:
https://reviews.freebsd.org and add me as a reviewer

https://wiki.freebsd.org/CodeReview might be useful to determine how to simply
use phabricator.

Do not hesitate to mail me if you need any help in that area.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: UTF-8 by default?

2016-07-20 Thread Baptiste Daroussin
On Wed, Jul 20, 2016 at 08:38:14PM +0200, Joel Dahl wrote:
> On Wed, Jul 20, 2016 at 04:07:41PM +0200, Baptiste Daroussin wrote:
> > On Wed, Jul 20, 2016 at 10:47:45AM -0230, Jonathan Anderson wrote:
> > > On 20 Jul 2016, at 9:13, Tim Čas wrote:
> > > 
> > > > So, without further ado:
> > > > 1) What are the reasons that UTF-8 isn't the default yet?
> > > > 2) Would it be possible to make this the default in 11.0? What about
> > > > 12.0?
> > > > 3) Assuming an effort is started towards making UTF-8 the default,
> > > > what changes would be required?
> > > 
> > > At least according to one of my students (who makes more extensive use of
> > > i18n than I do), enabling UTF-8 by default is pretty straightforward:
> > > 
> > > https://github.com/musec/freebsd/wiki/Common-setup#utf-8-support
> > 
> > the LC_COLLATE=C is not needed anymore with freebsd 11+
> > > 
> > > If there's anything missing there, I'd love to hear about it.
> > > 
> >   - unicode support in our old groff is pretty bad, I plan to replace it 
> > with
> > heirloom-doctools which does handle unicode propertly (as far I have 
> > tested
> > at least)
> 
> I haven't really been paying attention lately so things might have changed,
> but why can't we just remove groff now? We have mdocml, and for people that
> really need the groff functionality can just install it or heirloom-doctools
> from ports. The initial plan was to remove groff after we imported mdocml, but
> it was never removed and I lost interest, so again, things might have changed
> since then.

We have roff documentation in based, plus a lot of people argues that not havin
a roff toolchain in base is an issue for them.

heirloom doctools upstream is friendly, they fixed all the bugs I reported or
merged my fixes if needed, they have a good compatibility so the fallback in
man(1) could be done on something in base if mandoc cannot render properly some
manpages.

Upstream is CDDL but all new code is BSD licensed.

Importing is not hard, just need some motivation to finish all the required
makefiles :)

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: UTF-8 by default?

2016-07-20 Thread Baptiste Daroussin
On Wed, Jul 20, 2016 at 10:47:45AM -0230, Jonathan Anderson wrote:
> On 20 Jul 2016, at 9:13, Tim Čas wrote:
> 
> > So, without further ado:
> > 1) What are the reasons that UTF-8 isn't the default yet?
> > 2) Would it be possible to make this the default in 11.0? What about
> > 12.0?
> > 3) Assuming an effort is started towards making UTF-8 the default,
> > what changes would be required?
> 
> At least according to one of my students (who makes more extensive use of
> i18n than I do), enabling UTF-8 by default is pretty straightforward:
> 
> https://github.com/musec/freebsd/wiki/Common-setup#utf-8-support

the LC_COLLATE=C is not needed anymore with freebsd 11+
> 
> If there's anything missing there, I'd love to hear about it.
> 

Lot of work has been done during the 11.0 development the following issues were
fixed:

/bin/sh not able to handle utf-8 (fixed by fixing the bug in libedit)
no unicode collation: fixed but still very fresh code
vi: there was a potential corruption when opening a file in an encoding which is
not unicode in a unicode env, now is does not corrupt anything anymore but still
says it is unhappy
finger(1) has been fixed for multibytes names (I know noone care about that one
:))

On the list of still known issues:
* important:
  - csh does not handle unicode
  - regex in libc: it does not handle unicode right (except if I have missed
something) and needs to be either fixed either switch to libtre + custom
patches (there was a summer of code about it long ago and dfly went that
way)
  - unicode support in our old groff is pretty bad, I plan to replace it with
heirloom-doctools which does handle unicode propertly (as far I have tested
at least)
  - edit(1) does not handle multibyte

* medium (minor?)
  - login(1) does not handle unicode properly

* minor:
  - lots of base tools (minor one like nl and friends are not multibyte
aware in lot of cases, probably merging the work done by Ingo Schwarze on
those tools on OpenBSD might be useful, but I have no plan to do it)
  - vi needs improvement in multiencoding support I haven't checked the latest
modification on vi upstream about that

There might be more, but that is all that comes out of my head right now

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: ls eat high CPU time when LANG=zh_CN.UTF-8 and LC_ALL=zh_CN.UTF-8

2016-07-15 Thread Baptiste Daroussin
On Sun, Jul 10, 2016 at 04:23:03PM +0800, Huang Wen Hui wrote:
> I can reproduce it on a clean 11.0-BETA1 VM.
> 

fixed in r302916.

Will merge in 2 days in stable/11

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: ls eat high CPU time when LANG=zh_CN.UTF-8 and LC_ALL=zh_CN.UTF-8

2016-07-10 Thread Baptiste Daroussin
On Sun, Jul 10, 2016 at 04:23:03PM +0800, Huang Wen Hui wrote:
> I can reproduce it on a clean 11.0-BETA1 VM.
> 
> 
> 2016-07-09 9:03 GMT+08:00 Huang Wen Hui :
> 
> > For some reasons, r302324 seems not include in 11.0-ALPHA6?
> >
> > 2016-07-09 8:52 GMT+08:00 Huang Wen Hui :
> >
> >> Revert back r302324, Chinese locale problem is gone.
> >>
> >> Cheers
> >> Huang Wen Hui
> >>
Thanks, I can reproduce now, I'm looking into it.

Bapt


signature.asc
Description: PGP signature


Re: ls eat high CPU time when LANG=zh_CN.UTF-8 and LC_ALL=zh_CN.UTF-8

2016-07-05 Thread Baptiste Daroussin
On Tue, Jul 05, 2016 at 12:16:42PM +0800, Huang Wen Hui wrote:
> These 2 files can make ls suck:
> 
> touch 火灾1
> touch 火灾2
> 
> 2 files start with 2 same Chinese chars.
> 
I cannot reproduce on my head laptop, neither on a clean 11.0-ALPHA6 jail.

I'll try on a clean 11.0-ALPHA6 VM

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: ls eat high CPU time when LANG=zh_CN.UTF-8 and LC_ALL=zh_CN.UTF-8

2016-07-04 Thread Baptiste Daroussin
On Mon, Jul 04, 2016 at 02:51:46PM +0800, Huang Wen Hui wrote:
> 2016-07-04 14:41 GMT+08:00 Baptiste Daroussin <b...@freebsd.org>:
> 
> > On Mon, Jul 04, 2016 at 02:36:11PM +0800, Huang Wen Hui wrote:
> > > 2016-07-04 14:20 GMT+08:00 Baptiste Daroussin <b...@freebsd.org>:
> > >
> > > > On Mon, Jul 04, 2016 at 11:56:36AM +0800, Huang Wen Hui wrote:
> > > > > Hi,
> > > > > On very recent CURRENT, ls can eat high CPU time when
> > LANG=zh_CN.UTF-8
> > > > and
> > > > > LC_ALL=zh_CN.UTF-8:
> > > > >
> > > > > % uname -a
> > > > > FreeBSD mbp.gddsn.org.cn 11.0-ALPHA6 FreeBSD 11.0-ALPHA6 #121
> > r302331M:
> > > > Mon
> > > > > Jul  4 10:47:27 CST 2016 r...@mbp.gddsn.org.cn:
> > > > /usr/obj/usr/src/sys/MACBOOK
> > > > >  amd64
> > > > >
> > > > > top show:
> > > > > 4457 hwh   1 1000 16784K  4416K CPU44   0:22  98.86%
> > ls
> > > > >
> > > > > any ideas?
> > > > >
> > > > Is it in all directories or only in directories with files in chinese
> > > > characters?
> > > >
> > > Yes, the  directory contain Chinese characters.
> > >
> > > >
> > > > Is it only happening when you run ls with some arguments (in particular
> > > > -l) or
> > > > with any arguments?
> > > >
> > > I use  ls -wGl
> > >
> > > >
> > > > Do you see the same if you force any other locale like en_US.UTF-8?
> > > >
> > > There is no problem if set  en_US.UTF-8.
> > >
> > >
> > > > Best regards,
> > > > Bapt
> > > >
> >
> > Can you try:
> > env -i LANG=zh_CN.UTF-8 LC_COLLATE=C ls -l
> >
> > And tell me if it still happen?
> >
> No problem with this command.
> 

Ok so there might be an very inefficient code in the new chinese collation code
I will look into it thanks a lot for reporting.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: ls eat high CPU time when LANG=zh_CN.UTF-8 and LC_ALL=zh_CN.UTF-8

2016-07-04 Thread Baptiste Daroussin
On Mon, Jul 04, 2016 at 02:36:11PM +0800, Huang Wen Hui wrote:
> 2016-07-04 14:20 GMT+08:00 Baptiste Daroussin <b...@freebsd.org>:
> 
> > On Mon, Jul 04, 2016 at 11:56:36AM +0800, Huang Wen Hui wrote:
> > > Hi,
> > > On very recent CURRENT, ls can eat high CPU time when LANG=zh_CN.UTF-8
> > and
> > > LC_ALL=zh_CN.UTF-8:
> > >
> > > % uname -a
> > > FreeBSD mbp.gddsn.org.cn 11.0-ALPHA6 FreeBSD 11.0-ALPHA6 #121 r302331M:
> > Mon
> > > Jul  4 10:47:27 CST 2016 r...@mbp.gddsn.org.cn:
> > /usr/obj/usr/src/sys/MACBOOK
> > >  amd64
> > >
> > > top show:
> > > 4457 hwh   1 1000 16784K  4416K CPU44   0:22  98.86% ls
> > >
> > > any ideas?
> > >
> > Is it in all directories or only in directories with files in chinese
> > characters?
> >
> Yes, the  directory contain Chinese characters.
> 
> >
> > Is it only happening when you run ls with some arguments (in particular
> > -l) or
> > with any arguments?
> >
> I use  ls -wGl
> 
> >
> > Do you see the same if you force any other locale like en_US.UTF-8?
> >
> There is no problem if set  en_US.UTF-8.
> 
> 
> > Best regards,
> > Bapt
> >

Can you try:
env -i LANG=zh_CN.UTF-8 LC_COLLATE=C ls -l

And tell me if it still happen?

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Weekday missing in date(1) output

2016-07-04 Thread Baptiste Daroussin
On Sun, Jul 03, 2016 at 10:17:54PM +0200, Thomas Eberhardt wrote:
> % uname -a
> FreeBSD clarence.ocp.lan 11.0-ALPHA6 FreeBSD 11.0-ALPHA6 #0 r302324: Sun Jul  
> 3 21:27:41 CEST 2016 
> tho...@clarence.ocp.lan:/usr/obj/usr/src/sys/GENERIC-NODEBUG  amd64
> % env LANG=en_US.UTF-8 date
> Sunday, July  3, 2016 at 10:14:21 PM CEST
> % env LANG=de_DE.UTF-8 date
>  3. Juli 2016 um 22:14:22 CEST
> 
> stable10 gives:
> % env LANG=de_DE.UTF-8 date
> So  3 Jul 2016 19:34:18 CEST
> 
> Is this intentional?
> 
I have readded the weekday in most locales, I will recheck all of them to ensure
the weekday is readded everywhere it was expected

Bapt


signature.asc
Description: PGP signature


Re: ls eat high CPU time when LANG=zh_CN.UTF-8 and LC_ALL=zh_CN.UTF-8

2016-07-04 Thread Baptiste Daroussin
On Mon, Jul 04, 2016 at 11:56:36AM +0800, Huang Wen Hui wrote:
> Hi,
> On very recent CURRENT, ls can eat high CPU time when LANG=zh_CN.UTF-8 and
> LC_ALL=zh_CN.UTF-8:
> 
> % uname -a
> FreeBSD mbp.gddsn.org.cn 11.0-ALPHA6 FreeBSD 11.0-ALPHA6 #121 r302331M: Mon
> Jul  4 10:47:27 CST 2016 
> r...@mbp.gddsn.org.cn:/usr/obj/usr/src/sys/MACBOOK
>  amd64
> 
> top show:
> 4457 hwh   1 1000 16784K  4416K CPU44   0:22  98.86% ls
> 
> any ideas?
> 
Is it in all directories or only in directories with files in chinese
characters?

Is it only happening when you run ls with some arguments (in particular -l) or
with any arguments?

Do you see the same if you force any other locale like en_US.UTF-8?

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: pkg SAT_SOLVER bugs

2016-06-28 Thread Baptiste Daroussin
On Tue, Jun 28, 2016 at 05:52:32PM +0200, Hans Petter Selasky wrote:
> On 06/27/16 13:55, Baptiste Daroussin wrote:
> > On Mon, Jun 27, 2016 at 12:38:02PM +0200, Hans Petter Selasky wrote:
> > > Hi,
> > > 
> > > I found some bugs in PKG with regard to the SAT_SOLVER environment 
> > > variable.
> > > Please find patch attached :-)
> > > 
> > > Issues fixed:
> > > 1) No need to use hash table when generating SAT rules for external 
> > > solver.
> > > Variables are already in a linear array. Fix encoding and decoding of SAT
> > > data.
> > > 2) Endless variable loop caused pkg to crash.
> > > 3) it->inverse was checked for non-zero, while it should actually be 
> > > checked
> > > for -1 only. SAT rules produces were all negative.
> > > 
> > > How to verify:
> > > 
> > > make -C /usr/ports/math/picosat all install clean
> > > 
> > > env SAT_SOLVER=picosat pkg upgrade
> > > 
> > > --HPS
> > 
> > Thank you I will look into shortly
> > 
> 
> Hi Baptiste,
> 
> Are you handling this one or do you want me to create an issue at github.
> Thank you!
> 
I am handling it

BTW not that picosat is the one we use internally already :D

Best regards
Bapt


signature.asc
Description: PGP signature


Re: pkg SAT_SOLVER bugs

2016-06-27 Thread Baptiste Daroussin
On Mon, Jun 27, 2016 at 12:38:02PM +0200, Hans Petter Selasky wrote:
> Hi,
> 
> I found some bugs in PKG with regard to the SAT_SOLVER environment variable.
> Please find patch attached :-)
> 
> Issues fixed:
> 1) No need to use hash table when generating SAT rules for external solver.
> Variables are already in a linear array. Fix encoding and decoding of SAT
> data.
> 2) Endless variable loop caused pkg to crash.
> 3) it->inverse was checked for non-zero, while it should actually be checked
> for -1 only. SAT rules produces were all negative.
> 
> How to verify:
> 
> make -C /usr/ports/math/picosat all install clean
> 
> env SAT_SOLVER=picosat pkg upgrade
> 
> --HPS

Thank you I will look into shortly

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Date formatting with en_US locale

2016-06-17 Thread Baptiste Daroussin
On Fri, Jun 17, 2016 at 01:42:22PM -0500, Eric van Gyzen wrote:
> On 05/26/16 10:15 AM, Baptiste Daroussin wrote:
> > On Thu, May 26, 2016 at 11:55:08AM -0300, Otacílio wrote:
> >> Em 26/05/2016 11:49, Baptiste Daroussin escreveu:
> >>> On Thu, May 26, 2016 at 09:44:25AM -0500, Eric van Gyzen wrote:
> >>>> Baptiste and -current,
> >>>>
> >>>> I noticed two annoyances with date formatting on head, and I wonder how
> >>>> we can fix them.
> >>>>
> >>>> I have these settings:
> >>>>
> >>>>  LC_ALL=en_US.ISO8859-1
> >>>>  LANG=en_US.ISO8859-1
> >>>>
> >>>> First, Thunderbird displays the date as, for example:
> >>>>
> >>>>  03/ 6/16 ...
> >>>>
> >>>> The leading space on the day (6) looks weird.  I might even say it's
> >>>> simply wrong.  Zero-padding would better.  (/No/ padding would be best,
> >>>> but I don't think strftime supports that.)
> >>>>
> >>>> Second, date(1) no longer shows the day-of-week:
> >>>>
> >>>>  $ date
> >>>>  March 26, 2016 at 09:21:55 AM CDT
> >>>>
> >>>> For many years, I have been typing "date" to see the day-of-week (and
> >>>> other things).  I like the new human-friendly format, but I miss the
> >>>> day-of-week.
> >>>>
> >>>> Of course, I can fix these locally, but I wonder how we can fix them for
> >>>> everyone.  I see that the formats come from CLDR.  I also see that ume@
> >>>> restored the day-of-week for ja_JP in r292512.  Is this the best
> >>>> approach, or should we try to get them changed upstream (CLDR)?
> >>>>
> >>>> Thanks for your input,
> >>> I can hack cldr2def.pl to readd the week of day as it was before for 11.0 
> >>> still
> >>> the best approach is to push the change upstream.
> >>>
> >>> I will have a look at the cldr2def.pl hack this week end.
> >>>
> >>> Best regards,
> >>> Bapt
> >> LANG=pt_BR.UTF-8
> >>
> >> MM_CHARSET=UTF-8
> >>
> > By adding the hack I mean to do it for all locales not cherry picking
> 
> Thank you for fixing this!
> 
> Above, I mentioned two issues.  The other one is, the date format for
> en_US pads the month with a zero, but the day with a space.  So, June 7 is:
> 
> 06/ 7/16
> 
> That looks weird.  It should pad both with zeros.  I'd be happy to fix
> it, but I don't see how:  There isn't an "xformat" callback in the
> cldr2def.pl script, and it's not clear how to add one.  If you can
> explain, I'll do it.  If you can fix it, I'll be grateful.  ;)
> 
> Eric

This one I do not see how to fix, I'll dig into it but I promiss nothing

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Date formatting with en_US locale

2016-05-26 Thread Baptiste Daroussin
On Thu, May 26, 2016 at 11:55:08AM -0300, Otacílio wrote:
> Em 26/05/2016 11:49, Baptiste Daroussin escreveu:
> > On Thu, May 26, 2016 at 09:44:25AM -0500, Eric van Gyzen wrote:
> >> Baptiste and -current,
> >>
> >> I noticed two annoyances with date formatting on head, and I wonder how
> >> we can fix them.
> >>
> >> I have these settings:
> >>
> >>  LC_ALL=en_US.ISO8859-1
> >>  LANG=en_US.ISO8859-1
> >>
> >> First, Thunderbird displays the date as, for example:
> >>
> >>  03/ 6/16 ...
> >>
> >> The leading space on the day (6) looks weird.  I might even say it's
> >> simply wrong.  Zero-padding would better.  (/No/ padding would be best,
> >> but I don't think strftime supports that.)
> >>
> >> Second, date(1) no longer shows the day-of-week:
> >>
> >>  $ date
> >>  March 26, 2016 at 09:21:55 AM CDT
> >>
> >> For many years, I have been typing "date" to see the day-of-week (and
> >> other things).  I like the new human-friendly format, but I miss the
> >> day-of-week.
> >>
> >> Of course, I can fix these locally, but I wonder how we can fix them for
> >> everyone.  I see that the formats come from CLDR.  I also see that ume@
> >> restored the day-of-week for ja_JP in r292512.  Is this the best
> >> approach, or should we try to get them changed upstream (CLDR)?
> >>
> >> Thanks for your input,
> > I can hack cldr2def.pl to readd the week of day as it was before for 11.0 
> > still
> > the best approach is to push the change upstream.
> >
> > I will have a look at the cldr2def.pl hack this week end.
> >
> > Best regards,
> > Bapt
> 
> LANG=pt_BR.UTF-8
> 
> MM_CHARSET=UTF-8
> 
By adding the hack I mean to do it for all locales not cherry picking

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Date formatting with en_US locale

2016-05-26 Thread Baptiste Daroussin
On Thu, May 26, 2016 at 09:44:25AM -0500, Eric van Gyzen wrote:
> Baptiste and -current,
> 
> I noticed two annoyances with date formatting on head, and I wonder how
> we can fix them.
> 
> I have these settings:
> 
> LC_ALL=en_US.ISO8859-1
> LANG=en_US.ISO8859-1
> 
> First, Thunderbird displays the date as, for example:
> 
> 03/ 6/16 ...
> 
> The leading space on the day (6) looks weird.  I might even say it's
> simply wrong.  Zero-padding would better.  (/No/ padding would be best,
> but I don't think strftime supports that.)
> 
> Second, date(1) no longer shows the day-of-week:
> 
> $ date
> March 26, 2016 at 09:21:55 AM CDT
> 
> For many years, I have been typing "date" to see the day-of-week (and
> other things).  I like the new human-friendly format, but I miss the
> day-of-week.
> 
> Of course, I can fix these locally, but I wonder how we can fix them for
> everyone.  I see that the formats come from CLDR.  I also see that ume@
> restored the day-of-week for ja_JP in r292512.  Is this the best
> approach, or should we try to get them changed upstream (CLDR)?
> 
> Thanks for your input,

I can hack cldr2def.pl to readd the week of day as it was before for 11.0 still
the best approach is to push the change upstream.

I will have a look at the cldr2def.pl hack this week end.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: pkg chroot issues?

2016-05-22 Thread Baptiste Daroussin
On Sun, May 22, 2016 at 02:18:07PM -0700, Matthew Macy wrote:
> 
> 
> 
>   On Sun, 22 May 2016 13:43:13 -0700 Tim Kientzle  
> wrote  
>  >  
>  > > On May 22, 2016, at 1:28 PM, K. Macy  wrote: 
>  > >  
>  > >  
>  > >  
>  > > On Sunday, May 22, 2016, Tim Kientzle  wrote: 
>  > > Crochet has some experimental hooks to install packages onto the system 
> being built, but this seems to be hitting problems due to limitations in 'pkg 
> -c'.  In particular, it seems that pkg performs the chroot before it does any 
> network lookups.  This is a problem if the chroot is not a complete system 
> environment (which it cannot be when you're building an image for another 
> system). 
>  > >  
>  > > There's some further discussion on github: 
>  > >  
>  > >   https://github.com/freebsd/crochet/issues/141 
>  > >  
>  > > Any suggestions? 
>  > >  
>  > > Cheers, 
>  > >  
>  > > Tim 
>  > >  
>  > >  
>  > > Just like you need to mount devfs you should have a resolv.conf in your 
> chroot first. Just copy it over before running pkg. This works for me in my 
> image creation script. 
>  >  
>  > Sometimes the image does already have a resolv.conf, but if it does, it's 
> for the target environment (where the image will ultimately be running) and 
> may not be appropriate for the environment where the image is being built. 
>  
> Setting NAMESERVER to "10.0.1.1" crashed pkg for me. Maybe it's been fixed in 
> the meantime. If a resolv.conf already exists I would just rename it before 
> and then rename it back after the call to pkg -c.
> 
Setting nameserver will inject this name server directly via the libc resolv api
so it won't touch the resolv.conf at all no need to back it up

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: 11.0-RELEASE pkg base & base.txz file

2016-04-18 Thread Baptiste Daroussin
On Mon, Apr 18, 2016 at 05:14:54PM +0300, Slawa Olhovchenkov wrote:
> On Mon, Apr 18, 2016 at 10:05:12AM -0400, Nikolai Lifanov wrote:
> 
> > On 04/18/16 10:00, Ernie Luzar wrote:
> > > 11.0 will have pkg base, thats ok, but what does than mean for the
> > > base.txz file?
> > > 
> > > It it going to stay as part of FBSD install?
> > > 
> > > I have many scripts for creating jails which depend on the base.txz file.
> > 
> > It's even easier now:
> > 
> > # mkdir -p /usr/jails/new
> > # pkg -r /usr/jails/new install -r base -g '*'
> 
> And where /etc now?

What do you mean?

Bapt


signature.asc
Description: PGP signature


Re: Packaging the FreeBSD base system with pkg(8)

2016-04-05 Thread Baptiste Daroussin
On Tue, Apr 05, 2016 at 12:22:49PM +0200, Gary Jennejohn wrote:
> On Tue, 5 Apr 2016 10:22:04 +0100
> David Chisnall  wrote:
> 
> > On 5 Apr 2016, at 10:07, Gergely Czuczy  wrote:
> > > 
> > > Also, quite often entries from the base system are changed
> > > manually, think of root's/toor's password.  Are such cases
> > > going to be dealt with properly between upgrades, including
> > > self-built-and-packaged base systems?  Currently it can be a
> > > PITA with mergemaster to handle things like master.passwd
> > > properly between upgrades, automation so far wasn't famous on
> > > doing it properly. 
> > 
> > Mergemaster uses a 2-way merge.  It has the version that you
> > have installed and the version that's being proposed for
> > installation.  Etcupdate and pkg perform a 3-way merge.  It has
> > the pristine version, the version that you have made changes
> > to, and the new version.  If you have changed an entry and so
> > has the package, then you will get a conflict that you have to
> > resolve manually.  If you have added lines and so has the
> > upstream version, then that should cleanly apply.  Similarly,
> > if you and upstream have both modified different lines, then
> > there should be no problem.
> > 
> 
> Will there be an option not to merge?  I never update /etc when
> I do installworld because what I have works for me and I see no
> need to make any changes to a working system.

Yes pkg has an option to not merge and will give you some .pkgnew files

Bapt


signature.asc
Description: PGP signature


Re: FreeBSD_HEAD_amd64_gcc - Build #1144 - Failure

2016-04-04 Thread Baptiste Daroussin
On Mon, Apr 04, 2016 at 08:38:03PM +0800, Li-Wen Hsu wrote:
> On Mon, Apr 04, 2016 at 14:32:35 +0200, Baptiste Daroussin wrote:
> > On Mon, Apr 04, 2016 at 08:25:39PM +0800, Li-Wen Hsu wrote:
> > > On Mon, Apr 04, 2016 at 12:53:16 +0200, Baptiste Daroussin wrote:
> > > > On Mon, Apr 04, 2016 at 06:39:00PM +0800, Li-Wen Hsu wrote:
> > > > > On Mon, Apr 04, 2016 at 00:50:14 +0200, Baptiste Daroussin wrote:
> > > > > > On Mon, Apr 04, 2016 at 04:41:47AM +0800, Li-Wen Hsu wrote:
> > > > > > > I've checked this a little more, I found the return code of
> > > > > > > "installing a installed package" has been changed.
> > > > > > > 
> > > > > > > For pkg 1.6.4_1, it returns 0, but 1.7.1 returns 70, thus jenkins
> > > > > > > consider the build script execution failure.
> > > > > > > 
> > > > > > > 
> > > > > > It shoudl not return 70 if it returns 70 then there is a bug I can 
> > > > > > fix, I need
> > > > > > to know more about what is failing to be able to fix
> > > > > 
> > > > > What else information do you need?  My experiment is as following:
> > > > > 
> > > > > On a machine uses quarterly branch, where pkg is 1.6.4_1, install a
> > > > > installed package.  I choose rsync here, then echo $?, it returns 0.
> > > > > 
> > > > > And I switch to latest branch, also use `pkg install -y rsync`, but 
> > > > > this
> > > > > time $? is 70.
> > > > > 
> > > > > Jenkins "Execute shell" build step checks return value of each command
> > > > > in the shell script, if it is non-zero, it aborts the build and
> > > > > considers if failure.
> > > > 
> > > > Return 70 means an error occured. if I look at the log I can see that
> > > > powerpc64-gcc is not installed for sure, hence the error 70
> > > > 
> > > > What is strange it that no "error message is shown"
> > > 
> > > Very strange, no further error message is shown, even -d provided.
> > > 
> > > root@jenkins10a:~ # pkg -d install -y devel/amd64-xtoolchain-gcc
> > > DBG(1)[44393]> pkg initialized
> > > Updating FreeBSD repository catalogue...
> > > DBG(1)[44393]> PkgRepo: verifying update for FreeBSD
> > > DBG(1)[44393]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'
> > > DBG(1)[44393]> Fetch: fetching from: 
> > > http://pkgmir.pkg.freebsd.org/FreeBSD:10:amd64/latest/meta.txz with opts 
> > > "i"
> > > DBG(1)[44393]> Fetch: fetching from: 
> > > http://pkgmir.pkg.freebsd.org/FreeBSD:10:amd64/latest/packagesite.txz 
> > > with opts "i"
> > > FreeBSD repository is up-to-date.
> > > All repositories are up-to-date.
> > > DBG(1)[44393]> want to get an advisory lock on a database
> > > DBG(1)[44393]> release an advisory lock on a database
> > > root@jenkins10a:~ # echo $?
> > > 70
> > > 
> > > Is there anything I can help on debugging this?
> > > 
> > pkg info amd64-xtoolchain-gcc please
> 
> root@jenkins10a:~ # pkg -d info amd64-xtoolchain-gcc
> DBG(1)[44644]> pkg initialized
> amd64-xtoolchain-gcc-0.1
> Name   : amd64-xtoolchain-gcc
> Version: 0.1
> Installed on   : Tue Mar  3 07:32:25 2015 UTC
> Origin : devel/amd64-xtoolchain-gcc
> Architecture   : freebsd:10:x86:64
> Prefix : /usr/local
> Categories : devel
> Licenses   :
> Maintainer : b...@freebsd.org
> WWW: UNKNOWN
> Comment: Pre seeded toolchain to cross build FreeBSD base
> Annotations:
> repo_type  : binary
> repository : FreeBSD
> Flat size  : 225B
> Description:
> Pre seeded cross toolchain definition to cross build FreeBSD base
> 

Ok I see the issue, I will fix it thanks.

Bapt


signature.asc
Description: PGP signature


Re: FreeBSD_HEAD_amd64_gcc - Build #1144 - Failure

2016-04-04 Thread Baptiste Daroussin
On Mon, Apr 04, 2016 at 08:25:39PM +0800, Li-Wen Hsu wrote:
> On Mon, Apr 04, 2016 at 12:53:16 +0200, Baptiste Daroussin wrote:
> > On Mon, Apr 04, 2016 at 06:39:00PM +0800, Li-Wen Hsu wrote:
> > > On Mon, Apr 04, 2016 at 00:50:14 +0200, Baptiste Daroussin wrote:
> > > > On Mon, Apr 04, 2016 at 04:41:47AM +0800, Li-Wen Hsu wrote:
> > > > > I've checked this a little more, I found the return code of
> > > > > "installing a installed package" has been changed.
> > > > > 
> > > > > For pkg 1.6.4_1, it returns 0, but 1.7.1 returns 70, thus jenkins
> > > > > consider the build script execution failure.
> > > > > 
> > > > > 
> > > > It shoudl not return 70 if it returns 70 then there is a bug I can fix, 
> > > > I need
> > > > to know more about what is failing to be able to fix
> > > 
> > > What else information do you need?  My experiment is as following:
> > > 
> > > On a machine uses quarterly branch, where pkg is 1.6.4_1, install a
> > > installed package.  I choose rsync here, then echo $?, it returns 0.
> > > 
> > > And I switch to latest branch, also use `pkg install -y rsync`, but this
> > > time $? is 70.
> > > 
> > > Jenkins "Execute shell" build step checks return value of each command
> > > in the shell script, if it is non-zero, it aborts the build and
> > > considers if failure.
> > 
> > Return 70 means an error occured. if I look at the log I can see that
> > powerpc64-gcc is not installed for sure, hence the error 70
> > 
> > What is strange it that no "error message is shown"
> 
> Very strange, no further error message is shown, even -d provided.
> 
> root@jenkins10a:~ # pkg -d install -y devel/amd64-xtoolchain-gcc
> DBG(1)[44393]> pkg initialized
> Updating FreeBSD repository catalogue...
> DBG(1)[44393]> PkgRepo: verifying update for FreeBSD
> DBG(1)[44393]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'
> DBG(1)[44393]> Fetch: fetching from: 
> http://pkgmir.pkg.freebsd.org/FreeBSD:10:amd64/latest/meta.txz with opts "i"
> DBG(1)[44393]> Fetch: fetching from: 
> http://pkgmir.pkg.freebsd.org/FreeBSD:10:amd64/latest/packagesite.txz with 
> opts "i"
> FreeBSD repository is up-to-date.
> All repositories are up-to-date.
> DBG(1)[44393]> want to get an advisory lock on a database
> DBG(1)[44393]> release an advisory lock on a database
> root@jenkins10a:~ # echo $?
> 70
> 
> Is there anything I can help on debugging this?
> 
pkg info amd64-xtoolchain-gcc please

Bapt


signature.asc
Description: PGP signature


Re: FreeBSD_HEAD_amd64_gcc - Build #1144 - Failure

2016-04-04 Thread Baptiste Daroussin
On Mon, Apr 04, 2016 at 06:39:00PM +0800, Li-Wen Hsu wrote:
> On Mon, Apr 04, 2016 at 00:50:14 +0200, Baptiste Daroussin wrote:
> > On Mon, Apr 04, 2016 at 04:41:47AM +0800, Li-Wen Hsu wrote:
> > > I've checked this a little more, I found the return code of
> > > "installing a installed package" has been changed.
> > > 
> > > For pkg 1.6.4_1, it returns 0, but 1.7.1 returns 70, thus jenkins
> > > consider the build script execution failure.
> > > 
> > > 
> > It shoudl not return 70 if it returns 70 then there is a bug I can fix, I 
> > need
> > to know more about what is failing to be able to fix
> 
> What else information do you need?  My experiment is as following:
> 
> On a machine uses quarterly branch, where pkg is 1.6.4_1, install a
> installed package.  I choose rsync here, then echo $?, it returns 0.
> 
> And I switch to latest branch, also use `pkg install -y rsync`, but this
> time $? is 70.
> 
> Jenkins "Execute shell" build step checks return value of each command
> in the shell script, if it is non-zero, it aborts the build and
> considers if failure.

Return 70 means an error occured. if I look at the log I can see that
powerpc64-gcc is not installed for sure, hence the error 70

What is strange it that no "error message is shown"

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: FreeBSD_HEAD_amd64_gcc - Build #1144 - Failure

2016-04-03 Thread Baptiste Daroussin
On Mon, Apr 04, 2016 at 04:41:47AM +0800, Li-Wen Hsu wrote:
> On Mon, Apr 4, 2016 at 1:59 AM, Dimitry Andric  wrote:
> > On 03 Apr 2016, at 19:19, Andriy Gapon  wrote:
> >> On 03/04/2016 12:39, jenkins-ad...@freebsd.org wrote:
> >>> FreeBSD_HEAD_amd64_gcc - Build #1144 - Failure:
> >>>
> >>> Build information: 
> >>> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1144/
> >>> Full change log: 
> >>> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1144/changes
> >>> Full build log: 
> >>> https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_amd64_gcc/1144/console
> >>>
> >>
> >> I wasn't able to understand what was the failure here...
> >
> > The step to install the amd64-xtoolchain-gcc package failed, apparently 
> > because it decided to auto-upgrade pkg first, without actually installing 
> > the package that was asked for:
> >
> > + sudo pkg install -y devel/amd64-xtoolchain-gcc
> > Updating FreeBSD repository catalogue...
> > Fetching meta.txz: . done
> > Fetching packagesite.txz: .. done
> > Processing entries: .. done
> > FreeBSD repository update completed. 25093 packages processed.
> > New version of pkg detected; it needs to be installed first.
> > The following 1 package(s) will be affected (of 0 checked):
> >
> > Installed packages to be UPGRADED:
> > pkg: 1.6.4_1 -> 1.7.1
> >
> > The process will require 84 KiB more space.
> > 2 MiB to be downloaded.
> > Fetching pkg-1.7.1.txz: .. done
> > Checking integrity... done (0 conflicting)
> > [1/1] Upgrading pkg from 1.6.4_1 to 1.7.1...
> > [1/1] Extracting pkg-1.7.1: .. done
> > Updating FreeBSD repository catalogue...
> > Repo "FreeBSD" upgrade schema 2012 to 2013: Add vital field
> > FreeBSD repository is up-to-date.
> > All repositories are up-to-date.
> > Build step 'Execute shell' marked build as failure
> >
> > I'd consider this a bug in pkg?  Is it suppose to return a failure code in 
> > this case?
> 
> I've checked this a little more, I found the return code of
> "installing a installed package" has been changed.
> 
> For pkg 1.6.4_1, it returns 0, but 1.7.1 returns 70, thus jenkins
> consider the build script execution failure.
> 
> 
It shoudl not return 70 if it returns 70 then there is a bug I can fix, I need
to know more about what is failing to be able to fix

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: [CFT] packaging the base system with pkg(8)

2016-03-14 Thread Baptiste Daroussin
On Mon, Mar 14, 2016 at 12:38:55PM -0700, Bryan Drewery wrote:
> On 3/14/16 12:29 PM, Dag-Erling Smørgrav wrote:
> > Miroslav Lachman <000.f...@quip.cz> writes:
> >> Bryan Drewery  writes:
> >>> https://github.com/freebsd/pkg/blob/master/scripts/pkg_tree.sh
> >> Can you publish it as a port? I know there is one written in Perl but
> >> I like your sh without dependencies.
> > 
> > It's not very useful, in my opinion.  The relationships between packages
> > form a directed acyclic graph, not a tree, so pkg_tree.sh will either
> > show too little (without -r) or far, far too much (with -r).  If you
> > want to visualize the package graph, you can feed the output of 'pkg
> > query "%n %dn"' into something like graphviz.  For other tasks, you are
> > better off learning how to use 'pkg query' and possibly creating your
> > own aliases or scripts.  It's not that difficult; feel free to ask for
> > help.
> > 
> > (Just for kicks, I tried running 'pkg_tree.sh -rn' on my desktop, which
> > has 934 packages installed.  It's been running for ten minutes and has
> > printed over 90,000 lines, with no end in sight.)
> > 
> 
> Yeah that's why I never included it by default. It can be interesting
> sometimes but often is too noisy. I didn't want to encourage its use by
> default vs pkg info or pkg query.

Also note that if people likes graph we can now generate graphviz output from
the solver using pkg -o DOT_FILE="/something.dot"  then
play with graphviz.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: [CFT] packaging the base system with pkg(8)

2016-03-11 Thread Baptiste Daroussin
On Fri, Mar 11, 2016 at 04:10:56PM +0300, Slawa Olhovchenkov wrote:
> On Fri, Mar 11, 2016 at 01:05:11PM +0100, Baptiste Daroussin wrote:
> 
> > On Tue, Mar 08, 2016 at 05:35:59PM +, David Chisnall wrote:
> > > On 8 Mar 2016, at 15:14, Slawa Olhovchenkov <s...@zxy.spb.ru> wrote:
> > > > 
> > > 
> > > In terms of comparing packages, if you’re doing that visually then you 
> > > are likely to have problems anyway, unless your eyes and brain work far 
> > > better than most humans.  We can make that much easier by providing libxo 
> > > output in pkg and allowing you to have a simple jq script that tells you 
> > > what the differences are.
> > > 
> > pkg can already expose the entire content of a package in json or ucl via:
> > $ pkg info --raw --raw-format [json|json-conpact|yaml|ucl] name
> 
> Exposing  the entire content of a package is not a root of cause.
> Question in comapring of two different setup with different behaviour
> and search cause of difference.
> 
> Case of only a few monolitic packages is essentiality simple then case
> of 1000 combined packages.
pkg info -a on one diff with pkg info -a on the other
for the full content: pkg info -a --raw on both end and diff them.

That should cover your case, no?

Bapt




signature.asc
Description: PGP signature


Re: [CFT] packaging the base system with pkg(8)

2016-03-11 Thread Baptiste Daroussin
On Tue, Mar 08, 2016 at 05:35:59PM +, David Chisnall wrote:
> On 8 Mar 2016, at 15:14, Slawa Olhovchenkov  wrote:
> > 
> 
> In terms of comparing packages, if you’re doing that visually then you are 
> likely to have problems anyway, unless your eyes and brain work far better 
> than most humans.  We can make that much easier by providing libxo output in 
> pkg and allowing you to have a simple jq script that tells you what the 
> differences are.
> 
pkg can already expose the entire content of a package in json or ucl via:
$ pkg info --raw --raw-format [json|json-conpact|yaml|ucl] name

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: lldb input issue

2016-03-06 Thread Baptiste Daroussin
On Sun, Mar 06, 2016 at 10:55:27PM +0300, Roman Bogorodskiy wrote:
>   Baptiste Daroussin wrote:
> 
> > On Sun, Mar 06, 2016 at 07:44:34PM +0300, Roman Bogorodskiy wrote:
> > > Hi,
> > > 
> > > I'm seeing an issue with lldb: when I start it (without arguments) and
> > > try to type commands, it looks like this:
> > > 
> > > $  lldb
> > > (lldb) \U+7F68\U+7F65\U+7F6C\U+7F70\U+7F0A
> > > 
> > > So, basically, I cannot execute any command and cannot even exit from
> > > its shell, only by ctrl-z and killing it.
> > > 
> > > This happens to me on today's -CURRENT / amd64.
> > > 
> > > I was wondering if that's an issue with my user's locale, but the
> > > behavior is same for root.
> > > 
> > > Also, I can see exactly the same behavior with lldb on FreeBSD.
>^^^
> Oops, that's supposed to be 'freefall'.
> 
> > > Is that some known issue or maybe configuration problem?
> > > 
> > > Thanks,
> > > 
> > > Roman Bogorodskiy
> > 
> > 
> > 
> > Sounds like an issue with libedit, probably due to the latest import of 
> > libedit
> > (just a guess)
> 
> It could be. BTW, I've installed lldb38 using pkg and it works fine.
> 

Which is linked to libedit from ports (older that in base) so it seems to prove
that libedit update might be the issue there.

I have CC pfg@ who has updated libedit

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: lldb input issue

2016-03-06 Thread Baptiste Daroussin
On Sun, Mar 06, 2016 at 07:44:34PM +0300, Roman Bogorodskiy wrote:
> Hi,
> 
> I'm seeing an issue with lldb: when I start it (without arguments) and
> try to type commands, it looks like this:
> 
> $  lldb
> (lldb) \U+7F68\U+7F65\U+7F6C\U+7F70\U+7F0A
> 
> So, basically, I cannot execute any command and cannot even exit from
> its shell, only by ctrl-z and killing it.
> 
> This happens to me on today's -CURRENT / amd64.
> 
> I was wondering if that's an issue with my user's locale, but the
> behavior is same for root.
> 
> Also, I can see exactly the same behavior with lldb on FreeBSD.
> 
> Is that some known issue or maybe configuration problem?
> 
> Thanks,
> 
> Roman Bogorodskiy



Sounds like an issue with libedit, probably due to the latest import of libedit
(just a guess)

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: [CFT] packaging the base system with pkg(8)

2016-03-06 Thread Baptiste Daroussin
On Thu, Mar 03, 2016 at 10:27:00AM +, Matthew Seaman wrote:
> On 03/02/16 23:54, Glen Barber wrote:
> > Also note (as repeated below), running 'pkg delete -a' will implicitly
> > remove base system packages after they are installed.
> 
> This has the potential for many feet to be shot, given that up to now,
> 'pkg delete -a' would always leave you with a viable system.
> 
> We already make an exception for pkg itself -- you need 'pkg delete -fa'
> to actually remove pkg(8) as well.  (Note to self: this needs to be
> documented in the pkg-delete(8) man page.)
> 
> We should have similar exceptions for the essential bits of the base
> system -- at minimum everything you need to boot the system up and
> install stuff from a package repository.
> 
> We should also have a command line that will remove all ported software
> but leave the base intact.   Maybe by adding '-r reponame' functionality
> to 'pkg delete'?
> 

It is planned to have a "precious" flag for packages which will prevent pkg
delete -a from dropping them

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Dropping some locales/encodings?

2016-03-01 Thread Baptiste Daroussin
On Tue, Mar 01, 2016 at 10:58:37AM +0200, Daniel Kalchev wrote:
> 
> > On 1.03.2016 г., at 5:45, Sergey Manucharian <s...@ara-ler.com> wrote:
> > 
> > Excerpts from Andrey Chernov's message from Tue 01-Mar-16 05:47:
> >> On 01.03.2016 2:23, Baptiste Daroussin wrote:
> >>> 
> >>> I can properly generate almost any of the said locales/encodings but a 
> >>> few that
> >>> I would like to remove (provided that unicode version are available)
> >>> 
> >>> Here is the list of locales/encodings:
> >>> 
> >>> be_BY.CP1251
> >> 
> >> CP1251 is Windows native (single characters mode) and widely used to
> >> represent Cyrillic: Russian, Bulgarian, Serbian, Ukrainian, Belarusian
> >> (i.e. be_BY), Macedonian. IMHO it will be better to not remove it to
> >> make easy handling of native encoded texts comes from Windows.
> >> 
> > 
> > I agree with Andrey that CP1251 is needed as one of major Cyrillic
> > encodings.
> > 
> > Not sure how the locale existence/absence effects that. Definitely nobody
> > uses it as a locale for more than a decade.
> > 
> 
> I use daily bg_BG.CP1251 on a lot of workstations, primarily to handle old 
> text documents and source code that contains pre-UTF text. I could imagine 
> similar use case for be_BY.CP1251.
> 
> What benefit does it bring to remove an already existing locale?

The benefit is the fact we have no source for those and collation for one are
wrong for those. (We have added proper collation support in head).

But given the feedback I got I will provide them as static (aka non evolving
anymore) while others would get refreshed so no locales will be lost and that
will allow us to continue refreshing others.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Dropping some locales/encodings?

2016-03-01 Thread Baptiste Daroussin
On Tue, Mar 01, 2016 at 05:47:25AM +0300, Andrey Chernov wrote:
> On 01.03.2016 2:23, Baptiste Daroussin wrote:
> > Hi all,
> > 
> > I have updated few month ago the locales to cldr v27.0.1. I would like to
> > simplify the generation of those locales from cldr to POSIX locales that we 
> > do
> > provides.
> > 
> > I can properly generate almost any of the said locales/encodings but a few 
> > that
> > I would like to remove (provided that unicode version are available)
> > 
> > Here is the list of locales/encodings:
> > 
> > be_BY.CP1251
> 
> CP1251 is Windows native (single characters mode) and widely used to
> represent Cyrillic: Russian, Bulgarian, Serbian, Ukrainian, Belarusian
> (i.e. be_BY), Macedonian. IMHO it will be better to not remove it to
> make easy handling of native encoded texts comes from Windows.
> 
> Don't know about other mentioned.
> 

Ok Given the replies I will then change the way I'm looking at it and see if I
can provide static version of those instead of removing and generate all others
from cldr.

Thanks for the quick replies

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Dropping some locales/encodings?

2016-03-01 Thread Baptiste Daroussin
On Mon, Feb 29, 2016 at 08:41:08PM -0700, Sergey Manucharian wrote:
> Excerpts from Baptiste Daroussin's message from Tue 01-Mar-16 00:23:
> > 
> > I would like to remove (provided that unicode version are available)
> > 
> > Here is the list of locales/encodings:
> > 
> > be_BY.CP1251
> > be_BY.CP1131
> > hi_IN.ISCII-DEV
> > hy_AM.ARMSCII-8
> > zh_HK.Big5HKSCS
> > 
> > Provided that those should be covered by respectively:
> > be_BY.ISO8859-5 be_BY.UTF-8
> > hi_IN.UTF-8
> > hy_AM.UTF-8
> > zh_HK.UTF-8 zh_Hans_HK.UTF-8
> > 
> > Anyone has a strong opinion against this removal?
> > 
> 
> Well, as a locale hy_AM.ARMSCII-8 is definitely not needed. However, as
> an encoding it's still useful: sometimes I (and probably some other people)
> need to convert some "ancient" documents from ARMSCII-8 to UTF-8.
> 
> I do not think that the removing it will prevent such conversion anyway
> (e.g. with iconv), correct?

Yes iconv will still be able to do such conversion (I will double check)
> 
> Sergey
> 
> P.S. "hy" = Hayastan = Armenia
> 


signature.asc
Description: PGP signature


Dropping some locales/encodings?

2016-02-29 Thread Baptiste Daroussin
Hi all,

I have updated few month ago the locales to cldr v27.0.1. I would like to
simplify the generation of those locales from cldr to POSIX locales that we do
provides.

I can properly generate almost any of the said locales/encodings but a few that
I would like to remove (provided that unicode version are available)

Here is the list of locales/encodings:

be_BY.CP1251
be_BY.CP1131
hi_IN.ISCII-DEV
hy_AM.ARMSCII-8
zh_HK.Big5HKSCS

Provided that those should be covered by respectively:
be_BY.ISO8859-5 be_BY.UTF-8
hi_IN.UTF-8
hy_AM.UTF-8
zh_HK.UTF-8 zh_Hans_HK.UTF-8

Anyone has a strong opinion against this removal?

The reason to remove those is that they are not provided by cldr

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Packaging the FreeBSD base system with pkg(8)

2016-01-28 Thread Baptiste Daroussin
On Thu, Jan 28, 2016 at 02:58:52PM -0500, Nikolai Lifanov wrote:
> 
> 
> On January 28, 2016 1:23:22 PM EST, Glen Barber  wrote:
> >On Thu, Jan 28, 2016 at 09:12:53PM +0300, Slawa Olhovchenkov wrote:
> >> I see two hudge problem for support upgrades from older RELEASE
> >> versions (supported too): key (used for repo signing) change and need
> >> to access ports packages to install (installed outdated release can't
> >> find pkg packet for install).
> >> 
> >> Can you more detailed describe current propolsal of new packaging
> >> system? pkg is part of base installed packages? Or after installing
> >> base packages pkg installed from `ports`? For disc1.iso case.
> >> 
> >> How handled boot/device.hints and var/db/etcupdate?
> >> 
> >> How handled boot block updates?
> >
> >These are all valid questions, but let's take a step back for a bit,
> >and
> >not put the carriage in front of the horse.
> >
> >The initial email in this thread was intended to provide an update on
> >the status.  Some things, such as file merging, is already in place in
> >a few of the packages.  Not all, yet.
> >
> >Unexpected things are going to happen.  That is the only thing that is
> >a guarantee right now.  In other words, implementation (from what is
> >now
> >in the project branch) may change.  And yes, there needs to be a way to
> >upgrade from older releases.
> >
> >But let's not get too far ahead of ourselves.
> >
> >Glen
> 
> Can we have verbose and/or semi-interactive configuration merge, as offered 
> by etcupdate or mergemaster?

pkg has a merging mercanism (which one can disable if scared) which will
automatically merge everything it can and live a file .new if a conflict happen
(or merging disabled)
Would be easy to write an etcupdate like that walk through /etc and propose a
semi-interactive merging mechanism.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Packaging the FreeBSD base system with pkg(8)

2016-01-28 Thread Baptiste Daroussin
On Thu, Jan 28, 2016 at 12:46:39PM +, Thomas Mueller wrote:
> from Glen Barber:
> 
> > As many know, work has been in progress for quite some time to provide
> > the ability to package and upgrade the FreeBSD base system using pkg(8).
> > The majority of the initial implementation has provided much of the core
> > functionality to make this possible, however much work still needs to be
> > done.
> 
> (snip)
> 
> Would the base system all be one package?

multiple packages with meta packages to represent the whole base so you have the
best of both world :)
> 
> In Linux, everything is part of a package, even the kernel, but something 
> comparable to FreeBSD or NetBSD base system would have many packages.
> 
> Will it be possible to upgrade base system with portmaster or portupgrade, 
> and would that be better than the current procedure in UPDATING?

No but one will be able to simply run pkg upgrade (and built himself the
packages)
> 
> Would pkg then be able to show a package's required shared libraries 
> including shared libraries from the base system?  I was recently stung by pkg 
> not showing required shared libraries from the base system.

Yes, but but real usage of it would happen in a second step because of many
rought edges to be deal with. but yes the information would be available

see:
https://www.youtube.com/watch?v=Br6izhH5P1I
and
https://www.youtube.com/watch?v=v7px6ktoDAI

for a bigger view of what happened (note that some detail my have change a bit,
the overall remains the same)

Best regards,
Bapt
> 
> I just subscribed to freebsd-pkgb...@freebsd.org .
> 
> Tom
> 
> ___
> freebsd-pkgb...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase
> To unsubscribe, send any mail to "freebsd-pkgbase-unsubscr...@freebsd.org"


signature.asc
Description: PGP signature


Re: libcrypto.so.7 not found, needed (?) for X server

2016-01-19 Thread Baptiste Daroussin
On Tue, Jan 19, 2016 at 02:17:11PM +0100, Dimitry Andric wrote:
> On 19 Jan 2016, at 13:44, Thomas Mueller  wrote:
> > 
> > I just updated FreeBSD-current to r294248, and can no longer startx.
> > 
> > Error message is
> > 
> > xauth:  file /home/arlene/.serverauth.1177 does not exist
> > 
> > Shared object "libcrypto.so.7" not found, required by "X"
> > xinit: giving up
> > xinit: unable to connect to X server: Connection refused
> > xinit: server error
> > 
> > There is /usr/lib/libcrypto.so but notlibcrypto.so.7.
> > 
> > I also looked in /usr/local/lib, no libcrypto... files.
> > 
> > Now I run
> > 
> > pkg info -f xserver and get
> > 
> > root@amelia:~ # pkg info -f xserver
> > Shared object "libssl.so.7" not found, required by "pkg"
> > 
> > What happened here?  Bug in new FreeBSD?
> 
> This is explained by the UPDATING entry of October 2015:
> 
> 20151030:
> The OpenSSL has been upgraded to 1.0.2d.  Any binaries requiring
> libcrypto.so.7 or libssl.so.7 must be recompiled.
> 
Given how long ago this happen a simple pkg-static upgrade -f would do the trick
because all available packages has been rebuilt in the cluster since and have
been published.

All official packages for head are linked against newer libcrypto.so and
libssl.so

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: Base Packaging in 11

2015-12-18 Thread Baptiste Daroussin
On Fri, Dec 18, 2015 at 03:21:13PM -0800, Roger Marquis wrote:
> Forwarding this from freebsd-security in case anyone here can update us
> regarding the status of base packaging or has URLs for projects/release-pkg.
> 
> Roger
> 
Packaging base is happening here:
https://svnweb.freebsd.org/base/projects/release-pkg/

It is mostly stalled for month due to lack of time working on it.
The TODO list is mostly:
- implement priotity in plist for pkg to ensure the ordre files will be replaced
- finishing flexible dependencies and use it by default in pkg
- tracking down all issues from installworld that results files not installed by
  install(1) and files installed twice

In my opinion it should not be made the default mechanism for 11.0-RELEASE if we
are not able to provide our first packages for testing by the end of january to
leave enough time for testing and fixes before the release.

While I was pretty confident few month ago, things has changed since and I
cannot spend the necessary time on it for various reasons.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: pkg does not update the repo catalogue

2015-12-07 Thread Baptiste Daroussin
On Mon, Dec 07, 2015 at 01:25:09PM +0100, olli hauer wrote:
> On 2015-12-07 09:50, Matthias Apitz wrote:
> > 
> > Hello,
> > 
> > This is with 11-CURRENT and ports from July this year; I have the
> > packages which I build with poudriere on some other host in a dir
> > /usr/PKGDIR.20150726 and added 8 new packages there, the total number is
> > now 1691:
> > 
> > # ls *.txz | egrep -v 'packagesite.txz|meta.txz|digests.txz' | wc -l 
> > 1691
> > 
> > My repo definition is:
> > 
> > # cat /usr/local/etc/pkg/repos/myrepo.conf
> >FreeBSD: {
> >url: "file:/usr/PKGDIR.20150726",
> >enabled: true,
> >}
> > 
> > When I now want to update the 8 new packages to the repo catalogue, they
> > are not added (i.e. the number stays with 1683 and I also can not
> > install them with 'pkg instal ...'):
> > 
> > # pkg -v
> > 1.5.5
> > # pkg -R /usr/local/etc/pkg/repos/ update -f
> > Updating FreeBSD repository catalogue...
> > Fetching meta.txz: 100%260 B   0.3kB/s00:01
> > Fetching packagesite.txz: 100%  382 KiB 391.6kB/s00:01
> > Processing entries: 100%
> > FreeBSD repository update completed. 1683 packages processed.
> > 
> > What I'm missing here?
> > 
> > Thanks
> > 
> > matthias
> 
> Hi Matthias,
> 
> I see some possible issues.
> 
> - repo has same name as the official one (FreeBSD, see `cat 
> /etc/pkg/FreeBSD.conf')
> - file:/$path should be file:///$path

pkg should have yelled about that, I will fix that.

> - repo catalog was not updated with command `pkg repo'
> - REPO_AUTOUPDATE = true/false (in pkg.conf)
> 
> Does your repo match the output of the command
> $ pkg -vv (all below Repositories:)
> 
> 

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-25 Thread Baptiste Daroussin
On Wed, Nov 25, 2015 at 06:31:16PM +0300, Andrey Chernov wrote:
> On 25.11.2015 18:12, Andrey Chernov wrote:
> > On 25.11.2015 17:35, Baptiste Daroussin wrote:
> >>> BTW, array size looks suspicious:
> >>> static wchar_t wab_months[12][MAX_ABMON_WIDTH * 2 * MB_LEN_MAX];
> >>> what MB_LEN_MAX doing here? This constant is for multiple-bytes encoded,
> >>> not for wide chars.
> >> Bad copy/paste sorry it should be "MAX_ABMON_WIDTH * 2"
> > 
> > I don't check deep enough, it seems first array
> > MAX_ABMON_WIDTH * MB_LEN_MAX + 1
> > and second one
> > MAX_ABMON_WIDTH * 2 + 1
> > 
> 
> No. We can't assume anything here and should integrate limits from the
> locale for months fields instead. F.e. in abstract general case in wide
> array can be 100 zero-width characters + 5 of normal characters, so
> width-oriented sizes not prevents overflowing.
> 
> First array size should be from locale internals,

There is no max size for an abbreviated month in the locale internals
So maybe the best here is to consider your first proposal:
MAX_ABMON_WIDTH * MB_LEN_MAX + 1 and MAX_ABMON_WIDTH * 2 + 1
Check that it does not overflow and fallack if it overflows.

Having 100 zero-width characters in a locale have very little probability to
happen and if it does one day, it problably means something went wrong in the
code generation

Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-25 Thread Baptiste Daroussin
On Wed, Nov 25, 2015 at 04:34:17PM +0300, Andrey Chernov wrote:
> On 25.11.2015 15:53, Baptiste Daroussin wrote:
> > What I did for now is set max_month_width to -1 and in ls_strftime I 
> > fallback on
> > the plain strftime meaning you keep localized information but the 
> > alignement is
> > broken as of now.
> 
> It will be enough.
> 
> >> 3) wcwidth/wcswidth may return -1 too, it needs to be checked too.
> > done and truncate the name of the month to the latest valid character
> 
> I think there is no point for sofisticated truncating, if locale is not
> valid. F.e. you may truncate to just 1 char. The thing above will be
> enough for all such cases.
> 
> > Review updated (if you prefer a diff by mail just tell me, given you do not 
> > have
> > a phabricator account.)
> 
> I don't have phabricator.
> 
> This one
> if ((n = max_month_width - wab_months_width[i]) > 0) {
> wcslcat(wab_months[i], L" "/* MAX_ABMON_WIDTH */,
> max_month_width + 1);
> }
> should be
> if ((n = max_month_width - wab_months_width[i]) > 0)
> wcslcat(wab_months[i], L" "/* MAX_ABMON_WIDTH */, n);
> 
> I.e. you append n spaces and n is the difference between max and current.

With wcsncat I would agree, but not with wcslcat.

wcslcat works like strlcat:
to quote the manpage:
"It will append at most dstsize - strlen(dst) - 1 characters."
So here with your version it will be n - wcslen(wab_months[i]) -1
which won't fit what we want to do.

btw that makes me figure out that what I want is wcsncat because we do care to
make sure we have appended the right number of spaces at the end of the
abbreviated month based on character width, not the on the len of the wide
string

so I changed it

if ((n = max_month_width - wab_months_width[i]) > 0)
wcsncat(wab_months[i], L" "/* MAX_ABMON_WIDTH */, n);

If you do not have further comments, I will commit that later today.

Then I will work on libarchive and pax (would you like to review the patches for
those?)

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-25 Thread Baptiste Daroussin
On Wed, Nov 25, 2015 at 04:31:21AM +0300, Andrey Chernov wrote:
> On 25.11.2015 3:15, Baptiste Daroussin wrote:
> > On Sat, Nov 21, 2015 at 11:57:46PM +0300, Andrey Chernov wrote:
> >> On 21.11.2015 15:18, Ed Schouten wrote:
> >>> Hi Baptiste,
> >>>
> >>> I suppose you should use the wcswidth() function somewhere to compute
> >>> the visible width of the month name. Some characters may be
> >>> double-width, others may have no effective width at all.
> >>>
> >>
> >> I agree. Checking error return of wide chars functions with some
> >> fallback will be good too.
> > 
> > I have updated the code https://reviews.freebsd.org/D4239
> > 
> > Tested by modifying some locales to add double width and zero width unicode 
> > in
> > the locales
> > 
> > Also added the error checking for the return of wide chars functions. For 
> > now I
> > haven't added fallback, suggestions welcome if needed.
> 
> 1) For just 1 char in wcswidth(_months[i][j], 1); it is better to
> use another function wcwidth(wab_months[i][j]);

done
> 
> 2) By fallback I mean something which not stops ls working with
> incorrect for some reason locale, like setting max_width_month to
> MAX_ABMON_WIDTH on error return (from
> mbstowcs/wcwidth/wcswidth/wcswidth) and exit from
> populate_abbreviated_month().

Not that easy to provide a fallback (or better to say I can't find a proper way)
it if fails on mbstocws then ab_months is not populated so unusable.
What I did for now is set max_month_width to -1 and in ls_strftime I fallback on
the plain strftime meaning you keep localized information but the alignement is
broken as of now.

> 
> 3) wcwidth/wcswidth may return -1 too, it needs to be checked too.
done and truncate the name of the month to the latest valid character
> 
> 4) The whole processing looks overcomplicated and not effective. What
> about this instead?
> for (i = 0; i < 12; i++) {
> count wcswidth() of each month and store it in wab_months_width[].
> count max_width_month.
> }
> for (i = 0; i < 12; i++) {
> if ((n = max_width_month - wab_months_width[i]) > 0)
>   call wcscat(wab_months[i], L" ") n times.
> }

Done, along with your optimisation from your next mail
> 
> 5) If there is no %b is strftime() format, there is no sense to spend
> CPU cycles on from populate_abbreviated_month(), so it should be called
> only once inside ls_strftime() on first %b instead of calling it in
> printtime() for all cases.

done

Review updated (if you prefer a diff by mail just tell me, given you do not have
a phabricator account.)

Thanks for your patience! and reviews!

Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-25 Thread Baptiste Daroussin
On Wed, Nov 25, 2015 at 05:06:17PM +0300, Andrey Chernov wrote:
> On 25.11.2015 16:51, Baptiste Daroussin wrote:
> > wcslcat works like strlcat:
> > to quote the manpage:
> > "It will append at most dstsize - strlen(dst) - 1 characters."
> > So here with your version it will be n - wcslen(wab_months[i]) -1
> > which won't fit what we want to do.
> > 
> > btw that makes me figure out that what I want is wcsncat because we do care 
> > to
> > make sure we have appended the right number of spaces at the end of the
> > abbreviated month based on character width, not the on the len of the wide
> > string
> > 
> > so I changed it
> > 
> > if ((n = max_month_width - wab_months_width[i]) > 0)
> > wcsncat(wab_months[i], L" "/* MAX_ABMON_WIDTH */, n);
> 
> Sorry, I initially mean wcsncat() functionality here, wcslcat() is
> sneaked in due to wrong memorizing.
> 
> About width counting and fallback...
> Without attempt to nicely truncate data damaged by unknown way the code
> will be simpler. Instead all that loop adding width one by one wchar, just:
> 
> width = wcswidth(wab_months[i], MAX_ABMON_WIDTH);
> if (width == -1) {
> max_month_width = -1;
> return;
> }
> wab_months_width[i] = width;
> 
> About
> /* NULL terminate to simplify padding later */
> wab_months[i][j] = L'\0';
> You don't need it, mbstowcs() null-terminates result, if there is a room.

right if should be only set on the month we do truncate to MAX_ABMON_WIDTH

> BTW, array size looks suspicious:
> static wchar_t wab_months[12][MAX_ABMON_WIDTH * 2 * MB_LEN_MAX];
> what MB_LEN_MAX doing here? This constant is for multiple-bytes encoded,
> not for wide chars.
Bad copy/paste sorry it should be "MAX_ABMON_WIDTH * 2"


I will fix
Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-24 Thread Baptiste Daroussin
On Wed, Nov 25, 2015 at 01:15:13AM +0100, Baptiste Daroussin wrote:
> On Sat, Nov 21, 2015 at 11:57:46PM +0300, Andrey Chernov wrote:
> > On 21.11.2015 15:18, Ed Schouten wrote:
> > > Hi Baptiste,
> > > 
> > > I suppose you should use the wcswidth() function somewhere to compute
> > > the visible width of the month name. Some characters may be
> > > double-width, others may have no effective width at all.
> > > 
> > 
> > I agree. Checking error return of wide chars functions with some
> > fallback will be good too.
> 
> I have updated the code https://reviews.freebsd.org/D4239
> 
> Tested by modifying some locales to add double width and zero width unicode in
> the locales
> 
> Also added the error checking for the return of wide chars functions. For now 
> I
> haven't added fallback, suggestions welcome if needed.
> 
> Best regards,
> Bapt

Actually I can make the fallback on the C locale in case of failure. Would that
work for you?

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-24 Thread Baptiste Daroussin
On Sat, Nov 21, 2015 at 11:57:46PM +0300, Andrey Chernov wrote:
> On 21.11.2015 15:18, Ed Schouten wrote:
> > Hi Baptiste,
> > 
> > I suppose you should use the wcswidth() function somewhere to compute
> > the visible width of the month name. Some characters may be
> > double-width, others may have no effective width at all.
> > 
> 
> I agree. Checking error return of wide chars functions with some
> fallback will be good too.

I have updated the code https://reviews.freebsd.org/D4239

Tested by modifying some locales to add double width and zero width unicode in
the locales

Also added the error checking for the return of wide chars functions. For now I
haven't added fallback, suggestions welcome if needed.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-20 Thread Baptiste Daroussin
On Fri, Nov 20, 2015 at 11:05:56AM +0300, Sergey V. Dyatko wrote:
> Hi,
> 
> subj. http://i.imgur.com/F9QO29l.png
> it is on head@r290573:
> WTR:
> env LC_ALL=uk_UA.UTF-8 ls -la /usr/ports/databases/ or env LC_ALL=ru_RU.UTF-8
> ls -la /usr/ports/databases/
> 
> env LC_ALL=C ls -la /usr/ports/databases/ works fine
> also on old stable/10 (r286868)  as I can see 'month' field length 3 symbols 
> 
Thanks for reporting, I can reproduce the issue with some other locales. The
thing is there seems to be no standard for abbreviated length. Formerly we had a
3 character lenght for abbreviated month.

We now use CLDR which seems to follow the abbreviated rules from IBM:
"Each string must be of equal length and contain 5 characters or less"

There are 2 possible fixes: either always pad those in the locale definition
which seems wrong or modify ls so that it by itself pads properly.

Neither posix nor ISO-14652 defines the length of the abbreviated form

padding in the locales themself would be wrong so I do propose to pad in the ls
command. And padding with 5 characters.

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-20 Thread Baptiste Daroussin
On Fri, Nov 20, 2015 at 11:42:53AM +0100, Baptiste Daroussin wrote:
> On Fri, Nov 20, 2015 at 11:05:56AM +0300, Sergey V. Dyatko wrote:
> > Hi,
> > 
> > subj. http://i.imgur.com/F9QO29l.png
> > it is on head@r290573:
> > WTR:
> > env LC_ALL=uk_UA.UTF-8 ls -la /usr/ports/databases/ or env 
> > LC_ALL=ru_RU.UTF-8
> > ls -la /usr/ports/databases/
> > 
> > env LC_ALL=C ls -la /usr/ports/databases/ works fine
> > also on old stable/10 (r286868)  as I can see 'month' field length 3 
> > symbols 
> > 
> Thanks for reporting, I can reproduce the issue with some other locales. The
> thing is there seems to be no standard for abbreviated length. Formerly we 
> had a
> 3 character lenght for abbreviated month.
> 
> We now use CLDR which seems to follow the abbreviated rules from IBM:
> "Each string must be of equal length and contain 5 characters or less"
> 
> There are 2 possible fixes: either always pad those in the locale definition
> which seems wrong or modify ls so that it by itself pads properly.
> 
> Neither posix nor ISO-14652 defines the length of the abbreviated form
> 
> padding in the locales themself would be wrong so I do propose to pad in the 
> ls
> command. And padding with 5 characters.
> 
> Best regards,
> Bapt

For the record glibc/linux had the same problem:
https://sourceware.org/bugzilla/show_bug.cgi?id=9859

"fixed" in coreutils (gnu ls) the way I propose to do for us
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=612b647dd16d5abc03b295abe42d8b4a0fe660f7

Best regards,
Bapt


signature.asc
Description: PGP signature


Re: /bin/ls formatting broken for non-C(?) locales

2015-11-20 Thread Baptiste Daroussin
On Fri, Nov 20, 2015 at 01:23:52PM +0100, Jilles Tjoelker wrote:
> On Fri, Nov 20, 2015 at 12:02:13PM +0100, Baptiste Daroussin wrote:
> > On Fri, Nov 20, 2015 at 11:42:53AM +0100, Baptiste Daroussin wrote:
> > > On Fri, Nov 20, 2015 at 11:05:56AM +0300, Sergey V. Dyatko wrote:
> > > > subj. http://i.imgur.com/F9QO29l.png
> > > > it is on head@r290573:
> > > > WTR:
> > > > env LC_ALL=uk_UA.UTF-8 ls -la /usr/ports/databases/ or env 
> > > > LC_ALL=ru_RU.UTF-8
> > > > ls -la /usr/ports/databases/
> 
> > > > env LC_ALL=C ls -la /usr/ports/databases/ works fine
> > > > also on old stable/10 (r286868)  as I can see 'month' field length 3 
> > > > symbols 
> 
> > > Thanks for reporting, I can reproduce the issue with some other
> > > locales. The thing is there seems to be no standard for abbreviated
> > > length. Formerly we had a 3 character lenght for abbreviated month.
> 
> > > We now use CLDR which seems to follow the abbreviated rules from IBM:
> > > "Each string must be of equal length and contain 5 characters or less"
> 
> > > There are 2 possible fixes: either always pad those in the locale
> > > definition which seems wrong or modify ls so that it by itself pads
> > > properly.
> 
> > > Neither posix nor ISO-14652 defines the length of the abbreviated form
> 
> > > padding in the locales themself would be wrong so I do propose to
> > > pad in the ls command. And padding with 5 characters.
> 
> > For the record glibc/linux had the same problem:
> > https://sourceware.org/bugzilla/show_bug.cgi?id=9859
> 
> > "fixed" in coreutils (gnu ls) the way I propose to do for us
> > http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=612b647dd16d5abc03b295abe42d8b4a0fe660f7
> 
> Coreutils fixed it slightly better: it calculates the maximum width of
> the abbreviated month names and pads to that (with a maximum of 5). In
> particular, this ensures that the output does not change for locales
> that have 3-character abbreviations, such as the POSIX locale. I think
> this is valuable.
> 
> They also keep the list of month names from this calculation and they
> say this speeds up ls noticeably compared to having strftime() expand %b
> for every file.

Here is what I do propose (sorry for the ugly pad_to_col name, if one has better
please share :D

https://reviews.freebsd.org/D4239

Best regards,
Bapt


signature.asc
Description: PGP signature


  1   2   3   4   5   >