Re: Policy on closing bugs

[Jan Bramkamp]

On 24.05.19 12:07, Grzegorz Junka wrote:

Hey,

Is there any policy/document when a bug can be closed? For example, is 
it OK to close a bug that is fixed upstream but not yet in ports?


Thanks
GrzegorzJ

I don't know of any official policy, but as a user (and port maintainer) 
I would prefer an update to the FreeBSD PR stating that the bug has been 
fixed upstream. Closing the PR should wait until the fix made it into 
the port.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: irc/ngircd: typo in Makefile

[Jan Bramkamp]

Probably ;-) Please create a PR for this to get it fixed.

On 15.04.19 21:09, Serpent7776 wrote:

Hello,

I've just stumbled upon a possible typo in Makefile:

DEBUG_CONFIGURE_EBABLE=¦¦   debug

Shouldn't this be ENABLE instead of EBABLE?


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: python 3 subprocess performance

[Jan Bramkamp]

The reason is that that python does something stupid (tm). It tries to
close all file descriptors (except a few whitelisted ones) up to the
maximum file descriptor number. It does this by asking the kernel for
the maximum possible number and closing everything it doesn't want to
keep. Some time later someone came up with an optimization (read the
open file descriptors from /dev/fd). All of this pain and suffering is
caused by good old Ulrich Drepper braindamage:
https://sourceware.org/bugzilla/show_bug.cgi?id=10353.

Most Linux distros have lower default file descriptor limits than
FreeBSD making this workaround less painful. The correct solution would
be to teach python3 about closefrom(2).

On 11.04.19 16:39, Konstantin Schukraft wrote:
>> Please run python under truss -f. Does it try to close(2) all possible
>> file descriptors? Does the runtime decrease if fdescfs is mounted at
>> /dev/fd?
> truss -f -p 17407
> 
> 17407: read(5,0x1265d2e9120,5)   = 0 (0x0)
> 17407: close(5)  = 0 (0x0)
> 17407: lseek(3,0x0,SEEK_CUR) ERR#29 'Illegal seek'
> 17407: fstat(3,{ mode=p- ,inode=1610971,size=0,blksize=4096 }) =
> 0 (0x0)
> 17407: read(3,"FreeBSD midgard 12.0-STABLE-HBSD"...,8192) = 88 (0x58)
> 17407: read(3,0x1265ceda078,8104)    = 0 (0x0)
> 17407: close(3)  = 0 (0x0)
> 17407: wait4(1948,{ EXITED,val=0 },0x0,0x0)  = 1948 (0x79c)
> 17407: write(1,"63\n",3) = 3 (0x3)
> 17407: pipe2(0x63857dad22e8,O_CLOEXEC)   = 0 (0x0)
> 17407: fstat(3,{ mode=p- ,inode=1610975,size=0,blksize=4096 }) =
> 0 (0x0)
> 17407: ioctl(3,TIOCGETA,0x63857dad21a0)  ERR#25 'Inappropriate
> ioctl for device'
> 17407: lseek(3,0x0,SEEK_CUR) ERR#29 'Illegal seek'
> 17407: pipe2(0x63857dad21e8,O_CLOEXEC)   = 0 (0x0)
> 17407: sigprocmask(SIG_BLOCK,{
> SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|
> 
> SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
> 97164: 
> 17407: fork()    = 97164 (0x17b8c)
> 97164: thr_self(0x1265c7e9000)   = 0 (0x0)
> 17407: sigprocmask(SIG_SETMASK,{ },0x0)  = 0 (0x0)
> 97164: sigprocmask(SIG_SETMASK,{ },0x0)  = 0 (0x0)
> 97164: close(3)  = 0 (0x0)
> 97164: close(5)  = 0 (0x0)
> 17407: close(6)  = 0 (0x0)
> 97164: dup2(4,1) = 1 (0x1)
> 17407: close(4)  = 0 (0x0)
> 97164: dup2(4,2) = 2 (0x2)
> 97164: sigprocmask(SIG_SETMASK,{
> SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPRO
> 
> F|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
> 97164: sigaction(SIGPIPE,{ SIG_DFL 0x0 ss_t },{ SIG_IGN 0x0 ss_t }) = 0
> (0x0)
> 97164: sigprocmask(SIG_SETMASK,{ },0x0)  = 0 (0x0)
> 97164: sigprocmask(SIG_SETMASK,{
> SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPRO
> 
> F|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
> 97164: sigaction(SIGXFSZ,{ SIG_DFL 0x0 ss_t },{ SIG_IGN 0x0 ss_t }) = 0
> (0x0)
> 97164: sigprocmask(SIG_SETMASK,{ },0x0)  = 0 (0x0)
> 97164: fstatat(AT_FDCWD,"/dev",{ mode=dr-xr-xr-x
> ,inode=2,size=512,blksize=4096 },0x0) = 0 (0x0)
> 97164: fstatat(AT_FDCWD,"/dev/fd",{ mode=dr-xr-xr-x
> ,inode=135,size=512,blksize=4096 },0x0) = 0 (0x0)
> 97164: getrlimit(RLIMIT_NOFILE,{ cur=940563,max=940563 }) = 0 (0x0)
> 97164: close(3)  ERR#9 'Bad file
> descriptor'
> 97164: close(4)  = 0 (0x0)
> 97164: close(5)  ERR#9 'Bad file
> descriptor'
> 97164: close(7)  ERR#9 'Bad file
> descriptor'
> 97164: close(8)  ERR#9 'Bad file
> descriptor'
> 97164: close(9)  ERR#9 'Bad file
> descriptor'
> 
> The last error repeats several 1 times.
> 
> Mounting fdescfs does indeed speed python3 up to python2 levels.
> 
>> On 11.04.19 15:16, Alexander Zagrebin wrote:
>>> Hi!
>>>
>>> I've noticed the subprocess performance issue with python 3.
>>> For example, this simple script takes just 0,15 second to complete
>>> with python 2.7, but more than 5 sec with python 3.6:
>>>
>>> import subprocess
>>>
>>> for i in range(100):
>>>     p = subprocess.Popen(['uname', '-a'], stdout=subprocess.PIPE,
>>> stderr=subprocess.STDOUT)
>>>   

Re: python 3 subprocess performance

[Jan Bramkamp]

Please run python under truss -f. Does it try to close(2) all possible
file descriptors? Does the runtime decrease if fdescfs is mounted at
/dev/fd?

On 11.04.19 15:16, Alexander Zagrebin wrote:
> Hi!
> 
> I've noticed the subprocess performance issue with python 3.
> For example, this simple script takes just 0,15 second to complete
> with python 2.7, but more than 5 sec with python 3.6:
> 
> import subprocess
> 
> for i in range(100):
> p = subprocess.Popen(['uname', '-a'], stdout=subprocess.PIPE, 
> stderr=subprocess.STDOUT)
> (stdoutdata, stderrdata) = p.communicate()
> 
> Profiling with the cProfile shows, that this excessive 5 seconds was
> wasted in the "{built-in method posix.read}"
> 
> Could anybody confirm this issue?
> 
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: WireGuard for FreeBSD

[Jan Bramkamp]

On 25.05.18 15:11, Bernhard Fröhlich wrote:

On Fri, May 25, 2018 at 12:05 PM, Jan Bramkamp <cr...@rlwinm.de> wrote:



On 25.05.18 09:29, Bernhard Fröhlich wrote:


On Fri, May 25, 2018 at 12:24 AM, Chris H <portmas...@bsdforge.com> wrote:


On Thu, 24 May 2018 22:16:42 +0200 "Bernhard Froehlich"
<de...@bluelife.at>
said


Am 24.05.2018 21:06 schrieb Chris H <portmas...@bsdforge.com>:



On Thu, 24 May 2018 19:39:22 +0200 "Jason A. Donenfeld"
<ja...@zx2c4.com>
said >


Hi Chris, > > > > On Thu, May 24, 2018 at 3:38 PM, Chris H
<portmas...@bsdforge.com> wrote: > > > I should have no trouble
introducing
Wireguard to the ports system today.


I'm not a native fluent speaker of FreeBSDese, but my
understanding is: > > a) Bernhard committed the two new packages to
ports
today. > > b) If you update ports with portsnap, you can build them
locally.


c) If you run `pkg install wireguard`, it fails because the build




servers haven't gotten to them and won't for several days. > > > >
Does your
statement about "introducing WireGuard to the ports system" > >
mean that
you intend to rectify (c) immediately, so we don't have to > > wait
several
days for the build snapshot scripts to tick in cron? Or > > is it
mostly
just related to not realizing (a)? > Sigh... > It was my
understanding that
when I stepped up to adopt WireGuard, > and your ack to that. That
*I* would
be adding the port. I wasn't able > to produce the port that same,
or next
day, as I am already Maintainer > for nearly 150 ports. I have no
trouble
with that list, except that > clang/llvm v5, and shortly after v6
became the
default versions in $BASE. > Which introduced a few pr(1)'s I
needed to deal
with. > Now all the time I have spent researching, and staging to
build the
port > have been laid to waste. Apparently you rescinded, and gave
it to
Bernhard. > This project doesn't feel like a good match to me. > No
hard
feelings, Bernhard. Have fun with the port.


Hi Chris,

I'm sorry that I was confusing people which was really not my intention.
I
have also seen your ACK to create the ports and replied to you in
private
to
offer my help. Then I joined in IRC and just wanted to get an idea how
far
the FreeBSD support was. I ended up creating two very rough ports which
did
build but not pass poudriere and called it a day. I also did send you
and
the
list a mail to avoid duplicate work - and hoped you take it as a base.

But I did not get any reply on the next day so I kept going and finished
the
ports yesterday with some good help from upstream.

Sorry for how that developed but I hoped you get in contact with either
me
or
upstream which neither happened. We usually do not have the problem that
too
many people want to help out so I did not expect that this will be a
problem
for anyone.



Ahem. OK thank you for the kind words, and intentions, Bernhard. Like I
said;
no hard feelings. If you've already gotten that far. You might as well
finish.
FWIW while you *did* indeed shoot me, and the list a couple of notes. I
was
never under the impression you were going to take it so far. Which
*ultimately*
left everyone concerned believing *you* were going to maintain it.
I only mention it, in hopes all of us might use the --verbose switch in
the
future, in hopes of avoiding this sort of nonsense. :-) :-)

Thanks again, Bernhard!

--Chris

P.S. just in case it wasn't clear; feel free to finish, and submit your
work.
P.P.S. Just so you (and everyone else) knows; I'm already working on the
kernel module. Please keep in touch, should you also be interested, and
have
any work of your own.



Hi chris,

to be crystal clear about that. My motivation is not to be maintainer
of any specific
port or anything like that but only to have technology available on
FreeBSD that I
personally need and/or want.

Usually for more complex ports this did lead to team efforts on our
porting work
which was also what I did expect to happen for wireguard. Well it
turned out to be
easier than thought and upstream was also very helpful so in the end
that was more
like a one day of work effort to get the basic ports.

Nevertheless I would still be very happy to increase the bus factor
and team up with
multiple people to maintain wireguard. I think there will be more work
to be done in the
near future for wireguard on FreeBSD where a team effort would speed
up things for
sure:

- we need to support FreeNAS and pfsense to get it into their package
systems
- documentation is still needed because it differs a bit from upstream
documentation (Handbook page?)
- wireguard kernel module (can that work already be seen somewhere?
upstream will be interested for sure)
- rc script(s)
- the regular maintenance for the port



The wireguard userspace tooling isn't that simple to use reliably. You have
to spawn the wireguard-go process before th

Re: WireGuard for FreeBSD

[Jan Bramkamp]

On 24.05.18 13:07, Jason A. Donenfeld wrote:

On Thu, May 24, 2018 at 12:43 PM, Jan Bramkamp <cr...@rlwinm.de> wrote:

Did I understand correctly that both these ports are userspace
implementations and have a similar per packet overhead to OpenVPN and fastd?


Indeed they're userspace ports. Maybe down the line this will be
ported to the FreeBSD kernel like we have on Linux.
However, performance wise, even the userspace implementation seems to
have better performance than OpenVPN in my testing.


I tried wireguard-go on OpenBSD and FreeBSD. I want to use WireGuard as 
replacement for OpenVPN point to point tunnels with dynamic routing 
(OSPF, iBGP). Especially this requires the right interface flags for the 
tun interface. So far wireguard-go on *BSD configures the tun interfaces 
as multicast incapable, broadcast interface which confuses the OpenBSD 
OSPF daemon completely and doesn't make any sense for a point to point 
tunnel. I get that wireguard-go tries to fake point to multipoint 
support that way. Is there a better solution than changing the hardwired 
argument ioctl() in tun/tun_*bsd.go?

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: WireGuard for FreeBSD

[Jan Bramkamp]

On 25.05.18 09:29, Bernhard Fröhlich wrote:

On Fri, May 25, 2018 at 12:24 AM, Chris H  wrote:

On Thu, 24 May 2018 22:16:42 +0200 "Bernhard Froehlich" 
said


Am 24.05.2018 21:06 schrieb Chris H :


On Thu, 24 May 2018 19:39:22 +0200 "Jason A. Donenfeld"

said >

Hi Chris, > > > > On Thu, May 24, 2018 at 3:38 PM, Chris H
 wrote: > > > I should have no trouble introducing
Wireguard to the ports system today.

I'm not a native fluent speaker of FreeBSDese, but my
understanding is: > > a) Bernhard committed the two new packages to ports
today. > > b) If you update ports with portsnap, you can build them locally.

c) If you run `pkg install wireguard`, it fails because the build > >

servers haven't gotten to them and won't for several days. > > > > Does your
statement about "introducing WireGuard to the ports system" > > mean that
you intend to rectify (c) immediately, so we don't have to > > wait several
days for the build snapshot scripts to tick in cron? Or > > is it mostly
just related to not realizing (a)? > Sigh... > It was my understanding that
when I stepped up to adopt WireGuard, > and your ack to that. That *I* would
be adding the port. I wasn't able > to produce the port that same, or next
day, as I am already Maintainer > for nearly 150 ports. I have no trouble
with that list, except that > clang/llvm v5, and shortly after v6 became the
default versions in $BASE. > Which introduced a few pr(1)'s I needed to deal
with. > Now all the time I have spent researching, and staging to build the
port > have been laid to waste. Apparently you rescinded, and gave it to
Bernhard. > This project doesn't feel like a good match to me. > No hard
feelings, Bernhard. Have fun with the port.

Hi Chris,

I'm sorry that I was confusing people which was really not my intention. I
have also seen your ACK to create the ports and replied to you in private
to
offer my help. Then I joined in IRC and just wanted to get an idea how far
the FreeBSD support was. I ended up creating two very rough ports which
did
build but not pass poudriere and called it a day. I also did send you and
the
list a mail to avoid duplicate work - and hoped you take it as a base.

But I did not get any reply on the next day so I kept going and finished
the
ports yesterday with some good help from upstream.

Sorry for how that developed but I hoped you get in contact with either me
or
upstream which neither happened. We usually do not have the problem that
too
many people want to help out so I did not expect that this will be a
problem
for anyone.


Ahem. OK thank you for the kind words, and intentions, Bernhard. Like I
said;
no hard feelings. If you've already gotten that far. You might as well
finish.
FWIW while you *did* indeed shoot me, and the list a couple of notes. I was
never under the impression you were going to take it so far. Which
*ultimately*
left everyone concerned believing *you* were going to maintain it.
I only mention it, in hopes all of us might use the --verbose switch in the
future, in hopes of avoiding this sort of nonsense. :-) :-)

Thanks again, Bernhard!

--Chris

P.S. just in case it wasn't clear; feel free to finish, and submit your
work.
P.P.S. Just so you (and everyone else) knows; I'm already working on the
kernel module. Please keep in touch, should you also be interested, and have
any work of your own.


Hi chris,

to be crystal clear about that. My motivation is not to be maintainer
of any specific
port or anything like that but only to have technology available on
FreeBSD that I
personally need and/or want.

Usually for more complex ports this did lead to team efforts on our porting work
which was also what I did expect to happen for wireguard. Well it
turned out to be
easier than thought and upstream was also very helpful so in the end
that was more
like a one day of work effort to get the basic ports.

Nevertheless I would still be very happy to increase the bus factor
and team up with
multiple people to maintain wireguard. I think there will be more work
to be done in the
near future for wireguard on FreeBSD where a team effort would speed
up things for
sure:

- we need to support FreeNAS and pfsense to get it into their package systems
- documentation is still needed because it differs a bit from upstream
documentation (Handbook page?)
- wireguard kernel module (can that work already be seen somewhere?
upstream will be interested for sure)
- rc script(s)
- the regular maintenance for the port


The wireguard userspace tooling isn't that simple to use reliably. You 
have to spawn the wireguard-go process before the config can be loaded 
and it can die in the meantime and to you want to terminate it and 
destroy the tun interface if the config contains errors. Doing this 
without ugly hacks isn't possible given the interfaces offered by 
wireguard-go. It would be really nice to be able to terminate 

Re: WireGuard for FreeBSD

[Jan Bramkamp]

On 24.05.18 09:15, Bernhard Fröhlich wrote:

On Thu, May 24, 2018 at 3:06 AM, Jason A. Donenfeld  wrote:

We now have a release, so the full instructions for the packages are:

1. wireguard-tools, providing wg(8) and wg-quick(8)
Runtime dependencies: bash, wireguard-go
Buildtime dependencies: gmake, c compiler, libc
Build: gmake -C src/tools WITH_WGQUICK=yes
Install: gmake -C src/tools PREFIX=/usr/local install
URL: https://git.zx2c4.com/WireGuard/snapshot/WireGuard-0.0.20180524.tar.xz

2. wireguard-go
Runtime dependencies: libc
Buildtime dependencies: gmake, go, dep
Build: gmake
Install: gmake PREFIX=/usr/local install
URL: 
https://git.zx2c4.com/wireguard-go/snapshot/wireguard-go-0.0.20180524.tar.xz

I believe decke is already working on a port in his repository.


Ports are already updated on github. I will do some final checks and
expect to commit
the wireguard ports to the official tree today.


Did I understand correctly that both these ports are userspace 
implementations and have a similar per packet overhead to OpenVPN and fastd?

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: saslauthd ldap connection to Samba AD

[Jan Bramkamp]

I use saslauthd with GSSAPI (aka Kerberos) authentication inside jails. 
IIRC you have to install a second package to add GSSAPI support to 
saslauthd.


On 18.04.18 15:49, Andrea Venturoli wrote:

Hello.

I'm trying to setup saslauthd (in a jail) to authenticate to a Samba AD 
DC (in another jail) and I'm hitting my head on this.


I'm stuck at the connection stage where the two will refuse to talk to 
each other: before I spend too many hours on this: is it expected to work?


  bye & Thanks
 av.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: THANK YOU for flavors!

[Jan Bramkamp]

On 07.12.17 13:51, Baho Utot wrote:


users also don't want to wake up to the fact that what worked a few days 
ago not is working ie fetch/update port repo run synth and then get a " 
What the fuck has just happened now" only to be put down by the so 
called folks in charge here for even asking.


Shut the fuck up until you have something more than your whining to 
offer. *PLONK*

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Flavors *COMPLETELY* break the port system (synth and poudriere are useless)

[Jan Bramkamp]

On 06.12.17 23:23, Dave Horsfall wrote:

On Wed, 6 Dec 2017, Jan Bramkamp wrote:

Synth and poudriere are parallel build tools and as such are very 
taxing on the system. I suspect your system is unstable under such 
load, because of a configuration error or unreliable hardware. One 
such configuration error that bit me is tmpfs mounted without size 
limitation. Without size limits it can exhaust RAM + swap and crash 
the system. Limit the sum of all your tmpfs mounts to significantly 
less than RAM + swap.


What happens then?  Does the build process merely crash instead?  I ask 
because my minimal system (all I can afford on my "income") has 512MB 
memory (all it will take) ad 1GB swap; building Ruby etc kills it, so I 
use packages in that case i.e. no customisation if I wanted it.


In that case I wouldn't use tmpfs at all. One conceptual difference 
between the old portmaster/portupgrade tools and the newer tools like 
poudriere and synth is that the old tools try to minimize rebuilds by 
modifying the live system. This can break the system if the upgrade 
fails and will leave it inconsistent while the upgrade runs. Also some 
ports misbehave and interact with installed software they didn't list as 
dependency if it is installed. For these reasons the new tools create 
jail/chroot environments to build ports in a clean environment. 
Poudriere is designed around ZFS features (snapshots and cloning). Those 
can be emulated with UFS and overlay file systems, but Poudriere works 
best on a big ZFS based system with enough RAM to keep the build dirs in 
tmpfs (e.g. 8GB RAM per parallel builder and one builder per CPU 
core/thread). You can configure poudriere to use a single builder and 
UFS, but it will be a slow process.


Synth reduces that overhead somewhat by reusing the host system. Its 
focus is more on keeping a single system up to date instead of compiling 
sets of ports to custom repos for other systems and the curses UI is a 
nice touch. Also synth can try to avoid building ports by prefetching 
packages from an upstream repo.


To be honest few FreeBSD devs still care about self hosting FreeBSD on 
such tiny systems (0.5GB RAM, 1-2 cores). There are devs interested on 
optimizing FreeBSD for small embedded systems but you aren't expected to 
rebuild FreeBSD from source on a wireless access point or dedicated 
firewall appliance. If you have to compile and value your time get 
adequate hardware. Old 2U dual sockets servers may burn a lot of power 
but are quite cheap and you don't have to run a compile server 24/7.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Flavors *COMPLETELY* break the port system (synth and poudriere are useless)

[Jan Bramkamp]

On 05.12.17 10:53, Aryeh Friedman wrote:

First some background (my typical use cases for ports):

0. FreeBSD lilith 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321664: Fri Jul 28
23:35:18 EDT 2017 root@lilith:/usr/obj/usr/src/sys/GENERIC  amd64

1. Daily routine (current):

   cd /usr/src
   svn update (from 11.1-RELEASE)
   [make -DESTDIR=/ world kernel&& as needed]

   cd /usr/ports
   svn update (from HEAD)
   portmaster -ad
   [reboot if any Xorg/xfce or stuff in rc.d got modified]

2. I maintain devel/aegis (which as per Bug 219284 does not compile with
anything greater then GCC 5 [I don't have time to figure out how to patch
it is make it work {the upstream maintainer died a few years ago}]).

So what happens when I see UPDATING 20171130:

1. I decide to try poudriere since it seems to what people are raving
about.   What a 'ing confusing mess it is use After deciding it
is over kill I go to option 2

2. I installed synth (2.00) and *ATTEMPTED* to do a upgrade-system with the
following results (still not a successful run):

 a. Hard freezes the machine (not even a kernel panic) 4 times in a row

 b. Skips devel/aegis recompile because it can't understand the makefile
or something (see above).  *BUT* gives no clues as to why and gives me
nothing actionable on how to repair the port


Synth and poudriere are parallel build tools and as such are very taxing 
on the system. I suspect your system is unstable under such load, 
because of a configuration error or unreliable hardware. One such 
configuration error that bit me is tmpfs mounted without size 
limitation. Without size limits it can exhaust RAM + swap and crash the 
system. Limit the sum of all your tmpfs mounts to significantly less 
than RAM + swap.



3. The suggestion of using plainly old make install on each port is
unworkable because it is fundamentally error prone with my daily use case.


Nobody suggested doing that. You're just running abandonware and the 
ports tree broke compatiblity that was never promised. Your options are 
to move to a supported wrapper around the ports tree or to take up fix 
your existing wrapper to deal with the change.



TL;DR;
Flavors 'ed up ports and there are no good ways/alternates for how
to use the ports collection for normal everyday users/maintainers


For a single system synth works well and is easy to setup. If it crashes 
your system something is very wrong indeed. Userspace software shouldn't 
be able to crash the system. Especially not by accident. This a bug that 
should be fixed.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Spam on -ports

[Jan Bramkamp]

On 27.10.17 01:24, Mark Linimon wrote:

On Fri, Oct 27, 2017 at 07:50:40AM +1100, Dave Horsfall wrote:

Is it official FreeBSD policy to allow spammers free reign on this
list, or is the list owner merely incompetent?

or is it 3) spam is a never-ending arms race, and the volunteers are
struggling to keep up?
It is and low latency human moderation of all mailinglists is an 
unrealistic expectation.

Honestly, I do not understand why people make posts like this.
Most precise low maintenance spam filtering works on meta that gets 
mangled too badly by mailing lists to be usable afterwards. From the 
point of view of spam filters the messages relayed by the mailing list 
originate in from a valid MX of freebsd.org. Combining a strict postfix 
setup with postscreen and postgrey solved the spam problem for my 
domains with one exception: the FreeBSD mailing lists. To keep the 
signal to noise ratio on those acceptable I had to deploy a content 
based spam filter and have not yet found a good enough configuration for 
it. While not fair it is very tempting to blame the operators and I'm 
surprised how some spam campaigns manage to get unchanged content 
relayed through the FreeBSD mailing lists for months.

Speaking as someone who, behind the scenes, helps people try to stay
motivated, all you're doing is make other people's lives miserable --
including mine.
I would like to help fighting the spam problem on the FreeBSD.org 
mailing lists. Can you put me in contact with the right people?


-- Jan Bramkamp
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Chicken/egg problem with pkg

[Jan Bramkamp]

On 10/03/2017 17:04, Kurt Jaeger wrote:

Hi!


I have an old web server (10.1-RELEASE-p9) which is running for years


I had one with 9.3 something, and upgrading it from the pkg repo
took not a huge amount of time, yesterday evening...

So maybe upgrading is a good thing, if not too many fancy things
are active.


You can use pkg upgrade -F to prefetch the packages if you connection 
and/or mirror is slow.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Chicken/egg problem with pkg

[Jan Bramkamp]

On 10/03/2017 15:08, Konstantin Belousov wrote:

On Fri, Mar 10, 2017 at 02:55:13PM +0100, Jan Bramkamp wrote:

Among other
things FreeBSD 10.3 added a bunch of new *at() system calls like openat.
These *at() system calls are useful inside (capsicum) sandboxes. Your
old 10.1 kernel lacks those systems calls and your old 10.1 libc lacks
the stubs to call them anyways. It is this missing stub that causes the
new libpkg.so to fail to link.


Although (removed) rest of your mail is mostly accurate, the cited part
is explicitely false. The openat(2) syscall and friends exist even in
FreeBSD 8.x.  What has changed in 10.2->10.3 is that the version for
openat symbol in libc has to be bumped due to some issue with libthr. As
result, newer binaries require a symbol version which does not exist in
older libc.


Thanks for the correction.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Chicken/egg problem with pkg

[Jan Bramkamp]

On 10/03/2017 14:47, Guido Falsi wrote:

On 03/10/17 14:26, Hans de Hartog wrote:

I have an old web server (10.1-RELEASE-p9) which is running for years

without any upgrades. Now I want to install a simple port (trafshow, to

see what's going on).

It first want to upgrade pkg from 1.5.1 to 1.10.0_2 but that doesn't
work:

[1/1] Upgrading pkg from 1.5.1 to 1.10.0_2...
[1/1] Extracting pkg-1.10.0_2: 100%
/usr/local/lib/libpkg.so.4: Undefined symbol "openat"

Anything I try to do with pkg now gives me this error-message.

/var/log/messages says: pkg: pkg upgraded: 1.5.1 -> 1.10.0_2

How do I proceed from here (without upgrading everything, please)?



Have you tried using the "pkg-static" command? It's the same as pkg, but
statically linked, should sidestep your issue.


The old pkg-static might work, but it would wreck his system because he 
uses a repo with an ABI unsupported by his base + kernel system. He is 
lucky he immediately triggered the incompatibility.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Chicken/egg problem with pkg

[Jan Bramkamp]

On 10/03/2017 14:26, Hans de Hartog wrote:

I have an old web server (10.1-RELEASE-p9) which is running for years

without any upgrades. Now I want to install a simple port (trafshow, to

see what's going on).

It first want to upgrade pkg from 1.5.1 to 1.10.0_2 but that doesn't work:

[1/1] Upgrading pkg from 1.5.1 to 1.10.0_2...
[1/1] Extracting pkg-1.10.0_2: 100%
/usr/local/lib/libpkg.so.4: Undefined symbol "openat"

Anything I try to do with pkg now gives me this error-message.

/var/log/messages says: pkg: pkg upgraded: 1.5.1 -> 1.10.0_2

How do I proceed from here (without upgrading everything, please)?


You dug yourself a deep hole by letting your system fall that far 
behind. Both FreeBSD 10.1 and 10.2 are EoL since 2016-12-31 
(https://www.freebsd.org/security/unsupported.html). While FreeBSD is 
backward compatible (e.g. you can run binaries for 10.1 on 10.3) it 
isn't forward compatible (e.g. you can't run FreeBSD 10.3 binaries on 10.1).


In your case the problem is that the official package build jails run 
the oldest supported minor release inside each major release. After 
2016-12-31 the build jails migrated from 10.1 to 10.3. Among other 
things FreeBSD 10.3 added a bunch of new *at() system calls like openat. 
These *at() system calls are useful inside (capsicum) sandboxes. Your 
old 10.1 kernel lacks those systems calls and your old 10.1 libc lacks 
the stubs to call them anyways. It is this missing stub that causes the 
new libpkg.so to fail to link.


Running unpatched networked systems is a bad habit in most cases, but it 
is reckless to run an unpatched webserver, because webservers offer a 
large attack surface to the network.


Your simplest way forward is to update to the latest patchlevel and than 
migrate to a supported release (at least 10.3 as of writing). Afterward 
you can upgrade your packages from the default repos.


Upgrading to a newer minor release of the same major release is fast and 
painless with freebsd-update. Upgrading to the next major release 
shouldn't cause any problems, but you have to reinstall all packages. 
This was a major annoyance before pkgng, but these days changing the 
repo ABI and a single `pkg upgrade` does that in a few minutes.


Or you write this system off as probably compromised and rebuild it from 
scratch ;-).

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: The future of portmaster

[Jan Bramkamp]

On 17/02/2017 09:25, Matthieu Volat wrote:

On Fri, 17 Feb 2017 10:37:16 +0300
abi  wrote:


17.02.2017 00:22, Chris H пишет:

On Thu, 16 Feb 2017 15:48:57 -0500 Baho Utot  wrote


On 02/16/17 15:40, George Mitchell wrote:

On 02/16/17 15:33, Baho Utot wrote:


On 02/16/17 14:01, Lowell Gilbert wrote:

Baho Utot  writes:


On 02/16/17 06:08, Luca Pizzamiglio wrote:

I'm looking for constructive critics, feedbacks, anything that can
help me to make portmaster an actively maintained and used tool.

If you can have it build in a clean chroot or jail then you'll get my
attention

What kind of special support?

I use it with a chroot that mounts /usr/ports (and src) read-only, and
aside from the initial base system install, it took about fifteen
minutes to set up.


Using chroot or jails to build each individual package
[...]

While I understand the interest in chroot/jails as an optional
feature, I hope it doesn't become required.  The current non-use
of chroot/jails is, for me, a feature -- not a bug.-- George



Having built and packaged linux from scratch using the rpm package
manager, I came to find that if one is building packages to be used on
multiple machines, one needs to build each package in a chroot
environment or the package could inherit things from the parent not
found in the target machine.  Here by making the package unusable.

Hello. You shouldn't have any difficulty accomplishing your goal
by simply setting up a jail, and using portmaster within that jail(8).
portmaster really doesn't care where it's run. So long as it has
everything it needs to accomplish it's job(s). :-)


 From my point of view, jails are overkill. Chroot should be enough and
it would be nice if portmaster starts building in clean environment.


Just dropping privileges to a dedicated user for building would be a big step, 
but that's more a port feature (openbsd's ports do that, if I'm not wrong).


Yes dropping privileges would be a good *additional* step. The purpose 
of the jail/chroot isn't just for security. The real goal is to provide 
a reproducible, clean build environment. Lots of broken configure 
scripts out there include a lot of autodetection magic. And suddenly 
your binaries are link against additional libraries which are unknown to 
pkg. This becomes even funnier if your application uses fork+exec per 
connection. Suddenly you're left with a bound socket but each connection 
dies because the worker fails to link at runtime. This was the straw 
that broke the camels back for me. An other problem with portmaster is 
that it creates inconsistent during the upgrade by design. Of course pkg 
upgrade isn't atomic either but the time window is on the order of a few 
seconds instead of minutes to hours and is far less likely fail halfway 
through.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: openldap-client vs openldap-sasl-client

[Jan Bramkamp]

On 05/01/2017 11:56, Franco Fichtner wrote:



On 5 Jan 2017, at 11:44 AM, Julian Elischer <jul...@freebsd.org> wrote:

On 5/01/2017 6:30 PM, Jan Bramkamp wrote:

On 04/01/2017 18:32, Andriy Gapon wrote:


Do you I understand correctly that it is impossible now to install both samba44
and libreoffice using the official FreeBSD package repository?
Or samba44 and KDE?

If yes, then that sucks...


similar happened recently with the two jpeg libraries.
They can't be installed at the same time but some packages wanted one and some 
the other.


The OpenLDAP package state is a bit behind more modern ports framework
approaches.  Fixing the offending packages away from OpenLDAP is nice,
but eventually the issues will reappear port for port, time after time.

If we strive for default ports options that are sane for most users,
globally setting WANT_OPENLDAP_SASL=yes is the way to prevent that
from happening again.

There is probably a very valid historic reason for not having done so,
but people can still build their own ports without SASL if they want and
incompatibility issues are unlikely when the support is built in.  At
least we haven't seen anything in the past 6 months in OPNsense since we
switched to avoid this in our build runs.

And besides, having a package name flip-flop using arcane toggles should
be removed as it breaks POLA.

Long story short: make SASL an OPTION, add it to defaults, don't mess
with the package name anymore?


That would be my prefered short term solution. Can we get some input 
from the ports maintainer? Maybe there is still a good reason for the 
current state of affairs.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: openldap-client vs openldap-sasl-client

[Jan Bramkamp]

On 05/01/2017 12:04, Matthew Seaman wrote:

On 01/05/17 10:43, Kurt Jaeger wrote:

Hi!

[openldap-client vs openldap-sasl-client and libreoffice etc]

Yes and yes it sucks. The "solution" is to build your own repo and set
the right flags to always use the same LDAP client port. With binary
packages and the speed of modern x86_64 systems I for one no longer see
removing SASL support from OpenLDAP as useful enough to justify the
complexity.


The other question is: What's the use of SASL anyway ? I've seen it
for years in mailserver setups, etc, and it always caused trouble.



SASL effectively gets you a number of new authentication mechanisms.
Most of these are ways of proving you know a secret without sending the
actual secret (ie. password) over the net in plain text, but I think it
also adds the ability to use client TLS certificates for authentication.
IIRC.

I don't see much value in the extra mechanisms for secure login over
unencrypted links nowadays.  Pretty much everything I'm using currently
already requires TLS for good security reasons, so there's no real
downside to using plain LOGIN over the encrypted channel.  Plus the
'proof of knowledge' authentication mechanisms have a big downside: they
need the secret stored in the LDAP database in plain text, or in some
locally reversible encryption.  With LOGIN over TLS, I can use salted
password hashes in much the same way as Unix passwords.

SASL would be worth it for TLS client certificate functionality, if
that's the only way to enable that.


SASL external is required by OpenLDAP to extract authentication data 
from TLS client certificates and not all SASL mechs require plaintext 
passwords inside the database to avoid leaking the credentials over the 
wire e.g. SASL SCRAM (RFC 5802). You can also use SASL to authenticate 
against PAM.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: openldap-client vs openldap-sasl-client

[Jan Bramkamp]

On 05/01/2017 11:43, Kurt Jaeger wrote:

Hi!

[openldap-client vs openldap-sasl-client and libreoffice etc]

Yes and yes it sucks. The "solution" is to build your own repo and set
the right flags to always use the same LDAP client port. With binary
packages and the speed of modern x86_64 systems I for one no longer see
removing SASL support from OpenLDAP as useful enough to justify the
complexity.


The other question is: What's the use of SASL anyway ? I've seen it
for years in mailserver setups, etc, and it always caused trouble.


SASL is a authentication API. It allows an application to reuse the 
tested SASL protocol and code. SASL differs from PAM in that it allows 
its authentication mechanisms more possibilities as required to deal 
with X.509 client certificates and Kerberos tickets. Without SASL 
OpenLDAP is limited to password based authentication against passwords 
stored inside the LDAP database as plaintext or salted MD5/SHA1 hashes. 
With SASL you can the supported SASL mechanisms:


* SASL external: Let the transport layer provide authentication. TLS 
protected connections can reuse the client certificate common name and 
map it to a LDAP object. Local connections over UNIX domain sockets can 
use the effective client user and primary group id to search for a 
matching user object inside the LDAP database.


* SASL GSSAPI: Use Kerberos to authenticate against the LDAP service. 
Works great for interactive use if you already have Kerberos deployed. 
Can be used to centralize password storage in the Kerberos KDCs instead 
of the LDAP service while still supporting LDAP simple binds.


In theory you could use a bunch of additional SASL mechs but these are 
the two I have encountered in everyday use.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: openldap-client vs openldap-sasl-client

[Jan Bramkamp]

On 04/01/2017 18:32, Andriy Gapon wrote:


Do you I understand correctly that it is impossible now to install both samba44
and libreoffice using the official FreeBSD package repository?
Or samba44 and KDE?

If yes, then that sucks...


Yes and yes it sucks. The "solution" is to build your own repo and set 
the right flags to always use the same LDAP client port. With binary 
packages and the speed of modern x86_64 systems I for one no longer see 
removing SASL support from OpenLDAP as useful enough to justify the 
complexity. Are there any reasons other than saved build time to disable 
this dependency (e.g. a bad security track record/process, different 
licenses)?

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: harder and harder to avoid pkg

[Jan Bramkamp]

On 14/10/2016 09:39, Julian Elischer wrote:

On 13/10/2016 10:33 AM, RW via freebsd-ports wrote:

On Tue, 11 Oct 2016 11:59:47 -0700
Julian Elischer wrote:


As the number of dependencies between packages get ever higher, it
becomes more and more difficult to compile packages and the
dependence on binary precompiled packages is increased. However
binary packages are unsuitable for some situations.  We really need
to follow the lead of some of the Linux groups and have -runtime and
-devel versions of packages,  OR  we what woudlbe smarter, woudl be
to have several "sub manifests" to allow unpacking in different
environments.


A simple example:   libxml2

This package installs include files and libraries and dicumentation
etc.

yet if I build an appliance , I want it to only install a singe file.

/usr/local/lib/libxml2.so.2

What practical problem does installing the include files and man pages
cause you?

I have to delete them from the appliance I'm building up.
So I need to get the manifest, remove the files I want from it, and
delete every other file mentioned.

This is an appliance class machine. It has 2G of storage and that has
to  include 2 copies for the OS so we can ping-pong for upgrades.


I can get > 2GB CPU cache per system (spread over 8+ sockets) these 
days. Is it really reasonable to expect port maintainers to take up the 
work and classify their maintained ports for you to save you an 
additional 2GB of cheap flash storage? At a certain scale those 
trade-offs might make sense for you, but I suspect most FreeBSD port 
maintainers and FreeBSD users don't mind a few 100 kB of documentation 
and headers on their systems. Aren't there easier solutions which don't 
require a lot of manual work?


 * Documentation and source code compresses well. Can you use a 
read-only lzma or gzip compressed filesystem with GEOM uncompress?


 * Can you use snapshots (and rerooting) to rollback failed updates 
instead of keeping two full copies around?


-- Jan Bramkamp
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Mailman in a jail

[Jan Bramkamp]

On 21/04/16 17:21, Jim Ohlstein wrote:

Hello,

I'm trying to get Mailman working in a 10.3 amd64 jail. Everything
works, except Mailman doesn't talk to Postfix. Incoming mail works and
posts to the list's archives but no outgoing email is sent. I asked in
the Mailman list and they seem to think it's related to running in a jail.

If anyone's gotten this running in a jail I'd appreciate some input. I'm
not married to Postfix - willing to use a different MTA.


- Did you enable Postfix in the mailer.conf?
- Are Postfix and mailman in the same jail?
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Maintainer for multimedia/zoneminder

[Jan Bramkamp]

On 02/03/16 16:15, Kristian Kræmmer Nielsen wrote:

Hi,

This mail-address is written in "MAINTAINER" for multimedia/zoneminder -
does that actually mean the port has no maintainer?


Exactly.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Packages lagging for 10.2

[Jan Bramkamp]

On 08/09/15 14:45, Kubilay Kocak wrote:

On 8/09/2015 5:21 PM, Matthew Seaman wrote:

On 08/09/2015 07:46, Jimmy Renner wrote:

I don't know if this is the right list to post to but for me lots (a bit
over 100) of ports are updated but if I do a pkg upgrade not much is
updated. This has been the case since I upgraded to 10.2. No complaints,
I was just wondering if I have done anything wrong :-)


You haven't done anything wrong.  You're just using the quarterly
package branch, which is a change in the defaults since 10.1-RELEASE.

Cheers,

Matthew




Do you know if this was -announce'd to users?

If not, there's no such thing as too much communication.


It was part of the release message. The -latest repos switched to a 
daily instead of weekly build schedule as well.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"