Re: pkg builders order of jobs

[Stefan Esser]

Am 09.05.21 um 12:22 schrieb Ronald Klop:
> Hi,
> 
> On the pkg builders the list of builds is ordered on git hash.
> http://beefy18.nyi.freebsd.org/jail.html?mastername=main-amd64-default
> (NB: ipv6 only)
> 
> This ordering is totally random to me. With svn the list was naturally ordered
> because of the incremental commit id.
> 
> Is it possible to add the commit count to the job-name like 
> sys/conf/newvers.sh
> in base?
> git_cnt=$($git_cmd rev-list --first-parent --count HEAD 2>/dev/null)

Adding the start-time of the build in ISO format would also be highly
appreciated, at least by me ;-)

The date should reflect the time of the last commit that has been
considered. Such a time stamp would let me see with little effort
whether a failure is due to a long running build job that did not
catch-up with the latest fixes to a port, or whether there still is
an issue with a port (especially for the "exotic" architectures that
I do not test on my system and that often need a lot longer to complete
a build job).

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: Making a port to use OpenSSL of ports collection on FreeBSD 11.x

[Stefan Esser]

Am 05.05.21 um 18:00 schrieb Yasuhiro Kimura:
> You misunderstand my intention. What I would like to do is to make a
> port use security/openssl instead of base OpenSSL even if user sets
> 'DEFAULT_VERSIONS+=ssl=base' in /etc/make.conf (or user doesn't
> customize setting about ssl at all).

You can mark the port as broken if the user has selected "base" and
this is incompatible with the port's requirements. E.g.:

.include 

.if ${SSL_DEFAULT} == base
BROKEN_FreeBSD_11=  OpenSSL 1.1 required
.endif

.include 

You can of course add a better message, and the user can decide to
try the compilation again after changing the default. But this will
of course prevent the building of official packages for FreeBSD-11.

> As I wrote previous mail, if 'USES=python:3.8+' is specified in
> Makefile of a port, lang/python38 is used for the port event if user
> adds 'DEFAULT_VERSIONS+=python=3.7' in /etc/make.conf. I'm looking for
> similar way about ssl setting.

You could add a dependency on the openssl port and make sure, that
the port's include file and library is used in preference of the
base version on FreeBSD-11.

But you have to consider the risk of mixing references to the base and
the ports version in one binary (e.g. other libraries are linked in that
had been built against the base version).

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: [SOLVED?] Recovery of deleted ports fails due to pre-commit checks

[Stefan Esser]

Am 04.05.21 um 11:46 schrieb Mathieu Arnold:
> On Mon, May 03, 2021 at 09:54:36PM +0200, Stefan Esser wrote:
>> Am 03.05.21 um 09:01 schrieb Mathieu Arnold:
>>> On Sat, May 01, 2021 at 09:01:02PM +0200, Stefan Esser wrote:
>>>> The recovery of deleted ports in their previous form is rejected
>>>> by the pre-commit checks on the repository server:
>>>>
>>>> remote:
>>>> remote: 
>>>> remote: Do not commit ports without TIMESTAMP in their distinfo files.
>>>> remote: Rerun make makesum to add it.
>>>> remote: 
>>>> remote:
>>>>
>>>> I have tried to revert the deletion with unchanged files and then
>>>> updated the ports' Makefiles and distinfo files in a later commit.
>>>>
>>>> Pushing those commits all together fails with the message above,
>>>> and in order to not confuse GIT, deleted files should be committed
>>>> first, before applying any changes.
>>>
>>> This is not needed at all, Git cannot get confused by something it has
>>> no knowledge of. Once a file is deleted, or moved, the history tracking
>>> stops.
>>
>> I wanted to re-connect the resurrected files to the history of the port.
>> And that works best, if unmodified files are committed first, changes
>> applied and committed thereafter.
>>
>> Did you try "git log multimedia/transcode"?
>>
>> The history is there, back to 2002.
> 
> Yeah, but this has nothing to do with you commiting unmodified files.
> Git does not track file renames or moves (or resurrection), it blindly
> looks at what you told it and goes as far as it can find things.

Yes, sure, but the general advice when moving around files in GIT
repositories is: First move and commit unchanged, then modify in
place and commit again. And I was under the impression that the
same advice applies to files that have been deleted and are brought
back - GIT can identify and reconnect them in a way that preserves
history only by guessing, and I wanted to make it as easy as possible
for GIT, since I have watched GIT to get trivial operations of that
kind wrong in grotesque ways ...

https://github.blog/2020-12-17-commits-are-snapshots-not-diffs/#since-commits-arent-diffs-how-does-git-track-renames



OpenPGP_signature
Description: OpenPGP digital signature

Re: [SOLVED?] Recovery of deleted ports fails due to pre-commit checks

[Stefan Esser]

Am 03.05.21 um 09:01 schrieb Mathieu Arnold:
> On Sat, May 01, 2021 at 09:01:02PM +0200, Stefan Esser wrote:
>> The recovery of deleted ports in their previous form is rejected
>> by the pre-commit checks on the repository server:
>>
>> remote:
>> remote: 
>> remote: Do not commit ports without TIMESTAMP in their distinfo files.
>> remote: Rerun make makesum to add it.
>> remote: 
>> remote:
>>
>> I have tried to revert the deletion with unchanged files and then
>> updated the ports' Makefiles and distinfo files in a later commit.
>>
>> Pushing those commits all together fails with the message above,
>> and in order to not confuse GIT, deleted files should be committed
>> first, before applying any changes.
> 
> This is not needed at all, Git cannot get confused by something it has
> no knowledge of. Once a file is deleted, or moved, the history tracking
> stops.

I wanted to re-connect the resurrected files to the history of the port.
And that works best, if unmodified files are committed first, changes
applied and committed thereafter.

Did you try "git log multimedia/transcode"?

The history is there, back to 2002.



OpenPGP_signature
Description: OpenPGP digital signature

[SOLVED?] Recovery of deleted ports fails due to pre-commit checks

[Stefan Esser]

The recovery of deleted ports in their previous form is rejected
by the pre-commit checks on the repository server:

remote:
remote: 
remote: Do not commit ports without TIMESTAMP in their distinfo files.
remote: Rerun make makesum to add it.
remote: 
remote:

I have tried to revert the deletion with unchanged files and then
updated the ports' Makefiles and distinfo files in a later commit.

Pushing those commits all together fails with the message above,
and in order to not confuse GIT, deleted files should be committed
first, before applying any changes.

Apparently this is not possible, and I cannot re-order the commits
in such a way, that the distinfo change is merged in to the initial
commit.

I'm now going to try with 10 commits squashed into 1, to make the
distinfo update appear to be part of the initial commit. This will
make it much harder to distinguish the reversal of the deletion
from later changes, though.


The squashed commit has worked - the multimedia/transcode port and
dependent ports that had been deleted should be restored and fixed.

It is thus possible to commit a reverted port, but I had rather kept
the commit log entries separate and more comprehensible.

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: SVNWEB not updated for ports

[Stefan Esser]

Am 19.04.21 um 14:42 schrieb Helge Oldach:
> Stefan Esser wrote on Mon, 19 Apr 2021 12:11:40 +0200 (CEST):
>> After the switch to GIT, the ports SVN repository is in a frozen state.
>>
>> This causes accesses to https://svnweb.freebsd.org/ports/head/ to return
>> stale information.
> 
> I guess it might be more appropriate to encourage 
> https://cgit.freebsd.org/ports/?h=main instead...

No, I'm talking about the path to the actual files and have created
a review:

https://reviews.freebsd.org/D29829

The fix is trivial and a functionally equivalent replacement of the
SVNWEB link (I just noticed that I had suggested a path that lead to
a different representation of the content and have updated the patch
that is attached to this review).

I'd suggest to test the suggested path of cgit.freebsd.org/ports/plain
by directly patching the file on the web-server just for the purpose of
verification of accesses still working, then to commit it to the doc
repository.

I cannot locally test the change, but if it has bad side-effects, it
can easily be reverted.

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

SVNWEB not updated for ports

[Stefan Esser]

After the switch to GIT, the ports SVN repository is in a frozen state.

This causes accesses to https://svnweb.freebsd.org/ports/head/ to return
stale information.

But the real issue IMHO is that the "FreeBSD Ports Search" page on the
web server does return 404 errors for ports that have been created in
GIT and did not exist in SVN.

I have created a review for the suggested fix:

https://reviews.freebsd.org/D29829




OpenPGP_signature
Description: OpenPGP digital signature

Re: Ports INDEX

[Stefan Esser]

Am 14.04.21 um 21:33 schrieb Carmel:

I just switched from using 'portsnap' to 'git'. I removed everything in
the directory, then used 'git' to build it. That worked fine. I had to
manually recreate the 'distfiles' directory, but I expected to have to
do that.

Now, if I run a 'make search' in the directory, I receive an error
message that it needs the INDEX file. I ran "make index" and that
seems to work alright. My question is, is that the best way to handle
this? Couldn't 'git' download the INDEX file, too. I assume that
'portsnap' was doing something like that.


You do not want to bloat the ports repository with the INDEX file.

It does not contain any useful "historical" information, only the
current version is ever relevant.

A mechanism that applies the relatively small deltas to the INDEX
file instead of having to download the compressed file with "make
fetchindex" could reduce the amount of data to transfer, though.

A mechanism like CTM could provide that. Else "make fetchindex" is
the fastest way to get the index, but only "make index" does also
reflect dependencies affected by locally set non-default port options.



OpenPGP_signature
Description: OpenPGP digital signature

Re: Deprecation of portsnap

[Stefan Esser]

Am 14.04.21 um 02:43 schrieb Chris:

On 2021-04-13 15:53, Dave Horsfall wrote:

On Mon, 12 Apr 2021, Peter Jeremy via freebsd-ports wrote:

Except that git will arbitrarily and randomly decide that it needs to 
run 
"gc" - which is similarly extravagant in memory usage.  Last time I 
found 

one running, it thrashed that poor VM for 3 days.


Would this be a good time to mention the https://ohshitgit.com/ site? 
Warning: it

contains strong language...

It would!
And the language is very appropriate, thank you. :-)


If you disagree regarding the appropriateness of the language,
there is:

https://dangitgit.com/

And in the upper right corner you'll find a language selection
list that gives access to some 20 translations.

Or just try the URL with your 2-letter country code added ...

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: FreeBSD Port: owncloud-php80-10.6.0

[Stefan Esser]

Am 19.03.21 um 10:23 schrieb Erik Neubauer via freebsd-ports:

Hi,

Thanks for maintaining an ownCloud package!

I noticed that you are using PHP 8.0. However ownCloud doesn't support 
that version of PHP.


Unfortunately I don't have any experience with BSDs, but perhaps I can 
help you create a package based on PHP 7.4?


Hi Sunpoet,

the following patch prevents use of PHP-8.0 and the creation of
a package for that PHP version:

Index: www/owncloud/Makefile
===
--- www/owncloud/Makefile   (revision 568788)
+++ www/owncloud/Makefile   (working copy)
@@ -18,6 +18,7 @@
 USE_PHP=   bz2 ctype curl dom fileinfo filter gd hash iconv intl json \
mbstring pdo posix session simplexml xml xmlreader xmlwriter \
xsl zip zlib
+IGNORE_WITH_PHP=   80

 NO_ARCH=   yes
 NO_BUILD=  yes

In order to get the current owncloud-php80 package removed from the
package mirrors, a PORT_REVISION bump will be required.

Let me know if I should go ahead and commit that change ...

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: On 14-CURRENT: no ports options anymore?

[Stefan Esser]

Am 13.03.21 um 20:17 schrieb Hartmann, O.:

Since I moved on to 14-CURRENT, I face a very strange behaviour when trying to 
set
options via "make config" or via poudriere accordingly. I always get "===> 
Options
unchanged" (when options has been already set and I'd expect a dialog menu).
This misbehaviour is throughout ALL 14-CURRENT systems (the oldest is at FreeBSD
14.0-CURRENT #49 main-n245422-cecfaf9bede9: Fri Mar 12 16:08:09 CET 2021 amd64).

I do not see such a behaviour with 13-STABLE, 12-STABLE, 12.2-RELENG.

How to fix this? What happened?


Hi Oliver,

please check your TERM setting and test with a trivial setting
if it is not one of xterm, vt100 or vt320 (for example).

I had this problem when my TERM variable was xterm-color, which
used to be supported but apparently no longer is.

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: Creating port from pre-built package

[Stefan Esser]

Am 11.02.21 um 18:12 schrieb Shawn Webb:

On Thu, Feb 11, 2021 at 06:09:52PM +0100, Patrick M. Hausen wrote:

Hi all,


Am 11.02.2021 um 18:03 schrieb Chris :

On 2021-02-11 08:26, Shawn Webb wrote:

Hey all,
The Splunk universal forwarder for FreeBSD is distributed as a package
tarball that you can use `pkg add` on. I'm in a position where I'd
like to create a port of the package so that I can automate certain
tasks.

Reverse engineer a package? I think that will violate the NDA you
signed. ;-)

Really. Unless the package is simply a wrapper of a binary blob. You'll
need to *build* the port, as part of the package creation process.
That is, unless I've *completely* misunderstood your intent here. :-)


The binary package is provided by the software manufacturer without
source code.

Still one might want to have a port available so one can put the port
in poudriere and have it available in one's own package repo.



Right, I'm not aiming to recompile the software. I'm looking to simply
create a new package.txz with the pre-built artifacts included.


Adding to the previous mail on how a FreeBSD port might be used to
install this binary package:

Be sure to be allowed to re-package the binary distribution that
you want to use in this port. The license may explicitly forbid the
distribution of derived works of any kind.

Do also set the LICENSE and specifically the LICENSE_PERMS variable
in the port's Makefile to indicate whether the FreeBSD package that
will be built from this port may be created, distributed, possibly
also sold.

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: Creating port from pre-built package

[Stefan Esser]

Am 11.02.21 um 18:12 schrieb Shawn Webb:

On Thu, Feb 11, 2021 at 06:09:52PM +0100, Patrick M. Hausen wrote:

Hi all,


Am 11.02.2021 um 18:03 schrieb Chris :

On 2021-02-11 08:26, Shawn Webb wrote:

Hey all,
The Splunk universal forwarder for FreeBSD is distributed as a package
tarball that you can use `pkg add` on. I'm in a position where I'd
like to create a port of the package so that I can automate certain
tasks.

Reverse engineer a package? I think that will violate the NDA you
signed. ;-)

Really. Unless the package is simply a wrapper of a binary blob. You'll
need to *build* the port, as part of the package creation process.
That is, unless I've *completely* misunderstood your intent here. :-)


The binary package is provided by the software manufacturer without
source code.

Still one might want to have a port available so one can put the port
in poudriere and have it available in one's own package repo.



Right, I'm not aiming to recompile the software. I'm looking to simply
create a new package.txz with the pre-built artifacts included.


I shouldn't be too hard to create a port, just be careful to consider
the architecture dependency of the binary package you are using.

If multiple architectures and OS releases are supported, then you'll
have to create the package name from those parameters and provide
distinfo data for all supported combinations of architecture and
OS release. You can override the name of the DISTINFO_FILE in the
port's Makefile to have separate distinfo files per arch/version.

In the port's Makefile extract the contents of the correct package
into the stage directory, have dummy targets for phases that are not
reuquired (e.g. an empty do-stage to have the stage done marker set
in the ${WRKSRC} directory) and provide a pkg-plist (possibly multiple
variants selected by the PLIST variable, if there are arch/version
specific differences).

Use ONLY_FOR_ARCH and IGNORE to limit the package building to those
architectures and releases that are supported, to prevent package
fall-out messages and to give ports users a reasonable error message
when trying to build the port on an unsupported system.

The generated package will have the meta-data from your port. If
you want to preserve some meta-date from the pre-built package, then
extract it from the extracted distfile, e.g. in the build phase.
Else use NO_BUILD to suppress this and possibly other phases that
are not required (empty dummy targets will do as well).

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: portmaster new development

[Stefan Esser]

Am 28.12.20 um 22:07 schrieb Michael Grimm:>> On 28. Dec 2020, at 21:41, 
Stefan Esser  wrote:


Poudriere works best on sufficiently powerful build servers and it
often requires rebuilding dependencies over hours when I just want to
test a new port before committing it.


Excuse me, but that is not true in this generality. I do run poudriere on ZFS 
in a cloud instance with 7G of memory and a CPU of 2.1 MHz. Never had to wait 
for more then one hour for recompiling *all* of my 240+ ports for STABLE-12. 
YMMV, yes, but stating that in that totality is nonsense, at least IMHO.


My build host is much bigger and I often have to build
dependencies over night before I can test-build a new
port with poudriere.

Maybe your ports do not have as many big dependencies,
but building LLVM and GCC as a dependency for 3 release
versions takes its time and whenever these ports have
been updated I cannot run "poudriere test" for my new
port before the compilers are updated.

If there are no complex dependencies, you are right,
but MESA, KDE (or even Qt5) and all ports that need
a specific compiler version that is receiving updates
cause excessive delays before a port can be test built
with poudriere.

So yes, YMMV, but it depends on the complexity of the
dependencies. And I test with different options in
the different jails (e.g. with/without DOCS, one with
non-standard PREFIX, etc.) for a better test coverage.
Therefore I cannot fetch pre-compiled compilers and
other dependencies to speed up my port tests.

And this might be typical for poudriere users. If you
do not want to build with non-default options you are
better served by using pre-compiled packages.

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: portmaster new development

[Stefan Esser]

Am 28.12.20 um 15:04 schrieb abi via freebsd-ports:

On 28.12.2020 16:16, Stefan Esser wrote:
Am 28.12.20 um 11:11 schrieb abi via freebsd-ports:> I build my ports 
in poudriere in VM without zfs or ssd on pre-Sandy
Bridge CPU. I don't have enough memory or disk space, so I don't use 
tmpfs or ccache either. I migrated from portmaster when it was 
abandoned several years ago and don't think I'll come back, 
especially if new portmaster will be written on bash. The idea behind 
portmaster was zero dependencies, so it doesn't brake after major 
upgrades.


You are free to use poudriere and it definitely is the official tool
for FreeBSD package building (and I have to use it myself and it has
cost me a lot of time rebuilding broken poudriere jails and keeping
them in state that I can use them to test new ports on a number of
different releases as well as i386 plus amd64).

And while you are free to never again use portmaster, telling people
that it has been abandoned is just a _lie_ and I'd want to ask you to
stop telling it. It has been continuously maintained for decades.

I remember portmaster marked as deprecated in 2016. I've switched to 
poudriere because of that. So, it _was_ abandoned when I migrated. It is 
good that it is not, the more options - the better. But some people here 
telling that poudriere requires ZFS and powerful dedicated hardware, I 
just pointed that they are wrong.


The portmaster port had been marked deprecated by the author of synth
and this commit has been reverted by him on request of portmgr on the
next day.

The portmaster port has never been abandoned - it does not suffice that
some unrelated committer considers his port management tool superior
and decides to deprecate a "competing" one.

Poudriere works best on sufficiently powerful build servers and it
often requires rebuilding dependencies over hours when I just want to
test a new port before committing it.

My goal is to have portmaster build ports in a clean jail as synth
does (restricted to i386 and amd64 due to the dependency on Ada) and
poudriere do, but without giving up the ease of use of portmaster for
other use cases.

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: portmaster new development

[Stefan Esser]

Am 28.12.20 um 11:11 schrieb abi via freebsd-ports:> I build my ports in 
poudriere in VM without zfs or ssd on pre-Sandy
Bridge CPU. I don't have enough memory or disk space, so I don't use 
tmpfs or ccache either. I migrated from portmaster when it was abandoned 
several years ago and don't think I'll come back, especially if new 
portmaster will be written on bash. The idea behind portmaster was zero 
dependencies, so it doesn't brake after major upgrades.


You are free to use poudriere and it definitely is the official tool
for FreeBSD package building (and I have to use it myself and it has
cost me a lot of time rebuilding broken poudriere jails and keeping
them in state that I can use them to test new ports on a number of
different releases as well as i386 plus amd64).

And while you are free to never again use portmaster, telling people
that it has been abandoned is just a _lie_ and I'd want to ask you to
stop telling it. It has been continuously maintained for decades.

The next version will not be using bash but LUA, which is highly
portable and does not have problematic dependencies. I'm well aware
that a pure shell script has its advantages, but bringing down the
time to scan for updates on my system from 300 to less than 10 seconds
(for > 2000 installed packages) combined with the ability to build
ports in a clean jail might make it an attractive choice for current
users of the /bin/sh based version.

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: portmaster new development

[Stefan Esser]

Am 27.12.20 um 06:04 schrieb Thomas Mueller:

And as portsnap user I have a question: Do they planning deprecation of



portmaster too?



No, I'm actively working on portmaster and have rewritten it from
scratch for better performance (and additional features, e.g. building
in a clean chroot jail, similar to synth or poudriere).



I have been using that version for more than one year, but the
functionality is not complete, yet.



On a test system with > 2200 installed ports it takes less than 10
seconds to identify the ~600 out-of-date ports (that I keep in this
state for testing of the upgrade strategy function), which is more
than 30 times faster than the same operation with the "official"
portmaster.



Until completion of that version, I'll continue to maintain and
update the current portmaster port ...



Regards, STefan


Question about the relation of portsnap and portmaster reminds me of Java and 
Javascript, or potato and sweet potato (not closely related).


Yes, portsnap and portmaster do not share anything beyond the first 4 
letters of their names ...



Since my question is about a new portmaster, I rename the subject to "portmaster" or 
"portmaster new development", rather than hijack the "portsnap" thread.

Which portmaster do I get if I build and install what is currently in the ports 
tree?


This is a version that I became maintainer of when it lacked flavor 
support and the original developer (Doug Barton) had left the project.



amelia2# ls -l ports-mgmt/portmaster
total 16
-rw-r--r--  1 root  wheel  1479 Dec 27 02:01 Makefile
-rw-r--r--  1 root  wheel   184 Feb 28  2018 distinfo
drwxr-xr-x  2 root  wheel   512 Dec 27 02:01 files
-rw-r--r--  1 root  wheel  1189 May  6  2019 pkg-descr

from a fresh svn up of the ports tree.


Yes, I added a feature requested by Kevin Obermann, yesterday, who
had noticed that a locked port could be built but not installed, if
the user answers "yes" to the question whether the lock should be
ignored.


An improved portmaster arouses my interest.  Maybe modify the name so it can be added to 
the ports tree and coexist with the "official" portmaster.


After taking over the current portmaster I noticed that it is ancient
in its structure. It used global variables throughout and forked a lot
if sub-processes to allow for local state.

Since it is extremely cumbersom to work on a linear 4000 line shell
script, I re-wrote it from scratch using bash associative arrays and
used that version since the summer of 2019. That version did already
support clean builds in a chroot jail as an option.

But I noticed that better data structures would allow to substantially
speed-up the scanning phase and decided to port that bash based version
to LUA. It can be found in my GH repo, but I'm currently reworking the
planning phase to allow for multiple pre-decessors of a port (e.g. if
A-v1 is integrated into B-v2 and B-v1 is already installed).

My current (not pushed) development version locks up due to such a case
(deskutils/kdepim-apps-libs has been merged into net/akonadi-contacts
and akonadi-contacts is downlevel, leading two an attempt to get an
exclusive lock on the target package name twice, since the two update
tasks are not currently merged into one).

But other than that it has been usable for simple updates for a long
time already.


Desired features/options would be to keep going rather than stop when one port 
fails to build, and the ability to install build dependencies, which may be 
useful for building other software.


Yes, I mark failed updates as such and cancel planned updates of
dependencies, but unrelated updates go on.


With synth, I had a difficult time getting everything that was built to 
install, some packages like bison are needed in building other software.


The bash version supported 4 modes: traditional portmaster, delayed
installation of packages not depended on in the update run, jailed
builds with installation of packages after completion of all builds
and repository mode to just create packages in a jail.

I ran into issues when I tried to optimize the disk space used for
the jail by de-installing no longer required build dependencies
from the jail as soon as possible. I needed better data structures
than offered by bash to efficiently express these dependencies.

That was the point where I decided to migrate the code to LUA.


How is poudriere in that regard?  I never used poudriere, have been intimidated 
by not wanting to use zfs or dialog4ports, or such an elaborate setup just to 
update one or a few ports.


I have to use poudriere to test ports before commit, but I do not
like it. The build jails are quickly becoming stale and have to be
rebuilt (causing all previously built packages to be compiled again)
and I often run into issues where a build dependency fails for reasons
that ar enot obvious (e.g. gmake) and I have found that the easiest
method of recovery is to throw away the whole poudriere jail and to
start over with a 

Re: portsnap

[Stefan Esser]

Am 26.12.20 um 18:41 schrieb LuMiWa via freebsd-ports:

Hi!

Today I red again an email:

Subject:[HEADS UP] Planned deprecation of portsnap
From:   Steve Wills 
Date:   2020-08-04 18:43:20

And as portsnap user I have a question: Do they planning deprecation of
portmaster too?


No, I'm actively working on portmaster and have rewritten it from
scratch for better performance (and additional features, e.g. building
in a clean chroot jail, similar to synth or poudriere).

I have been using that version for more than one year, but the
functionality is not complete, yet.

On a test system with > 2200 installed ports it takes less than 10
seconds to identify the ~600 out-of-date ports (that I keep in this
state for testing of the upgrade strategy function), which is more
than 30 times faster than the same operation with the "official"
portmaster.

Until completion of that version, I'll continue to maintain and
update the current portmaster port ...

Regards, STefan



OpenPGP_signature
Description: OpenPGP digital signature

Re: Port build failure in stage-qa,Port build failure in stage-qa

[Stefan Esser]

Am 11.11.20 um 23:48 schrieb Yasuhiro KIMURA:

From: Stefan Esser 
Subject: Port build failure in stage-qa,Port build failure in stage-qa
Date: Wed, 11 Nov 2020 21:56:49 +0100


Since DEVELOPER=yes cannot be overridden on the command line, I'll
have to remove it from make.conf to get a kernel with matching
modules installed.


I add following lines in make.conf.

--
.ifndef NO_DEVELOPER
DEVELOPER=yes
.endif
--

And developer mode can be disabled from command line with

% make NO_DEVELOPER=yes

Just FYI.


Thank you! I should have got to this solution myself ;-)

Such an override should probably exist in the port system's
Makefiles, but your work-around resolves the issue I had.

Best regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature

Port build failure in stage-qa

[Stefan Esser]

Building virtualbox-ose-kmod failed for me today (for the first time)
due to REINPLACE_CMD not changing some files. This breaks my kernel
build that include the kernel modules:

> Compressing man pages (compress-man)
===> Staging rc.d startup script(s)
> Running Q/A tests (stage-qa)
Warning: Possible REINPLACE_CMD issues:
- - REINPLACE_CMD ran, but did not modify file contents: Config.kmk
- - REINPLACE_CMD ran, but did not modify file contents: configure
*** Error code 1

Stop.
make[1]: stopped in /usr/svn/ports/head/emulators/virtualbox-ose-kmod
*** Error code 1

Stop.
make: stopped in /usr/svn/ports/head/emulators/virtualbox-ose-kmod



This appears to be due to the following commit that affected qa.sh:

r554893 | mat | 2020-11-11 14:29:52 +0100 (Wed, 11 Nov 2020) | 13 lines

Add set pipefail in most framework scripts.

set pipefail changes the pipeline return status from being the return
status of the last command to the last non 0 exit status of any command
in the pipeline.  This is needed to make sure all the commands in a
pipeline did actually return a non 0 status and not only the last one.

PR: 250723
Exp-run by: antoine
Reviewed by:bapt
Approved by:bapt
Differential Revision:  https://reviews.freebsd.org/D27007


Since DEVELOPER=yes cannot be overridden on the command line, I'll
have to remove it from make.conf to get a kernel with matching
modules installed.

Is this an expected outcome of the change?

Regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature

Re: Donation to Foundation for Poudriere /opt builds! (Bounty?)

[Stefan Esser]

Am 09.11.20 um 00:40 schrieb Tatsuki Makino:

I think you need to rewrite all the files under /etc that have
/usr/local in them.
For example, ldconfig_paths in /etc/rc.conf.


I have committed that change a few days ago, and it was heavily
disputed by those who think that there never should be a path
other than /usr/local used for LOCALBASE.


Perhaps we need to apply it to both host and jail.


It is in -CURRENT and I could MFC to -STABLE, but it will take
some time to arrive in a release (with 12.2 just finished).


If the shell of the user root of a jail is csh, the PATH of /root/.cshrc
in jail may also be relevant.


There are a number of files that need to be adjusted if LOCALBASE
is not /usr/local, and I'm willing to put proposed patches up for
review and commit them if accepted.


In addition, /root/.profile is another file that defines the PATH.


Yes, and there are many more.

I have added _PATH_LOCALBASE to /usr/include/paths.h in -CURRENT
to be picked up by binaries.

There already is ${LOCALBASE} in the Makefile in /usr/src and it
is used in some isolated parts of the tree to support a LOCALBASE
other than /usr/local.

But /usr/local has been hard-coded in FreeBSD for decades (not in
many files and binaries, but in some critical ones) and it takes
effort to make this a parameter that can be easily adjusted.

But there are down-sides: Making this parameter variable can lead
to security issues, since an attacker might be able to circumvent
policy restrictions and authorization checks.

I'm all for making it easy to build a system for another value of
LOCALBASE, but I'm not convinced that being able to choose another
value at run-time is worth the vulnerabilities this may create.


However, when you do all that, it's already a different OS distribution
than FreeBSD, isn't it :) ?


No, I don't think so. It is still FreeBSD, but you have to understand
that it is FreeBSD without pre-built packages, since most of them can
be built for a different LOCALBASE (but not all!) but the official
packages won't run (need a re-compile).

This may change if packages start to use the proposed getlocalbase()
function to construct paths at run-time. Other files provided by a
port need to be patched at install time (may apply to configuration
files rc scripts, ...)

Making FreeBSD friendly to environments that have a need for another
LOCALBASE than /usr/local will take a lot of effort and contributions
are welcome, as long as they do not cause issues for the large majority
that will continue to use the default of /usr/local.

Regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature

Re: security/openvpn-auth-ldap-2.0.4 not compatible with security/openvpn-2.5.0

[Stefan Esser]

Am 06.11.20 um 07:10 schrieb xtouqh--- via freebsd-ports:

Nikita Druba wrote:

Hello!

After updating security/openvpn to last(2.5.0) version, it's not start 
anymore, because:


Updating how?

"2020-11-06 02:37:05 PLUGIN_INIT: could not load plugin shared object 
/usr/local/lib/openvpn-auth-ldap.so: /usr/local/lib/libobjc.so.4.6: 
Undefined symbol "ceilf"


This suggests you need to update/rebuild lang/libobjc2 dependency as well.


Does that mean that the PORTREVISION of lang/libobjc2 has
to be bumped?

Seems this is a port maintained by ports@ and nobody feels
responsible?


OpenPGP_signature
Description: OpenPGP digital signature

Re: What are the benefits of NO_ARCH?

[Stefan Esser]

Am 02.11.20 um 15:33 schrieb Mateusz Piotrowski:

Hi ports@,

I wonder if setting NO_ARCH=yes brings any significant benefits to how 
our ports collection works. I'd be grateful if you could shed some light 
on the importance of setting NO_ARCH whenever possible.


NO_ARCH means that there is no need to build packages for each of the
supported architectures, e.g. for pure interpreted scripts or data files
that do not depend on byte-order and word-size (e.g. many font file
formats).

The result is reduced resources spent on building the packages, network
traffic, disk space on mirrors and on distribution media.

Regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature

Port uploaded for review (was: Re: Availability of samba 4.12.x)

[Stefan Esser]

I have derived a port for Samba-4.12.7 from net/sambar411 and have made 
it available for testing on Phabrcator:


https://reviews.freebsd.org/D26491

The CONFLICTS line is commented out to allow compilation if an older
version of this port is already installed and the CONFLICTS lines of
the other versions have to be adjusted to include this new port.

It builds and works for me on amd64/CURRENT, but I have not been able
to test it on other releases or architectures.

Since there is a maintainer, I'm not going to commit this version,
unless approved by him (timur) - but I do of course leave it to the
maintainer to make further changes or adjustments to this port ...

Regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature

Re: Need help: nearly complete port of GitHub CLI

[Stefan Esser]

Am 19.09.20 um 12:49 schrieb Christoph Moench-Tegeder:

## Stefan Esser (s...@freebsd.org):


I have tried to create a port for the recently announced GitHub CLI 1.0.


Without having actually looked at your work - we do already have
devel/gh - "GitHub's official command line tool", which seems to
be the beta version of what you're porting. And it has what seems
to be a patch for your specific problem.


Thank you, I had not been aware of a port for the beta version.

I have updated the port to use the new dependency versions that I had
worked out for my port attempt and could build version 1.0.0 that way.

There is a non-committer registered as maintainer of the port, and I
have sent him the diff for review and plan to commit the update (as
is or with the changes he might request).

Best regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature

Need help: nearly complete port of GitHub CLI

[Stefan Esser]

I have tried to create a port for the recently announced GitHub CLI 1.0.

What I got so far can be downloaded from:

https://people.freebsd.org/~se/ports/github-cli-port.tar.bz2


The GH_TUPLE list of the dependencies was the part that caused most
effort, but it should be correct and complete, AFAICT.

The port is nearly ready, but there is a compilation error which
appears to be caused by a missing architecture dependent definition:

# github.com/muesli/termenv
vendor/github.com/muesli/termenv/termenv_unix.go:76:9: readfds.Bits 
undefined (type syscall.FdSet has no field or method Bits)
vendor/github.com/muesli/termenv/termenv_unix.go:87:12: readfds.Bits 
undefined (type syscall.FdSet has no field or method Bits)

*** Error code 2

Another missing function could be fixed by copying over the Linux
specific definition, but I assume it should apply to FreeBSD, too
(see the post-extract target in the Makefile).


Since I do not know much about Go, I'm stuck, but think it should
be very easy to get the missing definition fixed for somebody with
some knowledge of Go and its build infrastructure.

More issues may exist, but since I'm stuck with the compiler error
that prevents building of the binary, I can't tell ...

Any help is highly appreciated ...

Regards, STefan


OpenPGP_signature
Description: OpenPGP digital signature

Re: Massive PORTS_REVISION bump after making gcc-9.1 default

[Stefan Esser]

Am 28.07.19 um 01:26 schrieb Kevin Oberman:
> Today I was hit with 226 ports needing update. With one exception, all were
> the result of the bump or the default gcc version to 9.1. The problem is
> that 9.1 was not installed first, so over 43 of these ports were rebuilt
> with the exact same compiler it was built with before the rebuild, eating
> up 2:45 of time on my build system. I'm sure if was less for many as my
> build system is over 8 years old. It was non-trivial in any case.

Which port management tool did you use to rebuild the updated ports?

I just checked what portmaster does, and it appears to build gcc-9.1
before starting to update ports that depend on it.

It was the 78th port out of 653 out-dated ports (most of them due to
a bumped port revision because of the gcc update), and I checked with
"make all-depends-list" that none of the first 77 ports depended on
gcc ...

> Should an install of gcc9 preceded all updates? Perhaps a note in UPDATING?
> I certainly looked there before I started when I saw 226 ports in the list.

Updating required build tools before ports that require them should
be automatic, if a tool like portmaster, portupgrade or poudriere is
used.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: GSoC: Separation of Ports Build Process from Local Installation

[Stefan Esser]

Am 29.05.19 um 00:51 schrieb Theron:
> Hello All,
> 
> For Google Summer of Code 2019 I am working on FreeBSD's ports tree makefiles
> towards eliminating the dependency of the ports building process on the local
> system's installed packages.  Currently this level of separation can only be
> accomplished in practice through chroot or Jail.  The project will eliminate
> the need for cooperation of the root user since /usr/local will not need to be
> touched.
> 
> The major technical obstacle to be overcome is that ports expect to find files
> of their dependencies installed in /usr/local.  To support this without
> touching that location on the installed system, file accesses will be
> redirected to a location controlled by the ports build process through use of
> a library to intercept file accesses.
> 
> Once I have that working (well enough to build one port at a time) I will move
> on to modify bsd.port.mk itself (and related files) to utilize this mechanism
> for virtual installation of port dependencies during builds.
> 
> The full project proposal can be seen at
> https://docs.google.com/document/d/1B30U9csgY299W59tNraSX1LYjzsba2i04OrYAUpdIZs/edit
> .

What's wrong with using chroot to provide a clean build environment?

That is what synth does, and I have been using my re-implementation of
portmaster for this purpose for some time, which uses a chroot jail with
read-only null-mounts of all relevant file systems and a clean copy of
some files and directories in /etc and /var that can be written without
root privileges. The jail is set up in not measurable time (irrelevant
compared to the time required to build the port).

The only problem with this approach is that it requires extra disk space
for the build environment (e.g., the specific C compiler required by some
port) plus the work space for the actual port build process. I'm using
tmpfs file systems within the jail for the work directory and the copies
of parts of /etc and /var that need to be written to.

Is there a risk of mis-use of the interception library to attack the
system, BTW?

[Its use is not restricted to root and it might be used to re-map file
system paths for commands that check e.g. policy files to decide whether
some operation is authorized ... SUID programs should not be vulnerable
to such an attack (since they do not allow the library pre-load required
to intercept the file operations), but there might be application programs
that are restricted by non-writable files in hard-coded directories that
could be subverted this way ... (such a command would be ill-designed,
since any user could compile her own interception library, but providing
such a library with the system and possibly having hooks for it in libc
might simplify such an attack, especially if there is no compiler and
easy way to install such a library on a host).]

> My goal is that this work can be integrated well enough into /usr/ports/Mk so
> that unlike Jail, no set up work should be required for using ports tree to
> build a set of installable packages.

Yes, this might be beneficial. But there will be huge differences
compared to the current build process. And in the end you'll probably
have to put the logic used by, e.g., portmaster to track dependencies
and determine the availability of up-to-date packages (to use as build
dependencies) into the ports system.

> Please let me know if you are interested in this project; feedback is
> appreciated.  If someone would like to provide ongoing feedback or mentorship
> that would be especially helpful.  Bakul Shah is my mentor officially for GSoC
> but I would be happy to have additional support from someone who is
> experienced with internals of the port infrastructure makefiles.

I'd be interested to get further information about your approach and
the progress you make and my experience working on a somewhat similar
project with portmaster might allow me to answer questions or provide
some help ...

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Request to change ruby default version to 2.5

[Stefan Esser]

Am 20.04.19 um 11:05 schrieb Matthias Fechner:
> Am 20.04.2019 um 06:59 schrieb Walter Schwarzenfeld:
>> After this change:
>>
>> |portmaster -R -r ruby-2.5|
>>
>> |
>> ===>>> Port directory: /usr/ports/databases/ruby-bdb
>>
>>     ===>>> This port is marked BROKEN
>>     ===>>> does not build with Ruby 2.5
>>
>>
>>     ===>>> If you are sure you can build it, remove the
>>        BROKEN line in the Makefile and try again. 
> 
> you have here several possibilities:
> 
> - keep on ruby 2.4 till portmaster is fixed, see UPDATING

What is there to fix in portmaster?

Portmaster uses the BROKEN= in the Makefile to detect ports
that will not build and warns about this situation before
trying to start the actual build ...

If there is a request to upgrade dependent ports (with -r)
then portmaster will perform checks that are possible before
starting the builds.

Without this check, the build will fail later-on, possibly
after replacing a number of ports with ones build for the
new ruby version. (Which may or may not cause problems, this
cannot be mechanically checked by portmaster.)

You can exclude ports from being rebuild with portmaster by
using "-x ruby-bdb" (for this particular case). But this may
lead to inconsistencies (will keep the version for ruby-2.4
installed, which will lead to run-time errors).

> - write a bug report upstream for ruby-bdb to get it fixed for ruby 2.5

Yes, that is the only sensible approach (IMHO), if you are
not able to fix the ruby-bdb port yourself to work with the
new ruby version.

> - fix portmaster by yourself

What do you suggest to change in portmaster?

> - use a more recent package builder like poudriere

How does this help, if a port requires ruby-bdb and that is not
yet available for the ruby version used to by the other ports?

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: python

[Stefan Esser]

Please excuse the top-post - I have only one sentence to
add:

I have tried to reproduce the issue with print/hplip by
installing a QT4 flavored version from before the port
update, and updating that port the new the non-flavor
version with portmaster, which worked without issue - so
I'm not able to test any fix to portmaster, currently ...

Regards, STefan

Am 11.04.19 um 09:29 schrieb Stefan Esser:
> Am 11.04.19 um 00:03 schrieb ajtiM via freebsd-ports:
>> Hi!
>>
>> With portmaster I try to update todays ports and python default is
>> version 3.7 which is okay for me but not for hplip:
>>  ===>>> All >>
>> hplip-3.17.11_3 (5/43)
>>
>> ===>>> Returning to dependency check for print/hplip
>> ===>>> Dependency check complete for print/hplip
>>
>> ===>>> All >> hplip-3.17.11_3 (5/43)
>>
>> ===>  hplip-3.17.11_4 FLAVOR is defined (to py27) while this port does
>> not have FLAVORS..
>> *** Error code 1
>>
>> Stop.
>> make: stopped in /usr/ports/print/hplip
> 
> This is a bad effect caused by the removal of FLAVORs from an
> installed port.
> 
> Since the previous revision of this port required a flavor, it
> will be updated with that same flavor passed as a parameter to
> the build process. The build could just ignore the irrelevant
> flavor, but that is not what the ports system does ...
> 
> In this particular case the removal of QT4 is the cause. The
> port used to support flavors "qt4" and "qt5" and to fix this
> issue for portmaster, entries in MOVED should exist that make
> upgrades use print/hplip without flavor, whether the qt4 or
> qt5 version was installed.
> 
> I plan to add code to portmaster to verify that a FLAVOR that
> is to be used for a port still applies, before starting to
> build it.
> 
> But it is not obvious to me what to do for all of the cases
> that exist:
> 
> 1) FLAVORS removed from the port -> build without flavors
> 
> 2) Specific FLAVOR removed from the port -> build with default
>flavor (?)
> 
> Anyway, for the time being you'll have to deinstall the port
> (use "pkg delete -f hplip" to prevent recursive deletion of
> dependent ports) and then re-install it with portmaster (which
> will build it without flavor, then.
> 
> Another possibility is to remove the "flavor" annotation
> registered for this port in the PKGDB, then portmaster will
> upgrade the port without providing a FLAVOR to the build ...
> 
> I'll see that I fix this problem in portmaster, but it will take
> some time ...
> 
> Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: python

[Stefan Esser]

Am 11.04.19 um 00:03 schrieb ajtiM via freebsd-ports:
> Hi!
> 
> With portmaster I try to update todays ports and python default is
> version 3.7 which is okay for me but not for hplip:
>  ===>>> All >>
> hplip-3.17.11_3 (5/43)
> 
> ===>>> Returning to dependency check for print/hplip
> ===>>> Dependency check complete for print/hplip
> 
> ===>>> All >> hplip-3.17.11_3 (5/43)
> 
> ===>  hplip-3.17.11_4 FLAVOR is defined (to py27) while this port does
> not have FLAVORS..
> *** Error code 1
> 
> Stop.
> make: stopped in /usr/ports/print/hplip

This is a bad effect caused by the removal of FLAVORs from an
installed port.

Since the previous revision of this port required a flavor, it
will be updated with that same flavor passed as a parameter to
the build process. The build could just ignore the irrelevant
flavor, but that is not what the ports system does ...

In this particular case the removal of QT4 is the cause. The
port used to support flavors "qt4" and "qt5" and to fix this
issue for portmaster, entries in MOVED should exist that make
upgrades use print/hplip without flavor, whether the qt4 or
qt5 version was installed.

I plan to add code to portmaster to verify that a FLAVOR that
is to be used for a port still applies, before starting to
build it.

But it is not obvious to me what to do for all of the cases
that exist:

1) FLAVORS removed from the port -> build without flavors

2) Specific FLAVOR removed from the port -> build with default
   flavor (?)

Anyway, for the time being you'll have to deinstall the port
(use "pkg delete -f hplip" to prevent recursive deletion of
dependent ports) and then re-install it with portmaster (which
will build it without flavor, then.

Another possibility is to remove the "flavor" annotation
registered for this port in the PKGDB, then portmaster will
upgrade the port without providing a FLAVOR to the build ...

I'll see that I fix this problem in portmaster, but it will take
some time ...

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: category for VPN softwares?

[Stefan Esser]

Am 02.04.19 um 07:42 schrieb Koichiro Iwao:
> On Tue, Apr 02, 2019 at 06:41:51AM +0200, Kurt Jaeger wrote:
>> Create a real category vpn and move everything to it ?
> 
> Sounds better! Gentoo has net-vpn category. Just FYI, Gentoo also have
> net-dialup category. PPP/PPPoE/L2TP softwares are put under net-dialup
> but I feel that classification is too fine. At least creating vpn or
> net-vpn souds good.

How about a new "real" category vpn and preserving the current categories
of the ports as their additional categories (assuming that they are in net
vs. security for a reason).
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Can't compile www/node on rpi2

[Stefan Esser]

Am 27.03.19 um 02:46 schrieb bob prohaska:
>> All the
>>> port-management tools in ports-mgmt assume this, and build
>>> port-dependancies as required. When building ports, it is always best
>>> to use one of the build-tools (ie: poudriere, synth , portmaster)
>>> instead of by hand.
>>
> I've played a little with portmaster, and it seemed more prone to stopping
> unintentionally than a simple "make -DBATCH" in the port directory. IIRC,
> it always stopped on stale but installed dependencies. Perhaps I'm doing
> somthing dumb, but I couldn't figure out what it was. Could it merely be
> the fact that I'm using a Raspberry Pi 2 or 3?

Could you provide me with traces of portmaster runs for cases where it did
not upgrade stale dependencies?

The dependency resolution and build strategy of portmaster scans all ports
for potential upgrades and ought to upgrade dependencies first, even if
the currently installed versions satisfy version requirements stated in a
dependency.

Portmaster should only stop if a dependency cannot be upgraded (and it does
this even if an older version is installed that might be sufficient for the
build attempted) or if it detects conflicts that cannot be resolved without
user intervention.

There have been a number of problems in the ports collection recently, which
lead to problems for port management tools other than poudriere. E.g. due to
the removal of FLAVORS from ports that could be built with either qt4 or qt5,
which was not accompanied by entries in MOVED that provide a hint about the
prior category/port@qt5 now just being category/port again ...

I'd like to get such cases detected and correctly dealt with in portmaster,
but each additional special case will slow down general processing due to
tests that take time but fail in 99,99% of cases. And they consume my time
due to the need to develop and test work-arounds for such cases ...

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Django versions

[Stefan Esser]

Am 07.03.19 um 09:02 schrieb Matthew Seaman:
> On 07/03/2019 07:22, Antoine Brodin wrote:
>> Please don't use the django metaport,  this port should be removed and
>> people should stop using hacks.
>> Someone needs to integrate a USE_PYTHON=django in python.mk
> 
> Let me look at that.  I never did like leaving D12592 so unresolved.
> 
> Although it would be good to have a more general solution to
> multiple-flavourization.

If anything in that direction is discussed, then please include
me in the thread (since I do read the list, but I'm not able to
follow it closely, currently).

I had asked about multi-flavour support nearly one-and-a-half year
ago and I was told that lack of support for more than one flavour
was by design and considered to be the correct concept.

There are quite a number of other weaknesses in the way flavours
are currently implemented (and it cost me a lot of effort to work
around them, when I took over maintainer-ship of portmaster to make
it support flavours).

I had given up trying top get this fixed, but it may be the right
time (given some experience with flavoured ports) to reconsider
some of the design and implementation choices made ...

Regards, STefan



signature.asc
Description: OpenPGP digital signature

Re: thunderbird build error

[Stefan Esser]

Am 17.12.18 um 00:11 schrieb George Mitchell:
> On 12/16/18 5:24 PM, Stefan Esser wrote:
>> [...]
>> I have (my version of) portmaster mostly working in a clean chroot jail.
>> It is still a pure shell script (works with the FreeBSD /bin/sh and bash),
>> thus portable to all architectures supported by FreeBSD (e.g. ARM).
>>
>> There are a few edge cases that need further work, but my version does
>> already support 4 build modes:
>>
>> 1) direct build ("classic portmaster mode")
>>
>> 2) delayed installation (only BUILD_DEPENDS are immediately installed,
>>all other ports are installed or upgraded from saved packages at the
>>end of the portmaster run)
>>
>> 3) jailed build (everything is built in a chroot jail and installed after
>>all builds have finished, except for pure build dependencies, which are
>>only kept as packages for use in the next portmaster run)
>>
>> 4) repository mode (packages are saved and at the end the repository files
>>are updated to allow local and remote upgrades with "pkg upgrade")
>> [...]
>> My goal is to have portmaster build everything, but with some restrictions
>> compared to poudriere (only for the architecture and release of the base
>> system) and with the option to use the direct mode for simple cases and
>> jailed builds (which require extra disk space for the chroot jail) in case
>> the builds need to be performed in a clean environment.
>>
>> Regards, STefan
>> [...]
> 
> Wow!  I've been using "classic" mode and I didn't even realize the
> new modes were there.  THANK YOU for all your fine work!   -- George

Well, they are in my development version, not the one currently in
ports ...

I have worked on the portmaster rewrite since december 2017, when I
noticed that it was impossible to make the current port fully work
with flavors and complex dependency changes. It works for most simple
cases, but there have been changes (e.g. KDE4->KF5) where the current
version just can't pass the required state around (and that could not
be fixed without a lot or effort).

Anyway, I'm giving the current "beta" of the new portmaster to interested
developers, but there are still a number of details that need to be
cleaned up.

One of the nice features that are already working well is that a failed
portmaster run can be restarted with just "portmaster -R". All state and
options are read in from a restart file (but it is possible to add e.g.
an -x option to exclude some port that causes the failure from the current
run when restarting ...).

I had hoped to have finished the current version to a state that could
be released at least as portmaster-devel for interested parties to test,
a few months ago. But I have continued adding features (for example to
build the ports in a chroot jail), and I'm alternating between adding
features and cleaning and speeding-up the code ...

I'd really hope to commit that version as an alternative to the current
portmaster port within the next weeks (but I'm not sure that I'll have
much time to work on it over the holidays ...).

Thanks to all that have supported my work by testing draft versions and
who have given very valuable feedback!

Best regards, STefan



signature.asc
Description: OpenPGP digital signature

Re: thunderbird build error

[Stefan Esser]

Am 16.12.18 um 17:07 schrieb John Kennedy:
> On Sun, Dec 16, 2018 at 07:54:34AM -0500, George Mitchell wrote:
>> On 12/15/18 1:10 PM, George Mitchell wrote:
>>> I recently updated my port build machine to 11.2-RELEASE.  I'm in the
>>> process of recompiling my (previously) 10.4-based ports to 11.2, and
>>> perhaps I shouldn't be trying to do this incrementally.  [...]
>>
>> Sure enough, deleting all ports and starting on a fresh ports tree
>> fixed this problem.  But I'm still unable to get the Powder Keg set
>> up on my machine (and I'm still happy with portmaster anyway).
>> -- George
> 
> I was a happy portmaster user for a really long time, but eventually I ran 
> into
> problems.  Basically, once you get enough packages built (say, X11, 
> browser-of-
> choice and trimmings) and keep it up for long enough (like through some major
> version bumps of dependent packages) you will run into an issue two packages
> that are incompatible need to be installed at the same time.  That tends to 
> get
> caught and fixed for the general case (the FreeBSD-provided package build), 
> but
> others do not (like incompatible packages that are required to build but not 
> to
> be installed).
> 
> I wish I'd gotten poudriere to work before I got synth to work because synth
> isn't as portable (say, to ARM) and I apparently like to punish myself (by not
> cross-compiling... yet).

I have (my version of) portmaster mostly working in a clean chroot jail.
It is still a pure shell script (works with the FreeBSD /bin/sh and bash),
thus portable to all architectures supported by FreeBSD (e.g. ARM).

There are a few edge cases that need further work, but my version does
already support 4 build modes:

1) direct build ("classic portmaster mode")

2) delayed installation (only BUILD_DEPENDS are immediately installed,
   all other ports are installed or upgraded from saved packages at the
   end of the portmaster run)

3) jailed build (everything is built in a chroot jail and installed after
   all builds have finished, except for pure build dependencies, which are
   only kept as packages for use in the next portmaster run)

4) repository mode (packages are saved and at the end the repository files
   are updated to allow local and remote upgrades with "pkg upgrade")

> In any case, synth/poudriere seems to be good at rebuilding anything that 
> might
> need it, ready for a quick "pkg upgrade".  Sometimes it may *seem* like a bit
> much (like gcc7 -> gcc8, or upgrading ca_root_nss), but I've been burned by
> portmaster not always catching on to some more subtle changes that would break
> things (and that even assuming that was ever aspired to by portmaster).  For
> example, look at the advice we were given for perl5.26 -> 5.28, but now for a
> bunch of packages where you don't know the dependencies because you're not a
> master of ports.  I don't feel the need to periodically delete and reinstall
> all packages just to be sure.
> 
> tl;dr:  You can't build everything with portmaster.  You should be able to 
> with
> poudriere (and if not, someone will probably be working on it to figure out 
> why
> not).

My goal is to have portmaster build everything, but with some restrictions
compared to poudriere (only for the architecture and release of the base
system) and with the option to use the direct mode for simple cases and
jailed builds (which require extra disk space for the chroot jail) in case
the builds need to be performed in a clean environment.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

PKGNAME of perl package

[Stefan Esser]

Just noticed that the package name of the perl ports does not match the
PKGNAME variable:

$ make -C /usr/ports/lang/perl5.26/ -V PKGNAME
perl5-5.26.3

# make package
===>  Building package for perl5.26-5.26.3

There is a mismatch of names

perl5-5.26.3 vs. perl5.26-5.26.3

and this confuses (my version of) portmaster ...

Is this by design and what use is the PKGNAME variable, if the package
is actually created under another name?

Which variable should I use to get the actual package name from the
port's Makefile?

STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: ClamAV Port Version Regression?

[Stefan Esser]

Am 11.12.18 um 18:51 schrieb Janky Jay, III:
> Hi All,
> 
>   After a portsnap to pull ${latest), I'm seeing the following:
> 
> clamav-0.101.0_2   <   needs updating (index has 0.100.2,1)
> 
>   The version that is claiming to be the latest appears to be a version
> regression. Am I wrong here? I checked the ClamAV site and it appears
> version 0.101.0 is, in fact, the latest. So, I'm wondering if there was
> possibly a revision issue that's causing this?

The ,1 at the end of the port to be installed indicates a new port epoch,
which is used to enforce an "upgrade" to a lower version number.

This is typically done if a port upgrade has been revoked due to problems
with the new version.

In this particular case the commit log says:

r487064 | antoine | 2018-12-09 18:33:26 +0100 (Sun, 09 Dec 2018) | 6 lines

Downgrade to 0.100.2
Upstream plans to release version 0.101.1 which fixes header issues in January
2019

Regards, STefan



signature.asc
Description: OpenPGP digital signature

Re: Acme-client (Let's Encrypt)and OpenSSL

[Stefan Esser]

Am 26.11.18 um 19:58 schrieb Xavier:
> On 10/11/2018 17:27, Xavier wrote:
>> Hi,
>>
>> Following the upgrade fom 11-STABLE to 12-STABLE, I updated OpenSSL from
>> 1.0.1 to 1.1.1, since it seemd mandatory for pkg to work
>>
>> But when I check pacakages with pkg check -Bd, I've an error :
>>> acme-client is missing a required shared library: libssl.so.46
>>
>> Trying to rebuild acme-client, it insists to install libressl, which
>> obvously fails.
> 
> I fixed the problem by force upgrading acme-client using binaries (pkg
> upgrade -f acme-client)
> 
> But build problem from source is still present :
> 
> --
> [root@numenor acme-client]# grep openssl /etc/make.conf
> DEFAULT_VERSIONS+=ssl=openssl111
> 
> [root@numenor acme-client]# openssl version
> OpenSSL 1.1.1a-freebsd  20 Nov 2018
> 
> [root@numenor acme-client]# svn info
> Path: .
> Working Copy Root Path: /usr/ports
> URL: svn://svn.freebsd.org/ports/head/security/acme-client
> Relative URL: ^/head/security/acme-client
> Repository Root: svn://svn.freebsd.org/ports
> Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5
> Revision: 485950
> Node Kind: directory
> Schedule: normal
> Last Changed Author: tobik
> Last Changed Rev: 476218
> Last Changed Date: 2018-08-02 07:33:16 +0200 (Thu, 02 Aug 2018)
> 
> [root@numenor acme-client]# make config
> ===> No options to configure
> 
> [root@numenor acme-client]# make all
> ===>  License ISCL accepted by the user
> ===>   acme-client-0.1.16_4 depends on file: /usr/local/sbin/pkg - found
> ===> Fetching all distfiles required by acme-client-0.1.16_4 for building
> 
> ===>  libressl-2.8.2 conflicts with installed package(s):
>   openssl111-1.1.1a
> 
>   They install files into the same place.
>   You may want to stop build with Ctrl + C.
> --

Other ports overcome this problem by using a dependency of the form
"security/libressl:stage" and then linking against the static lib in
the stage directory.

Since libressl is not installed in that case, there is no install
conflict to care about ...

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: FreeBSD Port: firefox-63.0.1,1 multiple errors build

[Stefan Esser]

Am 01.11.18 um 17:43 schrieb Mathieu Arnold:
> On Thu, Nov 01, 2018 at 08:54:16AM +0100, Stefan Esser wrote:
>> Am 01.11.18 um 03:38 schrieb Jonathan Chen:
>>> On Thu, 1 Nov 2018 at 15:35, Montgomery-Smith, Stephen
>>>  wrote:
>>> [...]
>>>> I only started using portupgrade recently.  I find it rather unreliable.
>>>>  I think the problem is that some ports need to be deleted before they
>>>> can be built successfully.
>>>
>>> For reliable port builds, you need use port builders that use clean
>>> environments; ie poudriere or synth
>>
>> True, but we used to make ports build with a previous version installed,
>> whenever possible. The problems are generally caused by the build process
>> picking up include files or libraries from LOCALBASE instead of from the
>> port's source directory.
>>
>> I'd expect a port maintainer to check for easy fixes to such build problems.
>>
>> Maybe we should add a port variable that is true if a port conflicts with
>> earlier versions of itself. That would indicate to port build tools like
>> portmaster or portupgrade that the old version should be deleted before
>> starting the build of the new version (and to re-install the old version
>> if the build of the new one fails).
>>
>> E.g.:
>>
>> CONFLICTS_WITH_ITSELF=   yes
> 
> You can already set
> 
> CONFLICT_BUILD=   firefox-62.*

Besides overloading the CONFLICTS* variables with just another meaning,
this does not work at all.

You obviously have not tested this suggestion before posting!

The output of "make check-depends" is explicitly filtered to omit matches
that belong to the same origin as the port being built.

And BTW: Another problem with flavors: The check for same origin does also
suppress entries for conflicting packages built from that origin but with
a different flavor (even if the FLAVOR is also reflected in the package
name, e.g. by a prefix).

STefan



signature.asc
Description: OpenPGP digital signature

Re: Inkscape compiles but crashes on startup

[Stefan Esser]

Am 04.11.18 um 22:21 schrieb bob prohaska:
> On Sun, Nov 04, 2018 at 10:47:22AM +0100, T??l Coosemans wrote:
>> On Sat, 3 Nov 2018 22:31:34 -0700 bob prohaska  wrote:
>>> On Sat, Nov 03, 2018 at 07:18:52AM +0100, Walter Schwarzenfeld wrote:
 Wait, fix of the primal cause of it is committed right now.

 https://svnweb.freebsd.org/ports?view=revision=483878
>>>
>>> Alas, no luck. Updating the ports tree and recompiling inkscape got
>>> rid of the locale error, but inkscape still crashes with an otherwise
>>> similar error stream from Gtk.
>>
>> You have to rebuild devel/glib20.
> 
> That did the trick, thank you!

If some dependent port only runs after glib20 has been recompiled,
shouldn't that be reason to bump the port revision of glib20, even
though the port didn't change in any other way?

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: FreeBSD Port: firefox-63.0.1,1 multiple errors build

[Stefan Esser]

Am 01.11.18 um 03:38 schrieb Jonathan Chen:
> On Thu, 1 Nov 2018 at 15:35, Montgomery-Smith, Stephen
>  wrote:
> [...]
>> I only started using portupgrade recently.  I find it rather unreliable.
>>  I think the problem is that some ports need to be deleted before they
>> can be built successfully.
> 
> For reliable port builds, you need use port builders that use clean
> environments; ie poudriere or synth

True, but we used to make ports build with a previous version installed,
whenever possible. The problems are generally caused by the build process
picking up include files or libraries from LOCALBASE instead of from the
port's source directory.

I'd expect a port maintainer to check for easy fixes to such build problems.

Maybe we should add a port variable that is true if a port conflicts with
earlier versions of itself. That would indicate to port build tools like
portmaster or portupgrade that the old version should be deleted before
starting the build of the new version (and to re-install the old version
if the build of the new one fails).

E.g.:

CONFLICTS_WITH_ITSELF=  yes

I'd be willing to integrate support for such a functionality into portmaster,
if it was accepted in the ports framework.

Regards, STefan

PS: And yes, there are good reasons to keep support for tools that are
lighter-weight than poudriere and more portable than synth in the
ports system.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: RUN_DEPENDS and portmaster

[Stefan Esser]

Am 19.09.18 um 14:25 schrieb Kurt Jaeger:
> Hi!
> 
>> Why is Ada only available on i386/amd64?
> 
> Because nobody provided fixes for the build on other platforms up to now.

And given that the only use of Ada in FreeBSD ports seems to be synth,
it seems a lot easier to implement the synth functionality in some more
portable (or at least readily available) language.

There is nothing in Ada, that makes it specifically suitable for synth.
The synth code consists mostly of string processing and program invocations.
It uses Ada exception handling, but I did not spot anything that could not
easily be implemented in any other language.

In fact, due to the many invocations of external binaries, porting of
synth to a shell seems sensible. I do not consider the time-outs on all
the build phases a strict requirement, but even those could be implemented
with shell mechanisms,

The setup of the clean environment for the package builds is easy to extract
and I have considered adding that feature to portmaster, to support building
of ports that currently fail if a previous version is installed (generally
caused by include paths that prefer installed headers over those in the
sources of the new version).

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: RUN_DEPENDS and portmaster

[Stefan Esser]

Am 18.09.18 um 14:05 schrieb Matthias Fechner:
> Dear Stefan,
> 
> Am 17.09.2018 um 14:31 schrieb Stefan Esser:
>> But the behavior of portmaster will not be changed.
>> RUN_DEPENDS are dependencies required to run a port, not dependencies
>> required to install a port.
>>
>>
>> And I do not care whether bsd.port.mk treats RUN_DEPENDS as if they
>> were INSTALL_DEPENDS (which do not exist). The fact that bsd.port.mk
>> works in that way is due to the fact, that it generally executes sub
>> processes in a depth first manner. Portmaster distinguishes build and
>> run dependencies and makes sure, that build dependencies not only exist,
>> but are updated before the ports they depend on, while bsd.port.mk will
>> use any build dependency that satisfies the range requirements (if any)
>> and does not upgrade existing but outdated (in the sense that an upgrade
>> is available) dependencies. Portmaster will then upgrade any out-dated
>> run dependencies (again if an upgrade is available, not only if it is
>> strictly required). Thus portmaster guarantees, that a port is built
>> with the latest available build tools, and that run dependency upgrades
>> see the upgraded port that requires them, in case they depend on it.
> 
> I fully understand you.
> 
> Maybe it will be a good idea to phase portmaster out as it seems to be a
> unmaintable beast?
> 
> Maybe synth can replace it for users that are not used to poudriere?

I have been using a portmaster-rewrite for many months, which is ready
for release except for some performance tuning. (The portmaster in ports
is not un-maintainable, but it's hard to modify a monolithic 4000 line
shell script that uses global variables to pass state and recursive
invocation of itself to provide local state when required.)

The performance problems are caused by bad design of the FLAVOR feature,
which ignored the requirements of tools like portmaster (I've written
about this at length when FLAVOR support had been committed).

Synth is a non-starter for me, since it is written in Ada and only
available on i386/amd64. I have plans to implement the functionality
of synth in portmaster (not really hard, since the complex parts are
the logic that deals with moved ports and conflicts, while the actual
port building is simple). Portmaster can already create packages
without installing them (unless they are BUILD_DEPENDS of some later
port, of course) and you can populate your local repository with
portmaster.

Different from poudriere or synth, portmaster adapts to the preferences
of the user (and e.g. upgrades samba48 used by some port that specifies
a dependency on samba46, if the system already has an outdated samba48
installed).

Portmaster will use what's available on a system and does allow selective
upgrades (keeping some ports at a back-level on purpose, but still upgrade
other ports that depend on them), while a poudirere/pkg based upgrade will
typically require all dependencies to exactly match what was present at
the time the package was built (in a clean environment, not resembling the
system the packages are going to be installed on).

Both, portmaster and poudriere/pkg have their use-cases, and there is only
a partial overlap. There are quite a number of portmaster users, and they
use it since their use-cases are not well covered by other tools.

The way the ports system handles installs (in that it installs RUN_DEPENDS
before the port that needs them) is an artifact of the way Makefiles
naturally work, i.e. of the tool the ports system is based on.

I do not understand why you intend on having RUN_DEPENDS cause installation
of dependencies before your port. If you need those dependencies before the
port is installed, then they are not really run dependencies, but dependencies
of your particular build process.

Portmaster has worked for far more than a decade with >20,000 ports. I do
not see that your single port that expects run dependencies to be available
before the installation has completed is reasonable cause to change the way
portmaster works with dependencies. It pre-dates the "new" ports framework
by far, and it used to be common practice, that changes respect established
practices.

BTW: Your port-install target will not be run when installing from a package
(or building a package) anyway. Portmaster will take care of providing the
required dependencies, as will pkg.

So what's the reasoning that this test in do-install is required at all?

You already specify exact versions in the RUN_DEPENDS, which are checked
by the ports framework. Portmaster will take care, that all these ports
are re-built to the latest level (hopefully satisfying the version test)
after gitlab-ce has been installed. If you use pkg, the test is performed
at install time, too.

Are there any PRs due to lack of that test?

Regards,

Re: Error building net/librsync2

[Stefan Esser]

Am 18.09.18 um 12:23 schrieb Jakob Breivik Grimstveit:
> $ uname -a
> FreeBSD core2.grimstveit.no 11.2-RELEASE-p3 FreeBSD 11.2-RELEASE-p3 #7
> r338607: Wed Sep 12 13:24:12 CEST 2018
> r...@core2.grimstveit.no:/usr/obj/usr/src/sys/CORE2
> amd64
> 
> [...]
> [22/53] /usr/bin/cc -Drsync_EXPORTS -I/usr/local/include -Isrc/blake2 -Isrc
> -O2 -pipe -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2
> -pipe -march=native  -fstack-protector -fno-strict-aliasing -fPIC -MD -MT
> CMakeFiles/rsync.dir/src/scoop.c.o -MF CMakeFiles/rsync.dir/src/scoop.c.o.d
> -o CMakeFiles/rsync.dir/src/scoop.c.o   -c src/scoop.c
> [23/53] /usr/bin/cc -Drsync_EXPORTS -I/usr/local/include -Isrc/blake2 -Isrc
> -O2 -pipe -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2
> -pipe -march=native  -fstack-protector -fno-strict-aliasing -fPIC -MD -MT
> CMakeFiles/rsync.dir/src/version.c.o -MF
> CMakeFiles/rsync.dir/src/version.c.o.d -o
> CMakeFiles/rsync.dir/src/version.c.o   -c src/version.c
> [24/53] /usr/bin/cc -Drsync_EXPORTS -I/usr/local/include -Isrc/blake2 -Isrc
> -O2 -pipe -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2
> -pipe -march=native  -fstack-protector -fno-strict-aliasing -fPIC -MD -MT
> CMakeFiles/rsync.dir/src/util.c.o -MF CMakeFiles/rsync.dir/src/util.c.o.d
> -o CMakeFiles/rsync.dir/src/util.c.o   -c src/util.c
> [25/53] /usr/bin/cc -Drsync_EXPORTS -I/usr/local/include -Isrc/blake2 -Isrc
> -O2 -pipe -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2
> -pipe -march=native  -fstack-protector -fno-strict-aliasing -fPIC -MD -MT
> CMakeFiles/rsync.dir/src/mdfour.c.o -MF
> CMakeFiles/rsync.dir/src/mdfour.c.o.d -o
> CMakeFiles/rsync.dir/src/mdfour.c.o   -c src/mdfour.c
> [26/53] /usr/bin/cc -Drsync_EXPORTS -I/usr/local/include -Isrc/blake2 -Isrc
> -O2 -pipe -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2
> -pipe -march=native  -fstack-protector -fno-strict-aliasing -fPIC -MD -MT
> CMakeFiles/rsync.dir/src/trace.c.o -MF CMakeFiles/rsync.dir/src/trace.c.o.d
> -o CMakeFiles/rsync.dir/src/trace.c.o   -c src/trace.c
> [27/53] /usr/bin/cc -Drsync_EXPORTS -I/usr/local/include -Isrc/blake2 -Isrc
> -O2 -pipe -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2
> -pipe -march=native  -fstack-protector -fno-strict-aliasing -fPIC -MD -MT
> CMakeFiles/rsync.dir/src/tube.c.o -MF CMakeFiles/rsync.dir/src/tube.c.o.d
> -o CMakeFiles/rsync.dir/src/tube.c.o   -c src/tube.c
> [28/53] /usr/bin/cc -Drsync_EXPORTS -I/usr/local/include -Isrc/blake2 -Isrc
> -O2 -pipe -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2
> -pipe -march=native  -fstack-protector -fno-strict-aliasing -fPIC -MD -MT
> CMakeFiles/rsync.dir/src/whole.c.o -MF CMakeFiles/rsync.dir/src/whole.c.o.d
> -o CMakeFiles/rsync.dir/src/whole.c.o   -c src/whole.c
> [29/53] /usr/bin/cc  -I/usr/local/include -Isrc/blake2 -Isrc -O2 -pipe
> -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2 -pipe
> -march=native  -fstack-protector -fno-strict-aliasing -MD -MT
> CMakeFiles/sumset_test.dir/tests/sumset_test.c.o -MF
> CMakeFiles/sumset_test.dir/tests/sumset_test.c.o.d -o
> CMakeFiles/sumset_test.dir/tests/sumset_test.c.o   -c tests/sumset_test.c
> FAILED: CMakeFiles/sumset_test.dir/tests/sumset_test.c.o
> /usr/bin/cc  -I/usr/local/include -Isrc/blake2 -Isrc -O2 -pipe
> -march=native  -fstack-protector -fno-strict-aliasing -Wall -O2 -pipe
> -march=native  -fstack-protector -fno-strict-aliasing -MD -MT
> CMakeFiles/sumset_test.dir/tests/sumset_test.c.o -MF
> CMakeFiles/sumset_test.dir/tests/sumset_test.c.o.d -o
> CMakeFiles/sumset_test.dir/tests/sumset_test.c.o   -c tests/sumset_test.c
> In file included from tests/sumset_test.c:27:
> /usr/local/include/librsync.h:430:21: error: unknown type name 'FILE'
> void rs_mdfour_file(FILE *in_file, char *result);
> ^
> /usr/local/include/librsync.h:432:23: error: unknown type name 'FILE'
> rs_result rs_sig_file(FILE *old_file, FILE *sig_file,
>   ^
> /usr/local/include/librsync.h:432:39: error: unknown type name 'FILE'
> rs_result rs_sig_file(FILE *old_file, FILE *sig_file,
>   ^
> /usr/local/include/librsync.h:437:27: error: unknown type name 'FILE'
> rs_result rs_loadsig_file(FILE *, rs_signature_t **, rs_stats_t *);
>   ^
> /usr/local/include/librsync.h:441:43: error: unknown type name 'FILE'
> rs_result rs_delta_file(rs_signature_t *, FILE *new_file, FILE *delta_file,
> rs_stats_t *);
>   ^
> /usr/local/include/librsync.h:441:59: error: unknown type name 'FILE'
> rs_result rs_delta_file(rs_signature_t *, FILE *new_file, FILE *delta_file,
> rs_stats_t *);
>   ^
> /usr/local/include/librsync.h:443:25: error: unknown type name 'FILE'
> rs_result rs_patch_file(FILE *basis_file, FILE *delta_file, FILE *new_file,
> rs_stats_t *);
> ^
> 

Not a portmaster problem

[Stefan Esser]

Am 17.09.18 um 22:51 schrieb @lbutlr:
> I am using MariaDB 10.0 on FreeBSD 11.3
> 
> When trying to update rsyslogd via postmaster I get:
> 
> checking for mysql_config... mysql_config
> checking for mysql_init in -lmysqlclient... no
> configure: error: in `/usr/ports/sysutils/rsyslog8/work/rsyslog-8.37.0':
> configure: error: MySQL library is missing
> See `config.log' for more details
> ===>  Script "configure" failed unexpectedly.
> Please report the problem to matt...@freebsd.org [maintainer] and attach the
> "/usr/ports/sysutils/rsyslog8/work/rsyslog-8.37.0/config.log" including the
> output of the failure of your make command. Also, it might be a good idea to
> provide an overview of all packages installed on your system (e.g. a
> /usr/local/sbin/pkg-static info -g -Ea).
> *** Error code 1
> 
> Stop.
> make[1]: stopped in /usr/ports/sysutils/rsyslog8
> *** Error code 1
> 
> I’ve disabled the MySQL output for now, but this disconnect between MySQL and 
> Maria seems to be happening with a lot of ports.
> 
> Is there anything I can do to restore the MySQL output for rsylog?

AFAICT, Wolfgang Zenker correctly points at PR 230839.

Portmaster does not appear to play a role - the problem exists when
building without portmaster, too.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: RUN_DEPENDS and portmaster

[Stefan Esser]

Am 17.09.18 um 07:47 schrieb Matthias Fechner:
> Am 10.09.18 um 12:16 schrieb Mathieu Arnold:
>> Reading Mk/bsd.port.mk at line 5274, run-depends are installed before
>> do-install runs.
> 
> thanks, I see it the same way and created a PR for it, to get this fixed
> in portmaster.

You are of course free to create a PR against portmaster.

But the behavior of portmaster will not be changed.

RUN_DEPENDS are dependencies required to run a port, not dependencies
required to install a port.


And I do not care whether bsd.port.mk treats RUN_DEPENDS as if they
were INSTALL_DEPENDS (which do not exist). The fact that bsd.port.mk
works in that way is due to the fact, that it generally executes sub
processes in a depth first manner. Portmaster distinguishes build and
run dependencies and makes sure, that build dependencies not only exist,
but are updated before the ports they depend on, while bsd.port.mk will
use any build dependency that satisfies the range requirements (if any)
and does not upgrade existing but outdated (in the sense that an upgrade
is available) dependencies. Portmaster will then upgrade any out-dated
run dependencies (again if an upgrade is available, not only if it is
strictly required). Thus portmaster guarantees, that a port is built
with the latest available build tools, and that run dependency upgrades
see the upgraded port that requires them, in case they depend on it.


I have spent hundreds of hours to work around the bad design of the
FLAVOR support, which ignored the requirements of tools like portmaster
or portupgrade. Changes to the port infrastructure tend to ignore the
existence and requirements of build tools that have a decade long
history and use cases not covered by the port infrastructure alone.

I'm not going to spend any time on a change that made portmaster install
RUN_DEPENDS before executing "make install" for a port.

You are free to create a patch to that effect, but be aware that it is
extremely likely to break lots of upgrade scenarios, and I'll make you
responsible for fixing them (or back-out your assumed patch that treats
run dependencies as if they were build dependencies).

Regards, STefan



signature.asc
Description: OpenPGP digital signature

Re: RUN_DEPENDS and portmaster

[Stefan Esser]

Am 12.09.18 um 07:58 schrieb Matthias Fechner:
> Dear Stefan, Dear Mathieu,
> 
> Am 10.09.18 um 14:10 schrieb Stefan Esser:
>> This is a design decision in portmaster that has existed for at least
>> a decade.
>>
>> Use INSTALL_DEPENDS if you depend on a port being available and upgraded
>> before your port's do-install is invoked.
>>
>> Changing the current portmaster version in ports is no option, since it
>> does not offer to recursively upgrade or install any other port while
>> working on some port and it cannot easily be made to support such a
>> sequence of actions.
> 
> thanks a lot for your explanation, so it seems to be a problem with
> portmaster.
> But as I do not want to block all users from using gitlab-ce that are
> using portmaster I think it is ok to define all RUN_DEPENDS also as
> BUILD_DEPENDS?

Yes, adding them to BUILD_DEPENDS will cause all those ports to be built
and installed by portmaster before the port that executes those tests.

I had thought there also was INSTALL_DEPENDS, but now I see that in fact
there only is INSTALLS_DEPENDS, which is used internally in bsd.port.mk.

So, BUILD_DEPENDS is the variable to use in that case.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: RUN_DEPENDS and portmaster

[Stefan Esser]

Am 10.09.18 um 06:54 schrieb Matthias Fechner:
> Dear all,
> 
> I have a questions reagarding the RUN_DEPENDS and at which step
> dependencies should be installed.
> I'm the maintainer of gitlab-ce port and I added a feature that checks
> in the install target:
> do-install:
> (cd ${WRKSRC} && ${RM} Gemfile.lock && bundle install --local)
> ${FIND} ${WRKSRC} -name '*.orig' -delete
> ...
> 
> that all gems available in the system do match the version required by
> the Gemfile.
> Poudriere works fine with this, but portmaster fails.
> Regarding the documentation RUN_DEPENDS packages should be installed
> before the install is happening.

Hi Matthias,

this is the description of the sequence of actions performed by the
ports framework alone.

INSTALLS_DEPENDS covers the case of dependencies that are required
to be available when a port is being installed.

Portmaster installs RUN_DEPENDS only after the port that depends on
them, since it is assumed, that they are actually only required to
execute it after it has been completely installed.

The reason is, that portmaster is typically used to upgrade multiple
ports in such a way, that all BUILD_DEPENDS are up to date (not only
available) when some some dependent port is compiled.

In case that some upgraded port actually is a build dependency of
some other port, it will need to have its run dependencies installed
and upgraded, and portmaster will assure this is the case.

The sequence of upgrade actions was chosen to follow this scheme to
prevent dependency loops (which typically will consist of a mix of
build and run dependencies).

> Is this install related to the do-install target or the installation
> of the package itself?

I have re-implemented portmaster and have been using my version to
maintain my ports since May. Due to several bad design decisions (or
rather the lack of thorough design) of the FLAVOR feature, it took me
quite some effort and time to get performance of that version to an
acceptable level.

Currently I'm building and installing ports in the same order as the
current official portmaster version, but that could easily be changed.

I have considered following the same concept as synth does (i.e. build
ports in a clean environment), at least as an option. But I have not
started to implement such a feature, yet.

> Is this maybe a problem with portmaster as poudiere handles this correctly?

This is a design decision in portmaster that has existed for at least
a decade.

Use INSTALL_DEPENDS if you depend on a port being available and upgraded
before your port's do-install is invoked.

Changing the current portmaster version in ports is no option, since it
does not offer to recursively upgrade or install any other port while
working on some port and it cannot easily be made to support such a
sequence of actions.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Ports vs packages

[Stefan Esser]

Am 26.08.18 um 21:55 schrieb Gregory Byshenk:
> On Sun, Aug 26, 2018 at 01:01:24PM +0200, Jos Chrispijn wrote:
>> On 26-8-2018 2:07, Pete Wright wrote:
>>> one thing i do for my systems is if there is an update to a port i 
>>> need/want to test before the official build cluster is done is run a 
>>> "make package" in the port directory.? then i can install the updated 
>>> code as a pkg for future upgrade convenience.? this works great for 
>>> ports without many external dependencies at build-time, not so much 
>>> when things like llvm need to be build ;)
>>
>> I did that once myself but ended in total chaos because I found out that 
>> using ports and packages next to each other is not a good marriage.
>> Port options that may have been enabled may be overuled by packages 
>> (which are always built using the default options). Not for a specific 
>> port but with regards to the depencies is will us (and which may already 
>> been installed as packages).
>>
>> I am quite a nub on this, so perhaps the problems were otherwise. Since 
>> I completely switched to packages, these issues are gone.
> 
> If you are using packages by default, then this shouldn't
> really be a problem. Your packages should have default 
> options, so if you build one port - using the default 
> options! - then there should be no serious conflict. At
> least when there are few/no dependencies, as Pete notes.
> 
> Where you can get into problems is if you are building 
> using ports by default, along with non-standard options,
> and then try to add packages. That can get very ugly.

To be fully compatible with official packages, you have to build in
Poudriere with the same FreeBSD version as used of the official builds.

Building in your host environment might cause conflicts between shared
libraries, if the shared library version used for the packages (from the
oldest supported release of the FreeBSD version you use) might be too old
to match what you use in a newer release of that FreeBSD version.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

libkgapi-18.08 conflicts with kdepim-runtime-18.04.3_2

[Stefan Esser]

This might be no problem when upgrading from a package, but upgrading from a
port (e.g. with portmaster) fails with:

===>   Registering installation for libkgapi-18.08.0
Installing libkgapi-18.08.0...
pkg-static: libkgapi-18.08.0 conflicts with kdepim-runtime-18.04.3_2 (installs
files into the same place).  Problematic file:
/usr/local/lib/sasl2/libkdexoauth2.so
*** Error code 70

I had expected some information in UPDATING or a conflicts entry, which
matches kdepim-runtime<=18.04 or something like that ...

What is the correct way to deal with that problem?

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: rubgem update to 3.2.7

[Stefan Esser]

Am 09.07.18 um 05:04 schrieb Koichiro Iwao:
> On Mon, Jul 09, 2018 at 12:02:22AM +0200, Walter Schwarzenfeld via 
> freebsd-ports wrote:
>> Portmaster tries to build the update in the wrong order.
>>
>> Right order for me was (depend on which packages are installed):
>>
>> rubygem-glib2
>> rubygem-gobject-introspection
>> rubygem-gio2
>> rubygem-gdk_pixbuf2
>> rubygem-pango
>> rubygem-gdk3
>>
>> rubygem-gtk3
>>
>> rubygem-atk
> 
> I can also confirm the issue. I always perform portmaster in order that
> you shown by hand when rubygem-gtk ports updated. portmaster needs to be 
> fixed.

A quick check shows that this is a problem with the port, which obviously
declares actual build dependencies as run dependencies.

Please open a PR and let the ruby maintainers fix this issue.

E.g. in x11-toolkits/rubygem-gtk3/Makefile:

RUN_DEPENDS=rubygem-atk>=${PORTVERSION}:accessibility/rubygem-atk \
rubygem-gdk3>=${PORTVERSION}:x11-toolkits/rubygem-gdk3 \

rubygem-gdk_pixbuf2>=${PORTVERSION}:graphics/rubygem-gdk_pixbuf2 \
rubygem-gio2>=${PORTVERSION}:devel/rubygem-gio2 \
rubygem-glib2>=${PORTVERSION}:devel/rubygem-glib2 \

rubygem-gobject-introspection>=${PORTVERSION}:devel/rubygem-gobject-introspection
\
rubygem-pango>=${PORTVERSION}:x11-toolkits/rubygem-pango

Run dependencies are generally built after the port that needs them, since
they may themselves depend on resources provided by the respective port.

Build dependencies OTOH provide those files that need to be updated before
building a port. They may also be run dependencies (and often are), and in
that case they need to be in both the BUILD_DEPENDS and RUND_DEPENDS lists.

I cannot modify the behavior of portmaster (i.e. build run dependencies
before the port that needs them, since this would break quite a number of
other ports with correct dependency specifications: Then portmaster would
try to build run dependencies before their prerequisites are available).

So, I'm sorry, but portmaster will not be able to upgrade the rubygem ports
in the correct order, unless the port dependencies are fixed.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: net-p2p/qbittorrent needs to be unblocked by portmgr

[Stefan Esser]

Am 20.06.18 um 10:34 schrieb Tobias Kortkamp:
> On Wed, Jun 20, 2018, at 10:18, Yuri wrote:
>> Committing transaction...
>> svn: E165001: Commit failed (details follow):
>> svn: E165001: Commit blocked by pre-commit hook (exit code 1) with output:
>> Do not commit a port with FLAVORS without first
>> getting approval from portmgr.
>>
> 
> What happens when you add something like
> 
> Approved by:  portmgr (???)
> 
> to the commit message?

... after getting approval from portmgr to be allowed to state approval?

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Why portmaster uses g++ not g++6 nor clang++?

[Stefan Esser]

Am 18.05.18 um 22:17 schrieb Rozhuk Ivan:
> On Fri, 18 May 2018 20:30:38 +0200
> Stefan Esser <s...@freebsd.org> wrote:
> 
>> Sorry, this was my fault and I hope it is fixed with the follow-up
>> commit to portmaster version 3.19-10.
>>
> 
> Sorry for offtopic, but if portmaster install some build dep or run dep
> if does not mark it as autoinstalled, and:
> pkg query -e '%a = 0' %o
> show autotools, help2man and other build only crap as user install it,
> pkg autoremove
> does not remove this.
> 
> Also --delete-build-only broken.
> 
> portmaster -BgvDa -y --delete-build-only 
> --local-packagedir=/usr/ports/packages --packages-local
> if found some package - install it, and after install create package and 
> overwrite original.
> pkg create -n will prevent this, or additional checks, or just keep remember 
> that pkg allready
> exist because port just installed from it.

I'm working on a completely new re-implementation of portmaster and the new
version will get these points right. Fixing the current port version is too
hard and wasted effort, since only the features and command line options are
carried over, but none of the code of the current version.

The current port master port was written at the time of the "old" package
management tools (pre PKG-NG). It took me quite some time and effort to
implement flavors support in that version, and I found that it is much
harder to maintain that version than to rewrite it with the current package
tools in mind.

The new version is already able to upgrade ports, but it lacks some of the
features of the old version (e.g. installation from local packages). But it
is already better at tracking changes, e.g. as in the recent KDE4 port and
package renaming (where both port directory and package name were changed
at the same time and the current portmaster in ports has no way to track
this change and to upgrade the affected ports).

But I'm not going to implement all features of the current portmaster. E.g.
I have no plan to implement dependency tracking via the INDEX file or the
installation of packages from a remote repository (since mixing locally
compiled and official packages is not well supported). But I plan to offer
the installation of build dependencies from locally cached packages (and
the deinstallation after they are no longer required).

But the use case you described in your mail will be covered.

I hope to have the new version ready for testing as a portmaster-devel port
before the end of June.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Why portmaster uses g++ not g++6 nor clang++?

[Stefan Esser]

Am 18.05.18 um 17:13 schrieb Koichiro Iwao:
> Hi,
> 
> I'm building devel/qt5-make by portmaster. Somehow portmaster incorrectly
> detects gcc6 and uses g++ despite it is actually installed as g++6. If I > 
> build it without portmaster, clang++ is used and build finishes successfully.

Sorry, this was my fault and I hope it is fixed with the follow-up commit
to portmaster version 3.19-10.

I had received a proposed patch to significantly speed-up portmaster by
caching of a few parameters in environment variables. My commit was meant
to work around a side effect of the proposed patch (sourcing of a script
imported several unused subroutines into portmaster and I wanted to just
use the result of execution of the one relevant subroutine, but missed the
fact that the quoting came out wrong ...).

(If you are interested: The environment variable _CXXINTERNAL_acaad9ca
should contain the literal character sequence "-lc++" including the
double quotes, when I use eval as in the defective version, I either get
no quotes around -lc++ or I get extra double-quotes around the whole
variable, if I eval the script output piped through sed s:":\\:" ...)

Since I did not manage to get the correct result without sourcing the
script, I gave in and accept the fact, that these subroutines are now
imported into portmaster.

Anyway, I hope my latest commit has fixed the problem ...

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Should we Rust warnings suggesting adding work/stage/usr/local/bin to the PATH?

[Stefan Esser]

Am 17.05.18 um 14:42 schrieb Mateusz Piotrowski:
> Hello,
> 
> When I build Rust packages I get warnings like this one:
> 
>> warning: be sure to add 
>> `/usr/home/0mp/ports/games/genact/work/stage/usr/local/bin` to your PATH to 
>> be able to run the installed binaries
> 
> It is not very helpful when building ports. Do you think that we should
> try to mute it by default?

I have seen such a message when building a Haskell based port, too.

Seems there are more languages (or build systems) that perform such
a check and emit a misleading warning ...

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

portmaster plans (was: Re: portupgrade vs. portmaster)

[Stefan Esser]

Am 30.04.18 um 12:33 schrieb Thomas Mueller:
> Current portmaster, even before FLAVORS, was clumsy upgrading a large number 
> of ports, especially when there is an upgrade of perl or png.

The author of portmaster decided to abort the upgrade of all remaining ports,
if any dependency failed for any one port (i.e. did not try to build further
independent ports, in that case).

Or is there some other problem with e.g. perl upgrades?

I plan to introduce new build options, e.g. to rebuild all ports that depend
on libraries in some compat directory, or which have installed files in some
path (e.g. the perl ports that install in a sub-directory that depends on the
perl version).

What other criteria might be useful to select ports for upgrading?

> I see hardly any mention of synth on the freebsd-ports list.  Have synth 
> users become disenchanted?

I do not know how much development is occurring for synth. It is written in a
language that is only supported on amd64 (and i386?) and I'd rather use a tool
that works on all platforms.

> One downside is that synth fails to install build dependencies, so I have to 
> pkg install all of these separately, very annoying.

Pure build dependencies should not be required on a system, outside the build
process. If you need some compiler or build tool, then it is just a normal
program selected by the user.

> I was bitten just yesterday trying to build cross-compiling tools for Haiku 
> when make info was missing because texinfo was built but not installed.

This might be a problem with the specification of dependencies in the port.

> But I was able to recover with pkg install after checking my repository.

With correct dependencies, this should not have been necessary.

> With portmaster, I need to specify ports by category/portname rather than 
> just portname, for example
> portmaster www/seamonkey

Yes, this is intentional. It could easily be changed, but currently only a
parameter with / in it can lead to a new port being permanently installed
(instead of as a dependency that might be automatically deleted when no longer
required).

What is the proposed semantic of a parameter that might select a port origin
or some installed package, say if I invoke "portmaster bash" with or without
bash being installed?

> Will both the old and revamped portmaster be maintained, and what will be the 
> port names, since there can't be two ports both named portmaster?

I plan to create a portmaster-devel port, once the main-functionality of the
existing portmaster port is complete. That version is meant for testing and
as development version for new features. It should replace the current version
after a few (say 3 to 6) months of testing.


I'm willing to consider feature requests for the new version. My goal for now
is to make it automatically deal with situations like the recent KDE4 port and
package renaming, but also with scenarios not covered by the current version,
where an indirect dependency must be considered.

I'd like to implement a feature that builds all ports in a clean environment,
as synth does. But this is not a priority, since we have poudriere and that is
the official tool to build your own package repository. But a low overhead
mode that works without setup effort on a subset of what poudriere offers
might be quite useful. (And synth might offer that if it was written in a more
portable language and if it was guaranteed to be well supported and maintained
over the coming years.)

Best regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

portupgrade vs. portmaster (was: Re: Port directory not found:)

[Stefan Esser]

Am 30.04.18 um 05:45 schrieb Kevin Oberman:
> portmaster(8) operates very similarly to portupgrade(8). There are some
> differences that can bite you, though, so read the man page first.

I used to be a portupgrade user, long ago (years before the introduction
of the new package tools), but then mobed over to using portmaster.

When the package system (PKG-NG) war completely reworked, I heard that
portupgrade was better adapted to the new tools, but did not verify that
claim (and instead submitted a few fixes for portmaster).

I'm working on a complete rewrite on portmaster, since the original author
has left the FreeBSD project, years ago, and I found it very hard to wrap
my mind around his design when I implemented FLAVOR support in portmaster.

My time is now spent on completing that new portmaster version, but I still
fix problems reported in the current portmaster port (but will not implement
any changes that are not bug-related, to be able to concentrate on the new
version).

> For updating a single port: portmaster PORTNAME (e.g. portmaster
> chromium-65.0.3325.181_1)
> Note that portmaster(8) treats when you type like there is an '*' at the
> end, so, e.g. "portmaster virtualbox-ose" will result in both
> virtualbox-ose and virtualbox-ose-kmod being updated. Very useful for
> updating multiple related ports such as "portmaster p5-" to update all p5
> ports.

I know that portmaster and portupgrade offer differing features and options,
but do not have time to identify features missing in portmaster to make it
a fully functional replacement of portmaster.

I'd appreciate, if portupgrade users could let me know, what features or
behaviour they miss in portmaster.

The globbing performed by portmaster (as explained by Kevin) is one of
the portmaster features that I find surprising at times, and I think
I'll require "virtualbox-ose*" for the current behaviour - but I'll let
the portmaster users decide, whether this is a useful change.

> If you are installing, you must include both the ports
> subdirectory/portdirectory. E.g. graphics/ImageMagick

Does portupgrade locate the correct category for you, if you just specify
the sub-directory within some category?

That is something that I could add to portmaster, but currently the slash
in a parameter signals, that a port directory is meant instead of a
package name ...

> Options are similar, but not identical nd there are a lot of added ones. I
> find "-y --clean-dirtfiles" very handy.

Those dirtfiles are distfiles, I assume ;-)

That is one of the many features already re-implemented in my new version.

Missing are special features like the temporary installation of pure build
dependencies (say bison or some cross-gcc version) from packages (if those
are available at the expected version level) and de-installation of those
dependencies after a successful creation of packages that needed to be
upgraded (e.g. for use by the nanobsd release builder).


So, if you miss a portupgrade feature in portmaster, let me know or create
a feature request on bugzilla. I'll consider those for the new version,
that I hope to be able to release in a few weeks.

Best regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Conflicts due to renamed KDE4 ports

[Stefan Esser]

Am 17.04.18 um 20:29 schrieb Tobias C. Berner:
> Moin moin
> 
> Here's a script which should automatically fix the origin for the
> kde4-versioned ports (based on the MOVED entries of r465345):
>    http://people.freebsd.org/~tcberner/scripts/fix_kde4_origins.sh
> 
> It //should// set the origins properly for the moved ports, and the output
> should be on the lines of
> # ./fix_kde4_origins.sh
> [...]
> - sysutils/baloo-widgets [sysutils/baloo-widgets-kde4] is not installed.
> + Changing origin of nepomuk-core-4.14.3_14 from sysutils/nepomuk-core to
> sysutils/nepomuk-core-kde4.
> - sysutils/kfloppy [sysutils/kfloppy-kde4] is not installed.
> - sysutils/ksystemlog [sysutils/ksystemlog-kde4] is not installed.
> + Changing origin of baloo-4.14.3_5 from sysutils/baloo to 
> sysutils/baloo-kde4.
> + Changing origin of kfilemetadata-4.14.3_13 from sysutils/kfilemetadata to
> sysutils/kfilemetadata-kde4.
> [...]
> 
> 
> Please let me know if that works for you, or how I could improve it.

My suggested version (that does not depend on any hard-coded version
strings in the script) is:

#!/bin/sh

cd /usr/ports
for origin in */*-kde4; do
origin_old=$(dirname $origin)/$(basename $origin -kde4)
pkgname=$(make -C $origin -V PKGNAME)
pkg_glob="${pkgname%%.*}.*"
package=$(pkg query -g "%n-%v" "$pkg_glob")
pkg set -y -o $origin_old:$origin $package
done

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Conflicts due to renamed KDE4 ports

[Stefan Esser]

 the pain of getting from the mess to something better organized has 
> to happen at some point.

The scheme looks good to me and will work for fresh installs. And I have
already pointed out, how such conflicts are generally being dealt with: If
it cannot be via MOVE entries, the the necessary preparation steps are given
in an UPDATING entry. But they should be complete (i.e. not only "set the
new origins following this scheme for all affected ports, since that may be
tens and the user cannot easily list them, since there is no simple pkg query
command that would generate this list).

> I think we've been saying this -- that things were going to happen this way 
> -- 
> for nearly a year. Maybe not in all the right places, though. 

Well, you may have said int in the KDE lists, but I'm not following them and
I'm just interested in keeping portmaster working for all ports, KDE included
but in no way special (except for the breakage caused ;-) ).

> On Monday, 16 April 2018 21:13:29 CEST Tijl Coosemans wrote:
>> On Mon, 16 Apr 2018 20:11:33 +0200 Stefan Esser <s...@freebsd.org> wrote:
>>> Am 16.04.18 um 12:38 schrieb Tijl Coosemans:
>>>> On Sat, 14 Apr 2018 14:18:22 +0200 Stefan Esser <s...@freebsd.org> wrote:
>>>>> The way the new KDE5/KF5 ports have been introduced a few weeks back has
>>>>> caused me quite some effort (more than 100 hours of work), and now there
>>>>> have been further changes to implement KDE5 support (which I generally
>>>>> appreciate), which cause further complications and seem not to be well
>>>>> thought out.
>>>>>
>>>>> These problems affect at least all portmaster users, but I guess
>>>>> portupgrade is affected as well and a "pkg upgrade" dry-run indicates,
>>>>> that it will also cause breakage to binary upgrades of KDE4
>>>>> installations.
>>>>
>>>> Removing entries from MOVED after only a few weeks is a bad idea, but
>>>> it's not something you have to worry about.  As long as portmaster
>>>> behaves more or less the same as pkg you're good.
>>>
>>> I only tried a dry run, but it appears, that pkg does not deal with this
>>> situation correctly. Grzegorz Junka reported, that it did not work for
>>> him and he had to manually delete all KDE ports and re-install them from
>>> his repository built with poudriere.
>>>
>>>
>>> A correct decision is impossible given on the information in the ports.
>>>
>>> It is correct to upgrade e.g. databases/akonadi (akonadi-1.13.0_6) to
>>> databases/akonadi-kde4 (akonadi-kde4-1.13.0_7), but neither the origin
>>> nor package name nor a MOVED entry provide that information.
> 
> This is correct if, and only if, you want the migration path of staying-with-
> KDE4.

I'm not interested in whether the user wants to stay with KDE4. I'm
interested in portmaster upgrading the ports on a system that has KDE4
packages installed. And currently it fails, which now has lead to more
than 600 ports not being upgraded on my development system. I could
(and do for critical packages) upgrade them individually, but a simple
"portmaster -a" stops for each affected KDE4 port, with the remainder of
the work list dropped. (That is a weakness in portmaster, since it was a
basic decision to not make it continue in case any port failed that might
be used as a dependency of some other port. I'm considering to change that
in the new major portmaster release I'm working on - the way it works now
was chosen by the original author of portmaster.)

>>> It is not correct to replace databases/akonadi (akonadi-1.13.0_6) by
>>> databases/akonadi (akonadi-17.12.3), but this can only be seen by
>>> checking the forward and backward dependencies (which are for KDE5/QT5
>>> instead of KDE4/QT4 of the installed port).
> 
> . and this is the correct move if you want to go with KDE Applications (the 
> current releases). The package manager and the ports-management tools can't 
> know which one you want, I don't think.

The tools cannot know and cannot decide this. My thinking is, that the old
KDE4 ports shall be kept and updated if either manually installed (not as
a dependency) and if they are still required (there are still ports that
depend on them).

Since the KDE5 versions do not cause any conflicts, they can be installed
besides the KDE4 versions, and any port that depends on teh KDE5 versions
will get them installed as independent ports. But they are not going to
cause the KDE4 versions to be uninstalled (unless the KDE4 versions have
lost their role as dependencies and are deinstalled by an auto-delete).

> Consider vim -- it doesn't h

Re: Conflicts due to renamed KDE4 ports

[Stefan Esser]

Am 16.04.18 um 21:13 schrieb Tijl Coosemans:
> On Mon, 16 Apr 2018 20:11:33 +0200 Stefan Esser <s...@freebsd.org> wrote:
>> When not even pkg can deal with this situation, how should portmaster?
>>
>> The packages are built without consideration for the requirements of a
>> running system, and pkg sees all the meta-information of all installed
>> packages and the one being processed and can e.g. see, that files will
>> conflict (which portmaster can only do after completely building the
>> port, which means that this is long after the decision to use that port
>> has been required).
>>
>>
>> The lack of consideration given by port maintainers is quite frustrating,
>> since it requires a lot of effort to work around the issues caused.
> 
> Like I said, IMHO it's not your problem, so you don't need to work around
> it and you don't have to feel frustrated by it.  Without an entry in MOVED
> there's no way for portmaster or pkg to know that the old akonadi needs to
> be replaced with akonadi-kde4.  If any user contacts you about this you
> can forward them to kde@ because they created the problem.
> 
> IMHO, entries in MOVED should stay for at least a year, if not several
> years, so kde@ should restore the kde4 MOVED entries and give the kde5
> ports a -kde5 suffix or something.  Hopefully there aren't that many users
> yet because you can't create MOVED entries for this move.

Seems that I misunderstood your reply ...

Yes, adding -kde5 to all ports that have got origins previously used for
KDE4 ports, and MOVED entries for those KDE4 ports would solve the issue.

It would have helped, if either port origin or package name of the KDE4
ports had been kept, since that would have allowed to link the ports and
packages over the change by the unchanged attribute.

The absolute minimum would have been a complete set of "pkg set -o" commands
to adjust the registered origins of all affected and installed KDE4 ports in
an UPDATING entry.

I'm not sure that poudriere can create packages that let pkg upgrade succeed
without file conflicts, unless the relation is revealed by the MOVED entries.


So, I agree with your proposal (previously also suggested by me) of MOVED
entries for the KDE4 ports and new non-conflicting origins for those KDE5
ports that re-use the previous KDE4 origins (obviously without MOVED entries,
but since the package names (without the version) remain unchanged for those
KDE5 ports, automatic port and package upgrades can work for them.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Conflicts due to renamed KDE4 ports

[Stefan Esser]

Am 16.04.18 um 12:38 schrieb Tijl Coosemans:
> On Sat, 14 Apr 2018 14:18:22 +0200 Stefan Esser <s...@freebsd.org> wrote:
>> The way the new KDE5/KF5 ports have been introduced a few weeks back has
>> caused me quite some effort (more than 100 hours of work), and now there
>> have been further changes to implement KDE5 support (which I generally
>> appreciate), which cause further complications and seem not to be well
>> thought out.
>>
>> These problems affect at least all portmaster users, but I guess portupgrade
>> is affected as well and a "pkg upgrade" dry-run indicates, that it will also
>> cause breakage to binary upgrades of KDE4 installations.
> 
> Removing entries from MOVED after only a few weeks is a bad idea, but
> it's not something you have to worry about.  As long as portmaster
> behaves more or less the same as pkg you're good.

I only tried a dry run, but it appears, that pkg does not deal with this
situation correctly. Grzegorz Junka reported, that it did not work for
him and he had to manually delete all KDE ports and re-install them from
his repository built with poudriere.


A correct decision is impossible given on the information in the ports.

It is correct to upgrade e.g. databases/akonadi (akonadi-1.13.0_6) to
databases/akonadi-kde4 (akonadi-kde4-1.13.0_7), but neither the origin
nor package name nor a MOVED entry provide that information.

It is not correct to replace databases/akonadi (akonadi-1.13.0_6) by
databases/akonadi (akonadi-17.12.3), but this can only be seen by
checking the forward and backward dependencies (which are for KDE5/QT5
instead of KDE4/QT4 of the installed port).

The same considerations applied to another port may lead to different
results. While pkg requires exact dependencies to be installed, it is
possible to use alternatives to satisfy dependencies with portmaster.
And this feature is heavily used, e.g. to use a different version of
samba than the default hard-wired into package dependencies. But this
flexibility needs a basis for deciding, whether such a replacement is
valid and how to perform upgrades in that situation.


If akonadi is installed only as a dependency of other ports, then pkg will
install the akonadi-kde4 version. But since the old version is installed
as an in-use dependency of other KDE4 ports, it will not be removed before
the installation of the new version is attempted (which will lead to an
install conflict, since files of an installed port are to be overwritten).

It is possible to manually and forcefully delete akonadi-1.13.0_6 before
starting pkg upgrade for the KDE4 ports that depend on it. In that case,
there is no conflict. But pkg autodelete cannot be used, since to remove
the dependency on the old version, the (conflicting) new version must be
registered in the ports that depend on akonadi.

If akonadi has been directly installed and not (only) as a dependency,
the akonadi-17.12.3 will be considered to be an upgrade of akonadi-1.13.*
(same origin and same package name, except for the version numbers). This
will remove the required dependency from the KDE4 ports and will register
the KDE5 version as new dependency of those ports (although it completely
useless in that role).


When not even pkg can deal with this situation, how should portmaster?

The packages are built without consideration for the requirements of a
running system, and pkg sees all the meta-information of all installed
packages and the one being processed and can e.g. see, that files will
conflict (which portmaster can only do after completely building the
port, which means that this is long after the decision to use that port
has been required).


The lack of consideration given by port maintainers is quite frustrating,
since it requires a lot of effort to work around the issues caused.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Conflicts due to renamed KDE4 ports

[Stefan Esser]

Am 14.04.18 um 18:57 schrieb Steve Kargl:
> On Sat, Apr 14, 2018 at 02:30:09PM +, Carmel NY wrote:
>> On Sat, 14 Apr 2018 14:18:22 +0200, Stefan Esser stated:
>>
>> {truncated}
>>
>>> This is another case (after the implementation of FLAVOR support that does
>>> not seem well-designed and causes lots of effort and inefficiencies in port
>>> management tools like portmaster), which makes me consider giving up my
>>> efforts to keep portmaster alive.
>>
>> Have you tried using "synth"?
> 
> This discussion occurred with the introduction of FLAVORS,
> which broken all ports management tools except poudriere.

Yes, but I put literally hundreds of hours of effort into
understanding portmaster (which is one monolythic 4000 line
shell script with global state that recursively invokes itself
to implement local state, with hundreds of instances forked in
practice) and implementing FLAVOR support.

That worked, until the next breakage was introduced by port
and package renames, that make it impossible to automatically
track the history of a port and to upgrade it correctly.

While poudriere just rebuilds ports (using available packages
to satisfy dependencies) and does not care what dependencies
a user has previously used on a system (e.g. which of multiple
SSL libraries, for ports that offer a choice). Instead the
packages built by poudriere will enforce a certain set of
dependencies, giving the user no choice (unless the packages
were custom built with specific options).

But it seems that the packages built by poudriere are also
not suitable for a clean upgrade of installed KDE4 ports.
At least in a test run, some 10 KF5 ports were going to be
installed during an attempt to upgrade a KDE4 port from an
official package.

Portmaster was fully functional before this new breakage, and
I'm really annoyed, that the KDE port and package name changes
have been performed without any thought about the consequences
for port management tools.

It is possible to work around this problem by manually setting
certain parameters in the package DB for each affected port.

I had expected that at least a script that perform these
operations on the package DB was provided.

Now the best option appears to be to remove all installed KDE4
ports and then to rebuild them with portmaster (which will work,
but requires a lot of unnecessary effort and time).

I'm still trying to find a satisfactory heuristic that lets
portmaster DTRT.

But this is a problem, since there are situations where one
choice of action is correct, while it will lead to corruption
of the installed packages in other cases.

The problem is, that now there are systems that have KDE4 ports
with package names (sans version) and origin identical to KDE5
versions of the respective program (e.g. databases/akonadi used
to be a KDE4 port that built akonadi-1.*, now it is a KDE5 port
that builds akonadi-17.*, which is of no use on a KDE4 system and
not suitable for use with KDE4 ports.

Upgrading akonadi (and the tens of other KDE4 ports whose port
directory has been hijacked by KDE5 versions) will thus create
useless KDE5 versions (and the KDE4 version will be removed
when the KDE5 version is installed).

Later upgrades of ports that depend on akonadi-kde4 will install
the correct new dependency (but are broken up to that point). But
the useless KDE5 ports will not be automatically removed.

Hmmm, if they were installed as an automatic dependency (and not
directly by the user), I could use that as a sign, that no other
port depends on them (unless they are actually required by a KDE5
package). This will require extra effort, but I can try whether
this works reliably.

I'll see whether I find an algorithm that uses information not
currently required or used to resolve these cases.

But this will only be in a completely new portmaster, which
shares just the options processing (to stay as compatible as
possible with existing scripts that invoke it) with the current
version in ports.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Conflicts due to renamed KDE4 ports

[Stefan Esser]

The way the new KDE5/KF5 ports have been introduced a few weeks back has
caused me quite some effort (more than 100 hours of work), and now there
have been further changes to implement KDE5 support (which I generally
appreciate), which cause further complications and seem not to be well
thought out.

These problems affect at least all portmaster users, but I guess portupgrade
is affected as well and a "pkg upgrade" dry-run indicates, that it will also
cause breakage to binary upgrades of KDE4 installations.


I'm trying to get portmaster operational again, after it has been unusable
for systems with KDE4 ports for a few weeks.

Since the way portmaster works prevents an easy solution, I'm currently
rewriting it from scratch (and had it mostly working for the normal upgrade
tasks, with some of the more "special" like cleaning up stale distfiles
missing).

This new version is now again broken, because of changes that I do not
know how to automatically deal with. (And IMHO it is impossible to deal
with them, without hard-coding specific work-arounds for the affected
ports. This is against the spirit of a generic port management tool.)


A number of KDE4 ports have been moved to port directories that have an
extra -kde4 appended. The package names changed at the same time, and
that caused problems for portmaster, since if there was a dependency
that referred to the new origin with -kde4 attached, it would be in
conflict with the installed package. (The -kde4 version would be built
and the install would fail because the non-kde version was not recongnized
as a previous version of the same port and was not automatically deinstalled
first. This is fixed in my new portmaster version, which finds the old
origin in the MOVED file and checks whether there is a port that was
installed from some meanwhile "moved" origin.)

Individually updating those ports first, could solve this issue, but there
was not list of affected ports and thus it was up to the user to detect
the build failures as they occurred, delete the old version and restart
the portmaster run. (My new version deals with this, based on the MOVED
file, as described above.)


Now the situation has become much worse: Now there are KDE5 versions of
quite a number of prior KDE4 ports, which share the origins and package
names of these prior KDE4 programs, and can only be recognized by their
version numbers being different from 4.*).

This leads to breakage in a number of ways:

Ports depending on say KDE4 akonadi but installed at a time when the
origin still was databases/akonadi will (probably) break when akonadi
is upgraded and replaced by the KDE5 version of akonadi. There is no
way that portmaster could know, that the KDE4 version is now built from
databases/akonadi-kde4 and installed as akonadi-kde4-4.* (instead of
just akonade-4.*).

New ports will try to install akonadi-kde4 as a dependency, which may
succeed, after databases/akonadi has been upgraded to the KDE5 version,
but before that has happened, there will be conflicting files and the
dependency on akonadi-kde4 cannot be satisfied by the ports system.

OTOH, I now have a KDE5 version of akonadi, which has not been requested
by the user (who may want to stay at KDE4 at the moment). This KDE5 version
has been built, because there was a KDE4 version, and the port system did
not know, that these two ports share their origin and package name (sans
version), but are not directly related.

Again OTOH, the upgrade of akonadi to the KDE5 version will break all
ports, that rely on the KDE4 version being installed. These seem to be:

$ pkg info -r akonadi
akonadi-1.13.0_6:
kdepimlibs-kde4-4.14.10_15
smokekde-4.14.3_3
kde-workspace-4.11.22_14
kdeplasma-addons-4.14.3_6
kdepim-runtime-kde4-4.14.10_9
kdepim-kde4-4.14.10_11
ruby24-korundum-4.14.3_2
py27-pykde4-4.14.3_7
baloo-kde4-4.14.3_7

on my system ...


A number of KDE4 ports have been copied to new origins (with -kde4 attached)
with there old directories still present and functional. That does also cause
problems for automatic port build tools like portmaster. The old ports seems
to be still valid and is not marked to be in conflict with its copy. Since
the package names have been changed (by appending -kde4) it is not possible
to detect, that these are in fact just renamed versions of the previous port.
Since there is no MOVED entry, the last possibility that might provide a hint
is lost. (There can be no MOVED entry, since the old name is to be re-used for
the KDE5 version of that program.)


I think it is a very bad idea to do any of the following:

1) Rename a port without an entry in MOVED (even though dependencies are
   updated) if the package name (sans version) is changed at the same time.

2) Copy a port resulting in two origins that build the same package and that
   are not marked as mutually conflicting (whether with identical or changed
   package names - but the latter 

Re: Error while making sysutils/u-boot-beaglebone

[Stefan Esser]

Am 10.04.18 um 21:59 schrieb Udit agarwal:
> Hi,
> I am trying to make uboot for beaglebone black using
> sysutils/u-boot-beaglebone port package, but getting the following error.
> How to resolve this? I tried with FORCE_PKG_REGISTER also, but no success.
> 
> root@rtemsbuild1:/usr/ports/sysutils/u-boot-beaglebone # make install
> ===>   u-boot-beaglebone-2018.03 depends on executable: gsed - not found
> ===>   gsed-4.2.2_1 depends on executable: makeinfo - not found
> ===>   texinfo-6.5,1 depends on executable: help2man - not found
> ===>   help2man-1.47.6 depends on package: p5-Locale-gettext>=0 - not found
> ===>   p5-Locale-gettext-1.07 depends on executable: msgfmt - not found
> ===>   gettext-tools-0.19.8.1 depends on package: libiconv>=1.14_11 - not
> found
> ===>  Installing for libiconv-1.14_11
> ===>  Checking if libiconv already installed
> ===>   An older version of libiconv is already installed (libiconv-1.14_10)
>   You may wish to ``make deinstall'' and install this port again
>   by ``make reinstall'' to upgrade it properly.
>   If you really wish to overwrite the old port of libiconv
>   without deleting it first, set the variable "FORCE_PKG_REGISTER"
>   in your environment or the "make install" command line.
> *** Error code 1
> 
> Stop.
> make[11]: stopped in /usr/ports/converters/libiconv
> *** Error code 1

You have libiconv-1.14_10 installed and that does not satisfy
the requirement of this port.

You could just delete the installed libiconv with:

# pkg delete -f libiconv

which will allow the dependency to be installed from within the
sysutils/u-boot-beaglebone port.

Alternatively go to /usr/ports/converters/libiconv and upgrade
that library to the latest version with:

# cd /usr/ports/converters/libiconv
# make clean
# make all deinstall install clean

That will allow the u-boot port to build with the required
libiconv version.

I'd go with the first method (which is simpler) but the second
method reduces the time during which libiconv has been de-installed
but the new version is not ready for installation.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Should be fixed (was: Why portmaster try to install python 3.6 when I have 3.4 set in /etc/make.conf?)

[Stefan Esser]

Am 10.01.18 um 17:29 schrieb Lev Serebryakov:
> 
>  I have system with pre-FLAVORed ports installed, and try to update
> ports with new version of "portmaster". My "/etc/make.conf" contains:
> 
> DEFAULT_VERSIONS+=  python2=2.7
> DEFAULT_VERSIONS+=  python3=3.4
> DEFAULT_VERSIONS+=  python=3.4
> 
>  But "portmaster -a" try to install python 3.6:
> 
> ===>>> All >> py34-pip-9.0.1 (12/12)
> ===>>> The devel/py3-pip port moved to devel/py-pip@py36
> ===>>> Reason: Moved to a flavored, generic, version
> ===>>> Currently installed version: py34-pip-9.0.1
> ===>>> Port directory: /usr/ports/devel/py-pip@py36
> ===>>> Launching 'make checksum' for devel/py-pip@py36 in background
> ===>>> Gathering dependency list for devel/py-pip@py36 from ports
> ===>>> Launching child to install devel/py-setuptools@py36
> ===>>> All >> py34-pip-9.0.1 >> devel/py-setuptools@py36 (13/13)
> ===>>> Port directory: /usr/ports/devel/py-setuptools@py36
> ===>>> Launching 'make checksum' for devel/py-setuptools@py36 in background
> ===>>> Gathering dependency list for devel/py-setuptools@py36 from ports
> ===>>> Launching child to install lang/python36
> ===>>> All >> py34-pip-9.0.1 >> devel/py-setuptools@py36 >>
> lang/python36 (14/14)
> 
>  What do I do wrong?

Please test with the (just committed) portmaster version 3.18_12.

Best regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Why portmaster try to install python 3.6 when I have 3.4 set in /etc/make.conf?

[Stefan Esser]

Am 10.01.18 um 23:56 schrieb Jim Trigg:
> ISTR that there is a more generic variable that will cause make to work with
> all flavors instead of the default "common" set. Not where I can easily look
> up specifics right now...

FLAVOR support has many rough edges and I expect those to hurt us in one way
or the other in the future. A lot of them could easily be fixed, but a sound
design should have covered them from the beginning.

There is no generic ALL_FLAVORS variable, but some of the generic FLAVOR
support implemented in Mk/Uses does offer a way to access all FLAVORs. In the
case of Python ports, you have to pass -D BUILD_ALL_PYTHON_FLAVORS to make to
get FLAVOR set to all supported Python versions/flavors:

$ make -V FLAVORS
 py27 py36

$ make -V FLAVORS -D BUILD_ALL_PYTHON_FLAVORS
py27 py36 py35 py34

The upcoming Emacs flavor support will use a different parameter and for any
of the many individual ports, that support flavors directly in their
respective Makefiles, the variable will have a local name.

There seems to be a clear opinion in the portmgr team, that not all flavors
of each port shall be built by the package build cluster, but only those that
are most commonly used (e.g. py27 and py36), which I understand because else
there would be thousands of additional python packages (and later perl and
others) that are hardly ever used. But there should still be provisions to
obtain a full list of flavors in a generic way. (I had used "FLAVORS" to
denote a list of all supported flavors and e.g. PKG_FLAVORS for those that
are selected for the package builders, but FLAVORS for the selected set and
ALL_FLAVORS for the complete set will do as well ...)

I'm not going to work-around such deficiencies in the design of FLAVORs in
portmaster. I'd have to follow each new port that is converted to flavors to
check whether it has all flavors in FLAVORS and which other name it uses for
a list of possible FLAVOR values (if such a list explicitly exists at all,
it could also be implicitly present in the form of control structures in the
port's Makefile).

I'll ask portmgr for a few small changes (a generic ALL_FLAVORS variable and
a make target that lists all package names and corresponding flavors for a
port), and I hope I get approval for these changes.

For now I'll have to develop a method that at least partially compensates the
missing FLAVOR support features required for port management tools.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Why portmaster try to install python 3.6 when I have 3.4 set in /etc/make.conf?

[Stefan Esser]

Am 10.01.18 um 21:30 schrieb Stefan Esser:
> Am 10.01.18 um 17:29 schrieb Lev Serebryakov:
>>  I have system with pre-FLAVORed ports installed, and try to update
>> ports with new version of "portmaster". My "/etc/make.conf" contains:
>>
>> DEFAULT_VERSIONS+=  python2=2.7
>> DEFAULT_VERSIONS+=  python3=3.4
>> DEFAULT_VERSIONS+=  python=3.4
>>
>>  But "portmaster -a" try to install python 3.6:
>>
>> ===>>> All >> py34-pip-9.0.1 (12/12)
>> ===>>> The devel/py3-pip port moved to devel/py-pip@py36
>> ===>>> Reason: Moved to a flavored, generic, version
>> [...]
>> ===>>> All >> py34-pip-9.0.1 >> devel/py-setuptools@py36 >>
>> lang/python36 (14/14)
> 
> Another example that the FLAVOR feature is not well designed :(
> 
> The entry in /usr/ports/MOVED maps devel/py3-pip to devel/py-pip@py36
> even though building that port with FLAVOR=py34 builds the correct
> version with the expected dependency on Python-3.4.
> 
> If at least the output of "make pretty-flavors-package-names" did hint
> at the existence of a package for FLAVOR=py34, I could make portmaster
> override the wrong FLAVOR obtained from the MOVED file.
> 
> But the output of that command is:
> 
> $ cd /usr/ports/devel/py-py
> $ make pretty-flavors-package-names
> py27: py27-py-1.5.2
> py36: py36-py-1.5.2

I have just checked, that the ports infrastructure already kind of
supports this case. With PYTHON3_DEFAULT=3.4 set in /etc/make.conf,
it is possible to determine the flavor that corresponds to the
package to be upgraded:

$ PYTHON3_DEFAULT=3.4 make pretty-flavors-package-names
py27: py27-py-1.5.2
py34: py34-py-1.5.2

I'll try to get this feature implemented in portmaster, but it may
take a few days ...

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Why portmaster try to install python 3.6 when I have 3.4 set in /etc/make.conf?

[Stefan Esser]

Am 10.01.18 um 17:29 schrieb Lev Serebryakov:
>  I have system with pre-FLAVORed ports installed, and try to update
> ports with new version of "portmaster". My "/etc/make.conf" contains:
> 
> DEFAULT_VERSIONS+=  python2=2.7
> DEFAULT_VERSIONS+=  python3=3.4
> DEFAULT_VERSIONS+=  python=3.4
> 
>  But "portmaster -a" try to install python 3.6:
> 
> ===>>> All >> py34-pip-9.0.1 (12/12)
> ===>>> The devel/py3-pip port moved to devel/py-pip@py36
> ===>>> Reason: Moved to a flavored, generic, version
> [...]
> ===>>> All >> py34-pip-9.0.1 >> devel/py-setuptools@py36 >>
> lang/python36 (14/14)

Another example that the FLAVOR feature is not well designed :(

The entry in /usr/ports/MOVED maps devel/py3-pip to devel/py-pip@py36
even though building that port with FLAVOR=py34 builds the correct
version with the expected dependency on Python-3.4.

If at least the output of "make pretty-flavors-package-names" did hint
at the existence of a package for FLAVOR=py34, I could make portmaster
override the wrong FLAVOR obtained from the MOVED file.

But the output of that command is:

$ cd /usr/ports/devel/py-py
$ make pretty-flavors-package-names
py27: py27-py-1.5.2
py36: py36-py-1.5.2

I could special case the knowledge that if a port can be built for
py36 but the package name starts with py34-, that the FLAVOR is forced
to be py34.

But I'm not going to implement that kind of work-around in portmaster,
I expect that the FLAVOR support is fixed in the port system.

>  What do I do wrong?

You are not doing anything wrong, I'm afraid. The problem is in the
current state of the ports system, with FLAVOR support just used as
an alternative to slave ports, but without a sound design (IMHO).

Once you have installed the port built with manually specified
FLAVOR=py34, this information is recorded in the PKG DB, from where
portmaster will fetch it and use it to upgrade the port, when needed.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: some fix needed still

[Stefan Esser]

Am 20.12.17 um 13:08 schrieb Sergio de Almeida Lenzi:
> Version portmaster-3.17.11_9
> when you are in the port directory, the portmaster build wrong path
> /usr/ports/ports instead of /usr/ports
> =
> 
> 
> 
> [root@VMS /usr/ports/x11-themes/kf5-oxygen-icons5]# portmaster 
> 
>   ===>>> No /usr/ports/ports/Mk exists, and no information
>   ===>>> about ports/Mk can be found in /usr/ports/MOVED
> 
> ===>>> No valid installed port, or port directory given
> ===>>> Try portmaster --help

Thank you for the report. Fixed in portmaster-3.17.11_11.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Setting system user home directory

[Stefan Esser]

Am 20.12.17 um 15:12 schrieb Adam Vande More:
> On Sat, Dec 16, 2017 at 2:10 PM, Dmytro Bilokha  wrote:
> 
>> Guys, thanks for your help. I've managed to adjust user's homedir
>> using pkg-install script. Now I'll try to move everything writable
>> from /usr/local to /var (as Miroslav suggested), test and submit the new
>> port version.
> 
> I think you should do what makes sense for your application.  The /var/db
> stuff is not a hard fast rule and it doesn't work well for many
> situations.  Also it's mostly system related DB's that live there.  It's
> not only some java related ports that live mostly under /usr but also
> things like postgres(at least used to).
> 
> IMO, as long as you're not flagrantly violating hier(7), do what is best
> for your port.

For a port that needs quite a large database (password hashes from accounts
that have been leaked), I implemented an option to initialize the application
by downloading the data files after the user had a chance to select a place
for the data (by editing the config file). The default is in /var/db, but the
user may prefer a home directory or some sub-directory of /usr/local/ ...

Regards, STefan

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: pkg flavors breaks "make run-depends-list"

[Stefan Esser]

Am 18.12.17 um 23:34 schrieb lb:
> Hi,
> 
> Is it possible to add the flavors into the name returned by 
> build-depends-list?
> 
> An example:
> 
> Loic@FreeBSD:/usr/ports|⇒  make -C x11-toolkits/py-qt5-gui build-depends-list
> FLAVOR=py36
> /usr/ports/ports-mgmt/pkg
> /usr/ports/lang/python36
> /usr/ports/devel/py-sip
> ...
> 
> Should be /usr/ports/devel/py-sip@py36

I have already proposed the same (with patch), see

https://reviews.freebsd.org/D13535

You can download the patch for testing from:

https://reviews.freebsd.org/D13535?download=true

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: OSS Audio

[Stefan Esser]

Am 17.12.17 um 02:06 schrieb blubee blubeeme:
> This is why I am bringing up the issue and why I said the first step would
> be porting the proper 4Front OSS into the kernel.
> 
> It doesn't have to be the default at first but if it's not in the kernel
> why bother wasting time to have it rejected because of all the
> misconceptions I've been dealing with in these threads.
> 
> Add 4Front OSS to the kernel
> Use the 4.x API/ Documentation
> Add it to an audio programming section of FBSD Handbook
> Upstream changes that make sense
> Simplify audio programming, documentation, all those audio sound servers
> Port device drivers to the OSS Device Driver API:
> http://manuals.opensound.com/sources/drv_index.html

Do you propose to just update the code to what 4Front provides?

This may work for you as individual user, but the 4Front license makes
it impossible to commit that version to FreeBSD. (That was the reason
to stay at a reasonably licensed version, very long ago.)

Or do you propose a clean-room implementation that in the end is fully
compatible with 4Front OSS 4.x, but does not violate their license and
intellectual property rights?

You are welcome to start with such a clean-room implementation and it
may even be accepted into FreeBSD, once you are ready (provided there
really is no risk of legal problems in any part of the world).

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: new portmaster fails to build devel/py-libzfs@py36 because of failing cython

[Stefan Esser]

Am 17.12.17 um 05:47 schrieb Jim Trigg:
> 
> 
> On December 15, 2017 4:50:53 AM EST, Johan Hendriks  
> wrote:
>> Hello all.
>> First of all thank you for the update of portmaster, much appreciated.
>>
>> When i am updating my ports, portmaster fails with the following error.
>>
>> > Compressing man pages (compress-man)
>> ===>>> Starting check for runtime dependencies
>> ===>>> Gathering dependency list for lang/cython@py36 from ports
>> ===>>> Dependency check complete for lang/cython@py36
>>
>> ===>>> devel/py-libzfs@py36 1/10 >> lang/cython@py36 (1/1)
>>
>> ===>  Installing for py36-cython-0.26
>> ===>  Checking if py36-cython already installed
>> ===>   Registering installation for py36-cython-0.26 as automatic
>> Installing py36-cython-0.26...
>> pkg-static: py36-cython-0.26 conflicts with cython3-0.26 (installs
>> files
>> into the same place).  Problematic file: /usr/local/bin/cygdb-3.6
>> *** Error code 70
>>
>> Stop.
>> make: stopped in /usr/ports/lang/cython
>>
>> ===>>> Installation of py36-cython-0.26 (lang/cython@py36) failed
>> ===>>> Aborting update
>>
>> ===>>> Update for lang/cython@py36 failed
>> ===>>> Aborting update
>>
>> ===>>> Update for devel/py-libzfs@py36 failed
>> ===>>> Aborting update
>>
>>
>> Is there someting i can try or do?

Yes, try the latest version of portmaster that I just committed ...

Adding flavor support to portmaster was complicated by the way parameters are
passed from one phase (config options, dependency checks, ...) to the final
build process.

It worked if either the port origin or package name changed when a port was
upgraded, but not if both were changed simultaneously.

And that was just the case with cython, were the origin changed from
"lang/cython3" to "lang/cython" (with flavor py36) and the package name from
"cython3-0.26" to "py36-cython-0.26".

There was no way for portmaster to make the connection between the old and new
version if both changed at a time, and thus it failed to deinstall the old
version before installing the new one.

I plan to completely change the way state is maintained in portmaster (from
implicit by some global variables being set) to explicit state variables, and
at the same time pass parameters explicitly.

But this will be a major change, which will void any patches that have been
attached to PRs and Issues, and will make it much harder to work on them.

> Question: is 3.6 your default python version? I've discovered that the 
> current version 
> of portmaster uses python default version for dependencies when building a 
> flavored 
> port.

Dependencies should be built based on the different *_DEPENDS variables
defined in the port's Makefile and the *.mk files of the port framework.

Portmaster currently only accepts the notation of "origin@flavor" when a new
port of a given flavor is to be installed. I'm planning to add a --flavor
option for that purpose, but I have to be careful not to introduce any
regressions by doing so.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Can we have multiple flavors for a port?

[Stefan Esser]

Am 16.12.17 um 14:51 schrieb Matthew Seaman:
> 
> I have a review up to add a USES=django --
> https://reviews.freebsd.org/D12592 which I'm now modifying to be FLAVORS
> compatible given that has hit the tree.
> 
> Now, we currently have ports for 4 different versions of django (1.8,
> 1.10, 1.11, 2.0) and the obvious next move is to add django-based
> flavouring.  Currently, most django ports depend on django-1.8, with a
> few depending on django-1.10, but this I think is mostly due to inertia
> and django ports should be compatible with more recent versions.  Having
> four different versions of django in ports seems excessive -- so, for
> the sake of argument I'd cut that down to just django-1.11 and django-2.0
> 
> But this is all python code, and the python flavouring would also apply...
> 
> Given that django-2.0 requires python-3.5+ we end up with the following
> combinations:
> 
>py27 django111
>py36 django111
>py36 django20
> 
> and there's some 104 django ports which will mostly end up needing all
> three of these variants built -- so as combinatorial explosions go, it's
> not particularly earth-shattering.
> 
> However, my questions are:
> 
>   - Is it possible or desirable to have multiple flavourings like this?
> 
>   - If so, what should the syntax look like for specifying a particular
> combination?  eg. www/py-django-mezzanine@py27+django111 ?

I asked portmgr@ the same question when I started to work on portmaster
and did not want to implement a solution that does only support a single
flavor per port (or rather, waste time and effort on a portmaster version
that only supported 1 flavor per port, when there were plans to support
multiple flavors in the ports framework in the future).

The answer I received was very clear: There can only be 1 flavor per port
and there are no plans to remove that restriction.

I questioned the wise-ness of that decision, but had better use for my
time than to continue arguing, that multi-flavor support was probably
going to be required very soon.


My questions were specifically, how a dependency with multiple flavors
should be specified (e.g. "@py36@xyz") and how to query a port for this
case. The same notation should be used in the MOVED file in such a case.

The order of flavors added to an origin must either be ignored by all
tools operating on them (which precludes simple string comparisons to
see whether one origin+flavors is identical to another one) or should
be required to follow some (e.g.) lexical order.


I'd also like to see the flavor(s) (if any) become part of the port origin
(or some other variable in the PKG DB that can easily be queried). The
value should be identical to the format in dependencies and the MOVED
file to simplify tools like portmaster, which currently must execute a
complex series of queries to get at that information.

The origin@flavor does not encode the information previously contained
in just the origin.


IMHO, multi-flavor support should have been one of the initial considerations
when designing the flavor extension to the ports system!

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: new portmaster fails to build devel/py-libzfs@py36 because of failing cython

[Stefan Esser]

Am 16.12.17 um 07:02 schrieb Jim Trigg:
> On 12/15/2017 08:25 AM, Stefan Esser wrote:
>> Am 15.12.17 um 11:21 schrieb Kubilay Kocak:
>>> On 15/12/2017 8:50 pm, Johan Hendriks wrote:
>>>> Hello all.
>>>
>>> Hi Johan
>>>
>>>> First of all thank you for the update of portmaster, much appreciated.
>>>>
>>>> When i am updating my ports, portmaster fails with the following error.
>>>>
>>>> > Compressing man pages (compress-man)
>>>> ===>>> Starting check for runtime dependencies
>>>> ===>>> Gathering dependency list for lang/cython@py36 from ports
>>>> ===>>> Dependency check complete for lang/cython@py36
>>>>
>>>> ===>>> devel/py-libzfs@py36 1/10 >> lang/cython@py36 (1/1)
>>>>
>>>> ===>  Installing for py36-cython-0.26
>>>> ===>  Checking if py36-cython already installed
>>>> ===>   Registering installation for py36-cython-0.26 as automatic
>>>> Installing py36-cython-0.26...
>>>> pkg-static: py36-cython-0.26 conflicts with cython3-0.26 (installs files
> 
>> Well, actually portmaster is expected to deal with that conversion of the
>> cython3 port.
>>
>> I have tested the upgrade of individual ports and of all ports that are
>> either back-level or where the ORIGIN changed (as in the case of this
>> particular port).
>>
>> Hmmm, now I see what's the problem: The package name seems to have been
>> changed during the conversion to a flavored port (from "cython3-0.26.tbz"
>> to "py36-cython-0.26.tar.bz"). This is not typical of ports that use
>> flavors now, in general they generate packages under the same name as the
>> non-flavored port did.
>>
>> I have to see, whether I can easily detect this case - the current logic
>> that distinguishes between fresh installs and re-installs does not see
>> that the old version needs to be deleted before installing the new one.
>>
>> For now the advice to manually delete the cython3 port is right and will
>> let you install the new version.
> 
> Would using -o work? portmaster -o lang/cython@py36 cython3

No, sadly not. The -o option is implicitly passed to an intermediate recursive
call of portmaster, but then another child is launched for the actual build,
which does not receive that option. And if it was passed, then the logic would
probably lead to another recursive invocation (leading to endless recursion).

I could try passing the old origin in an environment variable to the child
process, which is used if other means to detect the previous origin fail. That
should not break the control flow and decision to actually build the port in
the child process.

I do not fully understand, why portmaster calls itself in such a way (with
ever less options passed, while ever more context is in the environment), and
I can only guess, that this is a relict from pre-PKG_NG times, when portmaster
had to perform many more actions in the script instead of relying on pkg DB
state.

I plan to change this completely, but this will make it very hard to work on
existing PRs and GitHub issues, since any patches provided with those will not
cleanly apply anymore.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: portmaster with FLAVOR support available for testing

[Stefan Esser]

Am 15.12.17 um 16:11 schrieb Rainer Hurling:
> Am 15.12.2017 um 15:48 schrieb Walter Schwarzenfeld:
>> Yes, if it don't work in the port the port is the problem.
>>
>> Rainer Hurling was filed a PR
>>
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223764
> 
> Yes, Walter, your problem is another ports problem as mine, first
> described in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219641,
> comment #23, than doubled by PR #223764.
> 
> In this thread, with portmaster, I asked why portmaster deinstalls
> py27-qt5-core before it tries to install py36-qt5-core. This is not the
> case with pure ports make mechanism.

Please try again with the version I just committed. It seems, that there
was a path through the program that just removed the passed flavor and
then proceeded to operate on the default flavor (py27 in this case).

Sorry for these problems with the port. There are so many possible cases
and it is impossible for me to achieve sufficient coverage in my tests.

But I'll try to always quickly fix such problems, when they are brought
to my attention.

This was a real problem in portmaster, but problems with ports that
install files in places that don't differ for different flavors will
continue to cause the error message reported as an assumed portmaster
bug multiple times in the last few days.

Whenever the error message indicates that files are installed in the
same place, the problem is in the port, not in portmaster.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: portmaster strange output after patch

[Stefan Esser]

Am 15.12.17 um 20:59 schrieb Walter Schwarzenfeld:
> Found it:
> 
> It is the shebang
> 
> -#!/bin/sh
> +#!/bin/sh -x
> 
> Stefan please remove "-x"

Yes, thanks for reporting that I forgot to remove the -x that I had added for
testing. Since portmaster executes itself repeatedly, that was the best way
to get trace output from child processes, it did not suffice to execute the
script with "sh -x portmaster".

Sorry for the breakage, it is fixed in the patch I committed 15v Minutes ago.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: new portmaster fails to build devel/py-libzfs@py36 because of failing cython

[Stefan Esser]

Am 15.12.17 um 14:25 schrieb Stefan Esser> Hmmm, now I see what's the problem:
The package name seems to have been
> changed during the conversion to a flavored port (from "cython3-0.26.tbz"
> to "py36-cython-0.26.tar.bz"). This is not typical of ports that use
> flavors now, in general they generate packages under the same name as the
> non-flavored port did.
> 
> I have to see, whether I can easily detect this case - the current logic
> that distinguishes between fresh installs and re-installs does not see
> that the old version needs to be deleted before installing the new one.
> 
> For now the advice to manually delete the cython3 port is right and will
> let you install the new version.

I'm sorry, but fixing this problem will take quite some effort ...

The problem is, that portmaster operates in phases and recursively
invokes itself with an always smaller set of options. While knows
that the port origin has moved in one invocation, it then goes on
and just calls itself to build and install with only the new origin
(in this case "lang/cython@py36").

It is not possible to derive the old package name from that information,
portmaster must assume, that the package name (sans version) did not
change, it can then lookup the nane of the package to be replaced in
the package DB. But in the case of this port, the origin changed and
the package name changed at the same time, and I'd need to have at
least one of these parameters unchanged to lookup the old package name
to de-install it before installing the new version.

I have plans to refactor portmaster and completely rewrite the logic
that builds and installs packages, but this is a major undertaking and
earlier maintainers have given up after trying.

So please accept that it may be necessary to manually remove a package
in case both port directory (origin) and package name change at the
same time. All information is there at a point, but the structure of
the program does not allow to easily access it when actually needed.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: new portmaster fails to build devel/py-libzfs@py36 because of failing cython

[Stefan Esser]

Am 15.12.17 um 11:21 schrieb Kubilay Kocak:
> On 15/12/2017 8:50 pm, Johan Hendriks wrote:
>> Hello all.
> 
> Hi Johan
> 
>> First of all thank you for the update of portmaster, much appreciated.
>>
>> When i am updating my ports, portmaster fails with the following error.
>>
>> > Compressing man pages (compress-man)
>> ===>>> Starting check for runtime dependencies
>> ===>>> Gathering dependency list for lang/cython@py36 from ports
>> ===>>> Dependency check complete for lang/cython@py36
>>
>> ===>>> devel/py-libzfs@py36 1/10 >> lang/cython@py36 (1/1)
>>
>> ===>  Installing for py36-cython-0.26
>> ===>  Checking if py36-cython already installed
>> ===>   Registering installation for py36-cython-0.26 as automatic
>> Installing py36-cython-0.26...
>> pkg-static: py36-cython-0.26 conflicts with cython3-0.26 (installs files
> 
> notice the reference to cython3 here ---^
> 
> lang/cython and lang/cython3 use to both exist, one for python2, the
> other for python3.
> 
> lang/cython3 was recently deleted, since lang/cython now supports
> multiple concurrent installations with different python versions.
> 
> pkg delete cython3, then go again.
> 
> Let us know how it goes

Well, actually portmaster is expected to deal with that conversion of the
cython3 port.

I have tested the upgrade of individual ports and of all ports that are
either back-level or where the ORIGIN changed (as in the case of this
particular port).

Hmmm, now I see what's the problem: The package name seems to have been
changed during the conversion to a flavored port (from "cython3-0.26.tbz"
to "py36-cython-0.26.tar.bz"). This is not typical of ports that use
flavors now, in general they generate packages under the same name as the
non-flavored port did.

I have to see, whether I can easily detect this case - the current logic
that distinguishes between fresh installs and re-installs does not see
that the old version needs to be deleted before installing the new one.

For now the advice to manually delete the cython3 port is right and will
let you install the new version.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: portmaster with FLAVOR support available for testing

[Stefan Esser]

Am 15.12.17 um 13:36 schrieb Walter Schwarzenfeld:
> I try to install py27-qt5-core and py35-qt5-core in the port, get:
> 
> ===>   Registering installation for py35-qt5-core-5.7.1
> Installing py35-qt5-core-5.7.1...
> pkg-static: py35-qt5-core-5.7.1 conflicts with py27-qt5-core-5.7.1 (installs
> files into the same place).  Problematic file:
> /usr/local/share/py-sip/PyQt5/QtCore/QtCoremod.sip
> *** Error code 70
> 
> It is the question is it a problem of the port or portmaster ? (or both?)

Hi Walter,

this appears to be a restriction of the port. It puts files into
${PYTHON_SITELIBDIR}, which depends on the Python version, but also
into ${PREFIX}/share/py-sip/PyQt5/QtCore/ independently of the Python
version.

If you want to test, whether portmaster plays a role, you can just
build and install the port directly.

You'll see, that the same message is printed and that portmaster is
not part of the problem ;-)

Gruß aus Köln, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

portmaster now supports FLAVORs

[Stefan Esser]

I have just committed the upgrade of portmaster to a version with flavor
support.

You need to upgrade portmaster on its own (e.g. by "portmaster portmaster")
and it should then be possible to use "portmaster -a" with flavored and
non-flavored ports (and automatically re-install ports, that have been
converted to generic versions with flavors).

Please let me know, if you notice any deviations from previous behavior.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

portmaster with FLAVOR support available for testing

[Stefan Esser]

I have created a new version of portmaster with FLAVOR support.

Before committing the changes to the ports repository, I'd like to receive
some feedback from users.

My tests have only covered port upgrades, not any of the other features
offered by portmaster. In fact, I'd like to remove several of the other
features, which may have been of use before PKG_NG (e.g. functions that
use the INDEX file, and in fact also the -P/-PP/--packages-* features).


*** Please let me know, if you want to receive the new version by personal
*** mail (I do not want to spam the mail-list by posting a 100KB+ file).


The following is example output from an portmaster upgrade run that I just
performed. It includes upgrades of flavored and non-flavored ports and the
re-installation of ports that have been converted to flavors Without wersion
update:

# portmaster -dgw -a

[...]

===>>> Launching child to update py27-werkzeug-0.12.2 to py27-werkzeug-0.13

===>>> All >> py27-werkzeug-0.12.2 (5/5)

===>>> Currently installed version: py27-werkzeug-0.12.2
===>>> Port directory: /usr/svn/ports/head/www/py-werkzeug

===>>> Launching 'make checksum' for www/py-werkzeug in background
===>>> Gathering dependency list for www/py-werkzeug from ports
===>>> Launching child to install security/py-openssl@py27

===>>> All >> py27-werkzeug-0.12.2 >> security/py-openssl@py27 (6/6)

===>>> Currently installed version: py27-openssl-17.3.0
===>>> Port directory: /usr/svn/ports/head/security/py-openssl@py27

===>>> Launching 'make checksum' for security/py-openssl@py27 in background
===>>> Gathering dependency list for security/py-openssl@py27 from ports
===>>> Launching child to install devel/py-six@py27

[...]

===>>> The following actions were performed:
Upgrade of avidemux-2.6.11_6 to avidemux-2.6.11_7
Upgrade of libva-intel-driver-1.8.3_1 to libva-intel-driver-2.0.0
Upgrade of nghttp2-1.28.0 to nghttp2-1.28.0_1
Upgrade of py27-psutil-5.4.1 to py27-psutil-5.4.2
Re-installation of py27-six-1.11.0
Re-installation of py27-cffi-1.7.0
Re-installation of py27-asn1crypto-0.22.0
Re-installation of py27-enum34-1.1.6
Re-installation of py27-idna-2.5
Re-installation of py27-ipaddress-1.0.18
Re-installation of py27-cryptography-2.0.3
Re-installation of py27-openssl-17.3.0
Upgrade of py27-werkzeug-0.12.2 to py27-werkzeug-0.13
Upgrade of sbcl-1.4.1,1 to sbcl-1.4.2,1
Upgrade of scons-2.5.1_1 to scons-3.0.1
Upgrade of xfce4-notifyd-0.4.0 to xfce4-notifyd-0.4.1

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Flavor part of package origin?

[Stefan Esser]

Am 11.12.17 um 16:10 schrieb RW via freebsd-ports:
> On Mon, 11 Dec 2017 14:38:59 +0100
> Mathieu Arnold wrote:
> 
>> Le 11/12/2017 à 10:12, Stefan Esser a écrit :
>>> $ pkg info -o '*setuptools*'
>>> py27-setuptools-36.5.0 devel/py-setuptools@py27
>>> py36-setuptools-36.5.0 devel/py-setuptools@py36  
>>
>>
>> I really do not like the look of this. The origin always has been a
>> directory name, with this change, it would become some abstract thing.

The origin is used by tools to identify the location in the ports tree that
allows to build a package. With FLAVOR support, knowing the directory is
insufficient to build a package - for that reason the @FLAVOR notation has
been used in a few places already, e.g. in the MOVED file and for dependency
declarations.

> It's not all that abstract if you can simply strip the end part off. 

The directory is /usr/ports/${origin%@*} in that case, and this stripping of
the FLAVOR is already required and used in e.g. bsd.ports.mk.

With origin@flavor, there is an alternate identifier that is equivalent to
the PKGNAME, and this information is used in port management tools, e.g.
when a port is to be rebuilt for a given PKGNAME.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: make reinstall does not work

[Stefan Esser]

Am 11.12.17 um 12:38 schrieb Stefan Esser:
> Am 09.12.17 um 00:29 schrieb Shawn Webb:
>> On Fri, Dec 08, 2017 at 10:35:50PM +0100, Walter Schwarzenfeld wrote:
>>> I had a strange error: Make reinstall does not work anymore in the port
>>> (10.3-amd64) also FORCE_PKG_REGISTER.
>>>
>>> All other make commands works fine. I don't know if it is related to
>>> FLAVORS.
>>>
>>> make reinstall ended up e.g. with:
>>>
>>> ??Registering installation for gtk2-2.24.31
>>> *** Error code 70
>>>
>>> Stop.
>>> make[2]: stopped in /usr/ports/x11-toolkits/gtk20
>>> *** Error code 1
>>>
>>> Stop.
>>> make[1]: stopped in /usr/ports/x11-toolkits/gtk20
>>> *** Error code 1
>>>
>>> Any idea or suggestion?
>>
>> This is due to this commit:
>> https://github.com/freebsd/pkg/commit/7991c49665419916210ad589d4a85fd2a7f58b37
>>
>> The standard procedure for reinstall is to do a deinstall first. I
>> guess it's pretty common just to issue `make reinstall` (which is what
>> I used to do as well). However, that's not the originally intended
>> behavior as designed in the Ports build framework.
>>
>> So: just do a `make deinstall reinstall`. It'll work that way.
> 
> "make reinstall" combines 2 actions:
> 
> 1) De-installation of the currently installed version of the port, if any.

Hmmm, seems I was wrong about this aspect ...

There was no implicit de-installation, just FORCE_PKG_REGISTER being passed
to the sub-sequent "make install", it appears. And forcing the installation
in that way does not seem to work anymore ...

> 2) Installation of  the port even if the work directory contains the tag-files
>that indicate that the port has already been installed after building.
> 
> If you did not complete an installation of a freshly built port, then
> "make deinstall reinstall" is equivalent to "make deinstall install" and ought
> to be equivalent to a plain "make reinstall" IMHO.
> 
> Could the "70" correspond to the SQLITE_INDEX_CONSTRAINT_ISNOTNULL error code
> returned by SQlite operating on the package DB?
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: make reinstall does not work

[Stefan Esser]

Am 09.12.17 um 00:29 schrieb Shawn Webb:
> On Fri, Dec 08, 2017 at 10:35:50PM +0100, Walter Schwarzenfeld wrote:
>> I had a strange error: Make reinstall does not work anymore in the port
>> (10.3-amd64) also FORCE_PKG_REGISTER.
>>
>> All other make commands works fine. I don't know if it is related to
>> FLAVORS.
>>
>> make reinstall ended up e.g. with:
>>
>> ??Registering installation for gtk2-2.24.31
>> *** Error code 70
>>
>> Stop.
>> make[2]: stopped in /usr/ports/x11-toolkits/gtk20
>> *** Error code 1
>>
>> Stop.
>> make[1]: stopped in /usr/ports/x11-toolkits/gtk20
>> *** Error code 1
>>
>> Any idea or suggestion?
> 
> This is due to this commit:
> https://github.com/freebsd/pkg/commit/7991c49665419916210ad589d4a85fd2a7f58b37
> 
> The standard procedure for reinstall is to do a deinstall first. I
> guess it's pretty common just to issue `make reinstall` (which is what
> I used to do as well). However, that's not the originally intended
> behavior as designed in the Ports build framework.
> 
> So: just do a `make deinstall reinstall`. It'll work that way.

"make reinstall" combines 2 actions:

1) De-installation of the currently installed version of the port, if any.

2) Installation of  the port even if the work directory contains the tag-files
   that indicate that the port has already been installed after building.

If you did not complete an installation of a freshly built port, then
"make deinstall reinstall" is equivalent to "make deinstall install" and ought
to be equivalent to a plain "make reinstall" IMHO.

Could the "70" correspond to the SQLITE_INDEX_CONSTRAINT_ISNOTNULL error code
returned by SQlite operating on the package DB?

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: make reinstall does not work

[Stefan Esser]

Am 09.12.17 um 00:22 schrieb Kevin Oberman:
> On Fri, Dec 8, 2017 at 2:52 PM, Greg 'groggy' Lehey 
> wrote:
> 
>> [rearranged]
>>
>> On Friday,  8 December 2017 at 23:42:40 +0100, Jaap Akkerhuis wrote:
 On Dec 8, 2017, at 22:35, Walter Schwarzenfeld <
>> w.schwarzenf...@utanet.at> wrote:

 I had a strange error: Make reinstall does not work anymore in the port
>> (10.3-amd64) also FORCE_PKG_REGISTER.

 All other make commands works fine. I don't know if it is related to
>> FLAVORS.

 make reinstall ended up e.g. with:

  Registering installation for gtk2-2.24.31
 *** Error code 70

 Stop.
>>>
>>> When I see this Error code 70 I do a make deinstall and then a make
>>> install. That worked for me
>>
>> Yes, I've seen this too and used the same workaround.
>>
>> Has something changed?  Does 'make reinstall' still work for others?
>>
>> Greg
>>
> 
> I regularly use 'make reinstall'  and it still worked yesterday.
> 
> # cd /usr/ports/devel/py-setuptool
> # make FLAVOR=py27
> # make FLAVOR=py27 deinstall

This step removed the package from the system ...

> # make FLAVOR=py27 reinstall

This is now equivalent to a plain "make install"

I.e. you are always performing the "deinstall install" sequence that has
been mentioned as a work-around for when "make reinstall" fails with error
code 70 ...

> # make FLAVOR=py27 clean

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Flavor part of package origin?

[Stefan Esser]

Am 10.12.17 um 21:20 schrieb Kurt Jaeger:
> Hi!
> 
>> Shouldn't the FLAVOR be part of the package origin?
>>
>> $ pkg info -o '*setuptools*'
>> py27-setuptools-36.5.0 devel/py-setuptools
>> py36-setuptools-36.5.0 devel/py-setuptools
> 
> Yes, but it seems this works:
> 
> $ pkg info -o '*setuptools*'
> py27-setuptools-36.5.0 devel/py27-setuptools
> py34-setuptools-36.5.0 devel/py34-setuptools
> py35-setuptools-36.5.0 devel/py35-setuptools
> py36-setuptools-36.5.0 devel/py36-setuptools
> 
> with pkg 1.10.99.5

But that output is wrong for a system with ports built after FLAVOR support
has been activated.

I have already asked portmgr, whether it might be possible, to consider the
flavor as a required part of the origin. That should lead to the following
output for the first command example:

$ pkg info -o '*setuptools*'
py27-setuptools-36.5.0 devel/py-setuptools@py27
py36-setuptools-36.5.0 devel/py-setuptools@py36

I'd also like to have a make target "list-pkgname-flavors", which should
provide somewhat similar output when run in a port directory:

$ cd /usr/ports/devel/py-setuptools
$ make list-pkgname-flavors
py27-setuptools-36.5.0 py27
py36-setuptools-36.5.0 py36

The purpose is to easily identify the FLAVOR to be passed when building some
package with a known name and is the counterpart to querying the FLAVOR of a
package from the pkg DB.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Working on FLAVOR support in portmaster

[Stefan Esser]

Am 10.12.17 um 18:47 schrieb Matthieu Volat:
> They do... but only if you commit and push something (even if it's only
> a personnal clone). If you just keep the changes on your computer, there's
> nothing.
The GitHub master version has changes, that are not yet in any release.

This is irrelevant as long as FLAVOR support is missing in portmaster,
since there is no version that fully supports flavors, right now.

> As much as I am defiant of github on certain aspects, I've found in quite
> some occasion the discussion/comment system around pull requests quite nice.

I'm working in FLAVOR support and I have a version that correctly builds
the Python ports, that have been converted.

But I'm currently trying to understand, where the information that the
ports is to be re-installed, gets lost. Debugging shell scripts is a lot
of work, since you cannot single step through them. Portmaster does call
itself recursively, which further complicates understanding and tracing
the execution. (Besides, portmaster is a main program of 4300+ lines with
functions sprinkled throughout the code. I have a local version, which
breaks this large main program in named subroutines, which makes it much
easier to understand the logic flow, but hides the actual changes when
creating diffs. I have backported the FLAVOR changes to a portmaster
version without those subroutines, to get the minimal functional patch,
but now I'm fighting with the install vs. upgrade distinction being lost.)

I can send you the current version in private mail (I do not want to spam
the mail-list with a 120k+ shell script).

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Working on FLAVOR support in portmaster

[Stefan Esser]

Am 05.12.17 um 00:43 schrieb Tatsuki Makino:
> By the way, where is the clever way to update to flavor?
> I am using portmaster.

I'm working on FLAVOR support in portmaster. My version did already build
all updated ports, the FLAVOR parameter is passed to build sub-processes,
but there is still some confusion between multiple flavored versions of the
same port (installing the py27 version wants to deinstall the py36 version
and vice versa), which I still have to fix.

I'm not sure that I have time to complete the fix today, but it is not too
hard. Ports need to complement the port origin with the FLAVOR, where
appropriate (e.g. when a flavored destination is found in MOVED). Already
installed packages are annotated with "flavor" and that must be passed to
the build command, when that port is updated. Most other logic in portmaster
remains unaffected.


My work version has all non PKG_NG support stripped, but that is mainly to
not waste effort fixing irrelevant sub-routines.

Is it acceptable, to have portmaster stop supporting the old package system?
AFAIK, there is no way that a modern ports tree with flavor support works
with a non-PKG_NG infrastructure?

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Firefox became much slower

[Stefan Esser]

Am 01.11.17 um 12:40 schrieb Guido Falsi:
> On 11/01/2017 12:15, Baho Utot wrote:
>>
>>
>> On 11/01/17 02:07, Gleb Popov wrote:
>>> Hello.
>>>
>>> I'm using Firefox on quite ancient machine (amd64, though) and after
>>> updating from firefox-56.0.1_3 to 56.0.2_3,1 it has become much more
>>> sluggish - whole UI hangs during page loading, scrolling isn't smooth
>>> anymore.
>>>
>>> Anyone also see this?
>>> ___
>>
>>
>> Yep FF is starting to get on my nerves.  Javascript brings it to it's knees
> 
> Most sites make an unreasonable use of javascript though. The recent
> news about sites and CDNs leveraging javascript to mine bitcoins make
> things even worse.
> 
> My personal suggestion is to use noscript, most sites work quite fine
> without javascript and even more sites work fine by allowing only a few
> origins, but blocking dozens of third party scripts.

I used to use NoScript, but have switched to uMatrix, meanwhile.

uMatrix has lots of pre-configured rules (blocks trackers and known
malware sites), but I'm using it in conjunction with uBlock from the
same developers.

Both are available as "new-style" web-extensions and will continue
to work with Firefox 57.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

new port security/cvechecker

[Stefan Esser]

Am 13.10.17 um 09:25 schrieb Torsten Zuehlsdorff:
> Aloha,
> 
>>> Why not
>>> teach pkg-audit(8) to query NVD based on CPE annotations in *binary*
>>> packages?
>>> Doing so would also provide a workaround for VuXML entries cancelled
>>> to reduce bloat.
>>
>> I agree, pkg-audit needs to be taught to do that. Along those lines,
>> we could create a port for cvechecker:
>>
>> https://github.com/sjvermeu/cvechecker
>>
>> But both solutions only handle installed packages.
>>
>> We would still need something to alert us to CVEs in non-installed
>> software, I think.
>>
>> Also, I've just looked and it seems only a little over 1000 ports have
>> CPE strings. Adding something to portlint that warned ports developers
>> to add any needed CPE info would be helpful. I think that type of
>> warning has helped us improve LICENSE entries.
> 
> One more thought on this topic: a cvececker isn't enough. Looking at
> security updates of piwik, gitlab, phpmailer and many more: most of the
> security issues fixed never got an CVE entry. But of course any of the
> issues could be exploited in one or another way.
> 
> But i think cvechecker is a step in the right direction. pkg audit is
> incredible helpful even with its current restrictions!

Well, and now cvechecker is in ports :)

Please let me know about any problems with the port.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: New pkg audit FNs

[Stefan Esser]

Am 10.10.17 um 00:03 schrieb Steve Wills:
> Hi,
> 
> On 10/09/2017 17:55, Jan Beich wrote:
>> Why not
>> teach pkg-audit(8) to query NVD based on CPE annotations in *binary*
>> packages?
>> Doing so would also provide a workaround for VuXML entries cancelled
>> to reduce bloat.
> 
> I agree, pkg-audit needs to be taught to do that. Along those lines, we
> could create a port for cvechecker:
> 
> https://github.com/sjvermeu/cvechecker

I have a mostly working port of cvechecker, which I plan to commit
soonish.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: [HEADUP] FLAVORS landing.

[Stefan Esser]

Am 27.09.17 um 22:57 schrieb Don Lewis:
> If at some point you run into a bug and need the debug files, you can
> pkg install the debug files for whatever packages that you need without
> disturbing your already installed runtime files, and then you can later
> deinstall the debug files when you are done with them without needing to
> reinstall the runtime files.  The same thing applies to docs.

This assumes, that a matching version of the debug files is still
available. Otherwise, you had to first install the latest version
of the package and to reproduce the situation with that version.

This may be seen as a feature (bug reports only for the version
currently in ports), but may be impractical in many cases.

The matching of versions of base package and sub-packages must be
more strict than by version number, since trivial changes might be
applied to a port without incrementing the PORTREVISION, but with
impact on the binary, e.g. if the port is to built with some gcc
version from ports and that gcc port has been updated, leading to
different object files and debug symbols than a previous version
of the port with identical version number.

A "build number" could be added to each (sub-)package and only if
this build number matches, a sub-package may be installed on top
of an already installed base package. The build time/date could of
course be used instead, if an identical value is used for all the
corresponding files.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: [HEADUP] FLAVORS landing.

[Stefan Esser]

Am 26.09.17 um 16:05 schrieb Mathieu Arnold:
> Hi,
> 
> **Do not commit FLAVORS to any ports, a hook will prevent it, that being
> said, do try it and test what can be done.**
> 
> To test this feature in poudriere, you need
> poudriere-devel-3.1.99.20170621 or later.
> 
> This has been a long awaiting feature, most of the work has been done by
> bapt, bdrewery and antoine, I am just the one actually doing the
> announce and commit and all.
> 
> All this information, and more to come are in the first link to our wiki
> in the bottom block. A roadmap is in the second link.
> 
> To define a different flavors in a port, before any include, set:
> 
> FLAVORS=  flavor1 flavor2 [...]
> 
> The first flavor in the list will be the default.

One question regarding the naming of packages:

The default flavour should probably(?) lead to a package with the same
name as created without flavours.

But any non-default flavour needs a distinct package name.

In the case of the Python ports, this will be the version prefix (py27-
vs. py36-).

But what is planned for the general case?

Will it be left to the port maintainer to add prefixes / postfixes to
package names that reflect the flavours by means of the options that
are enabled by some particular flavour?

Or will the flavour be automatically added to the package name, if
there would be name collisions, otherwise?

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: [HEADUP] FLAVORS landing.

[Stefan Esser]

Am 27.09.17 um 13:52 schrieb Julian Elischer:
> On 27/9/17 4:20 pm, Matthew Seaman wrote:
> 
> Before this gets too far down the road I would like to suggest that we
> quickly formalise some nomenclature
> or we will have 200 different ideas as to how to do the same thing;
> 
> I would like to propose the following possible "examples of official"
> flavours:
> -nodocs ..  nearly every port has a DOCS option..  a way to
> automatically turn it off globally and generate said pkgs would be good.
> -minimal ..  smallest possible feature set.. probably used just to
> satisfy some stupid dependency.
> -kitchensink    ..  speaks for itself .. options lit up like a christmas
> tree
> -runtime    ..  no .a files, include files, development
> documentation or sources ..
>     might only contain a single libxx.so.N file, or a
> single binary executable.

No, these are no good examples for flavours, as I understand them ...

These are possible typical sub-package categories, or rather you could
remove the DOCS from the base port, but offer a sub-package for them.


I'd rather think that NO-X11 might become a typical flavour, or the
dependency on a particular crypto library (e.g. openssl vs. libressl).


Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Porters Handbook section 4.4

[Stefan Esser]

Am 26.09.17 um 08:21 schrieb Kubilay Kocak:
> In section:
> 
> 4.4. Patching
> 
> - Add new section (at/numbered 4.4.2)
>   - Name: Automatic Patch Generation
> - Renumber sections (4.4.2 -> 4.4.3)
> 
> Text:
> 
> The ports framework provides a {{{makepatch}} target, which when run,
> automatically creates correctly named and formatted patch files in the
> correct location. The general process is as follows:
> 
> % cd 
> % make patch
> 
> Note: In the general case, {{{make patch}} is used (not just {{{make
> extract}}} to extract the DISTFILES), because ports that contain
> existing patches need to have the patches applied so that they are also
> generated (regenerated) in the last step.
> 
> ```
> % cd work/ (WRKSRC)
> ```
> 
> At this point, make the source changes in WRKSRC:
> 
> ```
> % cp  .orig

Do not perform this copy operation, IF the .orig file does already exist
(from the earlier "make patch" step).

I'd also use "cp -p" to preserve the original modification time, which
will be included in the patch ...

Else, your change will be against a patched file, without the patch that
lead to this state being conserved.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: make install for print/texinfo fails on -CURRENT

[Stefan Esser]

Am 12.07.17 um 18:11 schrieb David Naylor:
> On Tuesday, 11 July 2017 08:47:17 Bob Willcox wrote:
>> Hmm, I just tried running synth on my test system again (this is the system
>> that I successfully built and install texinfo on just a bit ago) and synth
>> still fails with the same build errors as before. I'm not all that familiar
>> with synth, but it appears that it may be working with some old stuff.
> 
> Hi
> 
> (Replying to a random email)
> 
> I've encountered the exact same issue while building the i386 packages for 
> wine.  Once I forced a rebuild of _all_ texinfo dependencies, installation 
> worked.  
> 
> I suspect an ABI change in -CURRENT that broke a dependency (sounds like perl 
> based on the thread).  

I had a problem with subversion after the upgrade to 1.9.6, and this
was due to an incompatibility with the apr library, which had not been
rebuilt for some time.

Seems that this is fall-out from the ino64 changes. Seems that while
symbol versioning allows old libraries to access struct stat as it was
before the change, this struct stat passed to a freshly built binary
(or library) by reference will cause obvious failures.


So, if you compile any port after the ino64 change, better make sure
that all referenced libraries (that operate on struct stat and might
pass such a struct to a fresh binary) have been rebuilt before.

As long as binary packages are built on a pre-ino64 machine, the
resulting packages ought to work on any -CURRENT system. But if you
build any port that references a library installed by such a package,
you risk mixing different versions of struct stat. This will continue
to be a problem as long as the -CURRENT package builders are pre-ino64.

If the package builders are upgraded to support ino64, all ports that
call stat() (or other functions with call signature affected by ino64)
should have PORTREVISION bumped, as should all ports depending on them.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Update failure E000022 after upgrade to subversion 1.9.6

[Stefan Esser]

Am 12.07.17 um 07:42 schrieb Rainer Hurling:
> Hi duglas@
> 
> I had the same problem. I completely solved it by rebuilding the
> dependency chain of devel/subversion:
> 
> portmaster serf-1.3.9_1 expat-2.2.1 gettext-runtime-0.19.8.1_1
> apr-1.5.2.1.5.4_2 sqlite3-3.19.3_1 subversion-1.9.6
> 
> Probably, only one of them is the culprit, but I haven't testet ;)

I have, and it appears to have been devel/apr1 ...

After rebuilding first apr and then subversion, the problem is resolved
on my system.

The cause seems to have been the ino64 change, which is visible from the
userland. Library versioning should take care of that, but that does not
work, if subversion wants the new version and apr needs the old one ...

Other ports could be affected as well, if they link against a library
that needs an old version of a C library function, while the binary was
built for the new call signature.

As long as ports are built with or without the ino64 change, they should
continue to work (and thus, -STABLE packages are not affected).

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Unable to upgrade from Samba43 to Samba44 or Samba45

[Stefan Esser]

Am 08.06.17 um 21:15 schrieb Dimitry Andric:
> On 8 Jun 2017, at 20:35, Mark Knight  wrote:
>>
>> On 08/06/2017 18:35, Dimitry Andric wrote:
>>> I'm guessing that it is confused by an existing samba installation.  Try
>>> removing all other samba installations before attempting to build this
>>> port.
>>
>> Having followed an off-list suggestion to create a package (thanks) as a 
>> backup, I removed samba43 and established that your "guess" was right. Thank 
>> you!
>>
>> Presumably this implies a bug in the port?
> 
> Not really, more a deficiency in portmaster.  The port itself is marked
> as conflicting with any other samba port:
> 
> CONFLICTS?= *samba3[2-6]-3.* samba4-4.0.* samba4[1-356]-4.* 
> p5-Parse-Pidl-4.*
> 
> E.g., you should normally not be able to build it when samba43 is
> installed.  Probably, portmaster is overriding this safety belt somehow.

I have always understood CONFLICTS to indicate, that some port
can not be *installed* at the same time as some of the ports listed
as conflicting.

Building such a port while a conflicting version is installed, is
possible most of the time. If such a build fails, it is most ofter
due to the build preferring the already installed headers over those
provided with the new version (i.e. a wrong ordering of include
paths).

Since port developers typically test ports in poudriere (to get a
clean build environment and e,g, to be sure, all dependencies are
correctly specified), they often don't notice such conflicts with
already installed versions of the same port.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"