Re: www/py-html5lib with FLAVOR=py27 failed to build

2020-07-27 Thread Stefan Eßer
Am 27.07.20 um 07:03 schrieb Yasuhito FUTATSUKI:
> In article <20200727.112301.1619120197420987885.y...@utahime.org>
> y...@utahime.org writes:
> 
>> From: KIRIYAMA Kazuhiko 
>> Subject: www/py-html5lib with FLAVOR=py27 failed to build
>> Date: Sat, 25 Jul 2020 15:17:04 +0900
>>
>>> www/py-html5lib with FLAVOR=py27 had failed to build:
>>
>> I tried `cd /usr/ports/www/py-html5lib; make FLAVOR=py27 install` with
>> following conditions,
>>
>> OS: 11.4-RELEASE, 12.1-RELEASE and 13-CURRENT r363475 (amd64)
>> Ports tree: head r543492
>>
>> And in all cases it compeletes without any error.
>>
>> Do you have any non-default setting about options or something related
>> to ports build?
> 
> www/py-html5lib@py27 run depends on devel/py-six@py27.
> devel/py-six@py27 test depends on devel/py-pytest@py27.
> devel/py-pytest@py27 test depends on devel/py-hypothesis@py27.
> devel/py-hypothesis dropped py27 support on r538898.
> 
> So it can't be built with test. I guess it is the reason.

I consider it quite an annoyance that ports depending on Python 2.7
are deleted before the EoL of the interpreter has actually occurred.

I'm currently trying to resurrect a port (textproc/scancode-toolkit)
which for quite some time has already been available in an upgraded
version for Python 3 on Github, but was deleted in our ports tree.

It depends on other ports that work with Python 3 if only the USES
clause in the ports Makefile is fixed to include 3.6+ (I do not have
older Python versions installed and they might work with 3.0+, but I
cannot easily test that assumption). Some of these dependencies have
also been deleted from ports, despite being ready for Python 3.

IMHO ports that are currently marked to require Python 2.7 should be
updated to a version that works with 3.x (and many will do without
any change to the port except removing the restriction in the port's
Makefile, as I have found when working on scancode-toolbox).

Removal of Python-32.7 specific ports that are depended on by other
ports instead of just updating them or their port Makefile causes
friction and work for maintainers of dependent ports, who may not
have much experience with those dependencies (e.g. because they are
written in Python whole the port maintainer is not well versed in
that language and especially not in the steps required to migrate
a port from 2.7 to 3.x or to test whether it has been fully migrated
by the upstream).
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: WITH_META_MODE and poudriere: *.meta files, where?

2020-06-23 Thread Stefan Eßer
Am 22.06.20 um 23:02 schrieb Torfinn Ingolfsen:
> On Mon, Jun 22, 2020 at 8:56 PM Michael Grimm  wrote:
>>
>> Hi,
>>
>> I do have activated 'WITH_META_MODE=yes' in > to>/poudriere.d/src-env.conf [1].
>>
>> Now, I would like to double-check if poudriere is following my "advice". For 
>> buildworld I do find *.meta files in /usr/obj/…, but I do not have any clue 
>> where poudriere would keep this information for subsequent compilations?
>>
> 
> The is some information here
> https://xmj.github.io/articles/sysadmin/builds_ccache_memcached.html
> and link to a presentation with more about (among otter things)  metamode.

Is this article noch valid with regards to WITH_DIRDEPS_BUILD?

I tried to enable it in /etc/src-env.conf with the result
that only the destroy targets were offered:

# make buildworld
ERROR: 'buildworld' is not a valid target for amd64.

You can see the targets which are valid for a given machine
by running 'env MACHINE= make show-valid-targets'
For other information, read:

# make show-valid-targets
Other targets:
  destroy
  destroy-arch
  destroy-host
  destroy-stage

The same behavior is observed with /usr/obj from a previous
buildworld with only WITH_META_MODE=yes or with a clean obj
directory ...

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: might need to bump version of python ports after recent openssl changes

2020-05-27 Thread Stefan Eßer
Am 27.05.20 um 04:24 schrieb Jan Beich:
> Pete Wright  writes:
> 
>> hello - on current i found myself in a situation where python37 was
>> unable to import ssl:
>>
>> $ python3.7
>> Python 3.7.7 (default, May  9 2020, 01:37:42)
>> [Clang 10.0.0 (g...@github.com:llvm/llvm-project.git
>> llvmorg-10.0.0-0-gd32170dbd on freebsd13
>> Type "help", "copyright", "credits" or "license" for more information.
> import ssl
>> Traceback (most recent call last):
>>   File "", line 1, in 
>>   File "/usr/local/lib/python3.7/ssl.py", line 98, in 
>>     import _ssl # if we can't import it, let the error
>> propagate
>> ImportError: /usr/local/lib/python3.7/lib-dynload/_ssl.so: Undefined
>> symbol "SSLv3_method@OPENSSL_1_1_0"
>
>>
>>
>> after a little digging it looks like we recently disabled SSLv3 on
>> CURRENT (huzzah!):
>> https://reviews.freebsd.org/D24945
>>
>> After forcing a re-install of python37 things are working again as it
>> looked like the pbuilder did rebuild python after this commit. But pkg 
>> upgrade didn't detect a new version, so I think it might be helpful to
>> bump the python version's so that people on CURRENT don't end up in
>> the same situation I was in?  Not sure what the usual process is for
>> stuff like this...
> 
> OSVERSION was already bumped in base r361410, so poudriere will
> force-rebuild all packages. Those who hack .jailversion to avoid
> rebuilds can only blame themselves.

OSVERSION bumps will be observed by poudriere, but not by other port
building tools.

Did I miss an announcement that all other methods to keep your system
in a workable state are now considered obsolete and unsupported?

A port version bump would have enabled rebuilding just the affected
ports, while a rebuild of all my ports based on OSVERSION will take
days to complete on my local build server.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: Are broken ports retested before performing scheduled removal?

2020-05-16 Thread Stefan Eßer
Am 16.05.20 um 10:44 schrieb Stefan Ehmann:
> On Thursday, May 14, 2020 1:24:17 AM CEST Edward Sanford Sutton, III wrote:
>>   games/oneko was recently removed as it was marked broken due to being
>> unfetchable. I tried an old copy of the ports tree I had where it was
>> marked as such and `make fetch -DTRYBROKEN` succeeded. Was there an
>> error recorded for this and was it ever retested? Is this just a side
>> effect of having a failure and no maintainer?
>>   If being removed as unfetchable, wouldn't a better MOVED entry be to
>> say it was unfetchable (for more than 6 months) instead of just broken
>> for more than 6 months?
>> Thanks for feedback
> 
> I've recently asked myself the same question:
> 
> multimedia/gpac-mp4box is flagged
> BROKEN=unfetchable
> DEPRECATED=Broken for more than 6 months
> 
> multimedia/gpac-libgpac uses the same distfile from the same mirror and is not
> marked broken.
> 
> I figured once a port is marked unfetchable, it will be deleted, even it is a
> transient failure. Of course, unless someone opens a bug report.

I have also had to resurrect ports that had been deleted as unfetchable
(and which were prerequisites of ports I was working on) and found that
they were either available from the MASTER_SITE given in the port (i.e.,
have most probably been only temporarily unavailablet) or had been moved
to Github (or some other repository) and now have better accessibility
than before, after an update of the ports' master sites.

IMHO, it shows disrespect to the creator or maintainer of a port, if it
is marked broken and deleted, in such a case, especially since there is
no guarantee that the maintainer receives a notification for ports
marked broken (for any reason, often easily fixed, but noted only when
the port is deleted if the maintainer has no need to update or rebuild
it during the time of deprecation).

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: qt5-webengine

2020-04-05 Thread Stefan Eßer
Am 05.04.20 um 02:50 schrieb John Kennedy:>   The thing that drove me
away from portmaster to synth and eventually to
> poudriere is incompatible dependencies.  I was running into those with just
> X11 dependencies (~600 packages in my full port rebuild, so not sure how you
> got lucky over that period of time).  Now, people keep on fixing portmaster
> and fixing dependencies, but at times I would have just been SOL for an
> indeterminate period of time.

I have for more than 1 year been using a rewrite of portmaster that
uses a completely different approach than the official version.

But due to "features" of the ports system, that only ever considered
whether they might break poudriere (and thus the package build cluster)
it is extremely complex to cover all special cases.

That version has been tested by a few people that asked, but I have
meanwhile given up on trying to get it working perfectly for all cases.
The logic is just too complex to express in Bourne Shell or Bash.

Main features beyond a better strategy logic is that this portmaster
can be used to build in a clean environment (idea stolen from synth)
and it can also be used to maintain a repository to upgrade from.

I have started porting that version to LUA a few months ago and have
been able to update ports with it (with the syntax of the shell version
just translated to LUA).

Since LUA offers quite advanced features for object oriented or function
programming styles, I have then re-implemented layer for layer in a
style that uses LUA features. This has also allowed a significant
speed-up: Checking my ~2400 ports for missing updates (and I have kept
the development system without updates to have a good test coverage)
results in 1373 actions (upgrades, new installations) and it takes 50s
to get all ports checked and another 10 seconds to identify collisions
of to-be-installed dependencies with existing ports. (And with 4 cores
this could be reduced to 20 seconds for this large set of ports, but
I'm currently trying to get the functionality complete, speed comes as
a secondary goal.)

>   I also got in the habit of rebuilding and reinstalling everything about
> once a month because of weird (dependency) breakages that portmaster (at
> least at the time) couldn't figure out and recompile itself.

I'm tracking not only version changes, but also whether a port relies
on an outdated library. It is possible to update all ports that have
an OSVERSION from before a kernel change that requires the port to be
re-installed (most relevant for -CURRENT).

I'm working on this in spare time (too little, currently) and the ports
system is not friendly to any tool besides poudriere. The implementation
of flavors is particular bad to deal with, if you do not just maintain
a package repository, but want to upgrade a system with installed ports
to the latest versions (with your options intact).

>   I'm really impatient, and have a compulsion to security-patch things, so
> thus I was finally driven to change (after I don't know how many years).
> 
>   Synth and poudriere avoided it because it was a build dependency, not a
> run-time dependency, and their build environments kept that very clean
> (which portmaster couldn't do, at least at the time).  It also let me have
> less packages loaded on the machine overall, so less surface area for attacks.
> Yeah, it recompiles a BUNCH of things that often don't get upgraded, but I've
> never felt the need to recompile everything in case something got missed.

It is already possible to install build dependencies from packages saved
from a prior portmaster session and delete build tools not required as
run dependencies at the end of the portmaster run. This does work with
the version in ports. I have added the ability to build in a clean jail
with build tools only ever available within the jail during the session.

>   I also find that port problems that break poudriere builds get caught
> quickly (vs more-rare synth problems, and way faster than portmaster), so
> I get to reap the advantages of what FreeBSD is building with.

Yes, the problem is that only problems that affect poudriere are caught
by the build cluster and even port system infrastructure changes may
impact tools like portmaster (or portupgrade) in such a way that they
completely break.

That has been the case with the (IMHO) badly designed flavor support,
which made portmaster unusable until I took over maintainership and
added flavor support (working most of the time, but not perfect).
There have been other changes, like renaming of ports and packages at
the same time when KDE4 was removed from ports: This makes it impossible
for a tool like portmaster to correctly identify the port directory to
use in order to upgrade some installed package - at least one of port
origin and package name are needed to perform a match (MOVED could have
helped, but it was not used when KDE4 was deleted for reasons that I can
understand).

Anyway, I'm working on the LUA 

Re: pkg: how to retrieve the messages ports emit after install?

2020-03-18 Thread Stefan Eßer
Am 18.03.20 um 15:18 schrieb Chris:
> On Wed, 18 Mar 2020 08:27:29 + Bob Eager r...@tavi.co.uk said
> 
>> On Tue, 17 Mar 2020 23:28:49 -0700
>> Chris  wrote:
>>
>> > OK I'm a by ports make install as a rule. But I needed to
>> > spin up a box quickly, and decided to use pkg(8). xorg,
>> > and another port (package) I installed, dumped some
>> > important information after the install. I stripped the
>> > text from the console/terminal, and tried to paste it into
>> > a fresh file. But the new graphics drivers don't allow
>> > that sort of thing (graphics vs text mode). So I was left
>> > with mostly gibberish. I need to get that information back.
>> > I just guessed that pkg message  might do it.
>> > But no joy. How can I retrieve that information?
>>
>> Simple!
>>
>>  pkg info -D pkgname
>>
>> (or pkg info --pkg-message pkgname)
> --pkg-message, argh... I was *so* close!
> Thanks, Bob. Really appreciated!
> and for the record. I *did* read the man page. But as others
> were also inclined to suggest; I thought the output would be
> related to *query*

I do actually prefer the version I suggested based on
pkg query, since I find it quite hard to see message
boundaries in the pkg info output.

With pkg query you can introduce any kind of separators
(even multi-line) in the format string (and also print
the origin and other information along with the message).

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: pkg: how to retrieve the messages ports emit after install?

2020-03-18 Thread Stefan Eßer
Am 18.03.20 um 07:28 schrieb Chris:
> OK I'm a by ports make install as a rule. But I needed to
> spin up a box quickly, and decided to use pkg(8). xorg,
> and another port (package) I installed, dumped some
> important information after the install. I stripped the
> text from the console/terminal, and tried to paste it into
> a fresh file. But the new graphics drivers don't allow
> that sort of thing (graphics vs text mode). So I was left
> with mostly gibberish. I need to get that information back.
> I just guessed that pkg message  might do it.
> But no joy. How can I retrieve that information?

You may be looking for some variant of:

# pkg query -- '--- %n-%v ---\n%M'

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: svn commit: r358166 - head

2020-02-22 Thread Stefan Eßer
Am 22.02.20 um 03:50 schrieb Mark Millard via freebsd-ports:
> 
> 
> On 2020-Feb-21, at 15:59, Kevin Oberman  wrote:
> 
>> On Fri, Feb 21, 2020 at 8:38 AM Mark Millard via freebsd-ports 
>>  wrote:
>> Based on the example from https://www.freebsd.org/cgi/man.cgi?ldd
>> there are commands such as the following that might help:
>>
>> . . .
>>
>> This can be a lot of files to go through (e.g., lib*) and so
>> can take a fair amount of time.
>>
>> ===
>> Mark Millard
>> marklmi at yahoo.com
>> ( dsl-only.net went
>> away in early 2018-Mar)
>>  
>> You can feed that list into 'pkg which', awk that to remove all of the 
>> commands and then all but the last space delimited string of the remainder, 
>> and uniq. I also sorted.
>> bison-3.5.2,1
>> gdbm-1.18.1_1
>> gettext-tools-0.20.1_1
>> gnuplot-5.2.8
>> llvm80-8.0.1_3
>> lua52-5.2.4
>> sqlite3-3.30.1,1
>> xterm-353_1
> 
> Cool.
> 
> In the style of my prior examples (including the change that
> found libedit and such), analogous would be:
> 
> # find /usr/local/*bin* /usr/local/lib* -type f \
> | xargs -n1 file -F ' ' | grep 'ELF' | cut -f1 -d' ' \
> | xargs ldd -f '%A %p\n' 2>&1 | grep ncurses\.so\.8 | cut -f1 -d' ' \
> | xargs pkg which | cut -f6 -d' ' | sort -u | more
> bison-3.5.2,1
> gdbm-1.18.1_1
> gettext-tools-0.20.1_1
> gnuplot-5.2.8
> libedit-3.1.20191211,1
> libtextstyle-0.20.1
> llvm10-10.0.0.r2
> llvm80-8.0.1_3
> lua52-5.2.4
> mesa-dri-18.3.2_9
> ruby-2.6.5,1
> sqlite3-3.30.1,1
> xterm-353_1
> 
> Looks like I'll be reasonably ready when I get to the point
> of wanting to deal with this.

While technically correct, the extra test for the ELF format of each
single file is not required for correct results, AFAICT.

The ldd command will just emit an error message for files it cannot
process, and the error output is trivially suppressed.


The following pipeline should only need a fraction of the time required
by the one suggested above:

# find /usr/local/*bin* /usr/local/lib* -type f \
| xargs ldd -f '%o %A\n' 2>/dev/null \
| grep ^libncurses\.so\.8 | cut -w -f2 \
| xargs pkg which -q | sort -u | more

It completes in 13 seconds on my system, and I plan to add it to
portmaster as another option to select the ports to be updated.

The test of each file for ELF format requires one invocation of the
file command for each, which leads to 217 seconds real time (178 user,
39 system) for me - so my version is faster by a factor of at least 15.

Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: mail/junkfilter is several broken

2020-01-02 Thread Stefan Eßer
Am 01.01.20 um 22:04 schrieb Steve Kargl:
> For users of mail/junkfilter, it now will filter all emails claiming
> a "Bad Date line".  The following patch seems to fix the problem for
> the next decade.

Hi Steve,

thank you for providing a patch. Since the maintainer (gsutter) has
not been active in FreeBSD for a long time (AFAICT) and due to the
difference in time zones, I have taken liberty to apply the fix to
the port.

I have sent mail to Gregory who probably will want to apply the fix
to the sourceforge repo and to remove the patch, but the fixed port
will allow to keep junkfilter working, meanwhile.

Regards, STefan

> --- junkfilter.three.orig 2020-01-01 12:59:56.005681000 -0800
> +++ junkfilter.three  2020-01-01 13:00:26.254199000 -0800
> @@ -56,7 +56,7 @@
>  * ! $ ^Date:$JFWS((Sun|Mon|Tue|Wed|Thu|Fri|Sat),$JFWS)?\
>  (0?[1-9]|[12][0-9]|3[01])$JFWS\
>  (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)$JFWS\
> -((19)?[789][0-9]|(20)?[01][0-9])$JFWS\
> +((19)?[789][0-9]|(20)?[012][0-9])$JFWS\
>  (0?[0-9]|1[0-9]|2[0-3]):(0?|[1-5])[0-9](:(0?|[1-5])[0-9])?$JFWS\
>  
> (([+-][0-1][0-4]([03]0|45))|("?\(?(UT|GMT|EST|EDT|CST|CDT|MST|MDT|PST|PDT|[A-I]|[K-Z])\)?"?))?
>  { JFMATCH="$JFSEC: Bad Date line" INCLUDERC=$JFDIR/junkfilter.match }
> 
> 
> Suggest either installing the patch or marking the port as broken.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"