Re: [PATCH] portgen(1) man page: Add py type
On Mon, May 13, 2019 at 02:51:54PM -0700, Andrew Hewus Fresh wrote:
> On Mon, May 13, 2019 at 10:05:16AM -0700, Andrew Hewus Fresh wrote:
> > On Mon, May 13, 2019 at 07:37:36AM +0100, Stuart Henderson wrote:
> > > As far as plists go, unless the port only supports py2 or only supports
> > > py3,
> > > it ought to generate the plist with FLAVOR=python3 and then prefix any
> > > lines
> > > ending in ${MODPY_PYCACHE}/ with ${MODPY_COMMENT}.
> >
> > I think kmos@ explained this to me correctly, but if someone wants to
> > write down the why this needs to happen again I'll happily add it as a
> > comment to this magic.
>
> So this pointed out an issue to me due to this patch. As I understood
> it, I could attach "${MODPY_FLAVOR}" indiscriminately to all the
> depends, but this error tells me that's probably not true. Do I need to
> only add the flavor if the dependent has a python3 flavor?
>
> ===> Updating plist for py3-pytest-4.5.0
> Fatal: Unknown flavor(s) python3 (in devel/py-funcsigs)
>(No flavors for this port). (in devel/py-funcsigs)
> *** Error 1 in /usr/ports/devel/py-funcsigs
> (/usr/ports/infrastructure/mk/bsd.port.mk:3564 '.BEGIN': @exit 1)
> Problem with dependency STEM->=1.0:devel/py-funcsigs,python3
This was a bit annoying, but kmos@ explained that he really meant that
we need to tack on the MODPY_FLAVOR only if the dependency *has* a
python3 flavor. This sounds a bit error-prone if a port suddenly
sprouts a python3 flavor, but that's unrelated to portgen.
Anyway, this patch checks the flavors of the dependency before adding
MODPY_FLAVOR, which is how I understood what kurt said.
Comments, OK?
Index: infrastructure/lib/OpenBSD/PortGen/Port/PyPI.pm
===
RCS file: /cvs/ports/infrastructure/lib/OpenBSD/PortGen/Port/PyPI.pm,v
retrieving revision 1.12
diff -u -p -r1.12 PyPI.pm
--- infrastructure/lib/OpenBSD/PortGen/Port/PyPI.pm 12 May 2019 20:23:33
- 1.12
+++ infrastructure/lib/OpenBSD/PortGen/Port/PyPI.pm 13 May 2019 22:48:50
-
@@ -21,8 +21,10 @@ use warnings;
use parent 'OpenBSD::PortGen::Port';
+use Cwd;
+
use OpenBSD::PortGen::Dependency;
-use OpenBSD::PortGen::Utils qw( module_in_ports );
+use OpenBSD::PortGen::Utils qw( base_dir ports_dir module_in_ports );
sub ecosystem_prefix
{
@@ -155,11 +157,30 @@ sub get_deps
next if @plat and join( " ", @plat ) !~ /OpenBSD/i;
- my $port = module_in_ports( $name, 'py-' )
- || $self->name_new_port($name);
+ my $port = module_in_ports( $name, 'py-' );
+ my $dep_dir = ports_dir() . '/' . $port;
+
+ # don't have it in tree, port it
+ unless ( $port ) {
+ $port = $self->name_new_port($name);
+ $dep_dir = base_dir() . '/' . $port;
+
+ my $o = OpenBSD::PortGen::Port::PyPI->new();
+ $o->port($name);
+ $self->add_notice( $o->notices );
+ }
my $base_port = $port;
- $port .= '${MODPY_FLAVOR}';
+
+ {
+ my $old_cwd = getcwd();
+ chdir $dep_dir || die "Unable to chdir $dep_dir: $!";
+ my $flavors = $self->make_show('FLAVORS');
+ chdir $old_cwd || die "Unable to chdir $old_cwd: $!";
+
+ # Attach the flavor if the dependency has one
+ $port .= '${MODPY_FLAVOR}' if $flavors =~ /\bpython3\b/;
+ }
if ( $phase eq 'build' ) {
$deps->add_build( $port, $req );
@@ -175,13 +196,6 @@ sub get_deps
"Added $base_port as 'run' dep, wanted
'$phase'")
unless $phase eq 'run';
$deps->add_run( $port, $req );
- }
-
- # don't have it in tree, port it
- if ( $port =~ m{^pypi/} ) {
- my $o = OpenBSD::PortGen::Port::PyPI->new();
- $o->port($name);
- $self->add_notice( $o->notices );
}
}
Re: [PATCH] portgen(1) man page: Add py type
On Mon, May 13, 2019 at 10:05:16AM -0700, Andrew Hewus Fresh wrote:
> On Mon, May 13, 2019 at 07:37:36AM +0100, Stuart Henderson wrote:
> > As far as plists go, unless the port only supports py2 or only supports py3,
> > it ought to generate the plist with FLAVOR=python3 and then prefix any lines
> > ending in ${MODPY_PYCACHE}/ with ${MODPY_COMMENT}.
>
> I think kmos@ explained this to me correctly, but if someone wants to
> write down the why this needs to happen again I'll happily add it as a
> comment to this magic.
So this pointed out an issue to me due to this patch. As I understood
it, I could attach "${MODPY_FLAVOR}" indiscriminately to all the
depends, but this error tells me that's probably not true. Do I need to
only add the flavor if the dependent has a python3 flavor?
===> Updating plist for py3-pytest-4.5.0
Fatal: Unknown flavor(s) python3 (in devel/py-funcsigs)
(No flavors for this port). (in devel/py-funcsigs)
*** Error 1 in /usr/ports/devel/py-funcsigs
(/usr/ports/infrastructure/mk/bsd.port.mk:3564 '.BEGIN': @exit 1)
Problem with dependency STEM->=1.0:devel/py-funcsigs,python3
Re: [PATCH] portgen(1) man page: Add py type
On Mon, May 13, 2019 at 07:37:36AM +0100, Stuart Henderson wrote:
> As far as plists go, unless the port only supports py2 or only supports py3,
> it ought to generate the plist with FLAVOR=python3 and then prefix any lines
> ending in ${MODPY_PYCACHE}/ with ${MODPY_COMMENT}.
I think kmos@ explained this to me correctly, but if someone wants to
write down the why this needs to happen again I'll happily add it as a
comment to this magic.
Comments, OK?
Index: infrastructure/lib/OpenBSD/PortGen/Port/PyPI.pm
===
RCS file: /cvs/ports/infrastructure/lib/OpenBSD/PortGen/Port/PyPI.pm,v
retrieving revision 1.12
diff -u -p -r1.12 PyPI.pm
--- infrastructure/lib/OpenBSD/PortGen/Port/PyPI.pm 12 May 2019 20:23:33
- 1.12
+++ infrastructure/lib/OpenBSD/PortGen/Port/PyPI.pm 13 May 2019 17:03:48
-
@@ -48,6 +48,38 @@ sub get_ver_info
return 1;
}
+sub make_plist
+{
+ my ($self, @args) = @_;
+
+ my $flavored3 = $self->{FLAVORS} && $self->{FLAVORS} =~ /\bpython3\b/;
+ local $ENV{FLAVOR} = 'python3' if $flavored3;
+
+ my $ret = $self->SUPER::make_plist(@args);
+
+ if ( $flavored3 && -e 'pkg/PLIST' ) {
+ open my $ifh, '<', 'pkg/PLIST'
+ or die "Unable to open PLIST: $!";
+ open my $ofh, '>', 'pkg/PLIST.new'
+ or die "Unable to open PLIST.new: $!";
+
+ while ( my $line = readline $ifh ) {
+ $line = '${MODPY_COMMENT}' . $line
+ if $line =~ m<\$\{MODPY_PYCACHE\}/?$>
+ and $line !~ m<^\$\{MODPY_COMMENT\}>;
+ print $ofh $line;
+ }
+
+ close $ifh;
+ close $ofh;
+
+ rename 'pkg/PLIST.new', 'pkg/PLIST'
+ or die "Unable to rename new PLIST: $!";
+ }
+
+ return $ret;
+}
+
sub name_new_port
{
my ( $self, $di ) = @_;
Re: [PATCH] portgen(1) man page: Add py type
On 2019/05/13 09:18, Kurt Mosiejczuk wrote:
> On Mon, May 13, 2019 at 07:37:36AM +0100, Stuart Henderson wrote:
>
> > As far as plists go, unless the port only supports py2 or only
> > supports py3, it ought to generate the plist with FLAVOR=python3
> > and then prefix any lines ending in ${MODPY_PYCACHE}/ with
> > ${MODPY_COMMENT}.
>
> Yes. I was describing that to afresh1 at one point. Someone else brought
> up setting REVISION while doing so. I had seen that danj's aliases for
> regenerating the plist set REVISION to 999, although I had a problem
> related to have done that at one point. Is setting the REVISION while
> doing so still necessary? If it is, what does it do? (It'd probably be
> nice to know the reasoning before making a tool incorporate it :) ).
>
> --Kurt
portgen should not set this.
Python ports usually include an egg-info file which includes the module
name + version, normally written as name-${MODPY_EGG_VERSION}. But in the
case where this is equal to FULLPKGNAME update-plist prefers that longer
match. This is most common in tools that happen to be written in python
(no py- prefix in the package name) - things like ansible, beets,
radiotray etc.
It initially looks like things work ok with FULLPKGNAME there, but then
later somebody makes a change and bumps REVISION, and packaging gets
broken.
The trick of forcing a bogus REVISION prevents the string in the egg-info
filename from matching FULLPKGNAME.
The proper way to fix that problem is remove FULLPKGNAME from the
default SUBST_VARS. This used to be needed for the filenames for
pkg-readmes, but these have been changed to use PKGSTEM instead of
FULLPKGNAME, so pkg-readmes isn't a problem for doing this any more.
However since then, lua ports (ports/*/lua* and ports/devel/lpeg)
started using FULLPKGNAME as a de-conflict mechanism for examples and
docs directories, so they would need changing.
Re: [PATCH] portgen(1) man page: Add py type
On Mon, May 13, 2019 at 07:37:36AM +0100, Stuart Henderson wrote:
> As far as plists go, unless the port only supports py2 or only
> supports py3, it ought to generate the plist with FLAVOR=python3
> and then prefix any lines ending in ${MODPY_PYCACHE}/ with
> ${MODPY_COMMENT}.
Yes. I was describing that to afresh1 at one point. Someone else brought
up setting REVISION while doing so. I had seen that danj's aliases for
regenerating the plist set REVISION to 999, although I had a problem
related to have done that at one point. Is setting the REVISION while
doing so still necessary? If it is, what does it do? (It'd probably be
nice to know the reasoning before making a tool incorporate it :) ).
--Kurt
Re: [PATCH] portgen(1) man page: Add py type
As far as plists go, unless the port only supports py2 or only supports
py3, it ought to generate the plist with FLAVOR=python3 and then prefix any
lines ending in ${MODPY_PYCACHE}/ with ${MODPY_COMMENT}.
--
Sent from a phone, apologies for poor formatting.
On 12 May 2019 21:30:34 Andrew Hewus Fresh wrote:
Kurt, you're welcome to commit this whenever you think it's good enough.
I say, with the improvements this week, OK afresh1@
I'm not quite sure what "plist smarts" it needs, but let me know if
there is something I'm missing.
On Sun, Feb 03, 2019 at 11:44:41PM +, Stuart Henderson wrote:
I asked tsg about this before, IIRC it was intentionally omitted as it
wasn't considered finished. (For starters it could do with some plist
smarts).
On Sun, Feb 03, 2019 at 09:22:31PM +, Linda Lapinlampi wrote:
/usr/ports/infrastructure/bin/portgen supports "py" type, as seen in the
source code:
if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::CPAN->new();
} elsif ( $type eq 'py' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
} else {
die "unknown module type\n";
}
This "py" type also works to generate ports from PyPI sources, mostly.
It's not documented in the man page though.
Attached diff adds a mention of the type to portgen(1) man page.
Index: portgen.1
===
RCS file: /cvs/src/share/man/man1/portgen.1,v
retrieving revision 1.1
diff -u -p -u -p -r1.1 portgen.1
--- portgen.1 26 Jun 2018 05:38:49 - 1.1
+++ portgen.1 3 Feb 2019 21:15:33 -
@@ -50,6 +50,8 @@ values:
.Bl -inset -offset indent -compact
.It Cm p5
for Perl modules on CPAN.
+.It Cm py
+for Python modules on PyPI.
.It Cm ruby
for Ruby gems.
.El
Re: [PATCH] portgen(1) man page: Add py type
Kurt, you're welcome to commit this whenever you think it's good enough.
I say, with the improvements this week, OK afresh1@
I'm not quite sure what "plist smarts" it needs, but let me know if
there is something I'm missing.
On Sun, Feb 03, 2019 at 11:44:41PM +, Stuart Henderson wrote:
> I asked tsg about this before, IIRC it was intentionally omitted as it
> wasn't considered finished. (For starters it could do with some plist
> smarts).
On Sun, Feb 03, 2019 at 09:22:31PM +, Linda Lapinlampi wrote:
> /usr/ports/infrastructure/bin/portgen supports "py" type, as seen in the
> source code:
>
> if ( $type eq 'p5' ) {
> $o = OpenBSD::PortGen::Port::CPAN->new();
> } elsif ( $type eq 'py' ) {
> $o = OpenBSD::PortGen::Port::PyPI->new();
> } elsif ( $type eq 'ruby' ) {
> $o = OpenBSD::PortGen::Port::Ruby->new();
> } else {
> die "unknown module type\n";
> }
>
> This "py" type also works to generate ports from PyPI sources, mostly.
> It's not documented in the man page though.
>
> Attached diff adds a mention of the type to portgen(1) man page.
> Index: portgen.1
> ===
> RCS file: /cvs/src/share/man/man1/portgen.1,v
> retrieving revision 1.1
> diff -u -p -u -p -r1.1 portgen.1
> --- portgen.1 26 Jun 2018 05:38:49 - 1.1
> +++ portgen.1 3 Feb 2019 21:15:33 -
> @@ -50,6 +50,8 @@ values:
> .Bl -inset -offset indent -compact
> .It Cm p5
> for Perl modules on CPAN.
> +.It Cm py
> +for Python modules on PyPI.
> .It Cm ruby
> for Ruby gems.
> .El
Re: [PATCH] portgen(1) man page: Add py type
I asked tsg about this before, IIRC it was intentionally omitted as it
wasn't considered finished. (For starters it could do with some plist smarts).
--
Sent from a phone, apologies for poor formatting.
On 3 February 2019 21:22:58 Linda Lapinlampi wrote:
/usr/ports/infrastructure/bin/portgen supports "py" type, as seen in the
source code:
if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::CPAN->new();
} elsif ( $type eq 'py' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
} else {
die "unknown module type\n";
}
This "py" type also works to generate ports from PyPI sources, mostly.
It's not documented in the man page though.
Attached diff adds a mention of the type to portgen(1) man page.
[PATCH] portgen(1) man page: Add py type
/usr/ports/infrastructure/bin/portgen supports "py" type, as seen in the
source code:
if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::CPAN->new();
} elsif ( $type eq 'py' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
} else {
die "unknown module type\n";
}
This "py" type also works to generate ports from PyPI sources, mostly.
It's not documented in the man page though.
Attached diff adds a mention of the type to portgen(1) man page.
Index: portgen.1
===
RCS file: /cvs/src/share/man/man1/portgen.1,v
retrieving revision 1.1
diff -u -p -u -p -r1.1 portgen.1
--- portgen.1 26 Jun 2018 05:38:49 - 1.1
+++ portgen.1 3 Feb 2019 21:15:33 -
@@ -50,6 +50,8 @@ values:
.Bl -inset -offset indent -compact
.It Cm p5
for Perl modules on CPAN.
+.It Cm py
+for Python modules on PyPI.
.It Cm ruby
for Ruby gems.
.El
