Re: Go and portgen(1)
> I can also generate a very large set of ports with it! If you're interested in examples of things that don't work with this, here are a couple I ran into. github.com/tomnomnom/gron github.com/gcla/termshark/v2
Re: Go and portgen(1)
On Sat, 18 Apr 2020 at 11:26:34 +0100, Stuart Henderson wrote:
> On 2020/04/18 10:00, Landry Breuil wrote:
> > can be polished in-tree if needed, as long as all existing go ports
>
> > still build fine with the go.port.mk changes i dont see harm in
>
> > committing it :)
>
> this is the key point and it needs testing before commit.
>
Here is a full diff of all the parts. This has gone through a proot'd dpb build
of all the Go ports with no failures.
I can also generate a very large set of ports with it!
diff --git a/infrastructure/bin/portgen b/infrastructure/bin/portgen
index ad5ab17f3cf..b7316d42b64 100755
--- a/infrastructure/bin/portgen
+++ b/infrastructure/bin/portgen
@@ -32,6 +32,7 @@ use lib ( "$portdir/infrastructure/lib",
"$FindBin::Bin/../lib" );
use OpenBSD::PortGen::Port::CPAN;
use OpenBSD::PortGen::Port::PyPI;
use OpenBSD::PortGen::Port::Ruby;
+use OpenBSD::PortGen::Port::Go;
my ( $type, $module ) = @ARGV;
@@ -44,6 +45,8 @@ if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
+} elsif ( $type eq 'go' ) {
+ $o = OpenBSD::PortGen::Port::Go->new();
} else {
die "unknown module type\n";
}
diff --git a/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
new file mode 100644
index 000..9c9a41edbd8
--- /dev/null
+++ b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
@@ -0,0 +1,284 @@
+# $OpenBSD: Go.pm,v 1.16 2019/05/16 16:01:10 afresh1 Exp $
+#
+# Copyright (c) 2019 Aaron Bieber
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+package OpenBSD::PortGen::Port::Go;
+
+use 5.028;
+use utf8;
+use warnings;
+use strict;
+use warnings qw(FATAL utf8);# fatalize encoding glitches
+use open qw(:std :encoding(UTF-8)); # undeclared streams in UTF-8
+use OpenBSD::PackageName;
+use OpenBSD::PortGen::Utils qw( fetch );
+
+use parent 'OpenBSD::PortGen::Port';
+
+use Carp;
+use Cwd;
+use File::Temp qw/ tempdir /;
+use Data::Dumper;
+
+use OpenBSD::PortGen::Dependency;
+
+my $license_map = {
+ 'BSD-2-Clause' => 'BSD',
+ 'BSD-3-Clause' => 'BSD3',
+};
+
+sub ecosystem_prefix
+{
+ my $self = shift;
+ return '';
+}
+
+sub base_url
+{
+ my $self = shift;
+ return 'https://proxy.golang.org/';
+}
+
+sub _go_lic_info
+{
+ my ( $self, $module ) = @_;
+ my $html = fetch("https://pkg.go.dev/mod/; . $module);
+ my $license = "unknown";
+ if ($html =~ m/(.+)<\/a>/) {
+ $license = $1;
+ $license = $license_map->{$license} || $license;
+ }
+ return $license;
+}
+
+sub _go_determine_name
+{
+ # Some modules end in "v1" or "v2", if we find one of these, we need
+ # to set PKGNAME to something up a level
+ my ( $self, $module ) = @_;
+ my $json = $self->get_json( $module . '/@latest' );
+
+ if ($json->{Version} =~ m/incompatible/) {
+ my $msg = "${module} $json->{Version} is incompatible with Go
modules.";
+ croak $msg;
+ }
+
+ if ($module =~ m/v\d$/) {
+ $json->{Name} = ( split '/', $module )[-2];
+ } else {
+ $json->{Name} = ( split '/', $module )[-1];
+ }
+
+ $json->{Module} = $module;
+
+ return $json;
+}
+
+sub get_dist_info
+{
+ my ( $self, $module ) = @_;
+
+ my $json = $self->_go_determine_name($module);
+
+ my %mods;
+ for ( $self->_go_mod_graph($json) ) {
+ my ($direct, $ephemeral) = @{$_};
+
+ for my $d ( $direct, $ephemeral ) {
+ next unless $d->{Version};
+ $mods{ $d->{Module} }{ $d->{Version} } ||= $d;
+ }
+ }
+
+ # Filter dependencies to the one with the highest version
+ foreach my $mod ( sort keys %mods ) {
+ # Sort semver numerically then the rest alphanumeric.
+ # This is overkill for sorting, but I already wrote it
+ my @versions =
+ map { $_->[0] }
+ sort {
+ $a->[1] <=> $b->[1]
+
Re: Go and portgen(1)
On Sat, Apr 18, 2020 at 10:00:45AM +0200, Landry Breuil wrote: > On Fri, Apr 17, 2020 at 05:26:58PM -0600, Aaron Bieber wrote: > > Round... 10? > Cant comment on the portgen perl bits (afresh1@ did :), but unless > someone has an opposition against it, i think it's a nice addition that > can be polished in-tree if needed, as long as all existing go ports > still build fine with the go.port.mk changes i dont see harm in > committing it :) Agreed, it's an improvement on what we have. OK afresh1@ l8rZ, -- andrew - http://afresh1.com Beta. Software undergoes beta testing shortly before it's released. Beta is Latin for "still doesn't work."
Re: Go and portgen(1)
On 2020/04/18 10:00, Landry Breuil wrote: > can be polished in-tree if needed, as long as all existing go ports > still build fine with the go.port.mk changes i dont see harm in > committing it :) this is the key point and it needs testing before commit.
Re: Go and portgen(1)
On Fri, Apr 17, 2020 at 05:26:58PM -0600, Aaron Bieber wrote: > On Wed, 15 Apr 2020 at 18:56:09 -0600, Aaron Bieber wrote: > > On Mon, 06 Apr 2020 at 17:15:17 -0600, Aaron Bieber wrote: > > > Round two! This time with espie@'s bsd.port.mk fix: > > > https://marc.info/?l=openbsd-ports-cvs=158618354824687=2 > > > > > > This resolves the issue with the conversion of [A-Z] letters to ![a-z] in > > > the > > > package path. > > > > > > With this diff I can generate ports for the following without issue: > > > > > > github.com/jrick/domain > > > github.com/jrick/ss > > > github.com/junegunn/fzf > > > github.com/qbit/gavin > > > golang.zx2c4.com/wireguard > > > humungus.tedunangst.com/r/honk > > > suah.dev/ogvt > > > > > > There are still some issues with things like github.com/restic/restic and > > > github.com/gohugoio/hugo . For some reason the build looks for some files > > > that > > > 'go mod graph' doesn't list. > > > > > > For most go apps that follow the module guidelines: > > > https://github.com/golang/go/wiki/Modules > > > > > > This method of building things should work pretty well. One will have to > > > go in > > > and add a "do-install" to grab other files like man pages, etc. > > > > > > > OK, this fixes some issues with determining the latest version of a given > > port. > > > > With this, I can build a port for restic! (it still needs > > 'ALL_TARGET=github.com/restic/restic/...' set manually) > > > > hugo still results in some missing mod/zip files. > > > > Round... 10? > > Changes: > - Handle determine latest version in a manner that is more inline with how Go >does it. > - Error out when we are asked to gen a port for an incompatible module. > - Detect licenses from pkg.go.dev. > - Update ALL_TARGET to include "/cmd/..." when a "cmd" directory is present in > a module. Cant comment on the portgen perl bits (afresh1@ did :), but unless someone has an opposition against it, i think it's a nice addition that can be polished in-tree if needed, as long as all existing go ports still build fine with the go.port.mk changes i dont see harm in committing it :)
Re: Go and portgen(1)
On Fri, Apr 17, 2020 at 05:26:58PM -0600, Aaron Bieber wrote:
> On Wed, 15 Apr 2020 at 18:56:09 -0600, Aaron Bieber wrote:
> > On Mon, 06 Apr 2020 at 17:15:17 -0600, Aaron Bieber wrote:
> > > Round two! This time with espie@'s bsd.port.mk fix:
> > > https://marc.info/?l=openbsd-ports-cvs=158618354824687=2
> > >
> > > This resolves the issue with the conversion of [A-Z] letters to ![a-z] in
> > > the
> > > package path.
> > >
> > > With this diff I can generate ports for the following without issue:
> > >
> > > github.com/jrick/domain
> > > github.com/jrick/ss
> > > github.com/junegunn/fzf
> > > github.com/qbit/gavin
> > > golang.zx2c4.com/wireguard
> > > humungus.tedunangst.com/r/honk
> > > suah.dev/ogvt
> > >
> > > There are still some issues with things like github.com/restic/restic and
> > > github.com/gohugoio/hugo . For some reason the build looks for some files
> > > that
> > > 'go mod graph' doesn't list.
> > >
> > > For most go apps that follow the module guidelines:
> > > https://github.com/golang/go/wiki/Modules
> > >
> > > This method of building things should work pretty well. One will have to
> > > go in
> > > and add a "do-install" to grab other files like man pages, etc.
> > >
> >
> > OK, this fixes some issues with determining the latest version of a given
> > port.
> >
> > With this, I can build a port for restic! (it still needs
> > 'ALL_TARGET=github.com/restic/restic/...' set manually)
> >
> > hugo still results in some missing mod/zip files.
> >
>
> Round... 10?
A few super tiny comments inline, only one I would fail the review for :-)
>
> Changes:
> - Handle determine latest version in a manner that is more inline with how Go
>does it.
> - Error out when we are asked to gen a port for an incompatible module.
> - Detect licenses from pkg.go.dev.
> - Update ALL_TARGET to include "/cmd/..." when a "cmd" directory is present in
> a module.
>
> diff --git a/infrastructure/bin/portgen b/infrastructure/bin/portgen
> index ad5ab17f3cf..b7316d42b64 100755
> --- a/infrastructure/bin/portgen
> +++ b/infrastructure/bin/portgen
> @@ -32,6 +32,7 @@ use lib ( "$portdir/infrastructure/lib",
> "$FindBin::Bin/../lib" );
> use OpenBSD::PortGen::Port::CPAN;
> use OpenBSD::PortGen::Port::PyPI;
> use OpenBSD::PortGen::Port::Ruby;
> +use OpenBSD::PortGen::Port::Go;
>
> my ( $type, $module ) = @ARGV;
>
> @@ -44,6 +45,8 @@ if ( $type eq 'p5' ) {
> $o = OpenBSD::PortGen::Port::PyPI->new();
> } elsif ( $type eq 'ruby' ) {
> $o = OpenBSD::PortGen::Port::Ruby->new();
> +} elsif ( $type eq 'go' ) {
> + $o = OpenBSD::PortGen::Port::Go->new();
> } else {
> die "unknown module type\n";
> }
> diff --git a/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
> b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
> new file mode 100644
> index 000..34668ea6fa7
> --- /dev/null
> +++ b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
> @@ -0,0 +1,288 @@
> +# $OpenBSD: Go.pm,v 1.16 2019/05/16 16:01:10 afresh1 Exp $
> +#
> +# Copyright (c) 2019 Aaron Bieber
> +#
> +# Permission to use, copy, modify, and distribute this software for any
> +# purpose with or without fee is hereby granted, provided that the above
> +# copyright notice and this permission notice appear in all copies.
> +#
> +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
> +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
> +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
> +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
> +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
> +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
> +
> +package OpenBSD::PortGen::Port::Go;
> +
> +use 5.028;
> +use utf8;
> +use warnings;
> +use strict;
> +use warnings qw(FATAL utf8);# fatalize encoding glitches
> +use open qw(:std :encoding(UTF-8)); # undeclared streams in UTF-8
> +use OpenBSD::PackageName;
> +use OpenBSD::PortGen::Utils qw( fetch );
> +
> +use parent 'OpenBSD::PortGen::Port';
> +
> +use Carp;
> +use Cwd;
> +use File::Temp qw/ tempdir /;
> +use Data::Dumper;
> +
> +use OpenBSD::PortGen::Dependency;
> +
> +my $license_map = {
> + 'BSD-2-Clause' => 'BSD',
> + 'BSD-3-Clause' => 'BSD3',
> +};
This could just go into "%good_licenses" in OpenBSD::PortGen::License.
https://github.com/openbsd/ports/blob/master/infrastructure/lib/OpenBSD/PortGen/License.pm
> +
> +sub ecosystem_prefix
> +{
> + my $self = shift;
> + return '';
> +}
> +
> +sub base_url
> +{
> + my $self = shift;
> + return 'https://proxy.golang.org/';
> +}
> +
> +sub _go_lic_info
> +{
> + my ( $self, $module ) = @_;
> + my $html = fetch("https://pkg.go.dev/mod/; . $module);
> + my $license = "unknown";
> + if ($html =~ m/(.+)<\/a>/) {
> + $license = $1;
> +
Re: Go and portgen(1)
On Wed, 15 Apr 2020 at 18:56:09 -0600, Aaron Bieber wrote:
> On Mon, 06 Apr 2020 at 17:15:17 -0600, Aaron Bieber wrote:
> > Round two! This time with espie@'s bsd.port.mk fix:
> > https://marc.info/?l=openbsd-ports-cvs=158618354824687=2
> >
> > This resolves the issue with the conversion of [A-Z] letters to ![a-z] in
> > the
> > package path.
> >
> > With this diff I can generate ports for the following without issue:
> >
> > github.com/jrick/domain
> > github.com/jrick/ss
> > github.com/junegunn/fzf
> > github.com/qbit/gavin
> > golang.zx2c4.com/wireguard
> > humungus.tedunangst.com/r/honk
> > suah.dev/ogvt
> >
> > There are still some issues with things like github.com/restic/restic and
> > github.com/gohugoio/hugo . For some reason the build looks for some files
> > that
> > 'go mod graph' doesn't list.
> >
> > For most go apps that follow the module guidelines:
> > https://github.com/golang/go/wiki/Modules
> >
> > This method of building things should work pretty well. One will have to go
> > in
> > and add a "do-install" to grab other files like man pages, etc.
> >
>
> OK, this fixes some issues with determining the latest version of a given
> port.
>
> With this, I can build a port for restic! (it still needs
> 'ALL_TARGET=github.com/restic/restic/...' set manually)
>
> hugo still results in some missing mod/zip files.
>
Round... 10?
Changes:
- Handle determine latest version in a manner that is more inline with how Go
does it.
- Error out when we are asked to gen a port for an incompatible module.
- Detect licenses from pkg.go.dev.
- Update ALL_TARGET to include "/cmd/..." when a "cmd" directory is present in
a module.
diff --git a/infrastructure/bin/portgen b/infrastructure/bin/portgen
index ad5ab17f3cf..b7316d42b64 100755
--- a/infrastructure/bin/portgen
+++ b/infrastructure/bin/portgen
@@ -32,6 +32,7 @@ use lib ( "$portdir/infrastructure/lib",
"$FindBin::Bin/../lib" );
use OpenBSD::PortGen::Port::CPAN;
use OpenBSD::PortGen::Port::PyPI;
use OpenBSD::PortGen::Port::Ruby;
+use OpenBSD::PortGen::Port::Go;
my ( $type, $module ) = @ARGV;
@@ -44,6 +45,8 @@ if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
+} elsif ( $type eq 'go' ) {
+ $o = OpenBSD::PortGen::Port::Go->new();
} else {
die "unknown module type\n";
}
diff --git a/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
new file mode 100644
index 000..34668ea6fa7
--- /dev/null
+++ b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
@@ -0,0 +1,288 @@
+# $OpenBSD: Go.pm,v 1.16 2019/05/16 16:01:10 afresh1 Exp $
+#
+# Copyright (c) 2019 Aaron Bieber
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+package OpenBSD::PortGen::Port::Go;
+
+use 5.028;
+use utf8;
+use warnings;
+use strict;
+use warnings qw(FATAL utf8);# fatalize encoding glitches
+use open qw(:std :encoding(UTF-8)); # undeclared streams in UTF-8
+use OpenBSD::PackageName;
+use OpenBSD::PortGen::Utils qw( fetch );
+
+use parent 'OpenBSD::PortGen::Port';
+
+use Carp;
+use Cwd;
+use File::Temp qw/ tempdir /;
+use Data::Dumper;
+
+use OpenBSD::PortGen::Dependency;
+
+my $license_map = {
+ 'BSD-2-Clause' => 'BSD',
+ 'BSD-3-Clause' => 'BSD3',
+};
+
+sub ecosystem_prefix
+{
+ my $self = shift;
+ return '';
+}
+
+sub base_url
+{
+ my $self = shift;
+ return 'https://proxy.golang.org/';
+}
+
+sub _go_lic_info
+{
+ my ( $self, $module ) = @_;
+ my $html = fetch("https://pkg.go.dev/mod/; . $module);
+ my $license = "unknown";
+ if ($html =~ m/(.+)<\/a>/) {
+ $license = $1;
+ $license = $license_map->{$license} || $license;
+ }
+ return $license;
+}
+
+sub _go_determine_name
+{
+ # Some modules end in "v1" or "v2", if we find one of these, we need
+ # to set PKGNAME to something up a level
+ my ( $self, $module ) = @_;
+ my $json = {};
+
+ $json = $self->get_json( $module . '/@latest' );
+
+ if ($json->{Version} =~ m/incompatible/) {
+ my $msg = "${module} $json->{Version} is incompatible with Go
modules.";
+ warn "$msg\n";
+ croak
Re: Go and portgen(1)
On Mon, 06 Apr 2020 at 17:15:17 -0600, Aaron Bieber wrote:
> Round two! This time with espie@'s bsd.port.mk fix:
> https://marc.info/?l=openbsd-ports-cvs=158618354824687=2
>
> This resolves the issue with the conversion of [A-Z] letters to ![a-z] in the
> package path.
>
> With this diff I can generate ports for the following without issue:
>
> github.com/jrick/domain
> github.com/jrick/ss
> github.com/junegunn/fzf
> github.com/qbit/gavin
> golang.zx2c4.com/wireguard
> humungus.tedunangst.com/r/honk
> suah.dev/ogvt
>
> There are still some issues with things like github.com/restic/restic and
> github.com/gohugoio/hugo . For some reason the build looks for some files that
> 'go mod graph' doesn't list.
>
> For most go apps that follow the module guidelines:
> https://github.com/golang/go/wiki/Modules
>
> This method of building things should work pretty well. One will have to go in
> and add a "do-install" to grab other files like man pages, etc.
>
OK, this fixes some issues with determining the latest version of a given port.
With this, I can build a port for restic! (it still needs
'ALL_TARGET=github.com/restic/restic/...' set manually)
hugo still results in some missing mod/zip files.
diff --git a/infrastructure/bin/portgen b/infrastructure/bin/portgen
index ad5ab17f3cf..b7316d42b64 100755
--- a/infrastructure/bin/portgen
+++ b/infrastructure/bin/portgen
@@ -32,6 +32,7 @@ use lib ( "$portdir/infrastructure/lib",
"$FindBin::Bin/../lib" );
use OpenBSD::PortGen::Port::CPAN;
use OpenBSD::PortGen::Port::PyPI;
use OpenBSD::PortGen::Port::Ruby;
+use OpenBSD::PortGen::Port::Go;
my ( $type, $module ) = @ARGV;
@@ -44,6 +45,8 @@ if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
+} elsif ( $type eq 'go' ) {
+ $o = OpenBSD::PortGen::Port::Go->new();
} else {
die "unknown module type\n";
}
diff --git a/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
new file mode 100644
index 000..75d5bf33244
--- /dev/null
+++ b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
@@ -0,0 +1,251 @@
+# $OpenBSD: Go.pm,v 1.16 2019/05/16 16:01:10 afresh1 Exp $
+#
+# Copyright (c) 2019 Aaron Bieber
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+package OpenBSD::PortGen::Port::Go;
+
+use 5.028;
+use utf8;
+use warnings;
+use strict;
+use warnings qw(FATAL utf8);# fatalize encoding glitches
+use open qw(:std :encoding(UTF-8)); # undeclared streams in UTF-8
+use OpenBSD::PackageName;
+
+use parent 'OpenBSD::PortGen::Port';
+
+use Carp;
+use Cwd;
+use File::Temp qw/ tempdir /;
+use Data::Dumper;
+
+use OpenBSD::PortGen::Dependency;
+
+sub ecosystem_prefix
+{
+ my $self = shift;
+ return '';
+}
+
+sub base_url
+{
+ my $self = shift;
+ return 'https://proxy.golang.org/';
+}
+
+sub get_dist_info
+{
+ my ( $self, $module ) = @_;
+
+ my $json = {};
+ if ($module =~ m/@/) {
+ my @parts = split("@", $module);
+ $json->{Name}= ( split '/', $parts[0] )[-1];
+ $json->{Module} = $parts[0];
+ $json->{Version} = $parts[1];
+ } else {
+ $json = $self->get_json( $module . '/@latest' );
+ $json->{Name} = ( split '/', $module )[-1];
+ $json->{Module} = $module;
+ }
+
+ my %mods;
+ for ( $self->_go_mod_graph($json) ) {
+ my ($direct, $ephemeral) = @{$_};
+
+ for my $d ( $direct, $ephemeral ) {
+ next unless $d->{Version};
+ $mods{ $d->{Module} }{ $d->{Version} } ||= $d;
+ }
+ }
+
+ # Filter dependencies to the one with the highest version
+ foreach my $mod ( sort keys %mods ) {
+ # Sort semver numerically then the rest alphanumeric.
+ # This is overkill for sorting, but I already wrote it
+ my @versions =
+ map { $_->[0] }
+ sort {
+ $a->[1] <=> $b->[1]
+|| $a->[2] <=> $b->[2]
+|| $a->[3] <=> $b->[3]
+|| $a->[4] cmp $b->[4]
+ }
+
Re: Go and portgen(1)
On Mon, Apr 13, 2020 at 08:44:47AM +0200, Landry Breuil wrote:
> On Mon, Apr 06, 2020 at 05:15:17PM -0600, Aaron Bieber wrote:
> > Round two! This time with espie@'s bsd.port.mk fix:
> > https://marc.info/?l=openbsd-ports-cvs=158618354824687=2
> >
> > This resolves the issue with the conversion of [A-Z] letters to ![a-z] in
> > the
> > package path.
> >
> > With this diff I can generate ports for the following without issue:
> >
> > github.com/jrick/domain
> > github.com/jrick/ss
> > github.com/junegunn/fzf
> > github.com/qbit/gavin
> > golang.zx2c4.com/wireguard
> > humungus.tedunangst.com/r/honk
> > suah.dev/ogvt
> >
> > There are still some issues with things like github.com/restic/restic and
> > github.com/gohugoio/hugo . For some reason the build looks for some files
> > that
> > 'go mod graph' doesn't list.
> >
> > For most go apps that follow the module guidelines:
> > https://github.com/golang/go/wiki/Modules
>
> I wanted to port vgrep so i gave this a shot:
>
> [08:29] c64:/usr/ports/ $/usr/ports/infrastructure/bin/portgen go
> github.com/vrothberg/vgrep
> Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
> mystuff/go/vgrep)
> *** Error 1 in /home/ports/mystuff/go/vgrep
> (/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
> Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
> mystuff/go/vgrep)
> *** Error 1 in /home/ports/mystuff/go/vgrep
> (/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
> Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
> mystuff/go/vgrep)
> *** Error 1 in /home/ports/mystuff/go/vgrep
> (/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
> Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
> mystuff/go/vgrep)
> *** Error 1 in /home/ports/mystuff/go/vgrep
> (/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
> Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
> mystuff/go/vgrep)
> *** Error 1 in /home/ports/mystuff/go/vgrep
> (/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
> Can't stat : No such file or directory
> at /usr/ports/infrastructure/lib/OpenBSD/PortGen/Port.pm line 251.
> Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
> mystuff/go/vgrep)
> *** Error 1 in /home/ports/mystuff/go/vgrep
> (/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
> Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
> mystuff/go/vgrep)
> *** Error 1 in /home/ports/mystuff/go/vgrep
> (/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
> Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
> mystuff/go/vgrep)
> *** Error 1 in /home/ports/mystuff/go/vgrep
> (/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
> /usr/ports/mystuff/go/vgrep
>
> that's a lot of yelling at a user :)
>
> that generated this (under mystuff/go/vgrep):
>
> # $OpenBSD$
>
> COMMENT = todo
>
> MODGO_MODNAME = github.com/vrothberg/vgrep
> MODGO_VERSION = v0.0.0-20200412142857-0810837a928d
>
> DISTNAME = vgrep-${MODGO_VERSION}
> PKGNAME = vgrep-20200412142857
>
> CATEGORIES =go
>
> # unknown license -> 'unknown license'
> PERMIT_PACKAGE =unknown license
> PERMIT_DISTFILES = unknown license
>
> MODULES = lang/go
>
> MODGO_MODULES = github.com/davecgh/go-spew
> v1.1.1 \
> github.com/jessevdk/go-flags
> v1.4.0 \
> github.com/konsorten/go-windows-terminal-sequences
> v1.0.2 \
> github.com/kr/pretty
> v0.1.0 \
> github.com/kr/pty
> v1.1.1 \
> github.com/kr/text
> v0.1.0 \
> github.com/nightlyone/lockfile
> v0.0.0-20180618180623-0ad87eef1443 \
> github.com/pmezard/go-difflib
> v1.0.0 \
> github.com/sirupsen/logrus
> v1.5.0 \
> github.com/stretchr/objx
> v0.1.0 \
> github.com/stretchr/testify
> v1.4.0 \
> golang.org/x/sys
> v0.0.0-20191218084908-4a24b4065292 \
> gopkg.in/check.v1
> v1.0.0-20180628173108-788fd7840127 \
> gopkg.in/yaml.v2
> v2.2.2
> MODGO_MODFILES =github.com/davecgh/go-spew
>v1.1.0 \
> github.com/davecgh/go-spew
>v1.1.1 \
>
Re: Go and portgen(1)
On Mon, Apr 06, 2020 at 05:15:17PM -0600, Aaron Bieber wrote:
> Round two! This time with espie@'s bsd.port.mk fix:
> https://marc.info/?l=openbsd-ports-cvs=158618354824687=2
>
> This resolves the issue with the conversion of [A-Z] letters to ![a-z] in the
> package path.
>
> With this diff I can generate ports for the following without issue:
>
> github.com/jrick/domain
> github.com/jrick/ss
> github.com/junegunn/fzf
> github.com/qbit/gavin
> golang.zx2c4.com/wireguard
> humungus.tedunangst.com/r/honk
> suah.dev/ogvt
>
> There are still some issues with things like github.com/restic/restic and
> github.com/gohugoio/hugo . For some reason the build looks for some files that
> 'go mod graph' doesn't list.
>
> For most go apps that follow the module guidelines:
> https://github.com/golang/go/wiki/Modules
I wanted to port vgrep so i gave this a shot:
[08:29] c64:/usr/ports/ $/usr/ports/infrastructure/bin/portgen go
github.com/vrothberg/vgrep
Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
mystuff/go/vgrep)
*** Error 1 in /home/ports/mystuff/go/vgrep
(/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
mystuff/go/vgrep)
*** Error 1 in /home/ports/mystuff/go/vgrep
(/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
mystuff/go/vgrep)
*** Error 1 in /home/ports/mystuff/go/vgrep
(/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
mystuff/go/vgrep)
*** Error 1 in /home/ports/mystuff/go/vgrep
(/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
mystuff/go/vgrep)
*** Error 1 in /home/ports/mystuff/go/vgrep
(/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
Can't stat : No such file or directory
at /usr/ports/infrastructure/lib/OpenBSD/PortGen/Port.pm line 251.
Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
mystuff/go/vgrep)
*** Error 1 in /home/ports/mystuff/go/vgrep
(/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
mystuff/go/vgrep)
*** Error 1 in /home/ports/mystuff/go/vgrep
(/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
Fatal: one category in go lang/go should match PKGPATH=mystuff/go/vgrep (in
mystuff/go/vgrep)
*** Error 1 in /home/ports/mystuff/go/vgrep
(/usr/ports/infrastructure/mk/bsd.port.mk:3691 '.BEGIN': @exit 1)
/usr/ports/mystuff/go/vgrep
that's a lot of yelling at a user :)
that generated this (under mystuff/go/vgrep):
# $OpenBSD$
COMMENT = todo
MODGO_MODNAME = github.com/vrothberg/vgrep
MODGO_VERSION = v0.0.0-20200412142857-0810837a928d
DISTNAME = vgrep-${MODGO_VERSION}
PKGNAME = vgrep-20200412142857
CATEGORIES =go
# unknown license -> 'unknown license'
PERMIT_PACKAGE =unknown license
PERMIT_DISTFILES = unknown license
MODULES = lang/go
MODGO_MODULES = github.com/davecgh/go-spew v1.1.1
\
github.com/jessevdk/go-flags v1.4.0
\
github.com/konsorten/go-windows-terminal-sequences v1.0.2
\
github.com/kr/pretty v0.1.0
\
github.com/kr/ptyv1.1.1
\
github.com/kr/text v0.1.0
\
github.com/nightlyone/lockfile
v0.0.0-20180618180623-0ad87eef1443 \
github.com/pmezard/go-difflibv1.0.0
\
github.com/sirupsen/logrus v1.5.0
\
github.com/stretchr/objx v0.1.0
\
github.com/stretchr/testify v1.4.0
\
golang.org/x/sys
v0.0.0-20191218084908-4a24b4065292 \
gopkg.in/check.v1
v1.0.0-20180628173108-788fd7840127 \
gopkg.in/yaml.v2 v2.2.2
MODGO_MODFILES =github.com/davecgh/go-spew
v1.1.0 \
github.com/davecgh/go-spew
v1.1.1 \
github.com/jessevdk/go-flags
v1.4.0 \
github.com/konsorten/go-windows-terminal-sequences
v1.0.1 \
github.com/konsorten/go-windows-terminal-sequences
v1.0.2 \
github.com/kr/pretty
Re: Go and portgen(1)
Round two! This time with espie@'s bsd.port.mk fix:
https://marc.info/?l=openbsd-ports-cvs=158618354824687=2
This resolves the issue with the conversion of [A-Z] letters to ![a-z] in the
package path.
With this diff I can generate ports for the following without issue:
github.com/jrick/domain
github.com/jrick/ss
github.com/junegunn/fzf
github.com/qbit/gavin
golang.zx2c4.com/wireguard
humungus.tedunangst.com/r/honk
suah.dev/ogvt
There are still some issues with things like github.com/restic/restic and
github.com/gohugoio/hugo . For some reason the build looks for some files that
'go mod graph' doesn't list.
For most go apps that follow the module guidelines:
https://github.com/golang/go/wiki/Modules
This method of building things should work pretty well. One will have to go in
and add a "do-install" to grab other files like man pages, etc.
diff --git a/infrastructure/bin/portgen b/infrastructure/bin/portgen
index ad5ab17f3cf..b7316d42b64 100755
--- a/infrastructure/bin/portgen
+++ b/infrastructure/bin/portgen
@@ -32,6 +32,7 @@ use lib ( "$portdir/infrastructure/lib",
"$FindBin::Bin/../lib" );
use OpenBSD::PortGen::Port::CPAN;
use OpenBSD::PortGen::Port::PyPI;
use OpenBSD::PortGen::Port::Ruby;
+use OpenBSD::PortGen::Port::Go;
my ( $type, $module ) = @ARGV;
@@ -44,6 +45,8 @@ if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
+} elsif ( $type eq 'go' ) {
+ $o = OpenBSD::PortGen::Port::Go->new();
} else {
die "unknown module type\n";
}
diff --git a/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
new file mode 100644
index 000..d7bed0daa43
--- /dev/null
+++ b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
@@ -0,0 +1,227 @@
+# $OpenBSD: Go.pm,v 1.16 2019/05/16 16:01:10 afresh1 Exp $
+#
+# Copyright (c) 2019 Aaron Bieber
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+package OpenBSD::PortGen::Port::Go;
+
+use 5.028;
+use utf8;
+use warnings;
+use strict;
+use warnings qw(FATAL utf8);# fatalize encoding glitches
+use open qw(:std :encoding(UTF-8)); # undeclared streams in UTF-8
+
+use parent 'OpenBSD::PortGen::Port';
+
+use Carp;
+use Cwd;
+use File::Temp qw/ tempdir /;
+
+use OpenBSD::PortGen::Dependency;
+
+sub ecosystem_prefix
+{
+ my $self = shift;
+ return '';
+}
+
+sub base_url
+{
+ my $self = shift;
+ return 'https://proxy.golang.org/';
+}
+
+sub get_dist_info
+{
+ my ( $self, $module ) = @_;
+
+ my $json = {};
+ if ($module =~ m/@/) {
+ my @parts = split("@", $module);
+ $json->{Name}= ( split '/', $parts[0] )[-1];
+ $json->{Module} = $parts[0];
+ $json->{Version} = $parts[1];
+ } else {
+ $json = $self->get_json( $module . '/@latest' );
+ $json->{Name} = ( split '/', $module )[-1];
+ $json->{Module} = $module;
+ }
+
+ my %mods;
+ for ( $self->_go_mod_graph($json) ) {
+ my ($direct, $ephemeral) = @{$_};
+
+ for my $d ( $direct, $ephemeral ) {
+ next unless $d->{Version};
+ $mods{ $d->{Module} }{ $d->{Version} } ||= $d;
+ }
+ }
+
+ # Filter dependencies to the one with the highest version
+ foreach my $mod ( sort keys %mods ) {
+ # Sort semver numerically then the rest alphanumeric.
+ # This is overkill for sorting, but I already wrote it
+ my @versions =
+ map { $_->[0] }
+ sort {
+ $a->[1] <=> $b->[1]
+|| $a->[2] <=> $b->[2]
+|| $a->[3] <=> $b->[3]
+|| $a->[4] cmp $b->[4]
+ }
+ map { $_->[1] =~ s/^v//; $_ }
+ map { [ $_, split /[\.-]/, $_, 4 ] }
+ keys %{ $mods{$mod} };
+
+ push @{ $json->{Dist} }, $mods{$mod}{ $versions[-1] };
+ push @{ $json->{Mods} }, map { $mods{$mod}{$_} } @versions;
+ }
+
+ return $json;
+}
+
+sub _go_mod_graph
+{
+ my ($self, $json) = @_;
+ my $dir =
Re: Go and portgen(1)
On Wed, 11 Dec 2019 at 15:30:43 +, Stuart Henderson wrote:
> The go.port.mk part of this breaks some existing ports e.g. net/dnscontrol,
> net/wireguard-go.
>
Here is a diff against the current ports tree - wireguard-go and dnscontrol
build fine.
I also tested other "MODGO_TYPE={bin,lib}" ports.
As a reminder - the `portgen go` stuff only works for Go projects that don't
import libs that have mixed case module names. Anything that uses something
like https://github.com/BurntSushi/toml will be unable to be "portgen"'d.
diff --git a/infrastructure/bin/portgen b/infrastructure/bin/portgen
index ad5ab17f3cf..b7316d42b64 100755
--- a/infrastructure/bin/portgen
+++ b/infrastructure/bin/portgen
@@ -32,6 +32,7 @@ use lib ( "$portdir/infrastructure/lib",
"$FindBin::Bin/../lib" );
use OpenBSD::PortGen::Port::CPAN;
use OpenBSD::PortGen::Port::PyPI;
use OpenBSD::PortGen::Port::Ruby;
+use OpenBSD::PortGen::Port::Go;
my ( $type, $module ) = @ARGV;
@@ -44,6 +45,8 @@ if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
+} elsif ( $type eq 'go' ) {
+ $o = OpenBSD::PortGen::Port::Go->new();
} else {
die "unknown module type\n";
}
diff --git a/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
new file mode 100644
index 000..8ef9280a506
--- /dev/null
+++ b/infrastructure/lib/OpenBSD/PortGen/Port/Go.pm
@@ -0,0 +1,225 @@
+# $OpenBSD: Go.pm,v 1.16 2019/05/16 16:01:10 afresh1 Exp $
+#
+# Copyright (c) 2019 Aaron Bieber
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+package OpenBSD::PortGen::Port::Go;
+
+use 5.028;
+use utf8;
+use warnings;
+use strict;
+use warnings qw(FATAL utf8);# fatalize encoding glitches
+use open qw(:std :encoding(UTF-8)); # undeclared streams in UTF-8
+
+use parent 'OpenBSD::PortGen::Port';
+
+use Carp;
+use Cwd;
+use File::Temp qw/ tempdir /;
+use Data::Dumper;
+
+use OpenBSD::PortGen::Dependency;
+
+sub ecosystem_prefix
+{
+ my $self = shift;
+ return '';
+}
+
+sub base_url
+{
+ my $self = shift;
+ return 'https://proxy.golang.org/';
+}
+
+sub get_dist_info
+{
+ my ( $self, $module ) = @_;
+
+ my $json = $self->get_json( $module . '/@latest' );
+ $json->{Name} = ( split '/', $module )[-1];
+ $json->{Module} = $module;
+
+ my %mods;
+ for ( $self->_go_mod_graph($json) ) {
+ my ($direct, $ephemeral) = @{$_};
+
+ for my $d ( $direct, $ephemeral ) {
+ #next if $d->{Module} eq $module;
+ next unless $d->{Version};
+ $mods{ $d->{Module} }{ $d->{Version} } ||= $d;
+ }
+ }
+
+ # Filter dependencies to the one with the highest version
+ foreach my $mod ( sort keys %mods ) {
+ # Sort semver numerically then the rest alphanumeric.
+ # This is overkill for sorting, but I already wrote it
+ my @versions =
+ map { $_->[0] }
+ sort {
+ $a->[1] <=> $b->[1]
+|| $a->[2] <=> $b->[2]
+|| $a->[3] <=> $b->[3]
+|| $a->[4] cmp $b->[4]
+ }
+ map { $_->[1] =~ s/^v//; $_ }
+ map { [ $_, split /[\.-]/, $_, 4 ] }
+ keys %{ $mods{$mod} };
+
+ push @{ $json->{Dist} }, $mods{$mod}{ $versions[-1] };
+ push @{ $json->{Mods} }, map { $mods{$mod}{$_} } @versions;
+ }
+
+ carp Dumper $json;
+
+ return $json;
+}
+
+sub _go_mod_graph
+{
+ my ($self, $json) = @_;
+ my $dir = tempdir(CLEANUP => 1);
+
+ my $mod = $self->get("$json->{Module}/\@v/$json->{Version}.mod");
+ my ($module) = $mod =~ /\bmodule\s+(.*)\n/;
+ $module =~ s/\s+$//;
+ unless ( $json->{Module} eq $module ) {
+ my $msg = "Module $json->{Module} doesn't match $module";
+ warn "$msg\n";
+ croak $msg;
+ }
+
+ {
+ open my $fh, '>', $dir . "/go.mod" or die $!;
+ print $fh $mod;
+ close $fh;
+ }
+
+ my $old_cwd =
Re: Go and portgen(1)
On Wed, 11 Dec 2019 at 15:30:43 +, Stuart Henderson wrote:
> The go.port.mk part of this breaks some existing ports e.g. net/dnscontrol,
> net/wireguard-go.
>
Here is a version of go.port.mk that fixes the GH_* ALL_TARGET stuff.
# $OpenBSD: go.port.mk,v 1.27 2019/11/19 09:17:06 sthen Exp $
ONLY_FOR_ARCHS ?= ${GO_ARCHS}
MODGO_BUILDDEP ?= Yes
MODGO_DIST_SUBDIR ?=go_modules
MASTER_SITE_ATHENS =https://proxy.golang.org/
MODGO_MASTER_SITESN = 9
MASTER_SITES${MODGO_MASTER_SITESN} ?= ${MASTER_SITE_ATHENS}
MODGO_RUN_DEPENDS = lang/go
MODGO_BUILD_DEPENDS = lang/go
.if ${NO_BUILD:L} == "no" && ${MODGO_BUILDDEP:L} == "yes"
BUILD_DEPENDS +=${MODGO_BUILD_DEPENDS}
.endif
.if ${MACHINE_ARCH} == "amd64"
_GOARCH = amd64
.elif ${MACHINE_ARCH} == "arm"
_GOARCH = arm
.elif ${MACHINE_ARCH} == "aarch64"
_GOARCH = arm64
.elif ${MACHINE_ARCH} == "i386"
_GOARCH = 386
.endif
MODGO_PACKAGE_PATH =${PREFIX}/go-pkg
MODGO_PACKAGES =go-pkg/pkg/openbsd_${_GOARCH}
MODGO_SOURCES = go-pkg/src
MODGO_TOOLS = go-pkg/tool/openbsd_${_GOARCH}
SUBST_VARS += MODGO_TOOLS MODGO_PACKAGES MODGO_SOURCES
MODGO_SUBDIR ?= ${WRKDIST}
MODGO_TYPE ?= bin
MODGO_WORKSPACE ?= ${WRKDIR}/go
MODGO_GOCACHE ?=${WRKDIR}/go-cache
MODGO_GOPATH ?= ${MODGO_WORKSPACE}:${MODGO_PACKAGE_PATH}
# We cannot assume that the maching running the built code will have SSE,
# even though the machine building the package has SSE. As such, we need
# to explicitly disable SSE on i386 builds.
MAKE_ENV += GO386=387
MAKE_ENV += GOCACHE="${MODGO_GOCACHE}"
MODGO_CMD ?=${SETENV} ${MAKE_ENV} go
MODGO_BUILD_CMD = ${MODGO_CMD} install ${MODGO_FLAGS}
MODGO_TEST_CMD =${MODGO_CMD} test ${MODGO_FLAGS} ${MODGO_TEST_FLAGS}
MODGO_BINDIR ?= bin
.if ! empty(MODGO_LDFLAGS)
MODGO_BUILD_CMD += -ldflags="${MODGO_LDFLAGS}"
MODGO_TEST_CMD += -ldflags="${MODGO_LDFLAGS}"
.endif
.if defined(MODGO_MODNAME)
EXTRACT_SUFX ?= .zip
PKGNAME ?= ${DISTNAME:S/-v/-/}
ALL_TARGET ?= ${MODGO_MODNAME}
DISTFILES =
${DISTNAME}${EXTRACT_SUFX}{${MODGO_VERSION}${EXTRACT_SUFX}}
MASTER_SITES ?= ${MASTER_SITE_ATHENS}${MODGO_MODNAME}/@v/
. for _modname _modver in ${MODGO_MODULES}
SUPDISTFILES +=
${MODGO_DIST_SUBDIR}/${_modname}/@v/${_modver}.zip{${_modname}/@v/${_modver}.zip}:${MODGO_MASTER_SITESN}
. endfor
. for _modname _modver in ${MODGO_MODFILES}
SUPDISTFILES +=
${MODGO_DIST_SUBDIR}/${_modname}/@v/${_modver}.mod{${_modname}/@v/${_modver}.mod}:${MODGO_MASTER_SITESN}
. endfor
MAKE_ENV += GOPROXY=file://${DISTDIR}/${MODGO_DIST_SUBDIR}
MAKE_ENV += GO111MODULE=on GOPATH="${MODGO_GOPATH}"
.else
# ports are not allowed to fetch from the network at build time; point
# GOPROXY at an unreachable host so that failures are also visible to
# developers who don't have PORTS_PRIVSEP and a "deny .. _pbuild" PF rule.
MAKE_ENV += GOPROXY=invalid://ports.should.not.fetch.at.buildtime/
MAKE_ENV += GO111MODULE=off GOPATH="${MODGO_GOPATH}"
. if defined(GH_ACCOUNT) && defined(GH_PROJECT)
ALL_TARGET ?= github.com/${GH_ACCOUNT}/${GH_PROJECT}
. endif
.endif
MODGO_TEST_TARGET ?=cd ${WRKSRC} && ${MODGO_CMD} test ${ALL_TARGET}
SEPARATE_BUILD ?= Yes
CATEGORIES += lang/go
MODGO_BUILD_TARGET =${MODGO_BUILD_CMD}
MODGO_FLAGS += -v -p ${MAKE_JOBS}
.if empty(DEBUG)
# by default omit symbol table, debug information and DWARF symbol table
MODGO_LDFLAGS +=-s -w
.else
MODGO_FLAGS += -x
.endif
.if empty(MODGO_MODNAME)
WRKSRC ?= ${MODGO_WORKSPACE}/src/${ALL_TARGET}
MODGO_SETUP_WORKSPACE = mkdir -p ${WRKSRC:H}; mv ${MODGO_SUBDIR} ${WRKSRC};
.else
WRKSRC ?= ${WRKDIR}/${MODGO_MODNAME}@${MODGO_VERSION}
MODGO_SETUP_WORKSPACE = ln -sf ${WRKSRC} ${WRKDIR}/${MODGO_MODNAME}
.endif
INSTALL_STRIP =
.if ${MODGO_TYPE:L:Mbin}
MODGO_INSTALL_TARGET = ${INSTALL_PROGRAM_DIR} ${PREFIX}/${MODGO_BINDIR} && \
${INSTALL_PROGRAM} ${MODGO_WORKSPACE}/bin/* \
${PREFIX}/${MODGO_BINDIR};
.endif
# Go source files serve the purpose of libraries, so sources should be included
# with library ports.
.if ${MODGO_TYPE:L:Mlib}
MODGO_INSTALL_TARGET += ${INSTALL_DATA_DIR} ${MODGO_PACKAGE_PATH} && \
cd ${MODGO_WORKSPACE} && \
find src pkg -type d -exec ${INSTALL_DATA_DIR} \
${MODGO_PACKAGE_PATH}/{} \; \
-o -type f -exec ${INSTALL_DATA} -p \
${MODGO_WORKSPACE}/{} \
${MODGO_PACKAGE_PATH}/{} \;
# This is required to force rebuilding of go libraries upon changes in
# toolchain.
RUN_DEPENDS += ${MODGO_RUN_DEPENDS}
.endif
.if empty(CONFIGURE_STYLE)
Re: Go and portgen(1)
The go.port.mk part of this breaks some existing ports e.g. net/dnscontrol, net/wireguard-go.
Re: Go and portgen(1)
On Sun, 08 Dec 2019 at 13:53:40 -0800, Andrew Hewus Fresh wrote: > On Sun, Dec 08, 2019 at 11:42:50AM -0700, Aaron Bieber wrote: > > Here is a diff that adds go support in portgen(1). It's a combination > > of diffs from my self and afresh1@. There are a few issues, but on the > > whole this is the direction I think we should take with regard to > > porting Go application. > > > > The biggest issue with the diff is it's inability to cope with the way > > Go escapes[1] uppercase letters in URLs. This means that this > > implementation can't package things like "github.com/gohugoio/hugo" as > > it has some dependencies like: "github.com/BurntSushi/toml". > > I would like to see this committed, so OK afresh1@, but some of the > changes are mine, so would definitely like to hear about regressions. > > I thought I had mailed out the Port.pm changes already, but it seems I > didn't actually do that. I think that should go in separately, and can > probably commit them on Thursday or so. The diff Aaron posted is > missing a bit of cleanup to Dependency.pm related to the improvements to > formatting, so the full patch for that is attached as: > > portgen-multi_value_values.patch Tested with perl, ruby and python ports. No regress as far as building goes. I did notice that at least BUILD_DEPENDS becomes "unaligned". Before: BUILD_DEPENDS = devel/p5-Module-Build>=0.28 RUN_DEPENDS = www/p5-CGI \ After: BUILD_DEPENDS = devel/p5-Module-Build>=0.28 RUN_DEPENDS = www/p5-CGI \ (spaces used above to preserve look) > > I do have a couple other patches out looking for OKs, or at least > "doesn't seem to break anything" reports. > > This should detect ports that need FIX_EXTRACT_PERMISSIONS and set it: > portgen-fix_extract_permissions.patch > https://marc.info/?l=openbsd-ports=157454704431952=2 This seem fine. OK abieber@ > > Then one that looks up existing ports by "stem" instead of the current > heuristic. The worry here would be finding the "wrong" existing port, > but with this new way it should be easier to munge things and detect > more existing ports if necessary. > > portgen-lookup_existing_ports_by_stem.patch > https://marc.info/?l=openbsd-ports=157515508030392=2 > This one doesn't seem to cause issues so OK from me, it would be super cool if someone else could test it though! :D
Re: Go and portgen(1)
On Sun, Dec 08, 2019 at 11:42:50AM -0700, Aaron Bieber wrote:
> Here is a diff that adds go support in portgen(1). It's a combination
> of diffs from my self and afresh1@. There are a few issues, but on the
> whole this is the direction I think we should take with regard to
> porting Go application.
>
> The biggest issue with the diff is it's inability to cope with the way
> Go escapes[1] uppercase letters in URLs. This means that this
> implementation can't package things like "github.com/gohugoio/hugo" as
> it has some dependencies like: "github.com/BurntSushi/toml".
I would like to see this committed, so OK afresh1@, but some of the
changes are mine, so would definitely like to hear about regressions.
I thought I had mailed out the Port.pm changes already, but it seems I
didn't actually do that. I think that should go in separately, and can
probably commit them on Thursday or so. The diff Aaron posted is
missing a bit of cleanup to Dependency.pm related to the improvements to
formatting, so the full patch for that is attached as:
portgen-multi_value_values.patch
I do have a couple other patches out looking for OKs, or at least
"doesn't seem to break anything" reports.
This should detect ports that need FIX_EXTRACT_PERMISSIONS and set it:
portgen-fix_extract_permissions.patch
https://marc.info/?l=openbsd-ports=157454704431952=2
Then one that looks up existing ports by "stem" instead of the current
heuristic. The worry here would be finding the "wrong" existing port,
but with this new way it should be easier to munge things and detect
more existing ports if necessary.
portgen-lookup_existing_ports_by_stem.patch
https://marc.info/?l=openbsd-ports=157515508030392=2
Index: infrastructure/lib/OpenBSD/PortGen/Dependency.pm
===
RCS file: /cvs/ports/infrastructure/lib/OpenBSD/PortGen/Dependency.pm,v
retrieving revision 1.2
diff -u -p -r1.2 Dependency.pm
--- infrastructure/lib/OpenBSD/PortGen/Dependency.pm12 May 2019 20:23:33
- 1.2
+++ infrastructure/lib/OpenBSD/PortGen/Dependency.pm8 Dec 2019 21:35:41
-
@@ -109,10 +109,6 @@ sub format
@{ $fmt{'test'} } = '${RUN_DEPENDS}';
}
- for my $type ( keys %fmt ) {
- $fmt{$type} = ( join " \\\n\t\t\t", @{ $fmt{$type} } ) || undef;
- }
-
return \%fmt;
}
Index: infrastructure/lib/OpenBSD/PortGen/Port.pm
===
RCS file: /cvs/ports/infrastructure/lib/OpenBSD/PortGen/Port.pm,v
retrieving revision 1.18
diff -u -p -r1.18 Port.pm
--- infrastructure/lib/OpenBSD/PortGen/Port.pm 12 Jun 2019 19:25:53 -
1.18
+++ infrastructure/lib/OpenBSD/PortGen/Port.pm 8 Dec 2019 21:35:41 -
@@ -203,9 +205,7 @@ sub set_build_deps
{
my ( $self, $build_deps ) = @_;
- # Makefile.template is missing a tab for BUILD_DEPENDS
- # and we want the port to be pretty, so add one
- $self->{BUILD_DEPENDS} = "\t" . $build_deps if $build_deps;
+ $self->{BUILD_DEPENDS} = $build_deps;
}
sub set_run_deps
@@ -344,6 +375,22 @@ sub write_makefile
delete $configs{EXTRACT_SUFX}
if $configs{EXTRACT_SUFX} and $configs{EXTRACT_SUFX} eq '.tar.gz';
+ my $format = sub {
+ my ($key, $value, %opts) = @_;
+
+ my $tabs = "\t" x ( $opts{tabs} || 1 );
+ $key .= $opts{equal} || $default_equal;
+
+ if (ref $value eq 'ARRAY') {
+ my $key_tabs = "\t" x ( length($key) / 8 );
+ $value = join " \\\n$key_tabs$tabs", @{ $value }
+ }
+
+ $key .= $tabs if length $value;
+
+ return $key . $value;
+ };
+
my @makefile;
foreach my $line (@template) {
next# no more than one blank line
@@ -358,10 +405,7 @@ sub write_makefile
next if $key !~ /^[\p{Upper}_]+(?:-\w+)?$/;
my $value = $configs{$key};
next unless defined $value;
-
- my $print_key = "$key$default_equal";
- $print_key .= "\t" if length $value;
- push @additions, "$print_key$value";
+ push @additions, $format->($key, $value);
}
if (@additions) {
push @makefile,
@@ -382,19 +426,12 @@ sub write_makefile
}
# If we didn't get a value, copy from the template
- if ( not $value and %copy_values ) {
- $value = $line->{value}
-
Go and portgen(1)
Hi,
Here is a diff that adds go support in portgen(1). It's a combination
of diffs from my self and afresh1@. There are a few issues, but on the
whole this is the direction I think we should take with regard to
porting Go application.
The biggest issue with the diff is it's inability to cope with the way
Go escapes[1] uppercase letters in URLs. This means that this
implementation can't package things like "github.com/gohugoio/hugo" as
it has some dependencies like: "github.com/BurntSushi/toml".
As for usage, one needs to get the "module" name from a given
program. This is the first line of the "mo.mod" file contained in the
program being ported. Once you have that it's simply a call to
portgen:
portgen go gthub.com/qbit/gavin
This causes portgen to:
1) Fetch https://proxy.golang.org/github.com/qbit/gavin/@latest
2) Grab the latest version.
3) Fetch the .mod file:
https://proxy.golang.org/github.com/qbit/gavin/@v/v0.0.0-20191129154004-5242047b11bb.mod
4) Call 'go mod graph' to get a list of all dependencies and their
versions. We need the .zip files and a handful of .mod files (as
described in [1]) to complete the build.
5) Generate the port.
We are using SUPDISTFILES for the .mod and .zip files as Go doesn't
need the zip files to be extracted on disk.
With this diff I can package a number of applications (wireguard,
honk and a number of pet projects I have) without issue!
I'd like to get this diff into the tree to make collaboration
easier. As it is now, I can build other ports (with portgen) fine, and
in-tree go packages build as expected.
[1] https://tip.golang.org/cmd/go/#hdr-Module_proxy_protocol
** MMITYWTR (Much More Info Than You Wanted To Read)
Here is an outline of the current methods for creating a Go port and a
bit of reasoning as to why I went with the 'go mod' method.
1) Using GOPATH.
*Advantages*:
- Minimal changes to our tree.
- Easy.
*Disadvantages*:
- Going away in future releases.
- Prevents Go from using versioned modules (everything basically
builds off of 'master' of what ever dependency. This will likely
introduce issues that seem like upstream problems, but are
actually local to our port.
2) Using the vendor directory.
*Advantages*:
- Easy.
- Currently in use.
- Maintains "build integrity" by keeping versioned dependencies in
the 'vendor' directory.
*Disadvantages*:
- Requires building / hosting of 'vendor'ed distfiles.
- Might go away in the future.
3) Using 'go mod'.
*Advantages*:
- The New© Go⑨ Way™. Likely to stay the defacto method for some time.
- Maintains "build integrity". The binaries we produce in the
ports tree are the same as the binaries upstream will produce.
*Disadvantages*:
- More complex to implement.
- Makes patching an application's dependencies difficult[2]. But
one could still fall back to the 'vendor'ed method if patches
are absolutely required (this breaks "build integrity" though).
For portgen(1) I have opted to use method 3.
[2] I see this as an advantage, as it will encourage people to
upstream diffs first. This raises awareness of OpenBSD and lubes the
interactions between program and library developers.
diff --git a/infrastructure/bin/portgen b/infrastructure/bin/portgen
index ad5ab17f3cf..b7316d42b64 100755
--- a/infrastructure/bin/portgen
+++ b/infrastructure/bin/portgen
@@ -32,6 +32,7 @@ use lib ( "$portdir/infrastructure/lib",
"$FindBin::Bin/../lib" );
use OpenBSD::PortGen::Port::CPAN;
use OpenBSD::PortGen::Port::PyPI;
use OpenBSD::PortGen::Port::Ruby;
+use OpenBSD::PortGen::Port::Go;
my ( $type, $module ) = @ARGV;
@@ -44,6 +45,8 @@ if ( $type eq 'p5' ) {
$o = OpenBSD::PortGen::Port::PyPI->new();
} elsif ( $type eq 'ruby' ) {
$o = OpenBSD::PortGen::Port::Ruby->new();
+} elsif ( $type eq 'go' ) {
+ $o = OpenBSD::PortGen::Port::Go->new();
} else {
die "unknown module type\n";
}
diff --git a/infrastructure/lib/OpenBSD/PortGen/Port.pm
b/infrastructure/lib/OpenBSD/PortGen/Port.pm
index dc4c6bd6b7b..06814357f67 100644
--- a/infrastructure/lib/OpenBSD/PortGen/Port.pm
+++ b/infrastructure/lib/OpenBSD/PortGen/Port.pm
@@ -203,9 +203,7 @@ sub set_build_deps
{
my ( $self, $build_deps ) = @_;
- # Makefile.template is missing a tab for BUILD_DEPENDS
- # and we want the port to be pretty, so add one
- $self->{BUILD_DEPENDS} = "\t" . $build_deps if $build_deps;
+ $self->{BUILD_DEPENDS} = $build_deps;
}
sub set_run_deps
@@ -344,6 +342,22 @@ sub write_makefile
delete $configs{EXTRACT_SUFX}
if $configs{EXTRACT_SUFX} and $configs{EXTRACT_SUFX} eq '.tar.gz';
+ my $format = sub {
+ my ($key, $value, %opts) = @_;
+
+ my $tabs = "\t" x ( $opts{tabs} || 1 );
+
