Re: [OE-Core][PATCH v2 4/4] recipetool: Add handler to create go recipes
On Tue, 2023-10-17 at 16:00 +0200, Lukas Funke wrote: > Hi Richard, > > On 17.10.2023 15:53, Richard Purdie wrote: > > On Tue, 2023-10-17 at 15:26 +0200, Lukas Funke wrote: > > > From: Lukas Funke > > > > > > Signed-off-by: Lukas Funke > > > --- > > > scripts/lib/recipetool/create_go.py | 730 > > > 1 file changed, 730 insertions(+) > > > create mode 100644 scripts/lib/recipetool/create_go.py > > > > > > diff --git a/scripts/lib/recipetool/create_go.py > > > b/scripts/lib/recipetool/create_go.py > > > new file mode 100644 > > > index 00..e0254f111b > > > --- /dev/null > > > +++ b/scripts/lib/recipetool/create_go.py > > > @@ -0,0 +1,730 @@ > > > +# Recipe creation tool - go support plugin > > > +# > > > +# Copyright (C) 2023 Weidmueller GmbH & Co KG > > > +# Author: Lukas Funke > > > +# > > > +# SPDX-License-Identifier: GPL-2.0-only > > > +# > > > +# Copyright (c) 2009 The Go Authors. All rights reserved. > > > +# > > > +# SPDX-License-Identifier: BSD-3-Clause > > > +# > > > > Can you clarify what this license information means please? Two > > different license identifier lines seems rather confusing and > > problematic. > > > > I've not looked into the rest of the patches yet, this just caught my > > eye. > > Some of the ideas/code was ported from the original golang code to > python here. Thus, I had to copy the license information as well (I > guess?). If this is wrong or could be written in another way please > provide an example how it's done. If the code is close enough to the original to be based off it, the new code would be under the same license? We can't tell which is the old code and which is the new code so the license header is rather problematic and confusing so this does need to be fixed. Perhaps this means it is all under BSD-3-Clause? That sounds easier but we don't don't have a top level LICENSE.BSD-3-Clause to match this and I'd prefer to keep the code under MIT/GPL-2.0 and not add another license to the mix if we can help it. Or is the code sufficiently different that you can say you took inspiration from that code but the result is under GPL-2.0 as it was different enough you can license it as you wish, the original license becoming not relevant? Either way the header needs some explanation of where this other copyright comes from too. I would probably just credit it as a source of inspiration but I don't know how similar it is/isn't. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189364): https://lists.openembedded.org/g/openembedded-core/message/189364 Mute This Topic: https://lists.openembedded.org/mt/102017392/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v2 4/4] recipetool: Add handler to create go recipes
Hi Richard, On 17.10.2023 15:53, Richard Purdie wrote: On Tue, 2023-10-17 at 15:26 +0200, Lukas Funke wrote: From: Lukas Funke Signed-off-by: Lukas Funke --- scripts/lib/recipetool/create_go.py | 730 1 file changed, 730 insertions(+) create mode 100644 scripts/lib/recipetool/create_go.py diff --git a/scripts/lib/recipetool/create_go.py b/scripts/lib/recipetool/create_go.py new file mode 100644 index 00..e0254f111b --- /dev/null +++ b/scripts/lib/recipetool/create_go.py @@ -0,0 +1,730 @@ +# Recipe creation tool - go support plugin +# +# Copyright (C) 2023 Weidmueller GmbH & Co KG +# Author: Lukas Funke +# +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (c) 2009 The Go Authors. All rights reserved. +# +# SPDX-License-Identifier: BSD-3-Clause +# Can you clarify what this license information means please? Two different license identifier lines seems rather confusing and problematic. I've not looked into the rest of the patches yet, this just caught my eye. Some of the ideas/code was ported from the original golang code to python here. Thus, I had to copy the license information as well (I guess?). If this is wrong or could be written in another way please provide an example how it's done. Best regards, Lukas Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189336): https://lists.openembedded.org/g/openembedded-core/message/189336 Mute This Topic: https://lists.openembedded.org/mt/102017392/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v2 4/4] recipetool: Add handler to create go recipes
On Tue, 2023-10-17 at 15:26 +0200, Lukas Funke wrote: > From: Lukas Funke > > Signed-off-by: Lukas Funke > --- > scripts/lib/recipetool/create_go.py | 730 > 1 file changed, 730 insertions(+) > create mode 100644 scripts/lib/recipetool/create_go.py > > diff --git a/scripts/lib/recipetool/create_go.py > b/scripts/lib/recipetool/create_go.py > new file mode 100644 > index 00..e0254f111b > --- /dev/null > +++ b/scripts/lib/recipetool/create_go.py > @@ -0,0 +1,730 @@ > +# Recipe creation tool - go support plugin > +# > +# Copyright (C) 2023 Weidmueller GmbH & Co KG > +# Author: Lukas Funke > +# > +# SPDX-License-Identifier: GPL-2.0-only > +# > +# Copyright (c) 2009 The Go Authors. All rights reserved. > +# > +# SPDX-License-Identifier: BSD-3-Clause > +# Can you clarify what this license information means please? Two different license identifier lines seems rather confusing and problematic. I've not looked into the rest of the patches yet, this just caught my eye. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189334): https://lists.openembedded.org/g/openembedded-core/message/189334 Mute This Topic: https://lists.openembedded.org/mt/102017392/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-Core][PATCH v2 4/4] recipetool: Add handler to create go recipes
From: Lukas Funke
Signed-off-by: Lukas Funke
---
scripts/lib/recipetool/create_go.py | 730
1 file changed, 730 insertions(+)
create mode 100644 scripts/lib/recipetool/create_go.py
diff --git a/scripts/lib/recipetool/create_go.py
b/scripts/lib/recipetool/create_go.py
new file mode 100644
index 00..e0254f111b
--- /dev/null
+++ b/scripts/lib/recipetool/create_go.py
@@ -0,0 +1,730 @@
+# Recipe creation tool - go support plugin
+#
+# Copyright (C) 2023 Weidmueller GmbH & Co KG
+# Author: Lukas Funke
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (c) 2009 The Go Authors. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+
+
+from collections import namedtuple
+from enum import Enum
+from html.parser import HTMLParser
+from recipetool.create import RecipeHandler, handle_license_vars
+from recipetool.create import guess_license, tidy_licenses, fixup_license
+from urllib.error import URLError
+
+import bb.utils
+import json
+import logging
+import os
+import re
+import subprocess
+import sys
+import shutil
+import tempfile
+import urllib.parse
+import urllib.request
+
+
+GoImport = namedtuple('GoImport', 'root vcs url suffix')
+logger = logging.getLogger('recipetool')
+CodeRepo = namedtuple(
+'CodeRepo', 'path codeRoot codeDir pathMajor pathPrefix pseudoMajor')
+
+tinfoil = None
+
+# Regular expression to parse pseudo semantic version
+# see https://go.dev/ref/mod#pseudo-versions
+re_pseudo_semver = re.compile(
+
r"^v[0-9]+\.(0\.0-|\d+\.\d+-([^+]*\.)?0\.)(?P\d{14})-(?P[A-Za-z0-9]+)(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$")
+# Regular expression to parse semantic version
+re_semver = re.compile(
+
r"^v(?P0|[1-9]\d*)\.(?P0|[1-9]\d*)\.(?P0|[1-9]\d*)(?:-(?P(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+(?P[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$")
+
+
+def tinfoil_init(instance):
+global tinfoil
+tinfoil = instance
+
+
+class GoRecipeHandler(RecipeHandler):
+"""Class to handle the go recipe creation"""
+
+@staticmethod
+def __ensure_go():
+"""Check if the 'go' command is available in the recipes"""
+recipe = "go-native"
+if not tinfoil.recipes_parsed:
+tinfoil.parse_recipes()
+try:
+rd = tinfoil.parse_recipe(recipe)
+except bb.providers.NoProvider:
+bb.error(
+"Nothing provides '%s' which is required for the build" %
(recipe))
+bb.note(
+"You will likely need to add a layer that provides '%s'" %
(recipe))
+return None
+
+bindir = rd.getVar('STAGING_BINDIR_NATIVE')
+gopath = os.path.join(bindir, 'go')
+
+if not os.path.exists(gopath):
+tinfoil.build_targets(recipe, 'addto_recipe_sysroot')
+
+if not os.path.exists(gopath):
+logger.error(
+'%s required to process specified source, but %s did not
seem to populate it' % 'go', recipe)
+return None
+
+return bindir
+
+def __resolve_repository_static(self, modulepath):
+"""Resolve the repository in a static manner
+
+The method is based on the go implementation of
+`repoRootFromVCSPaths` in
+
https://github.com/golang/go/blob/master/src/cmd/go/internal/vcs/vcs.go
+"""
+
+url = urllib.parse.urlparse("https://; + modulepath)
+req = urllib.request.Request(url.geturl())
+
+try:
+resp = urllib.request.urlopen(req)
+# Some modulepath are just redirects to github (or some other vcs
+# hoster). Therefore, we check if this modulepath redirects to
+# somewhere else
+if resp.geturl() != url.geturl():
+bb.debug(1, "%s is redirectred to %s" %
+ (url.geturl(), resp.geturl()))
+url = urllib.parse.urlparse(resp.geturl())
+modulepath = url.netloc + url.path
+
+except URLError as url_err:
+# This is probably because the module path
+# contains the subdir and major path. Thus,
+# we ignore this error for now
+logger.debug(
+1, "Failed to fetch page from [%s]: %s" % (url, str(url_err)))
+
+host, _, _ = modulepath.partition('/')
+
+class vcs(Enum):
+pathprefix = "pathprefix"
+regexp = "regexp"
+type = "type"
+repo = "repo"
+check = "check"
+schemelessRepo = "schemelessRepo"
+
+# GitHub
+vcsGitHub = {}
+vcsGitHub[vcs.pathprefix] = "github.com"
+vcsGitHub[vcs.regexp] = re.compile(
+
r'^(?Pgithub\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/(?P[A-Za-z0-9_.\-]+))*$')
+vcsGitHub[vcs.type] = "git"
+vcsGitHub[vcs.repo] = "https://\\g"
+
+# Bitbucket
+vcsBitbucket = {}
+
