Martin,
first of all: thank you for testing the patches. As usual the corner
cases are the most tricky ones.
tl;dr: I'm working on an updated patch series to address the reported
issues.
I've looked into the issues and would like to give some explanation.
1) the first module you mention ('gonum.org/v1/gonum
<http://gonum.org/v1/gonum>') does not obey the go documentation for
resolving the modules source-code repository (see
https://go.dev/ref/mod#go-mod-download , Section 'Version control
systems'). The documentation states that there should be a html page,
queried by "?go-get=1", with a <meta>-tag which contains the original
URL to the source-code repository. For 'gonum' there is only a 404-page.
This page contains javascript-code that redirects to the actual gonum
package page. Unnecessary hard to process. I'll open an issue for that.
The second module 'code.cloudfoundry.org/clock
<http://code.cloudfoundry.org/clock>' has a broken certificate. I
already opend an issue on github for this.
I addressed this issue above by maintaining a list of modules and their
actual repo URLs inside the go-recipetool as an absolute fallback. Would
do you think about this solution?
2) yes, my bad :)
3) The problem is that some repositories have a prefix in their tags.
The tags usually only contain the semantic version string. I'm curretly
working on this issue.
Best regards
Lukas
Am 22.12.2022 um 20:08 schrieb Martin Jansa via lists.openembedded.org:
On Thu, Dec 22, 2022 at 7:39 PM Vyacheslav Yurkov <[email protected]>
wrote:
On 22.12.2022 17:48, Bruce Ashfield wrote:
> On Thu, Dec 22, 2022 at 11:20 AM Richard Purdie
> <[email protected]> wrote:
>> This patch is not in master and is not a backport therefore not
>> eligible for kirkstone/langdale.
My bad, the intention was to have it in all three branches: master,
kirkstone, and langdale.
> 'nor should it go to master.
>
> If someone wants to allow go to fetch sources during builds, it
needs
> to be done in their own layers.
>
> Bruce
Probably I misunderstood the outcome of the discussion in my first
link.
Bruce and Richard, you both suggested to have it in go-mod with the
warning. I agree that it breaks reproducibility, and is considered
a bad
practice in general (I'm aware of a few more build systems with
the same
shortcoming).
My point is, until a proper solution is in place, it should be at
least
documented somewhere that this should be done in own layer/recipe.
Any
suggestions where this can be documented if not in OE-Core?
Have you tried the changes submitted by Lukas/Stefan in:
https://lists.openembedded.org/g/openembedded-architecture/message/1539
?
It's not perfect, I was testing it on
https://github.com/influxdata/telegraf/blob/master/go.mod and I've
found some corner cases where it failed.
But it seems like very good start and we should work with Lukas/Stefan
to get it merged in master. Then all branches could consume recipes
created in master and only the exceptions would need to have network
access (instead of the bbclass enabling it for every go-mod user -
go-vendor inherits go-mod as well).
Lukas/Stefan: is there something we can do to help with v2 of your
patches?
I wanted to submit better review on submitted patches after more
debugging, but here it is:
1) to resolve 2 deps:
{'Path': 'gonum.org/v1/gonum <http://gonum.org/v1/gonum>', 'Version':
'v0.12.0'} due to 404 "ERROR: Error while fetching redirect page: HTTP
Error 404: Not Found"
{'Path': 'code.cloudfoundry.org/clock
<http://code.cloudfoundry.org/clock>', 'Version': 'v1.0.0',
'Indirect': True} due to this being redirect to github.com
<http://github.com> and failing with "ERROR: Error while fetching
redirect page: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: Hostname mismatch, certificate is not valid
for 'code.cloudfoundry.org <http://code.cloudfoundry.org>'. (_ssl.c:996)>"
parse
2) some issues with LICENSEs:
INFO: Please add the following line for
'docs/LICENSE_OF_DEPENDENCIES.md' to a 'lib/recipetool/licenses.csv'
and replace `Unknown` with the license:
c4807c5b058a56c02f592207b4e056b1,Unknown
Traceback (most recent call last):
File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
111, in <module>
ret = main()
^^^^^^
File "/OE/build/oe-core/openembedded-core/scripts/recipetool", line
100, in main
ret = args.func(args)
^^^^^^^^^^^^^^^
File
"/OE/layers/openembedded-core/scripts/lib/recipetool/create.py", line
746, in create_recipe
handler.process(srctree_use, classes, lines_before, lines_after,
handled, extravalues)
File
"/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 349, in process
self._rewrite_lic_uri(lines_before)
File
"/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 380, in _rewrite_lic_uri
updated, newlines = bb.utils.edit_metadata(lines_before,
['LIC_FILES_CHKSUM'], varfunc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1382, in
edit_metadata
if handle_var_end():
^^^^^^^^^^^^^^^^
File "/OE/build/oe-core/bitbake/lib/bb/utils.py", line 1280, in
handle_var_end
(newvalue, newop, indent, minbreak) = varfunc(in_var, full_value,
op, newlines)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/OE/layers/openembedded-core/scripts/lib/recipetool/create_go.py",
line 372, in varfunc
uri, chksum = license.split(';', 1)
^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)
(the license was just '/')
3) some SRCREVs set to None leading to parsing failure:
File "/OE/layers/bitbake/lib/bb/fetch2/git.py", line 762, in
_latest_revision
raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git
repository in git ls-remote output for %s" % \
bb.data_smart.ExpansionError: Failure expanding variable SRCPV,
expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception
FetchError: Fetcher failure: Unable to resolve 'None' in upstream git
repository in git ls-remote output for
github.com/googleapis/google-cloud-go
<http://github.com/googleapis/google-cloud-go>
The variable dependency chain for the failure is: SRCPV -> PV -> BP ->
FILESPATH
ERROR: Parsing newly created recipe failed, moving recipe to
/OE/build/oe-core/workspace/recipes/telegraf/telegraf_git.bb.parsefailed
for reference. If this looks to be caused by the recipe itself, please
report this error.
It takes long time to fetch all telegraf dependencies, so I'll create
some test recipe to more easily debug these 3 issues in isolation.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#175360):
https://lists.openembedded.org/g/openembedded-core/message/175360
Mute This Topic: https://lists.openembedded.org/mt/95827446/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
