AdrianVovk opened a new issue, #1851:
URL: https://github.com/apache/buildstream/issues/1851
Trying to port carbonOS to bst2 (specifically, 2.0.1). Using `project.refs`.
Running in a `fedora:38` distrobox container, w/ buildstream `pip install`'d
into /usr/local
Some packages I track will error out with bugs like this:
```
[00:00:25][????????][ track:pkgs/fonts/twemoji.bst ] BUG Track
An unhandled exception occured:
Traceback (most recent call last):
File
"/usr/local/lib64/python3.11/site-packages/buildstream/_projectrefs.py", line
144, in _lookup
node = element_list.mapping_at(source_index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "src/buildstream/node.pyx", line 1262, in
buildstream.node.SequenceNode.mapping_at
File "src/buildstream/node.pyx", line 1278, in
buildstream.node.SequenceNode.mapping_at
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/usr/local/lib64/python3.11/site-packages/buildstream/_scheduler/jobs/job.py",
line 438, in child_action
result = self.child_process() # pylint:
disable=assignment-from-no-return
^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib64/python3.11/site-packages/buildstream/_scheduler/jobs/elementjob.py",
line 91, in child_process
return self._action_cb(self._element)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib64/python3.11/site-packages/buildstream/_scheduler/queues/trackqueue.py",
line 65, in _track_element
return element._track()
^^^^^^^^^^^^^^^^
File
"/usr/local/lib64/python3.11/site-packages/buildstream/element.py", line 1390,
in _track
return self.__sources.track(self._get_workspace())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib64/python3.11/site-packages/buildstream/_elementsources.py",
line 97, in track
new_ref = source._track()
^^^^^^^^^^^^^^^
File
"/usr/local/lib64/python3.11/site-packages/buildstream/source.py", line 1214,
in _track
self._set_ref(new_ref, save=False)
File
"/usr/local/lib64/python3.11/site-packages/buildstream/source.py", line 1065,
in _set_ref
node = toplevel_refs.lookup_ref(project.name, element_name,
element_idx, write=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib64/python3.11/site-packages/buildstream/_projectrefs.py", line
113, in lookup_ref
node = self._lookup(self._toplevel_save, project, element,
source_index, ensure=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib64/python3.11/site-packages/buildstream/_projectrefs.py", line
150, in _lookup
node = element_list.mapping_at(source_index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "src/buildstream/node.pyx", line 1262, in
buildstream.node.SequenceNode.mapping_at
File "src/buildstream/node.pyx", line 1278, in
buildstream.node.SequenceNode.mapping_at
IndexError: list index out of range
```
Tracking this element:
```
# Check for updates here:
https://koji.fedoraproject.org/koji/packageinfo?packageID=26306
sources:
- kind: remote
url:
https://kojipkgs.fedoraproject.org//packages/twitter-twemoji-fonts/14.0.2/3.fc38/noarch/twitter-twemoji-fonts-14.0.2-3.fc38.noarch.rpm
filename: twemoji.rpm
- kind: local
path: files/fontconfig/75-twemoji.conf
# manual-updates
# For extracting the RPM
- kind: git
url: github:rpm-software-management/rpm
directory: rpm
```
This bug is oddly inconsistent. I can reliably reproduce it by deleting
`project.refs`. Then it'll fail to track (same packages every time). After it
fails, I type `r` to make buildstream retry. Then it "succeeds", but
incorrectly tracks the `kind: remote` source. So I re-track the package, and it
tracks everything correctly. See the log
```
$ rm project.refs
$ bst source track pkgs/fonts/twemoji.bst
... (error from above) ...
Track failure on element: pkgs/fonts/twemoji.bst
...
Choice: [continue]: r
Retrying failed job
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] START
carbonOS/pkgs-fonts-twemoji/????????-track.2231251.log
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] START
Tracking
https://kojipkgs.fedoraproject.org//packages/twitter-twemoji-fonts/14.0.2/3.fc38/noarch/twitter-twemoji-fonts-14.0.2-3.fc38.noarch.rpm
[00:00:00][????????][ track:pkgs/fonts/twemoji.bst ] SUCCESS
Tracking
https://kojipkgs.fedoraproject.org//packages/twitter-twemoji-fonts/14.0.2/3.fc38/noarch/twitter-twemoji-fonts-14.0.2-3.fc38.noarch.rpm
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] START
Tracking master from https://github.com/rpm-software-management/rpm
[00:00:00][????????][ track:pkgs/fonts/twemoji.bst ] SUCCESS
Tracking master from https://github.com/rpm-software-management/rpm
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] INFO Found
new revision: ('2043f2ac57ae2690cc1b11242f040085df8eed83', [])
[00:00:01][????????][ track:pkgs/fonts/twemoji.bst ] SUCCESS
carbonOS/pkgs-fonts-twemoji/????????-track.2231251.log
[00:00:02][ ][ main:core activity ] SUCCESS Track
...
$ cat project.refs
projects:
carbonOS:
pkgs/fonts/twemoji.bst:
- {}
- {}
- ref: 2043f2ac57ae2690cc1b11242f040085df8eed83
$ bst source track pkgs/fonts/twemoji.bst
...
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] START
carbonOS/pkgs-fonts-twemoji/????????-track.2231963.log
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] START
Tracking
https://kojipkgs.fedoraproject.org//packages/twitter-twemoji-fonts/14.0.2/3.fc38/noarch/twitter-twemoji-fonts-14.0.2-3.fc38.noarch.rpm
[00:00:00][????????][ track:pkgs/fonts/twemoji.bst ] SUCCESS
Tracking
https://kojipkgs.fedoraproject.org//packages/twitter-twemoji-fonts/14.0.2/3.fc38/noarch/twitter-twemoji-fonts-14.0.2-3.fc38.noarch.rpm
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] INFO Found
new revision: a842447e8610bce8a0564a91604a2988220285b5f60b0396440ead230ac6b1ec
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] START
Staging local files into CAS
[00:00:00][????????][ track:pkgs/fonts/twemoji.bst ] SUCCESS
Staging local files into CAS
[--:--:--][????????][ track:pkgs/fonts/twemoji.bst ] START
Tracking master from https://github.com/rpm-software-management/rpm
[00:00:01][????????][ track:pkgs/fonts/twemoji.bst ] SUCCESS
Tracking master from https://github.com/rpm-software-management/rpm
[00:00:02][????????][ track:pkgs/fonts/twemoji.bst ] SUCCESS
carbonOS/pkgs-fonts-twemoji/????????-track.2231963.log
[00:00:02][ ][ main:core activity ] SUCCESS Track
...
$ cat project.refs
projects:
carbonOS:
pkgs/fonts/twemoji.bst:
- ref: a842447e8610bce8a0564a91604a2988220285b5f60b0396440ead230ac6b1ec
- {}
- ref: 2043f2ac57ae2690cc1b11242f040085df8eed83
```
This issue is not exclusive to this one element. it also happens w/ others:
```
sources:
- kind: git_tag
url: github:systemd/zram-generator
track: main
exclude:
- v*-rc*
- v*-beta*
- kind: gen-cargo-lock
- kind: cargo
```
```
sources:
- kind: git_tag
url: github:cpuguy83/go-md2man
- kind: patch
path: patches/md2man/fix-go-ver.patch
- kind: go-vendor
```
```
sources:
- kind: tar
url:
https://static.crates.io/crates/gst-plugin-dav1d/gst-plugin-dav1d-0.10.0.crate
- kind: gen-cargo-lock
- kind: cargo
```
The `gen-cargo-lock` and `go-vendor` plugins are custom to carbonOS.
[Sources here](https://gitlab.com/carbonOS/build-meta/-/tree/35f91b6d/plugins).
I am not suspecting them, however, since another element that uses none of them
exhibits the same issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
