Bug#972371: debian-reference: Add CI to ensure that git is buildable
Control: tags -1 + pending Holger Wansing wrote: > Hi, > > xiao sheng wen (肖盛文) wrote: > > Good news! > > > > The newest git commit run Pipeline #187465 sucess!* > > Tagging this bug as pending. Hrrr, forgot the Control: line -- Holger Wansing PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Hi, xiao sheng wen (肖盛文) wrote: > Good news! > > The newest git commit run Pipeline #187465 sucess!* Tagging this bug as pending. Holger -- Holger Wansing PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Hi,Osamu: Thanks for your explanation. I get it. make entity is not run in CI pipeline now. 在 2020/10/19 下午10:03, Osamu Aoki 写道: H... the entity is somewhat fragile part. This is because it tends to organize package name etc. That's why we do not run this when we build package. Please consider this as a tool to make extracted database for the package related entity from Debian web site. I do it manually for every release. That's why I don't run this part of code when building package. I run this code manually and save the result. When I have time to fix some oddity with hackish work arounds, I update this data. It has more regrex so it is becoming more automated process but not a perfect one yet. For automated test, checking HTML build success for English is a good test. I also think testing for PO is good idea since I saw more broken data update in translation We tends to have more breakage on PDF building due to external tool chain issue. So skip testing the build of PDF to save CPU load. Osamu -- 肖盛文 xiao sheng wen Faris Xiao 微信(wechat)：atzlinux 《铜豌豆 Linux》 基于 Debian 的 Linux 中文桌面操作系统：https://www.atzlinux.com Debian QA page: https://qa.debian.org/developer.php?login=atzlinux%40sina.com GnuPG Public Key: 0x339240CB
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Hi, On Mon, 2020-10-19 at 07:35 +, Holger Wansing wrote: > Hi, > > Am Montag, 19. Oktober 2020 schrieb xiao sheng wen (肖盛文): > > run "make entity" also let us find any errors in the whole > > building > > process.(example is Bug#972437) > > > > entity tag also used very heavy in the project, if every commit > > can > > run"make entity"in pipelne, we will find any error immediately. H... the entity is somewhat fragile part. This is because it tends to organize package name etc. That's why we do not run this when we build package. Please consider this as a tool to make extracted database for the package related entity from Debian web site. > Hmm, maybe Osamu can comment on this, if there are any drawbacks > on running make entity everytime... I do it manually for every release. That's why I don't run this part of code when building package. I run this code manually and save the result. When I have time to fix some oddity with hackish work arounds, I update this data. It has more regrex so it is becoming more automated process but not a perfect one yet. For automated test, checking HTML build success for English is a good test. I also think testing for PO is good idea since I saw more broken data update in translation We tends to have more breakage on PDF building due to external tool chain issue. So skip testing the build of PDF to save CPU load. Osamu > Holger > > > > If "make entity" can success run in pipeline, the new version > > release > > can became more smoothly and automatically. > > > > As "make entity" can't run at present, let us comment it out at > > first. > > > > I hope "make entity" can run success in the further after that add > > > 'popcon.debian.org' into trusted.| > > > > 在 2020/10/19 上午12:59, Holger Wansing 写道: > > > Hi, > > > > > > I think it is not useful/not needed to execute "make entity" on > > > every build. It's only required when new packages are added to > > > the manual, or when preparing for the next stable release. > > > I would advise to remove it from the pipeline again... > > > > > > > > > Holger > > > > -- > > 肖盛文 xiao sheng wen Faris Xiao > > 微信(wechat)：atzlinux > > 《铜豌豆 Linux》 > > 基于 Debian 的 Linux 中文桌面操作系统：https://www.atzlinux.com > > Debian QA page: > > https://qa.debian.org/developer.php?login=atzlinux%40sina.com > > GnuPG Public Key: 0x339240CB > > > > > > -- > Sent from my Jolla phone > http://www.jolla.com/
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Hi, Am Montag, 19. Oktober 2020 schrieb xiao sheng wen (肖盛文): > run "make entity" also let us find any errors in the whole building > process.(example is Bug#972437) > > entity tag also used very heavy in the project, if every commit can > run"make entity"in pipelne, we will find any error immediately. Hmm, maybe Osamu can comment on this, if there are any drawbacks on running make entity everytime... Holger > > If "make entity" can success run in pipeline, the new version release > can became more smoothly and automatically. > > As "make entity" can't run at present, let us comment it out at first. > > I hope "make entity" can run success in the further after that add > |'popcon.debian.org' into trusted.| > > 在 2020/10/19 上午12:59, Holger Wansing 写道: > > Hi, > > > > I think it is not useful/not needed to execute "make entity" on > > every build. It's only required when new packages are added to > > the manual, or when preparing for the next stable release. > > I would advise to remove it from the pipeline again... > > > > > > Holger > > -- > 肖盛文 xiao sheng wen Faris Xiao > 微信(wechat)：atzlinux > 《铜豌豆 Linux》 > 基于 Debian 的 Linux 中文桌面操作系统：https://www.atzlinux.com > Debian QA page: https://qa.debian.org/developer.php?login=atzlinux%40sina.com > GnuPG Public Key: 0x339240CB > > -- Sent from my Jolla phone http://www.jolla.com/
Bug#972371: debian-reference: Add CI to ensure that git is buildable
在 2020/10/18 下午6:54, xiao sheng wen (肖盛文) 写道: when I run : make entity grep --binary-files=text -e '^Package: [a-z0-9][-+a-z0-9.]*[ 0-9]*$' < all-popcon-results.txt >all-popcon-pkgs.txt sed -n -e 's/^Submissions: *\([^ ]*\) *$/\1/p' < all-popcon-results.txt >all-popcon-submissions.txt # GENERATE bin/replace_package (sed script) echo "# This is autogenerated by Makefile. Update this with \"make rawxml\"" \ > bin/replace_package bin/genreplace emacs >> bin/replace_package *** emacs is missing *** The make process will stop here. There is a new bugreport for this problem: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=972437 This problem fixed in git now. https://salsa.debian.org/debian/debian-reference/-/commit/46b27cef6b8de97f346bc1e995672183dd71a2cc I use Debian 10.6 , I see CI pipeline also success in salsa. Perhaps It's a bug in some package in 10.6, the bug is resolved in testing. It's not a bug in 10.6.
Bug#972371: debian-reference: Add CI to ensure that git is buildable
run "make entity" also let us find any errors in the whole building process.(example is Bug#972437) entity tag also used very heavy in the project, if every commit can run"make entity"in pipelne, we will find any error immediately. If "make entity" can success run in pipeline, the new version release can became more smoothly and automatically. As "make entity" can't run at present, let us comment it out at first. I hope "make entity" can run success in the further after that add |'popcon.debian.org' into trusted.| 在 2020/10/19 上午12:59, Holger Wansing 写道: Hi, I think it is not useful/not needed to execute "make entity" on every build. It's only required when new packages are added to the manual, or when preparing for the next stable release. I would advise to remove it from the pipeline again... Holger -- 肖盛文 xiao sheng wen Faris Xiao 微信(wechat)：atzlinux 《铜豌豆 Linux》 基于 Debian 的 Linux 中文桌面操作系统：https://www.atzlinux.com Debian QA page: https://qa.debian.org/developer.php?login=atzlinux%40sina.com GnuPG Public Key: 0x339240CB
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Hi, Am Sonntag, 18. Oktober 2020 schrieb xiao sheng wen (肖盛文): > hi,Stuart, > > I had add make entity CI in pipeline, but meet the errors, please > help see: I think it is not useful/not needed to execute "make entity" on every build. It's only required when new packages are added to the manual, or when preparing for the next stable release. I would advise to remove it from the pipeline again... Holger > > https://salsa.debian.org/debian/debian-reference/-/jobs/1084031 > > Connecting to popcon.debian.org (popcon.debian.org)|209.87.16.63|:443... > connected. > | > ERROR: The certificate of 'popcon.debian.org' is not trusted. > ||ERROR: The certificate of 'popcon.debian.org' doesn't have a known > issuer.| > > Could you help to add |'popcon.debian.org' into trusted?| > > Thanks! > > xiao sheng wen > > -- Sent from my Jolla phone http://www.jolla.com/
Bug#972371: debian-reference: Add CI to ensure that git is buildable
hi,Stuart, I had add make entity CI in pipeline, but meet the errors, please help see: https://salsa.debian.org/debian/debian-reference/-/jobs/1084031 Connecting to popcon.debian.org (popcon.debian.org)|209.87.16.63|:443... connected. | ERROR: The certificate of 'popcon.debian.org' is not trusted. ||ERROR: The certificate of 'popcon.debian.org' doesn't have a known issuer.| Could you help to add |'popcon.debian.org' into trusted?| Thanks! xiao sheng wen
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Hi, xiao sheng wen (肖盛文) wrote: > Hi, > > After Holger and me do some git commit, the fr language still can't be > build in my local computer: > > make test "LANGPO=fr" has some error: It works fine here. The errors you are seing are all regarding "pop-xxxyyy" entities; those are generated with "make entity" (if that has not been done some time before). Does this fix the problem for you? Holger > > [...] > > ``` > > mkdir -p /home/atzlinux/devel/offical/debian-reference/tmp > xsltproc --novalid --nonet --xinclude --stringparam base.dir > /home/atzlinux/devel/offical/debian-reference/tmp/ \ > --stringparam html.ext .fr.html \ > xslt/style-html.xsl debian-reference.fr.xml > debian-reference.fr.xml:1757: parser error : Entity 'pop-emacs' not defined > url="http://qa.debian.org/popcon-graph.php?packages=@emacs@;> > ^ > debian-reference.fr.xml:28760: parser error : Entity 'pop-libphonon' not > defined > url="http://qa.debian.org/popcon-graph.php?packages=@libphonon@;> > ^ > debian-reference.fr.xml:28760: parser error : Detected an entity > reference loop > url="http://qa.debian.org/popcon-graph.php?packages=@libphonon@;> > ^ > debian-reference.fr.xml:38864: parser error : Entity 'pop-libpopplerjf' > not defined > "http://qa.debian.org/popcon-graph.php?packages=libpoppler95;> > ^ > debian-reference.fr.xml:38864: parser error : Detected an entity > reference loop > "http://qa.debian.org/popcon-graph.php?packages=libpoppler95;> > ^ > unable to parse debian-reference.fr.xml > make: *** > [Makefile:383：/home/atzlinux/devel/offical/debian-reference/tmp/index.fr.html] > > ``` > > > make test "LANGPO=zh-cn" also has some error output,but It can success > get the html files. > > ``` > > debian-reference.zh-cn.xml:1528: parser error : Entity 'pop-emacs' not > defined > url="http://qa.debian.org/popcon-graph.php?packages=@emacs@;> > ^ > debian-reference.zh-cn.xml:25146: parser error : Entity 'pop-libphonon' > not defined > url="http://qa.debian.org/popcon-graph.php?packages=@libphonon@;> > ^ > debian-reference.zh-cn.xml:34227: parser error : Entity > 'pop-libpopplerjf' not defined > "http://qa.debian.org/popcon-graph.php?packages=libpoppler95;> > ``` > > We need more work to fix these error output at first,then we can use the > CI pipeline. > > Thanks for Stuart! > > 在 2020/10/18 上午10:13, Stuart Prescott 写道: > > make > > wrap; make po; make test > > -- > 肖盛文 xiao sheng wen Faris Xiao > 微信(wechat)：atzlinux > 《铜豌豆 Linux》 > 基于 Debian 的 Linux 中文桌面操作系统：https://www.atzlinux.com > Debian QA page: https://qa.debian.org/developer.php?login=atzlinux%40sina.com > GnuPG Public Key: 0x339240CB > -- Holger Wansing PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Bug#972371: debian-reference: Add CI to ensure that git is buildable
在 2020/10/18 下午3:43, Holger Wansing 写道: It works fine here. The errors you are seing are all regarding "pop-xxxyyy" entities; those are generated with "make entity" (if that has not been done some time before). Does this fix the problem for you? Holger when I run : make entity grep --binary-files=text -e '^Package: [a-z0-9][-+a-z0-9.]*[ 0-9]*$' < all-popcon-results.txt >all-popcon-pkgs.txt sed -n -e 's/^Submissions: *\([^ ]*\) *$/\1/p' < all-popcon-results.txt >all-popcon-submissions.txt # GENERATE bin/replace_package (sed script) echo "# This is autogenerated by Makefile. Update this with \"make rawxml\"" \ > bin/replace_package bin/genreplace emacs >> bin/replace_package *** emacs is missing *** The make process will stop here. I use Debian 10.6 , I see CI pipeline also success in salsa. Perhaps It's a bug in some package in 10.6, the bug is resolved in testing.
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Good news! The newest git commit run Pipeline #187465 sucess!* * https://salsa.debian.org/debian/debian-reference/-/jobs/1083139 haha! :-D 在 2020/10/18 下午1:33, xiao sheng wen (肖盛文) 写道: Hi, After Holger and me do some git commit, the fr language still can't be build in my local computer: make test "LANGPO=fr" has some error: [...] -- 肖盛文 xiao sheng wen Faris Xiao 微信(wechat)：atzlinux 《铜豌豆 Linux》 基于 Debian 的 Linux 中文桌面操作系统：https://www.atzlinux.com Debian QA page: https://qa.debian.org/developer.php?login=atzlinux%40sina.com GnuPG Public Key: 0x339240CB
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Hi, After Holger and me do some git commit, the fr language still can't be build in my local computer: make test "LANGPO=fr" has some error: [...] ``` mkdir -p /home/atzlinux/devel/offical/debian-reference/tmp xsltproc --novalid --nonet --xinclude --stringparam base.dir /home/atzlinux/devel/offical/debian-reference/tmp/ \ --stringparam html.ext .fr.html \ xslt/style-html.xsl debian-reference.fr.xml debian-reference.fr.xml:1757: parser error : Entity 'pop-emacs' not defined url="http://qa.debian.org/popcon-graph.php?packages=@emacs@;> ^ debian-reference.fr.xml:28760: parser error : Entity 'pop-libphonon' not defined url="http://qa.debian.org/popcon-graph.php?packages=@libphonon@;> ^ debian-reference.fr.xml:28760: parser error : Detected an entity reference loop url="http://qa.debian.org/popcon-graph.php?packages=@libphonon@;> ^ debian-reference.fr.xml:38864: parser error : Entity 'pop-libpopplerjf' not defined "http://qa.debian.org/popcon-graph.php?packages=libpoppler95;> ^ debian-reference.fr.xml:38864: parser error : Detected an entity reference loop "http://qa.debian.org/popcon-graph.php?packages=libpoppler95;> ^ unable to parse debian-reference.fr.xml make: *** [Makefile:383：/home/atzlinux/devel/offical/debian-reference/tmp/index.fr.html] ``` make test "LANGPO=zh-cn" also has some error output,but It can success get the html files. ``` debian-reference.zh-cn.xml:1528: parser error : Entity 'pop-emacs' not defined url="http://qa.debian.org/popcon-graph.php?packages=@emacs@;> ^ debian-reference.zh-cn.xml:25146: parser error : Entity 'pop-libphonon' not defined url="http://qa.debian.org/popcon-graph.php?packages=@libphonon@;> ^ debian-reference.zh-cn.xml:34227: parser error : Entity 'pop-libpopplerjf' not defined "http://qa.debian.org/popcon-graph.php?packages=libpoppler95;> ``` We need more work to fix these error output at first,then we can use the CI pipeline. Thanks for Stuart! 在 2020/10/18 上午10:13, Stuart Prescott 写道: make wrap; make po; make test -- 肖盛文 xiao sheng wen Faris Xiao 微信(wechat)：atzlinux 《铜豌豆 Linux》 基于 Debian 的 Linux 中文桌面操作系统：https://www.atzlinux.com Debian QA page: https://qa.debian.org/developer.php?login=atzlinux%40sina.com GnuPG Public Key: 0x339240CB
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Control: tags -1 + patch Hi Holger thanks for the quick fix to the po files Here's a simple CI pipeline to test that the po files are usable using "make wrap; make po; make test". https://salsa.debian.org/debian/debian-reference/-/merge_requests/5 It could be extended to also try building the PDFs, to publish the latest version of the files on pages.debian.net, or to publish translation statistics for translated/fuzzy/missing strings. Note that until the tools art ported to Python 3 (see #942942) that the CI will fail, as the necessary dependencies aren't even installable at present. regards Stuart -- Stuart Prescotthttp://www.nanonanonano.net/ stu...@nanonanonano.net Debian Developer http://www.debian.org/ stu...@debian.org GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Hi,
Stuart Prescott wrote:
> The repo currently contains committed git conflict markers which are not
> valid .po syntax:
>
> #. type: Content of:
> #, no-wrap
> msgid ""
> "$ zcat \"/usr/share/doc/base-passwd/users-and-groups.txt.gz\" |
> sensible-pager\n"
> "$ sensible-browser \"/usr/share/doc/base-passwd/users-and-groups.html\"\n"
> "$ sensible-browser \"https://www.debian.org\";
> msgstr ""
> <<< HEAD
> "$ zcat \"/usr/share/doc/base-passwd/users-and-groups.txt.gz\" |
> sensible-pager\n"
> ===
> "$ zcat \"/usr/share/doc/base-passwd/users-and-groups.txt.gz\" | sensible-"
> "pager\n"
> >>> weblate
> "$ sensible-browser \"/usr/share/doc/base-passwd/users-and-groups.html\"\n"
> "$ sensible-browser \"https://www.debian.org\";
>
>
> At present:
>
> $ grep -c -e '<<< HEAD' po/{es,ja}.po
> po/es.po:6
> po/ja.po:13
I have fixed the po files es.po and ja.po now.
And pt also failed to build, due to several problems. Also fixed.
debian-reference now builds fine again here.
Adding CI for salsa to be done...
Holger
--
Holger Wansing
PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Bug#972371: debian-reference: Add CI to ensure that git is buildable
Source: debian-reference
Version: 2.76
Severity: important
Dear Maintainer,
While looking at #942942, I needed to be able to build the debian-reference
package to check that my changes made sense. Unfortunately, the git 'master'
branch is not currently buildable as many po syntax errors have been
committed to git.
The repo currently contains committed git conflict markers which are not
valid .po syntax:
#. type: Content of:
#, no-wrap
msgid ""
"$ zcat \"/usr/share/doc/base-passwd/users-and-groups.txt.gz\" |
sensible-pager\n"
"$ sensible-browser \"/usr/share/doc/base-passwd/users-and-groups.html\"\n"
"$ sensible-browser \"https://www.debian.org\";
msgstr ""
<<< HEAD
"$ zcat \"/usr/share/doc/base-passwd/users-and-groups.txt.gz\" |
sensible-pager\n"
===
"$ zcat \"/usr/share/doc/base-passwd/users-and-groups.txt.gz\" | sensible-"
"pager\n"
>>> weblate
"$ sensible-browser \"/usr/share/doc/base-passwd/users-and-groups.html\"\n"
"$ sensible-browser \"https://www.debian.org\";
At present:
$ grep -c -e '<<< HEAD' po/{es,ja}.po
po/es.po:6
po/ja.po:13
Of course there are other sorts of syntax errors beyond git conflict markers
that could be added to git. The gitlab repo on salsa.debian.org could perform
CI builds of the debian-reference package (or even just run a checker over the
.po files) to ensure that the repo is in good shape, and to flag when
mistakes have crept in.
regards
Stuart
