Re: Looking for feedback on a recent upload
Dear Andreas, My apologies for the regrettable git pushing. The debian mechanics is hard to decipher when you are not used to it. In all cases, many thanks for the additional clarification. I understand you are busy, so don't spend time reading / answering this mail. My answers for the list: Le 03/07/2018 à 11:59, Andreas Tille a écrit : > Hi Thomas, > > On Tue, Jul 03, 2018 at 10:18:19AM +0200, Thomas Schiex wrote: >> Thanks a lot, the uscan worked fine! Excited with this, I did this: >> >> - I integrated your commit in our upstream git repo (with thanks :-) > Why? Its fully sufficient in > >https://salsa.debian.org/science-team/toulbar2 > > since this is definitely no upstream change I'm pushing as much as I can in the upstream repo because I'd like the "debianizability" to be stored upstream for ease of management (it already was mostly there initially because we prepared our own .deb archives, without the terrible scrutiny of "lintian -EviI --pedantic"). >> - prepared a new GitHub release file including it > I do not understand this. Do you want to strip those files *generally* > from the upstream tarball? Than in turn you do not need the > Files-Exclude in debian/copyright since nothing needs exclusion any > more. Adding Files-exclude: in debian/copyright should not remove the excluded files from the source tarball that is produced by GitHub. So the tarball will still need "repacking" but the repacking "magic" will be stored upstream and the debian patching minimal (almost empty: just a one liner quilt patch). >> - gbp imported the tarball in an empty git repo and pushed it to salsa. > Why are you using an empty git repository? The workflow would have been > as follows: > > gbp clone g...@salsa.debian.org:science-team/toulbar2.git > cd toulbar2 > uscan --verbose --force-download > gbp import-orig --pristine-tar ../toulbar2_1.0.0+dfsg.orig.tar.xz > > Your action seems to have messed up the Git repository on Salsa. My apologies for this. I do not have a clear idea of who is using these repos when. I'm afraid the best course now that I messed up the salsa toulbar2 git repo is to redo it cleanly. I'm not a total git noob but not a pro either. Here is my plan: 0) I will wait for debian-science messages saying "No please don't do that" for 24 hours and then do the following (if no message arrives): I have a fresh clone of the upstream repo of toulbar2 in the directory gh-toulbar2. This includes the watch file with repacking that uscan likes and that Andreas provided. 1) Start form an empty git repo (mkdir deb-toulbar2; cd deb-toulbar2; git init; cd ..) 2) recover the "repacked archive prepared by uscan/watch magic from the upstream repo cd gh-toulbar2 uscan --verbose --force-download cd .. 3) import it in the empty git repo: cd deb-toulbar2 gbp import-orig --pristine-tar ../toulbar2_1.0.0+dfsg.orig.tar.xz 4) push it unconditionally on salsa: git remote add origin g...@salsa.debian.org:science-team/toulbar2.git git push --all --tags origin -f Hopefully this should lead to a correctly origanized and sanitized repo. Next time we have a new release out (let say version "ver"), it is still unclear to me whether I need to do: cd deb-toulbar2 uscan --verbose --force-download gbp import-orig --pristine-tar ../toulbar2_$ver+dfsg.orig.tar.xz git push --all --tags origin Or if the watch file will watch upstream and do all this on its own on salsa when a new higher version pops up. https://wiki.debian.org/debian/watch says it downloads but does not say if it imports. All comments welcome. Kind regards, Thomas
Re: Looking for feedback on a recent upload
Hi Thomas, On Tue, Jul 03, 2018 at 10:18:19AM +0200, Thomas Schiex wrote: > Thanks a lot, the uscan worked fine! Excited with this, I did this: > > - I integrated your commit in our upstream git repo (with thanks :-) Why? Its fully sufficient in https://salsa.debian.org/science-team/toulbar2 since this is definitely no upstream change > - prepared a new GitHub release file including it I do not understand this. Do you want to strip those files *generally* from the upstream tarball? Than in turn you do not need the Files-Exclude in debian/copyright since nothing needs exclusion any more. > - gbp imported the tarball in an empty git repo and pushed it to salsa. Why are you using an empty git repository? The workflow would have been as follows: gbp clone g...@salsa.debian.org:science-team/toulbar2.git cd toulbar2 uscan --verbose --force-download gbp import-orig --pristine-tar ../toulbar2_1.0.0+dfsg.orig.tar.xz Your action seems to have messed up the Git repository on Salsa. If I try $ gbp pull gbp:info: Fetching from default remote for each branch gbp:warning: Skipping non-fast forward of 'master' - use --force or update manually gbp:warning: Skipping non-fast forward of 'pristine-tar' - use --force or update manually gbp:warning: Skipping non-fast forward of 'upstream' - use --force or update manually Whatever you did - you are doing something very different than others. Sorry, I have no time to check further but your usage of empty git repositories sounds very suspicious. I have currently no time to track this down further. Hope this helps so far / somebody else will be able to help Andreas. -- http://fam-tille.de
Re: Looking for feedback on a recent upload
On Mon, Jul 02, 2018 at 11:02:32PM +0200, Thomas Schiex wrote: > Thanks Andreas. Didn't know such repack was possible (discovered > https://wiki.debian.org/BenFinney/software/repack following your mail). > > Looks like a perfect fit for what I need ! Please note thet this Wiki link is **different** than my suggested solution. My suggestion is based on https://wiki.debian.org/UscanEnhancements and from what I know the recommended way. It **should** work out of the box as I wrote (but I have not tested). Did you tried the uscan command I recommended? Kind regards Andreas. > Thomas > > > Le 02/07/2018 à 16:32, Andreas Tille a écrit : > > You need to repack the upstream source. I've commited an *untested* > > example to remove the win32 dir. Please do > > > >uscan --verbose --force-download > > > > and you get a tarball with version 1.0.0+dfsg which is lacking the > > win32 dir. You can exclude more stuff in Files-Excluded. > > > > Hope this helps > > > > Andreas. > -- http://fam-tille.de
Re: Looking for feedback on a recent upload
On Mon, Jul 02, 2018 at 01:21:11PM +0200, Thomas Schiex wrote: > > When I try to build the package, there is a series of messages: > > dpkg-source: warning: ignoring deletion of directory lib > dpkg-source: warning: ignoring deletion of directory lib/win32 > dpkg-source: warning: ignoring deletion of directory lib/win32/gmp > dpkg-source: warning: ignoring deletion of directory lib/win32/gmp/info > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info-4, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info-6, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info-5, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info-2, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info-3, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info-8, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info-1, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/info/gmp.info-7, use --include-removal to override > dpkg-source: warning: ignoring deletion of directory lib/win32/gmp/include > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/include/gmp.h, use --include-removal to override > dpkg-source: warning: ignoring deletion of directory lib/win32/gmp/lib > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/lib/libgmp.la, use --include-removal to override > dpkg-source: warning: ignoring deletion of file > lib/win32/gmp/lib/libgmp.a, use --include-removal to override > > which I would like to repair but do not know how. Google did not help. > > My problem is that the upstream tar ball contains a directory with > binary libraries for Windows with bad licencing info. I need to remove > them in the debian packaging (which is Ok for toulbar2 that does not use > this except on Windows). > > I did this with a "git rm -r lib"+commit in a branch deb/removeWin32lib > and merged in deb/integration. > > This needs to be done otherwise it raises: > license-problem-gfdl-invariants in lintian (critical). > > Is there a way to remove a directory for the debian branch ? You need to repack the upstream source. I've commited an *untested* example to remove the win32 dir. Please do uscan --verbose --force-download and you get a tarball with version 1.0.0+dfsg which is lacking the win32 dir. You can exclude more stuff in Files-Excluded. Hope this helps Andreas. -- http://fam-tille.de
Re: Looking for feedback on a recent upload
Lumin wrote on 29/06/2018: > 1. Git repository layout in question > >File debian/source/format writes "3.0 (quilt)", so the master >branch should hold the "packaging commits" instead of the >"upstream commits". By the way, please always keep the >"upstream" and "pristine-tar" branches up to date. In order >to achieve that, you can use this command when importing >a new upstream verison: > >│ >│ $ gbp import-orig --pristine-tar XXX_YYY.orig.tar.gz >│ Dear Lumin, I maintain a couple of package in debian-science using the classic workflow you describe, but I normally prefer the "pure git" workflow and the DEP14 branch layout. DEP14 clearly conflicts with the debian-science policy, but I'm not sure about the "pure git" approach. By "pure git" I basically mean having the upstream repository as a remote, fetching and merging new upstream versions via git, and using (possibly signed) tags instead of tarballs. It is still possible to maintain a pristine-tar branch, if desired, and uscan has a very handy ‘mode=git’ option (see for example [1]). Is this workflow to be considered incompatible debian-science's policy? Cheers, Paride [1] https://salsa.debian.org/debian-phototools-team/imv/blob/debian/sid/debian/watch
Re: Looking for feedback on a recent upload
Dear Lumin, I hope is still do not disturb. I'm coming back to you for one more advice. Here is point by point list of what I did with one question: > I indeed spotted something that doesn't match the debian-science > conventions: > > 1. Git repository layout in question > >File debian/source/format writes "3.0 (quilt)", so the master >branch should hold the "packaging commits" instead of the >"upstream commits". By the way, please always keep the >"upstream" and "pristine-tar" branches up to date. In order >to achieve that, you can use this command when importing >a new upstream verison: > >│ >│ $ gbp import-orig --pristine-tar XXX_YYY.orig.tar.gz >│ > >More details can be found in Debian Science Team Policy[1]. I created a new tarball on GitHub and used it to create a pristine-tar which I imported. > 2. "watch" file not working. > >We often use "uscan"[2] to conveniently obtain an tarball from upstream. >However the present watch file doesn't work anymore. Try to fix it >like this: > >│ >│ version=4 >│ > opts="filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%toulbar2-$1.tar.gz%" \ >│ https://github.com/toulbar2/toulbar2/tags \ >│ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian uupdate >│ > >I can obtain a tarball via this watch file. I used the watch file you provided. Thanks a lot. uscan does not complain. > 3. Reproducibility problem. > >With the upstream tarball downloaded by uscan. The package doesn't >build because the contents of git HEAD doesn't match with the >tarball. > >Actually this makes it harder for the others to review your work :-) This should be fine now since I follow the pristine-tar approach > 4. Policy version and debhelper compatibility level. > >debian/control: "Standards-Version: 3.9.8" >This version is quite old. Please check the policy check list [3] >and upgrade it. I upgraded to 4.1.4. Browse the policy page but didn't find any issue (I just swapped to lines in the control file to use the same order as in the policy, probably useless). >Apart from that, the compat level is 9. This is just a >recommend-to-fix. Have a look at manpage debhelper(7) and check >the "COMPATIBILITY LEVELS" part if you are interested in this. Upgraded to 10 > 5. Failed to build from source. > >I changed the source format to native[4] and tried to build it >in docker. However it failed to build. > >│ >│ Start 200: Phase2_10_1.bep_2 >│ 200/200 Test #200: Phase2_10_1.bep_2 .. > Passed0.01 sec >│ >│ 99% tests passed, 1 tests failed out of 200 >│ >│ Total Test time (real) = 43.65 sec >│ >│ The following tests FAILED: >│ 39 - Phase1_Toulbar_enum/latin0 (Failed) >│ Errors while running CTest >│ make[1]: *** [Makefile:155: test] Error 8 >│ make[1]: Leaving directory > '/home/whalebuilder/toulbar2/obj-x86_64-linux-gnu' >│ dh_auto_test: cd obj-x86_64-linux-gnu && make -j1 test ARGS\+=-j1 > returned exit code 2 >│ make: *** [debian/rules:3: build] Error 2 >│ dpkg-buildpackage: error: debian/rules build subprocess returned exit > status 2 >│ debuild: fatal error at line 1152: >│ dpkg-buildpackage -rfakeroot -us -uc -ui -j4 failed >│ > >Currently I have no idea about it at all. The problem is produced >from git HEAD[5] of the salsa repo. This was caused by a bug in the test system that depends on "bc" (binary calculator). Build dependency added in control. > 6. You have an RC bug. > >https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831148 > >It is FTBFS with GCC6. Since the default compiler is GCC7 now, >Does it reporduce with GCC-7? >If the bug doesn't exist anymore, you can close it with a changelog >entry like this: > >│ >│ * import upstream release 1.0.0 . (Closes: #XX) >│ Got it. It has been killed in upstream. The changelog now includes a "Close" for it. > 7. Please try to build it on Debian sid/unstable. > >Lintian didn't report anything about the standards version, which >indicates that your building system is Debian stable. It ships >an ancient lintian. > >A package can be built for Debian unstable with one of the following: > > Docker, sbuild, pbuilder, schroot > >I can see several lintian warnings and errors from the source package >check. Please run lintian like this: > >│ >│ $ lintian -EviI --pedantic XXX_YYY.changes >│ Tried with docker, Fixed various stuff. When I try to build the package, there is a series of messages: dpkg-source: warning: ignoring deletion of directory lib dpkg-source: warning: ignoring deletion of directory lib/win32 dpkg-source: warning: ignoring deletion of directory lib/win32/gmp dpkg-source: warning: ignoring deletion of directory lib/win32/gmp/info dpkg-source: warning: ignoring deletion of file lib/win32/gmp/info/gmp.info-4,
Re: Looking for feedback on a recent upload
Hi Thomas, Glad to see you came back to maintain this package. This package gives an overview of your package status in Debian: https://tracker.debian.org/pkg/toulbar2 > Would it be possible to have any feedback on this: > > - is it packaging fine on salsa (am I finished) ? I indeed spotted something that doesn't match the debian-science conventions: 1. Git repository layout in question File debian/source/format writes "3.0 (quilt)", so the master branch should hold the "packaging commits" instead of the "upstream commits". By the way, please always keep the "upstream" and "pristine-tar" branches up to date. In order to achieve that, you can use this command when importing a new upstream verison: │ │ $ gbp import-orig --pristine-tar XXX_YYY.orig.tar.gz │ More details can be found in Debian Science Team Policy[1]. 2. "watch" file not working. We often use "uscan"[2] to conveniently obtain an tarball from upstream. However the present watch file doesn't work anymore. Try to fix it like this: │ │ version=4 │ opts="filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%toulbar2-$1.tar.gz%" \ │ https://github.com/toulbar2/toulbar2/tags \ │ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian uupdate │ I can obtain a tarball via this watch file. 3. Reproducibility problem. With the upstream tarball downloaded by uscan. The package doesn't build because the contents of git HEAD doesn't match with the tarball. Actually this makes it harder for the others to review your work :-) 4. Policy version and debhelper compatibility level. debian/control: "Standards-Version: 3.9.8" This version is quite old. Please check the policy check list [3] and upgrade it. Apart from that, the compat level is 9. This is just a recommend-to-fix. Have a look at manpage debhelper(7) and check the "COMPATIBILITY LEVELS" part if you are interested in this. 5. Failed to build from source. I changed the source format to native[4] and tried to build it in docker. However it failed to build. │ │ Start 200: Phase2_10_1.bep_2 │ 200/200 Test #200: Phase2_10_1.bep_2 .. Passed0.01 sec │ │ 99% tests passed, 1 tests failed out of 200 │ │ Total Test time (real) = 43.65 sec │ │ The following tests FAILED: │ 39 - Phase1_Toulbar_enum/latin0 (Failed) │ Errors while running CTest │ make[1]: *** [Makefile:155: test] Error 8 │ make[1]: Leaving directory '/home/whalebuilder/toulbar2/obj-x86_64-linux-gnu' │ dh_auto_test: cd obj-x86_64-linux-gnu && make -j1 test ARGS\+=-j1 returned exit code 2 │ make: *** [debian/rules:3: build] Error 2 │ dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 │ debuild: fatal error at line 1152: │ dpkg-buildpackage -rfakeroot -us -uc -ui -j4 failed │ Currently I have no idea about it at all. The problem is produced from git HEAD[5] of the salsa repo. 6. You have an RC bug. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831148 It is FTBFS with GCC6. Since the default compiler is GCC7 now, Does it reporduce with GCC-7? If the bug doesn't exist anymore, you can close it with a changelog entry like this: │ │ * import upstream release 1.0.0 . (Closes: #XX) │ 7. Please try to build it on Debian sid/unstable. Lintian didn't report anything about the standards version, which indicates that your building system is Debian stable. It ships an ancient lintian. A package can be built for Debian unstable with one of the following: Docker, sbuild, pbuilder, schroot I can see several lintian warnings and errors from the source package check. Please run lintian like this: │ │ $ lintian -EviI --pedantic XXX_YYY.changes │ And fix those Errors and Warnings. > - can I test or have direct feedback on this by myself? 8. http://debomatic-amd64.debian.net/ If you ask, people in the list would be happy to help you upload if you don't have access to it. We can continue the discussion when a reproducible source tarball becomes available. [1] https://science-team.pages.debian.net/policy/ [2] "uscan" comes from "devscripts" package. [3] https://www.debian.org/doc/debian-policy/#document-upgrading-checklist [4] This is dirty hack. Don't do this. [5] 905ff12b91d39c4e690d232441b3a39d9d080b4f
