Sponsorship request: python-ping3
Hi, I ITP simplemonitor (#1016113), so I started with one of its dependencies (actually is a "soft" dependency, optional but better to have) (two more to come). So, I RFS for ping3: https://mentors.debian.net/package/python-ping3/ https://mentors.debian.net/debian/pool/main/p/python-ping3/python-ping3_4.0.4-1.dsc Also in: https://salsa.debian.org/python-team/packages/python-ping3 This is my first package for Debian. Reviewing only, or reviewing + sponsorship, are very appreciated. I'd like to get this one as right as possible to do the next Python3 packages as good as possible. If it suits anyone better: I'm cpina on freenode (#debian-python for example). Thank you very much for any advise! -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
Re: Sponsorship request: python-ping3
Hi,
On 20 Oct 2023 at 01:06:01, Pierre-Elliott Bécue wrote:
> Carles Pina i Estany wrote on 18/10/2023 at 01:56:46+0200:
> >> Tell me when you're fine with your work and I'll upload.
> >
> > To me, it can be uploaded :-)
> >
> > Let me know if I need or can do anything else.
>
> Uploaded, remember to put a tag on the latest commit. :)
Excellent!
Tagged via:
$ git tag -s debian/4.0.4-1
("Initial release" the commit message)
$ git push upstream debian/4.0.4-1
I've just realised that there is "gbp tag" :-/ I'll use it next time.
Reading the description in man gbp-tag: I was in the right branch and
the verifications would have been ok! (last commit in the debian
branch).
Thanks!
--
Carles Pina i Estany
https://carles.pina.cat
signature.asc
Description: PGP signature
Looking for a sponsor (python-cloudscraper)
Hi, I plan to package simplemonitor (https://simplemonitor.readthedocs.io/). I need to package two dependencies before. One is python-cloudscraper: https://mentors.debian.net/package/python-cloudscraper/ https://mentors.debian.net/debian/pool/main/p/python-cloudscraper/python-cloudscraper_1.2.69-1.dsc https://salsa.debian.org/python-team/packages/python-cloudscraper/ There were a bit of discussion regarding tests about 3 weeks ago, I think that the current situation should be as good as it can get for now. Thank you very much for any feedback, sponsorship, etc. PS: CCed a Pierre-Elliott who sponsored another package of mine, and Thomas Goirand who showed some interest. I think that both might be quite busy at the moment, I'm keen to move python-cloudscraper :-) , I'm happy to work with anyone who is happy :-) -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
SALSA_CI_LINTIAN_FAIL_WARNING / SALSA_CI_LINTIAN_ARGS in salsa-ci.yml
Hi, I created python-ping3 package a month ago. At some point I didn't see some lintian warnings from salsa. To avoid missing warnings again I added in salsa-ci.yml: variables: SALSA_CI_LINTIAN_FAIL_WARNING: 1 https://salsa.debian.org/python-team/packages/python-ping3/-/blob/debian/unstable/debian/salsa-ci.yml#L7 Is there any convention on doing this for python-team maintained packages? Also, I would be happy to have: SALSA_CI_LINTIAN_ARGS: --pedantic --fail-on pedantic (added there as well) IIRC, lintian pedantic's reports are shown in mentors.debian.net and I thought that would be good to have them in salsa - with the fail-on pedantic to avoid missing them. Any +1 / -1 of these options for python-team packages? Like "do not do this" or "if it works for you, feel free to do it". Thanks! -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
non-working debian/watch: create new version of package or not
Hi, I recently uploaded (via sponsor) python-ping3. The file debian/watch works locally but not in: https://qa.debian.org/cgi-bin/watch?pkg=python-ping3 The reason seems to be that the uscan version in the server is different to my local one: https://lists.debian.org/debian-qa/2023/10/msg00010.html When I've fixed it: should I upload the package to mentors.debian.net and request a re-upload (with a new version). Or just fix it, and no need to re-upload, until some other change in the package? (new upstream version, etc.). Thank you, -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)
Hi Thomas, list, On 02 Oct 2023 at 14:11:54, Thomas Goirand wrote: > On 10/2/23 09:11, Carles Pina i Estany wrote: > > > > Hi, > > > > I've uploaded my first package (python-cloudscraper). I've filled a RFS > > (#1053332). I have some questions (some specific to debian-python > > organisation) > > > > * Question 1: Git repo > > > > I pushed the code to > > https://salsa.debian.org/carlespina/python-cloudscraper/ . Should I, > > instead, push it already to > > https://salsa.debian.org/python-team/packages/python-cloudscraper/ ? > > Yes please. Done: https://salsa.debian.org/python-team/packages/python-cloudscraper/ (including the CI/CD setup) > > That might be related to Question 2. > > > > * Question 2: debian/control, Maintainers and Uploaders > > > > I've read > > https://salsa.debian.org/python-team/tools/python-modules/-/blob/master/policy.rst#maintainership > > and I think that my favourite, "long term" (does not need to be now), > > would be: > > > > Maintainer: Debian Python Team > > Uploader: Carles Pina i Estany > > Yes. Note that not everyone in the team agree with the Python team policy of > having the team as Uploader, some, including myself, think that if you don't > want other people from the team to upload, you shouldn't push the package to > the team at all. YMMV... Changed. I didn't want to add the team in "Maintainer:" before double checking with the team. I will do it straight away on the next package (python-pyaarlo, the other missing dependency of simplemonitor) I will create a new version of the package and upload it into mentors.debian.net when I finish this email. For reference: it will be the version 1.2.69-3. > > So far I've done: > > Maintainer: Carles Pina i Estany > > And no Uploader (will be the sponsor on the first time). > > > > Is that correct? > > It's probably preferable to directly put the team as Maintainer. :-) didn't want to impose it on my first attempt before seeing the dynamic. > > * Question 3: allow failing tests from upstream in dh_auto_test > > > > Upstream has 4 failing unit tests. Besides working with upstream to fix > > them what I've done is, in debian/rules: > > - > > override_dh_auto_test: > > # Disable tests failing from upstream > > pytest -k "not (test_bad_interpreter_js_challenge1_16_05_2020 or > > test_bad_solve_js_challenge1_16_05_2020 or > > test_Captcha_challenge_12_12_2019 or test_reCaptcha_providers)" > > - > > > > The reason is that I don't want to disable or even ignore failing unit > > tests in the salsa-ci pipeline. If there is a new one failing I'd like > > to know. The override_dh_auto_test is my way of having "allowed to fail" > > for a specific list. Is there any other, better / recommended / standard > > way? > > That's very good, and that's exactly what you should do, indeed: have as > many tests running as possible, so that you avoid regression. I would also > strongly suggest to use autopkgtest, to avoid that someone breaks your > package when uploading some of your dependencies. In my first version (not published) I wrote a simple autopkgtest with an "import cloudscraper" (I know that this is not fully testing everything but at least something!). Then I realised that it's not needed... without debian/tests it's just doing it: https://salsa.debian.org/python-team/packages/python-cloudscraper/-/jobs/4762103#L180 """ autopkgtest [22:24:36]: test autodep8-python3: set -e ; for py in $(py3versions -r 2>/dev/null) ; do cd "$AUTOPKGTEST_TMP" ; echo "Testing with $py:" ; $py -c "import cloudscraper; print(cloudscraper)" ; done """ I don't see this mentioned in "man autodep8": https://manpages.debian.org/testing/autodep8/autodep8.1.en.html (should it be there?) But it's done this way: https://salsa.debian.org/ci-team/autodep8/-/blob/master/support/python/generate#L78 > > * Question 4 > > > > Any casual comments on the package would be welcomed (even in a > > non-sponsor level). For sponsoring: the main reason of packaging this is > > that it's an indirect dependency of "simplemonitor". Related bugs: > > > > RFS: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053134 > > ITP: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053134 > > ITP of simplemonitor: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016113 > > Sorry, I wont have time for this right now, but if nobody does it, feel free > to ping me in a week or 2. Very appreciated, I take note. -- Carles Pina i Estany https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)
Hi, I've uploaded my first package (python-cloudscraper). I've filled a RFS (#1053332). I have some questions (some specific to debian-python organisation) * Question 1: Git repo I pushed the code to https://salsa.debian.org/carlespina/python-cloudscraper/ . Should I, instead, push it already to https://salsa.debian.org/python-team/packages/python-cloudscraper/ ? That might be related to Question 2. * Question 2: debian/control, Maintainers and Uploaders I've read https://salsa.debian.org/python-team/tools/python-modules/-/blob/master/policy.rst#maintainership and I think that my favourite, "long term" (does not need to be now), would be: Maintainer: Debian Python Team Uploader: Carles Pina i Estany So far I've done: Maintainer: Carles Pina i Estany And no Uploader (will be the sponsor on the first time). Is that correct? * Question 3: allow failing tests from upstream in dh_auto_test Upstream has 4 failing unit tests. Besides working with upstream to fix them what I've done is, in debian/rules: - override_dh_auto_test: # Disable tests failing from upstream pytest -k "not (test_bad_interpreter_js_challenge1_16_05_2020 or test_bad_solve_js_challenge1_16_05_2020 or test_Captcha_challenge_12_12_2019 or test_reCaptcha_providers)" - The reason is that I don't want to disable or even ignore failing unit tests in the salsa-ci pipeline. If there is a new one failing I'd like to know. The override_dh_auto_test is my way of having "allowed to fail" for a specific list. Is there any other, better / recommended / standard way? * Question 4 Any casual comments on the package would be welcomed (even in a non-sponsor level). For sponsoring: the main reason of packaging this is that it's an indirect dependency of "simplemonitor". Related bugs: RFS: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053134 ITP: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053134 ITP of simplemonitor: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016113 Thank you, -- Carles Pina i Estany https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
Debian Python Team membership
Hi, I'd like to join the team. The reason of wanting to join is that I started packaging simplemonitor (#1016113) and two missing dependencies: pyaarlo (#1053132) and cloudscraper (#1053134). My salsa login is carlespina I've read https://salsa.debian.org/python-team/tools/python-modules/blob/master/policy.rst and I accept it. Thank you very much, -- Carles Pina i Estany https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)
Hi Andrey, list, On 03 Oct 2023 at 12:25:27, Andrey Rakhmatullin wrote: > On Mon, Oct 02, 2023 at 11:32:31PM +0100, Carles Pina i Estany wrote: > > I will create a new version of the package and upload it into > > mentors.debian.net when I finish this email. For reference: it will be > > the version 1.2.69-3. > Note that the usual practice is not bumping Debian versions for packages > that were not uploaded yet and always uploading -1. Note taken, thanks! It will stay in -1 until uploaded. Makes it even easier. > > In my first version (not published) I wrote a simple autopkgtest with an > > "import cloudscraper" (I know that this is not fully testing everything > > but at least something!). Then I realised that it's not needed... > Yeah. > And you should try "Testsuite: autopkgtest-pkg-pybuild" instead, to run Ha! I didn't know of autopkgtest-pkg-pybuild. Thanks for this! > upstream tests. Though I think it won't see your explicit `pytest -k` and > you should replace the override with a PYBUILD_TEST_ARGS var. Done this way: https://salsa.debian.org/python-team/packages/python-cloudscraper/-/commit/78a83fbb0fe5fdfba78136921b919a11c8c9bc43 When I added it, the automatic simple test from autodep8 (importing the module) stopped being added. pytest, as run automatically when having "Testsuite: autopkgtest-pkg-pybuild": runs in the directory "/tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build" doing "python3.11 -m pytest -k ...". The contents of the directory via "ls -l" can be seen here: https://salsa.debian.org/carlespina/python-cloudscraper/-/jobs/4766353#L357) Because there is the sub-directory /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper : for what I can tell, pytest is running the tests with the code from /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper and not the installed package in /usr/lib/python3/dist-packages/cloudscraper/ . Generally speaking, I prefer to use the installed code (as done via the basic "import"). Once, in a similar situation, some files were shipped but not installed and the tests were running but not for the user. So I still added this: --- Test-Command: set -e ; cd "$AUTOPKGTEST_TMP" ; python3 -c "import cloudscraper" Depends: python3-cloudscraper Features: test-name=import-cloudscraper --- (simplified from the the original code done by autodep8) > > without debian/tests it's just doing it: > > > > https://salsa.debian.org/python-team/packages/python-cloudscraper/-/jobs/4762103#L180 > > """ > > autopkgtest [22:24:36]: test autodep8-python3: set -e ; for py in > > $(py3versions -r 2>/dev/null) ; do cd "$AUTOPKGTEST_TMP" ; echo "Testing > > with $py:" ; $py -c "import cloudscraper; print(cloudscraper)" ; done > > """ > > > > I don't see this mentioned in "man autodep8": > It says "each supported package type is tried against a set of heuristics, > based on packages names, build dependencies. specific files under debian/, > or a combination of those", or do you mean something else? I read it differently / expressed myself badly :-( TL;DR: I understood it now. I'm just explaining below what I had read and understood wrongly. In the section just above your text (AUTOMATIC USAGE BY AUTOPKGTEST) it says: "autodep8 can be automatically called by autopkgtest(1). To achieve that, you must set the Testsuite: field in the source package paragraph to autopkgtest-pkg-TYPE," Because debian/control didn't have the line "Testsuite: " I thought that the rest of the text didn't really apply and I didn't expect the automatic "import" to happen (which I liked, not complaining!). In the next section: "autodep8 will first look for Testsuite: autopkgtest-pkg-TYPE field in debian/control. if TYPE is a known package type, then that is used. If not, each supported package type is tried against a set of heuristics,"... I read (yesterday) "if TYPE is a known package type, then that is used. If is not a known package type, then the heuristics..." Basically I had understood that I needed "Testsuite: autopkgtest-pkg-SOMETHING" (where SOMETHING is a non-supported one) to enable the automatic heuristics. But that "Testsuite: " line was required to enable it. Anyway, all clear on this front! Thanks very much! -- Carles Pina i Estany https://carles.pina.cat
Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)
Hi Andrey, list On 03 Oct 2023 at 23:52:58, Carles Pina i Estany wrote: > On 03 Oct 2023 at 12:25:27, Andrey Rakhmatullin wrote: > > On Mon, Oct 02, 2023 at 11:32:31PM +0100, Carles Pina i Estany wrote: > > > I will create a new version of the package and upload it into > > > mentors.debian.net when I finish this email. For reference: it will be > > > the version 1.2.69-3. > > Note that the usual practice is not bumping Debian versions for packages > > that were not uploaded yet and always uploading -1. > > Note taken, thanks! It will stay in -1 until uploaded. Makes it even > easier. > > > > In my first version (not published) I wrote a simple autopkgtest with an > > > "import cloudscraper" (I know that this is not fully testing everything > > > but at least something!). Then I realised that it's not needed... > > Yeah. > > And you should try "Testsuite: autopkgtest-pkg-pybuild" instead, to run > > Ha! I didn't know of autopkgtest-pkg-pybuild. Thanks for this! > > > upstream tests. Though I think it won't see your explicit `pytest -k` and > > you should replace the override with a PYBUILD_TEST_ARGS var. > > Done this way: > https://salsa.debian.org/python-team/packages/python-cloudscraper/-/commit/78a83fbb0fe5fdfba78136921b919a11c8c9bc43 > > When I added it, the automatic simple test from autodep8 (importing the > module) stopped being added. > > pytest, as run automatically when having "Testsuite: > autopkgtest-pkg-pybuild": runs in the directory > "/tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build" doing > "python3.11 -m pytest -k ...". > > The contents of the directory via "ls -l" can be seen here: > https://salsa.debian.org/carlespina/python-cloudscraper/-/jobs/4766353#L357) > > Because there is the sub-directory > /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper > : for what I can tell, pytest is running the tests with the code from > /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper > and not the installed package in > /usr/lib/python3/dist-packages/cloudscraper/ . Generally speaking, I > prefer to use the installed code (as done via the basic "import"). > > Once, in a similar situation, some files were shipped but not installed > and the tests were running but not for the user. > > So I still added this: > --- > Test-Command: set -e ; cd "$AUTOPKGTEST_TMP" ; python3 -c "import > cloudscraper" > Depends: python3-cloudscraper > Features: test-name=import-cloudscraper > --- > (simplified from the the original code done by autodep8) Recap: pytest executed from "pybuild-autopkgtest", in the python-cloudscraper package, would use the src cloudscrapper instead of the installed one. So, to make sure that pytest uses the installed one, I added in debian/rules: - before-pybuild-autopkgtest: mv cloudscraper $(AUTOPKGTEST_TMP) after-pybuild-autopkgtest: mv $(AUTOPKGTEST_TMP) cloudscraper - https://salsa.debian.org/carlespina/python-cloudscraper/-/commit/c4ad00d05f9a7cf95c1c62d9fc791bf2cf0f222d Then I could delete (seems redundant now) what I did last night (debian/tests/control): - Test-Command: set -e ; cd "$AUTOPKGTEST_TMP" ; python3 -c "import cloudscraper" Depends: python3-cloudscraper Features: test-name=import-cloudscraper - Which was testing the import from the installed package. Any thoughts? -- Carles Pina i Estany https://carles.pina.cat
Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)
Hi,
On 05 Oct 2023 at 16:38:29, Andrey Rakhmatullin wrote:
> On Wed, Oct 04, 2023 at 07:58:11AM +0100, Carles Pina i Estany wrote:
> > Recap: pytest executed from "pybuild-autopkgtest", in the
> > python-cloudscraper package, would use the src cloudscrapper instead of
> > the installed one.
> >
> > So, to make sure that pytest uses the installed one, I added in
> > debian/rules:
> >
> > -
> > before-pybuild-autopkgtest:
> > mv cloudscraper $(AUTOPKGTEST_TMP)
> >
> > after-pybuild-autopkgtest:
> > mv $(AUTOPKGTEST_TMP) cloudscraper
> This looks very wrong, assuming it even runs.
Well, luckily $(AUTOPKGTEST_TMP) has no spaces :-) (I'll add " ")
Besides the lack of quotes: why does it look wrong? It's documented (or
this is how I understood it) in pybuild-autopkgtest(1), and it's
implemented in /usr/bin/pybuild-autopkgtest:
---
$ENV{PYBUILD_AUTOPKGTEST} = "1";
if (system("grep -q ^before-pybuild-autopkgtest: debian/rules") == 0) {
doit($run, "before-pybuild-autopkgtest");
}
doit($run, "pybuild-autopkgtest");
if (system("grep -q ^after-pybuild-autopkgtest: debian/rules") == 0) {
doit($run, "after-pybuild-autopkgtest");
}
---
Thanks!
--
Carles Pina i Estany
https://carles.pina.cat
Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)
Hi, On 05 Oct 2023 at 16:36:17, Andrey Rakhmatullin wrote: > On Tue, Oct 03, 2023 at 11:52:58PM +0100, Carles Pina i Estany wrote: > > > upstream tests. Though I think it won't see your explicit `pytest -k` and > > > you should replace the override with a PYBUILD_TEST_ARGS var. > > > > Done this way: > > https://salsa.debian.org/python-team/packages/python-cloudscraper/-/commit/78a83fbb0fe5fdfba78136921b919a11c8c9bc43 > Now you don't need to override dh_auto_test, it should call pytest with > correct args automatically. Thanks, deleted and tested: all good. Doing something late night I thought that PYBUILD_TEST_ARGS was only for the autopkgtest step and not for the pytest run after the build. > > pytest, as run automatically when having "Testsuite: > > autopkgtest-pkg-pybuild": runs in the directory > > "/tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build" doing > > "python3.11 -m pytest -k ...". > > > > The contents of the directory via "ls -l" can be seen here: > > https://salsa.debian.org/carlespina/python-cloudscraper/-/jobs/4766353#L357) > This gives 404. Apologies, for some reason, the settings of the repo was in Private. The relevant URL for the "ls -l" that I mentiond is here: https://salsa.debian.org/carlespina/python-cloudscraper/-/jobs/4766353#L190 This is how I generated that output: https://salsa.debian.org/carlespina/python-cloudscraper/-/commit/c8a01fca7bb8748da4529c57f2248db9ddb45bc7 > > Because there is the sub-directory > > /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper > It shouldn't be there. I think (and I see with one more package) that it's there if upstream created a directory named PACKAGE_NAME in the root of the repository (instead of doing src/PACKAGE_NAME). Which I think it's quite common, or at least I've seen it other times. I thought that I did something wrong with python-cloudscraper so I got another random one (recently modified, and with the same layout). I added these autopkgtests: -- Test-Command: set -e ; pwd ; ls -l . airspeed; cd "$AUTOPKGTEST_TMP" ; python3 -c "import airspeed ; print(airspeed.__file__)" Depends: python3-airspeed Features: test-name=import-airspeed-system Test-Command: set -e ; python3 -c "import airspeed ; print(airspeed.__file__)" Depends: python3-airspeed Features: test-name=import-airspeed-build -- (seen here: https://salsa.debian.org/carlespina/python-airspeed/-/commit/6ff20a68595be56101b63fbd4db720dee7c90ac9 ) The test with the "cd" to TMP: it imports the system wide airspeed: "/usr/lib/python3/dist-packages/airspeed/__init__.py" (see the output of the ls -l, airspeed, etc. and the airspeed.__file__ in https://salsa.debian.org/carlespina/python-airspeed/-/jobs/4773820#L277 ) The test without the "cd" it imports the bundled one: "/tmp/autopkgtest-lxc.yn973old/downtmp/build.9NT/src/airspeed/__init__.py" (seen in https://salsa.debian.org/carlespina/python-airspeed/-/jobs/4773820#L330) That's why I did my work around of moving "cloudscraper" build directory temporary before running autopkgtest. That lines in debian/rules: before-pybuild-autopkgtest: mv cloudscraper $(AUTOPKGTEST_TMP) after-pybuild-autopkgtest: mv $(AUTOPKGTEST_TMP) cloudscraper > > : for what I can tell, pytest is running the tests with the code from > > /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper > > and not the installed package in > > /usr/lib/python3/dist-packages/cloudscraper/ . Generally speaking, I > > prefer to use the installed code (as done via the basic "import"). > Sure, autopkgtests need to use the installed code. I'm afraid that for some packages layout this is not happening. I am happy to discuss it in another thread (I can open it). I think that, meanwhile, the workaround using "before-pybuild-autopkgtest" in debian/rules makes sense? If I missed, or might have missed something let me know and I will dig more... Thanks for all the feedback! -- Carles Pina i Estany https://carles.pina.cat
Re: Sponsorship request: python-ping3
Hi, On 18 Oct 2023 at 00:08:43, Pierre-Elliott Bécue wrote: > Carles Pina i Estany wrote on 17/10/2023 at 23:42:27+0200: > > > On 17 Oct 2023 at 22:13:05, Pierre-Elliott Bécue wrote: > >> Hi, > >> > >> Carles Pina i Estany wrote on 16/10/2023 at > >> 21:27:33+0200: > >> > >> LGTM. Just for DEP-14, you should have the main branch named > >> debian/unstable and not debian/master. > > > > Oops! I actually followed > > https://salsa.debian.org/python-team/tools/python-modules/blob/master/policy.rst#branch-names > > in the section "Branch names" and mentions "debian/master". Perhaps that > > should be updated? > > > > Anyway, thanks for changing it! > > I could be wrong, but ISTR that the branch name was debian/master for a > long time in the policy (dates back to 2016 at least?) > > I'm not sure that DEP-14 was providing a recommendation for sid branches > at that time. > > Anyway, I think that indeed we should offer in the policy a choice > between debian/unstable, debian/latest or debian/master although I'd not I actually really prefer debian/unstable to debian/master, so I'm happy how it is now in the ping3 package :-) -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
Re: Sponsorship request: python-ping3
Hi, On 17 Oct 2023 at 22:13:05, Pierre-Elliott Bécue wrote: > Hi, > > Carles Pina i Estany wrote on 16/10/2023 at 21:27:33+0200: > > > [[PGP Signed Part:No public key for A802884F60A55F81 created at > > 2023-10-16T21:27:33+0200 using RSA]] > > > > Hi, > > > > I ITP simplemonitor (#1016113), so I started with one of its > > dependencies (actually is a "soft" dependency, optional but better to > > have) (two more to come). > > > > So, I RFS for ping3: > > https://mentors.debian.net/package/python-ping3/ > > https://mentors.debian.net/debian/pool/main/p/python-ping3/python-ping3_4.0.4-1.dsc > > > > Also in: > > https://salsa.debian.org/python-team/packages/python-ping3 > > > > This is my first package for Debian. Reviewing only, or reviewing + > > sponsorship, are very appreciated. I'd like to get this one as right as > > possible to do the next Python3 packages as good as possible. > > > > If it suits anyone better: I'm cpina on freenode (#debian-python for > > example). > > > > Thank you very much for any advise! > > LGTM. Just for DEP-14, you should have the main branch named > debian/unstable and not debian/master. Oops! I actually followed https://salsa.debian.org/python-team/tools/python-modules/blob/master/policy.rst#branch-names in the section "Branch names" and mentions "debian/master". Perhaps that should be updated? Anyway, thanks for changing it! > I pushed a debian/unstable branch and modified gbp.conf. > > 1. Regarding packaging, lintian is happy and the files look good to > me. You can install devscripts and use wrap-and-sort to make some > things a bit more readable (IMHO). (have a look at devscripts in > general, it's resourceful) Thanks for showing wrap-and-sort! Note taken and I will look at other interesting things in devscripts. > 2. Regarding testing, this package is a bit a mess. First you probably > realized that you can't run tests at buildtime because a raw socket > requires root privilege. I see you designed custom autopkgtest to yep... [...] > From there you have two options: the first one is to drop the > Testsuite: field and keep the two tests you designed and call it a > day, or you drop it and write a third test stanza in > debian/tests/control with a shell script you'd also have to write > that moves the tests to the tmp dir autopkgtest creates, puts > localhost in /etc/hosts and then run tests. In that case you need > to add pytest to the dependencies of this test stanza. Sounds doable no problem, I'll try it this evening and see how it goes. > Tell me when you're fine with your work and I'll upload. thanks very much for the information, will let you know something. -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
Re: Sponsorship request: python-ping3
Hi, On 17 Oct 2023 at 22:42:27, Carles Pina i Estany wrote: [...] > > 2. Regarding testing, this package is a bit a mess. First you probably > > realized that you can't run tests at buildtime because a raw socket > > requires root privilege. I see you designed custom autopkgtest to > > yep... > > [...] > > > From there you have two options: the first one is to drop the > > Testsuite: field and keep the two tests you designed and call it a > > day, or you drop it and write a third test stanza in > > debian/tests/control with a shell script you'd also have to write > > that moves the tests to the tmp dir autopkgtest creates, puts > > localhost in /etc/hosts and then run tests. In that case you need > > to add pytest to the dependencies of this test stanza. > > Sounds doable no problem, I'll try it this evening and see how it goes. From doable to "I'm 99% sure that not possible". I cannot send pings from autopkgtest in salsa with any software. Side note: I remembered that when using "needs-internet": HTTP GET requests (even using hostnames) works. This is my first package for Debian, but I've used autopkgtest with needs-internet to run a sphinx "linkcheck" and it was working correctly. I've checked that no ICMP replies even using the standard ping binary from iputils-ping: - Test-Command: set +e ; ping -c 4 8.8.8.8 ; ping -c 4 example.com ; curl -s -I https://en.wikipedia.org ; curl -s -L https://en.wikipedia.org | head -5 Depends: python3-ping3, iputils-ping, curl Restrictions: needs-root, needs-internet Features: test-name=test-real-ping - That's in: https://salsa.debian.org/carlespina/python-ping3/-/blob/autopkgtest-connectivity/debian/tests/control The output: https://salsa.debian.org/carlespina/python-ping3/-/jobs/4822521#L213 4 packets transmitted, 0 received, 100% packet loss, time 3059ms 4 packets transmitted, 0 received, 100% packet loss, time 3079ms Even more: I think that curl -I (--head) (HTTP HEAD) might not work? but curl HTTP GET works. I'm not sure about the curl -I but I don't think it is relevant for this discussion. Somewhere, I think, I had read something that it implied that autopkgtest needs-internet was using a proxy? I cannot find it anyway and ICMP seems that cannot be used. So, for now, I've: -Used wrap-and-soft (excellent!). -Removed "set -e" in one of my Test-Commands: it's the default in autopkgtest (I discovered with the "pings..." and then documenation) -Fixed a cosmetic line in d/tests/control (s/features/Features) -Removed "Testsuite: autopkgtest-pkg-pybuild" from d/control because ICMP is not available anyway -Ran dch -r to update the date -"dput --force mentors python-ping3_4.0.4-1_amd64.changes" I've also discovered that there are a few unit tests in upstream that do not work. Some have an easy fix, I will do a MR of the fixes that I've done for some of them (separately, in GitHub, not tonight). > Your call. > > Tell me when you're fine with your work and I'll upload. To me, it can be uploaded :-) Let me know if I need or can do anything else. Thank you! -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
Re: What is the best method to build deb package?
Hi, On 19 Oct 2023 at 13:08:31, Miguel de Dios Matias wrote: > I have tiny side/pet projects in python. And they have setup.py the > build method and I publish to pypi. > > But I want to do some deb packages (for my pet projects), yes I can do > by hand (several years ago I do a debian package for PHP app). > > But What is the best method to build deb package? I look for in the I don't know poetry or its options. I don't know what's the "best" way, but I can tell how I start (and this is in a Debian system, gpg configured correctly and signing the packages which you might disable). Probably each person on the list has a different way? :-) I'm happy to read suggestions as well or tips. File .gbp.conf: [import-dsc] sign-tags = True [DEFAULT] pristine-tar = True debian-branch = debian/unstable mkdir debian_package_name cd debian_package_name git init git checkout -b upstream # I was creating a package for cloudscraper gbp import-orig https://github.com/VeNoMouS/cloudscraper/archive/refs/tags/1.2.68.tar.gz # answer the questions export DEBEMAIL=car...@pina.cat dh_make --createorig --packagename python-cloudscraper -p python-cloudscraper_1.2.68 Package type: "p" (python) Edit / delete files in debian/ (some are templates, some not-relevant for the package etc.) git add debian/* git commit -a -m "Add initial debian/* files" gbp buildpackage At the end it will run "lintian". Fix errors / warnings that you will see. But you might get *.dsc, *.deb in the parent directory where you are When you need to try again: debian/rules clean ; gbp buildpackage # probably there is some options in gbp to do this :-) And the rest that I have is to publish this in salsa, probably not relevant... I hope that it helps to have a start... at least with one of the tools. Cheers, -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
Re: Sponsorship request: python-ping3
Hi, On 20 Oct 2023 at 09:03:07, Santiago Ruano Rincón wrote: > El 18/10/23 a las 00:56, Carles Pina i Estany escribió: > > I've checked that no ICMP replies even using the standard ping binary > > from iputils-ping: > > - > > Test-Command: set +e ; ping -c 4 8.8.8.8 ; ping -c 4 example.com ; curl -s > > -I https://en.wikipedia.org ; curl -s -L https://en.wikipedia.org | head -5 > > Depends: python3-ping3, iputils-ping, curl > > Restrictions: needs-root, needs-internet > > Features: test-name=test-real-ping > > - > > That's in: > > https://salsa.debian.org/carlespina/python-ping3/-/blob/autopkgtest-connectivity/debian/tests/control > > > > The output: > > https://salsa.debian.org/carlespina/python-ping3/-/jobs/4822521#L213 > > > > 4 packets transmitted, 0 received, 100% packet loss, time 3059ms > > 4 packets transmitted, 0 received, 100% packet loss, time 3079ms > > ... > > AFAIU, there are network restrictions on salsa. Reaching HTTPS in the > outside world should work. But I am not sure about ICMP. > > FWIW, this is probably overengineering, but it is also possible to use > namespaces to avoid reaching INET, pinging from one namespace to the > other. > > As example: > https://salsa.debian.org/debian/isc-dhcp/-/blob/master/debian/tests/client-server I liked the idea of using namespaces! Thanks very much, I might use it in the future. Like you, I think that for python-ping3 unit tests this is overengineering (but I might play with this for python-ping3 or somehting else in the future...). In python-ping3, upstream has some IPs and hostnames hardcoded. The hostnames are easy to "tweak" via /etc/hosts (even to ping 127.0.0.1). The IPs: last night, I thought that another approach would be to use iptables (or nftables) and destination NAT. Redirect 8.8.8.8 to 127.0.0.1. It has the advantage that I could remove needs-internet, keeping all the unit tests 100% local. Cheers, -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)
Hi, I've investigated a bit more and found where I was confused. TL;DR: I'm deleting the non-needed workaround and found where things are happening. On 05 Oct 2023 at 16:36:17, Andrey Rakhmatullin wrote: > On Tue, Oct 03, 2023 at 11:52:58PM +0100, Carles Pina i Estany wrote: > > > upstream tests. Though I think it won't see your explicit `pytest -k` and > > > you should replace the override with a PYBUILD_TEST_ARGS var. > > > > Done this way: > > https://salsa.debian.org/python-team/packages/python-cloudscraper/-/commit/78a83fbb0fe5fdfba78136921b919a11c8c9bc43 > Now you don't need to override dh_auto_test, it should call pytest with > correct args automatically. > > > pytest, as run automatically when having "Testsuite: > > autopkgtest-pkg-pybuild": runs in the directory > > "/tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build" doing > > "python3.11 -m pytest -k ...". > > > > The contents of the directory via "ls -l" can be seen here: > > https://salsa.debian.org/carlespina/python-cloudscraper/-/jobs/4766353#L357) > This gives 404. > > > Because there is the sub-directory > > /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper > It shouldn't be there. They are not there if using all the automatic way. I was using at some point a manual invocation of pytest and then things are not as nice :-) So, using the automatic (generated via autodep8, etc.)...: autodep8 generates: Test-Command: pybuild-autopkgtest Depends: @, pybuild-plugin-autopkgtest, @builddeps@, Restrictions: allow-stderr, skippable, Features: test-name=pybuild-autopkgtest man pybuild-autopkgtest(1): pybuild-autopkgtest will run the tests in exactly the same way as pybuild will dur‐ ing the build, with the exception that the tests are not run in the build directory. The test files are copied to a temporary directory, so that the tests will run against the installed Python modules, and not against anything in the source tree. And yes, it happens in /usr/share/dh-python/dhpython/build/base.py So, all good as long as pybuild-autopkgtest is used (and not pytest invoked manually). I'm happy deleting the d/rules workaround. Cheers, -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
Re: Package testing with Salsa CI for new packages
Hi, Quick answer for now... On 19 Aug 2023 at 19:04:09, Paul Boddie wrote: > On Friday, 18 August 2023 19:54:55 CEST Carles Pina i Estany wrote: > > > > Ha! I wasn't aware of the aptly option > > (https://salsa.debian.org/salsa-ci-team/pipeline/#using-automatically-built-> > > apt-repository and SALSA_CI_DISABLE_APTLY=0). > > I think that I might have re-invented the wheel in a tiny part of > > Debusine CI/CD. > > It is certainly a way of propagating packages to those that might need > them. However, the instructions indicating how a package might access > these dependencies appear to be deficient. I haven't used aptly on salsa but on Monday I want to give it a try, to see if I can replace my solution... > It does not appear to be sufficient to merely specify the dependency > package repositories and mark them as trusted. Doing that will just > cause the repositories to be regarded as ignored and the GPG keys > signalled as unrecognised. > > So, the GPG keys need to be obtained. This is a hassle because something like > wget is needed to do that, and then apt has to be persuaded not to fail in an > opaque way. So the modified recipe is something like this: > > before_script: > - apt-get update > - NON_INTERACTIVE=1 apt-get install -y wget > - echo "deb https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/ > artifacts/raw/aptly unstable main" | tee /etc/apt/sources.list.d/pkga.list > ... > - wget -q -O /etc/apt/trusted.gpg.d/emeraldtree.asc https:// > salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/aptly/ > public-key.asc I'm ensure that this is correct. I have: - deb [arch=amd64 signed-by=/usr/share/keyrings/oracle-virtualbox-2016.gpg] https://download.virtualbox.org/virtualbox/debian bookworm contrib - So I thought that the "signed-by" was needed; and that the files that I have there are binary files (not .asc) of this type: carles@pinux:~$ file /etc/apt/trusted.gpg.d/skype.gpg /etc/apt/trusted.gpg.d/skype.gpg: OpenPGP Public Key Version 4, Created Wed Jun 22 09:36:35 2016, RSA (Encrypt or Sign, 2048 bits); User ID; Signature; OpenPGP Certificate carles@pinux:~$ I don't know if a .asc files are allowed. Actually, from my .bash_history: --- wget -O- https://www.virtualbox.org/download/oracle_vbox_2016.asc | sudo gpg --dearmor --yes --output /usr/share/keyrings/oracle-virtualbox-2016.gpg --- (I was following instructions, I didn't try leaving the .asc file there) > ... > - apt-get update > > This seems to make the various jobs happy, but the one that I was most > concerned with remains unhappy! I don't actually need the dependencies > for anything other than autopkgtest, but that job employs its own > environment where the above recipe has no effect. You could try using: --- variables: SALSA_CI_AUTOPKGTEST_ARGS: '--setup-commands=ci/setup-the-repo.sh' --- (and write and ship the ci/setup-the-repo.sh in the repo, do whatever you need there) Or, probably even better but less flexible: --- variables: SALSA_CI_AUTOPKGTEST_ARGS: '--add-apt-source="deb http://MIRROR SUITE COMPONENT' --- (I found add-apt-source via "man autopkgtest" in TEST BED SETUP OPTIONS. I haven't used add-apt-source, I don't know what happens with the gpg keys... but you could use [trusted=yes] :-| ) For the MIRROR you have salsa variables that might help if you need to specify the pipeline. > So, what I now need to do is to find out how I can make the new packages > available to autopkgtest specifically. We will get there! (one way or another) > > I will point out at some things that might safe some time to you > > (great) or not (ignore! :-) ): > > > > debusine's .gitlab-ci.yml: > > https://salsa.debian.org/freexian-team/debusine/-/blob/devel/.gitlab-ci.yml > > This is definitely useful for examples of the CI definition syntax and how > hidden jobs can be used to provide additional script fragments. > > [...] > > > The job autopkgtest, via debci command, runs autopkgtest: > > https://salsa.debian.org/freexian-team/debusine/-/jobs/4574458#L65 > > The difficult thing appears to be the configuration of this testing > environment. One approach might involve overriding the existing job > definitions to incorporate the injection of new packages. hopefully SALSA_CI_AUTOPKGTEST_ARGS will help you. I added this in salsa-ci/pipeline last year because I was trying to do a similar thing to what you are doing (I had to pin a package from backports). (I'm just a user of salsa-ci/pipeline, not a member of the team neither I can speak for them!) > [...] > > > It's possible for sure. Other people in this list might come up with a > > different idea. I don't have almost any experience with De
Re: Package testing with Salsa CI for new packages
Hi, On 20 Aug 2023 at 00:49:20, Paul Boddie wrote: > On Saturday, 19 August 2023 20:32:59 CEST Carles Pina i Estany wrote: > > > > Quick answer for now... > > And a very quick reply from me... :-) > > [...] > > > I don't know if a .asc files are allowed. Actually, from my > > .bash_history: > > --- > > wget -O- https://www.virtualbox.org/download/oracle_vbox_2016.asc | sudo gpg > > --dearmor --yes --output /usr/share/keyrings/oracle-virtualbox-2016.gpg --- > > > > (I was following instructions, I didn't try leaving the .asc file there) > > They are allowed in recent versions of apt, as confirmed by the man page. > > [...] Cool! I'll try next time (I cannot see it on my man apt-get, apt or sources.list on bookworm but I might need a newer apt) > > You could try using: > > --- > > variables: > > SALSA_CI_AUTOPKGTEST_ARGS: '--setup-commands=ci/setup-the-repo.sh' > > --- > > (and write and ship the ci/setup-the-repo.sh in the repo, do whatever > > you need there) > > This could be very useful. Also available (and also added last year for a similar case): - variables: SALSA_CI_PIUPARTS_PRE_INSTALL_SCRIPT: 'ci/pin-django-from-backports.sh' - (and SALSA_CI_PIUPARTS_POST_INSTALL_SCRIPT) > > Or, probably even better but less flexible: > > --- > > variables: > > SALSA_CI_AUTOPKGTEST_ARGS: '--add-apt-source="deb http://MIRROR SUITE > > COMPONENT' --- > > (I found add-apt-source via "man autopkgtest" in TEST BED SETUP OPTIONS. > > I haven't used add-apt-source, I don't know what happens with the gpg > > keys... but you could use [trusted=yes] :-| ) > > > > For the MIRROR you have salsa variables that might help if you need to > > specify the pipeline. > > So could this. > > > > So, what I now need to do is to find out how I can make the new packages > > > available to autopkgtest specifically. > > > > We will get there! (one way or another) > > Well, I have engineered something very inelegant, revealed in full here: > > https://salsa.debian.org/moin-team/moin/-/blob/debian/master/debian/salsa-ci.yml I, more or less, see what you did! Interesting approach, but if you could use the variables and ship the shell script it might reduce some duplication between jobs (if possible, I haven't look into too much detail in your case) and more importantly you might be able to use the standard "autopkgtest" or "piuparts" (instead of redefining them). Just last week I created a new autopkgtest (I have it running on bookworm and bullseye). It's done this way: -- autopkgtest-bullseye: extends: .test-autopkgtest variables: RELEASE: "bullseye-backports" SALSA_CI_AUTOPKGTEST_ARGS: '--setup-commands=ci/pin-django-from-backports.sh --skip-test=debusine-doc-linkcheck' -- Or, for more context: https://salsa.debian.org/freexian-team/debusine/-/blob/add-bullseye-support/.gitlab-ci.yml#L84 It ends up having "autopkgtest" and "autopkgtest-bullseye". But using the variable SALSA_CI_DISABLE_AUTOPKGTEST I could disable the one without my extends. [...] > > hopefully SALSA_CI_AUTOPKGTEST_ARGS will help you. I added this in > > salsa-ci/pipeline last year because I was trying to do a similar thing > > to what you are doing (I had to pin a package from backports). > > > > (I'm just a user of salsa-ci/pipeline, not a member of the team neither > > I can speak for them!) > > If this can simplify what I've done, which is really quite horrible, then I > will adopt it instead. The way that the artefacts of the dependencies are > bound up in specifically numbered jobs is also particularly unfortunate. Of If it's in the same pipeline: you don't need the numbers to get the artifacts (the .deb files from build). For the aptly generated artifact: I will investigate this tomorrow (to fetch it and try to use) (I want to replace some code that I did with the aptly repo, if possible). Cheers, -- Carles Pina i Estany https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
Re: Package testing with Salsa CI for new packages
Hi,
On 20 Aug 2023 at 23:16:57, Paul Boddie wrote:
> On Sunday, 20 August 2023 14:06:37 CEST Carles Pina i Estany wrote:
[...]
Thanks for sharing your path here, and I'm happy that you are reaching
your destination :-)
> autopkgtest:
> extends: .test-autopkgtest
> variables:
> SALSA_CI_AUTOPKGTEST_ARGS: '--setup-commands=debian/salsa/add-
> repositories.sh'
>
> piuparts:
> extends: .test-piuparts
> variables:
> SALSA_CI_PIUPARTS_PRE_INSTALL_SCRIPT: 'debian/salsa/add-repositories.sh'
>
> You can see that by defining the variables to customise the tools, I am able
> to work with the existing job definitions. This simplifies the CI description
> file considerably:
>
> https://salsa.debian.org/moin-team/moin/-/blob/debian/master/debian/salsa-ci.yml
>
> The script is pretty straightforward, too:
>
> https://salsa.debian.org/moin-team/moin/-/blob/debian/master/debian/salsa/add-repositories.sh
If you want, you can simplify more (it's not exactly the same, so it
might or might not help). There is a way on GitLab to point to the
latest build of a job. For example, you have the following URL for one
of the git repos:
https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/aptly
You could use instead (to avoid the pipeline number):
https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/debian/master/raw/aptly?job=aptly
Which is a redirect to the latest pipeline. Currently:
$ curl -s -I
"https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/debian/master/raw/aptly?job=aptly;
| grep -E -i "^(http|location)"
HTTP/2 302
location:
https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/aptly
Follows this format:
BRANCH=debian/master
DIRECTORY=aptly
JOB_NAME=aptly
https://salsa.debian.org/moin-team/emeraldtree/-/jobs/artifacts/${BRANCH}/raw/${DIRECTORY}?job=${JOB_NAME}
Just a side note: be careful about expiring artifacts. In some projects
(settings dependant) only the latest artifact is kept and older ones
might be expired (deleted) after some time. I don't think that this is
the case of the moin-team/emeraldtree after a quick check... but I'm
unsure where this is properly checked on GitLab.
[...]
Cheers,
--
Carles Pina i Estany
https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
Re: Package testing with Salsa CI for new packages
Hi, On 18 Aug 2023 at 13:16:15, Paul Boddie wrote: > On Friday, 18 August 2023 09:51:29 CEST Carles Pina i Estany wrote: > > > > I'm not a Debian developer but I have some experience on Salsa CI, so I > > thought that I might be able to help... but then I was confused by a > > specific part of the message: > > > > On 17 Aug 2023 at 17:10:08, Paul Boddie wrote: > > > > [...] > > > > > For another package I have been working on, the Salsa CI facility has > > > proven to be usable, configured using files in debian/test, particularly > > > as it allows test-related dependencies to be specified independently. > > > However, this other package has no dependencies that are currently > > > unpackaged in Debian. Meanwhile, the testing of this new Moin package > > > depends on brand new packages somehow being made available. > > > > If this dependencies are available on the "build" step: could they be > > made available on the autopkgtest? I didn't quite understand why this is > > not possible. I've found the autopkgtest quite flexible (since the tests > > are scripts that could prepare some environment) > > The package has dependencies on installation but these dependencies > are not strictly necessary when building. However, if I wanted to run I have the same situation on this side. Two solutions for two places. Debusine uses https://salsa.debian.org/salsa-ci-team/pipeline approach (extending it) > the test suite when building, I would indeed need to pull in these > dependencies as build dependencies so that the software being tested > can run without import errors. In our repo I have two solutions (or hacks, or workarounds) for this situation. AUTOPKGTEST TIME (after build) == In the autopkgtest job (from salsa-ci/pipeline), autopkgtest runs a shell script (below integration-tests-generic). Some of them set things up: -- Tests: integration-tests-generic Depends: debusine-client, debusine-server, debusine-worker, postgresql, redis-server, sudo, nginx Restrictions: allow-stderr, needs-root -- It could pull things from the internet (Pypi, git...) if it also has, in Restrictions: "needs-internet" Alternatively (and perhaps better, to keep things separated?), you could use salsa CI variable: SALSA_CI_AUTOPKGTEST_ARGS with this format: SALSA_CI_AUTOPKGTEST_ARGS: '--setup-commands=ci/pin-django-from-backports.sh Instead of pinning you could install packages. I don't know if it needs "needs-internet" in Restrictions. Note that the debusine-* packages are made available automatically thanks to Salsa CI pipeline and that tests-autopkgtest has: -- needs: - job: build artifacts: true -- So I don't need to do any repository. The build job just creates the *.deb packages (in a specific directory?), they are saved as artifacts and made available to "autopkgtest" job. UNIT-TESTS (before build) = We have a job before build doing (I'm simplifying): - unit-tests: stage: upstream-tests dependencies: [] script: - NON_INTERACTIVE=1 bin/quick-setup.sh install_packages - make coverage # to run the tests with coverage - In "bin/quick-setup.sh" it installs the required packages (some from Debian, some from Pypi if need to be) In here we execute tests before packaging them (because we are upstream as well). > I have to add that the other package I refer to has a test suite that takes a > long time to run, so that is another reason why I chose Salsa CI for that > package instead of letting autopkgtest do its work: Note that when I say "autopkgtest" above I always think of autopkgtest ran by Salsa CI. > One can imagine having a common storage area holding these newly Until now I see "the whole internet" as a "common storage area" (I know, not ideal! could be closer to the process, more robust, faster, etc.) > introduced packages that the CI scripts could access in preference to > the usual archives. In fact, this would be something that might also > affect existing packages. Consider the situation where fixes to a > dependency are required to fix functionality in a particular package. > One would have to wait for the fixed dependency to become integrated > into the unstable archive before the principal package's tests would > start to work again. Semi-offtopic: MMmmm... Debusine is the project that I mentioned before where I implemented different things in .gitlab-ci.yml. What you mentioned here is a thing that debusine, in the future (not impelmented yet) might help. But other people can talk better about this than me... and it's not available right now. > > I also created a repo and hosted it on a Salsa CI page fo
Re: Package testing with Salsa CI for new packages
Hi, I'm not a Debian developer but I have some experience on Salsa CI, so I thought that I might be able to help... but then I was confused by a specific part of the message: On 17 Aug 2023 at 17:10:08, Paul Boddie wrote: [...] > For another package I have been working on, the Salsa CI facility has proven > to be usable, configured using files in debian/test, particularly as it > allows > test-related dependencies to be specified independently. However, this other > package has no dependencies that are currently unpackaged in Debian. > Meanwhile, the testing of this new Moin package depends on brand new packages > somehow being made available. If this dependencies are available on the "build" step: could they be made available on the autopkgtest? I didn't quite understand why this is not possible. I've found the autopkgtest quite flexible (since the tests are scripts that could prepare some environment) > So, I would appreciate guidance on how I might enable testing of this new > Moin > package, along with its new dependencies, in the Salsa environment or any > other appropriate environment that would satisfy Debian packaging needs and > policies. It would also be quite helpful if built packages might be published > somewhere for people to test, but this is a separate consideration even if > such packages would obviously need to be generated as part of the testing > regime. For me, the packages on the build job are made available via a Salsa artifact automatically (and easy to download as a .zip of the *.deb). I think that this happens on all the "build" jobs on Salsa CI. They can be downloaded as a .zip file (e.g. https://salsa.debian.org/freexian-team/debusine/-/jobs/4564890, right hand side "Job artifacts" and click "Download"). Would that be enough? I also created a repo and hosted it on a Salsa CI page for internal testing but this is a bit of a workaround. This is in a new job but just download the artifacts (via a Salsa CI dependency) and run dpkg-scanpackages and copy the files to the right place). Cheers, -- Carles Pina i Estany https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
Re: Package testing with Salsa CI for new packages
Hi, On 18 Aug 2023 at 19:03:48, Paul Boddie wrote: > On Friday, 18 August 2023 16:12:19 CEST Carles Pina i Estany wrote: > > > > For the jobs it is happening via > > https://salsa.debian.org/salsa-ci-team/pipeline/#using-automatically-built- > apt-repository > > Reviewing this documentation is actually more helpful than I thought it would > be. I had noticed the "aptly" task in the YAML files, and I had started to > wonder if that meant that some kind of repository publishing was occurring > somewhere. Ha! I wasn't aware of the aptly option (https://salsa.debian.org/salsa-ci-team/pipeline/#using-automatically-built-apt-repository and SALSA_CI_DISABLE_APTLY=0). I think that I might have re-invented the wheel in a tiny part of Debusine CI/CD. I will point out at some things that might safe some time to you (great) or not (ignore! :-) ): debusine's .gitlab-ci.yml: https://salsa.debian.org/freexian-team/debusine/-/blob/devel/.gitlab-ci.yml Latest CI build: https://salsa.debian.org/freexian-team/debusine/-/pipelines/566796 In the "build" job: https://salsa.debian.org/freexian-team/debusine/-/jobs/4574451 You can see what's doing and it's using salsa-ci/pipeline's defaults in the build. All comes from https://salsa.debian.org/salsa-ci-team/pipeline/ , specifically from: https://salsa.debian.org/salsa-ci-team/pipeline/-/blob/master/salsa-ci.yml#L201 (I sometimes grep salsa-ci-team/pipeline for commands that I see in debusine's CI output to understand the flow...) In the autopkgtest job it sets things up and invokes autopkgtest: https://salsa.debian.org/freexian-team/debusine/-/jobs/4574458 The *.deb from build are made available via "artifacts" (and that autopkgtest needs the artifact of build). The job autopkgtest, via debci command, runs autopkgtest: https://salsa.debian.org/freexian-team/debusine/-/jobs/4574458#L65 And, to be honest, I didn't look into these details for long time. From my point of view I just write debian/tests/control with the tests (commands) to run: https://salsa.debian.org/freexian-team/debusine/-/blob/devel/debian/tests/control And that things run in a container/chroot (depends), etc. You can invoke autopkgtest locally as well... > > > In the Salsa CI environment, I would need to have the built packages > > > (found in the artefacts for each package's build job) copied somewhere > > > that can then be found by the Moin package's pipeline jobs and the > > > scripts creating a special repository of new packages. > > > > Archiving artifacts should happen automatically on the "build" step of > > Salsa CI (salsa-ci/pipeline). If I understand correctly what you > > wanted... > > I think you have a good understanding of what I am trying to achieve. If I > can > get the new package dependencies (emeraldtree, feedgen, and so on) to yield > installable packages when built that can then be referenced by Salsa CI as it > runs the build jobs for Moin, I have a chance of running the test suite. yep! > I'm currently persuading the CI system to run the "aptly" task and to publish > package repositories. I will then augment a customised YAML file for the Moin I might look more about this on Monday (I will write it down now) because I am using "dpkg-scanpackages" and I might be able to avoid it via SALSA_CI_DISABLE_APTLY: 0 I see the salsa-CI like a fancy environment where to run scripts, generate files, pass files between scripts, refactor scripts (in YAML in my .gitlab-ci.yml or to upstream salsa-ci/pipeline), scheduled jobs, etc. so almost anything that you can think of you can do it. Either with artifacts to keep and retrieve files between "jobs" or via GitHub Pages. If you need help creating a new job (a new bubble in the pipeline) let me know (even if you want off-list, if this is getting off-topic of debian-python). Or look at the https://salsa.debian.org/freexian-team/debusine/-/blob/devel/.gitlab-ci.yml where were created the stage "upstream-tests" with things like "code-linting". So You could even create dependencies of your task before the tests are executed and pass the .deb files via artifacts (or even repositories). > package with references to these repositories and see if the test > suite can be invoked somewhat more successfully as a consequence. It's possible for sure. Other people in this list might come up with a different idea. I don't have almost any experience with Debian packaging, but I have some experience on the salsa CI. So I might be giving you solutions that might be sub-optimal! :-) Cheers, -- Carles Pina i Estany https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
Re: Strangely rare pytest 7.x bug report
Hi,
I'm a lurker of debian-python@lists.debian.org but seeing Python+Qt I
wanted to have a look. I don't have a solution (I might look more
another time if time permits) but I might have something that might help
someone who knows the tools better.
I am not familiar with Python Debian packaging details/tools neither
with pytest :-( so take all of this with a pinch of salt.
If it helps the error comes from:
/usr/lib/python3.9/importlib/__init__.py in the functin "import_modules"
it does:
"""
if name.startswith('.'):
if not package:
msg = ("the 'package' argument is required to perform a relative "
"import for {!r}")
raise TypeError(msg.format(name))
"""
When the import fails the "name" parameter of "import_modules" function
is: '.pybuild.cpython3_3.9_qtpy.build.qtpy.tests' , which is derived
from the hidden dirctory ".pybuild" as created by default by "pybuild".
I think that the initial "." is used only as a directory name but Python
assumes that is a relative import requiring the package parameter.
Just to check my thoughts, and after running dpkg-buildpackage and
failing let's try again:
$ cd .pybuild/cpython3_3.9_qtpy/build; python3.9 -m pytest qtpy/tests ; cd -
Fails with the:
TypeError: the 'package' argument is required to perform a relative import for
'.pybuild.cpython3_3.9_qtpy.build.qtpy.tests'
/home/carles/git/python-qtpy
Then let's try to avoid the initial "." confusion:
$ mv .pybuild pybuild
$ cd pybuild/cpython3_3.9_qtpy/build; python3.9 -m pytest qtpy/tests ; cd -
It works.
I don't know why this is the only package affected by this though...
Hopefully it helps a bit!
On Jul/04/2022, Julian Gilbey wrote:
> Dear all,
>
> I wonder whether you might have any clue about
> https://bugs.debian.org/1013700
> I have mostly worked out the "cause" of the bug, but I haven't quite
> got to the bottom of it.
>
> When running the command
> PYTHONPATH=. python3.10 -m pytest qtpy/tests
> in the directory .pybuild/cpython3_3.10_qtpy/build, I get the error
> message:
>
> ImportError while loading conftest
> '/home/jdg/debian/spyder-packages/qtpy/build-area/python-qtpy-2.1.0/.pybuild/cpython3_3.10_qtpy/build/qtpy/tests/conftest.py'.
> TypeError: the 'package' argument is required to perform a relative import
> for '.pybuild.cpython3_3.10_qtpy.build.qtpy.tests'
>
> If the directory .pybuild is renamed to pybuild, the tests run without
> a problem. So there seems to be something funny about conftest.py
> (and removing all of the other files from the qtpy/tests directory
> except for the empty __init__.py gives the same error); here's a link
> to it:
>
> https://salsa.debian.org/python-team/packages/python-qtpy/-/blob/master/qtpy/tests/conftest.py
>
> But there doesn't seem to be anything out of the ordinary about this.
> So I am mystified: why does pytest 7.x seem to not give this error on
> any other Debian package?
>
> The only solution I currently have for this package is skip the tests
> at build time and rely on autopkgtest to do them.
>
> Best wishes,
>
>Julian
--
Carles Pina i Estany
https://carles.pina.cat
Re: Strangely rare pytest 7.x bug report
Hi Julian,
On Jul/04/2022, Julian Gilbey wrote:
> Hi Carles,
>
> Thanks for your thoughts! Yes, indeed that seems to be the issue.
> But what I don't understand is why the import is turned into
> .pybuild.cpython3_3.9_qtpy.build.qtpy.tests and not just qtpy.tests or
I see how pytest does it (but keep reading)
> a longer path, and why only this package fails in this way. Perhaps
> this is the only package that has an import statement in
> pytest_configure?
This I don't know and I'm curious, and it might help disecting the issue
(or understanding it). Do you know of any other python3 package that you
expected to fail? (using pytest in a similar way).
I might try to get both and follow what they do different (to hopefully
know what is python-qtpy doing different :-) )
I'm sure that there are tons of packages that use pytest :-) I'm
wondering if you had a good candidate.
Best regards,
>
> Best wishes,
>
>Julian
>
> On Mon, Jul 04, 2022 at 04:03:39PM +0100, Carles Pina i Estany wrote:
> >
> > Hi,
> >
> > I'm a lurker of debian-python@lists.debian.org but seeing Python+Qt I
> > wanted to have a look. I don't have a solution (I might look more
> > another time if time permits) but I might have something that might help
> > someone who knows the tools better.
> >
> > I am not familiar with Python Debian packaging details/tools neither
> > with pytest :-( so take all of this with a pinch of salt.
> >
> > If it helps the error comes from:
> > /usr/lib/python3.9/importlib/__init__.py in the functin "import_modules"
> > it does:
> > """
> > if name.startswith('.'):
> > if not package:
> > msg = ("the 'package' argument is required to perform a
> > relative "
> >"import for {!r}")
> > raise TypeError(msg.format(name))
> > """
> >
> > When the import fails the "name" parameter of "import_modules" function
> > is: '.pybuild.cpython3_3.9_qtpy.build.qtpy.tests' , which is derived
> > from the hidden dirctory ".pybuild" as created by default by "pybuild".
> >
> > I think that the initial "." is used only as a directory name but Python
> > assumes that is a relative import requiring the package parameter.
> >
> > Just to check my thoughts, and after running dpkg-buildpackage and
> > failing let's try again:
> >
> > $ cd .pybuild/cpython3_3.9_qtpy/build; python3.9 -m pytest qtpy/tests ; cd -
> > Fails with the:
> >
> > TypeError: the 'package' argument is required to perform a relative import
> > for '.pybuild.cpython3_3.9_qtpy.build.qtpy.tests'
> > /home/carles/git/python-qtpy
> >
> > Then let's try to avoid the initial "." confusion:
> >
> > $ mv .pybuild pybuild
> > $ cd pybuild/cpython3_3.9_qtpy/build; python3.9 -m pytest qtpy/tests ; cd -
> >
> > It works.
> >
> > I don't know why this is the only package affected by this though...
> >
> > Hopefully it helps a bit!
> >
> > On Jul/04/2022, Julian Gilbey wrote:
> > > Dear all,
> > >
> > > I wonder whether you might have any clue about
> > > https://bugs.debian.org/1013700
> > > I have mostly worked out the "cause" of the bug, but I haven't quite
> > > got to the bottom of it.
> > >
> > > When running the command
> > > PYTHONPATH=. python3.10 -m pytest qtpy/tests
> > > in the directory .pybuild/cpython3_3.10_qtpy/build, I get the error
> > > message:
> > >
> > > ImportError while loading conftest
> > > '/home/jdg/debian/spyder-packages/qtpy/build-area/python-qtpy-2.1.0/.pybuild/cpython3_3.10_qtpy/build/qtpy/tests/conftest.py'.
> > > TypeError: the 'package' argument is required to perform a relative
> > > import for '.pybuild.cpython3_3.10_qtpy.build.qtpy.tests'
> > >
> > > If the directory .pybuild is renamed to pybuild, the tests run without
> > > a problem. So there seems to be something funny about conftest.py
> > > (and removing all of the other files from the qtpy/tests directory
> > > except for the empty __init__.py gives the same error); here's a link
> > > to it:
> > >
> > > https://salsa.debian.org/python-team/packages/python-qtpy/-/blob/master/qtpy/tests/conftest.py
> > >
> > > But there doesn't seem to be anything out of the ordinary about this.
> > > So I am mystified: why does pytest 7.x seem to not give this error on
> > > any other Debian package?
> > >
> > > The only solution I currently have for this package is skip the tests
> > > at build time and rely on autopkgtest to do them.
> > >
> > > Best wishes,
> > >
> > >Julian
--
Carles Pina i Estany
https://carles.pina.cat
Re: Strangely rare pytest 7.x bug report
Hi,
On Jul/04/2022, Julian Gilbey wrote:
> It is utterly, utterly bizarre. But I think I've found the problem.
[...]
> Thanks for your help!
You're welcome!
Cheers,
>
> Best wishes,
>
>Julian
>
> On Mon, Jul 04, 2022 at 08:39:32PM +0100, Carles Pina i Estany wrote:
> >
> > Hi Julian,
> >
> > On Jul/04/2022, Julian Gilbey wrote:
> > > Hi Carles,
> > >
> > > Thanks for your thoughts! Yes, indeed that seems to be the issue.
> > > But what I don't understand is why the import is turned into
> > > .pybuild.cpython3_3.9_qtpy.build.qtpy.tests and not just qtpy.tests or
> >
> > I see how pytest does it (but keep reading)
> >
> > > a longer path, and why only this package fails in this way. Perhaps
> > > this is the only package that has an import statement in
> > > pytest_configure?
> >
> > This I don't know and I'm curious, and it might help disecting the issue
> > (or understanding it). Do you know of any other python3 package that you
> > expected to fail? (using pytest in a similar way).
> >
> > I might try to get both and follow what they do different (to hopefully
> > know what is python-qtpy doing different :-) )
> >
> > I'm sure that there are tons of packages that use pytest :-) I'm
> > wondering if you had a good candidate.
> >
> > Best regards,
> >
> > >
> > > Best wishes,
> > >
> > >Julian
> > >
> > > On Mon, Jul 04, 2022 at 04:03:39PM +0100, Carles Pina i Estany wrote:
> > > >
> > > > Hi,
> > > >
> > > > I'm a lurker of debian-python@lists.debian.org but seeing Python+Qt I
> > > > wanted to have a look. I don't have a solution (I might look more
> > > > another time if time permits) but I might have something that might help
> > > > someone who knows the tools better.
> > > >
> > > > I am not familiar with Python Debian packaging details/tools neither
> > > > with pytest :-( so take all of this with a pinch of salt.
> > > >
> > > > If it helps the error comes from:
> > > > /usr/lib/python3.9/importlib/__init__.py in the functin "import_modules"
> > > > it does:
> > > > """
> > > > if name.startswith('.'):
> > > > if not package:
> > > > msg = ("the 'package' argument is required to perform a
> > > > relative "
> > > >"import for {!r}")
> > > > raise TypeError(msg.format(name))
> > > > """
> > > >
> > > > When the import fails the "name" parameter of "import_modules" function
> > > > is: '.pybuild.cpython3_3.9_qtpy.build.qtpy.tests' , which is derived
> > > > from the hidden dirctory ".pybuild" as created by default by "pybuild".
> > > >
> > > > I think that the initial "." is used only as a directory name but Python
> > > > assumes that is a relative import requiring the package parameter.
> > > >
> > > > Just to check my thoughts, and after running dpkg-buildpackage and
> > > > failing let's try again:
> > > >
> > > > $ cd .pybuild/cpython3_3.9_qtpy/build; python3.9 -m pytest qtpy/tests ;
> > > > cd -
> > > > Fails with the:
> > > >
> > > > TypeError: the 'package' argument is required to perform a relative
> > > > import for '.pybuild.cpython3_3.9_qtpy.build.qtpy.tests'
> > > > /home/carles/git/python-qtpy
> > > >
> > > > Then let's try to avoid the initial "." confusion:
> > > >
> > > > $ mv .pybuild pybuild
> > > > $ cd pybuild/cpython3_3.9_qtpy/build; python3.9 -m pytest qtpy/tests ;
> > > > cd -
> > > >
> > > > It works.
> > > >
> > > > I don't know why this is the only package affected by this though...
> > > >
> > > > Hopefully it helps a bit!
> > > >
> > > > On Jul/04/2022, Julian Gilbey wrote:
> > > > > Dear all,
> > > > >
> > > > > I wonder whether you might have any clue about
> > > > > https://bugs.debian.org/1013700
> > > > > I have mostly worked out the "cause" of the bug, but I haven't quite
> > > > > got to the bottom of it.
> > > > >
&
Re: Failing autopkgtest for jupyter-notebook
Hi, On Aug/11/2022, julien.pu...@gmail.com wrote: > So far, so good, but I didn't unbreak the autopkg tests. Here is the > relevant part of the log: do you have a link to the full log on salsa and the repo? I'd like to verify something... Cheers, -- Carles Pina i Estany https://carles.pina.cat
Re: dh_python3: file in /usr/lib/python3.12/dist-packages ?
Hi,
During the "gbp buildpackage" (or "dpkg-buildpackage") I see in Salsa:
python3.11 -m build --skip-dependency-check --no-isolation --wheel
--outdir
/builds/python-team/packages/python-ping3/debian/output/source_dir/.pybuild/cpython3_3.11
[...]
python3.12 -m build --skip-dependency-check --no-isolation --wheel --outdir
/builds/python-team/packages/python-ping3/debian/output/source_dir/.pybuild/cpython3_3.12
The first one includes, in top_level.txt:
debian
ping3
And the second one:
build
debian
ping3
Where "ping3" is the expected module. "debian" is there because of the
debian/ directory (I'm super sure, and AFAIK should not be there!) and
"build" is there on the second time since, I guess, it exists at that
time.
So, even in the package in testing, it contains "debian" which is wrong:
$ cat ./dist-packages/ping3-4.0.4.dist-info/top_level.txt
debian
ping3
And in salsa it contains the difference, making it more obvious.
The file top_level.txt generated by "python3 -m build" in upstream
checkout does not contain "debian".
In my system, I see[1] more packages that might have the same problem.
So now, I guess that the question is:
a) How to make "python3 -m build" to not include "debian" or "build"?
b) What would be the best way to address this (if possible, via d/rules
changes). I mean, besides post-processing the generated file which I
guess that is possible, but not the best idea.
[1]: Via carles@pinux:/usr/lib/python3$ find . -iname "top_level.txt" -exec
grep -l ^debian$ {} \;
On 25 Feb 2024 at 23:32:09, Carles Pina i Estany wrote:
>
> Hi,
>
> On 25 Feb 2024 at 01:35:41, stefa...@debian.org wrote:
> > Hi Carles (2024.02.25_00:14:46_+)
> > > It generates a .deb file with a directory:
> > >
> > > /usr/lib/python3.12/dist-packages/ping3-4.0.4.dist-info/
> > >
> > > With two files there:
> > > -INSTALLER
> > > -top_level.txt
> >
> > If there are files in /usr/lib/python3.*/dist-packages/ after running
> > dh_python3, it means they differed between python 3.x versions.
>
> yes...
>
> > So to investigate, diff the files against the ones in
> > /usr/lib/python3/dist-packages/ and see if you can spot why they are not
> > matching.
>
> $ ls python3.12/dist-packages/ping3-4.0.4.dist-info/
> INSTALLER top_level.txt
>
> $ ls python3/dist-packages/ping3-4.0.4.dist-info/
> entry_points.txt INSTALLER METADATA top_level.txt WHEEL
>
> The file INSTALLER is the same:
>
> $ diff -u python3.12/dist-packages/ping3-4.0.4.dist-info/INSTALLER
> python3/dist-packages/ping3-4.0.4.dist-info/INSTALLER
>
> The file top_level.txt is different:
>
> $ diff -u python3.12/dist-packages/ping3-4.0.4.dist-info/top_level.txt
> python3/dist-packages/ping3-4.0.4.dist-info/top_level.txt
> --- python3.12/dist-packages/ping3-4.0.4.dist-info/top_level.txt
> 2023-11-06 22:53:00.0 +
> +++ python3/dist-packages/ping3-4.0.4.dist-info/top_level.txt 2023-11-06
> 22:53:00.0 +
> @@ -1,2 +1,3 @@
> +build
> debian
> ping3
>
> > It's probably something non-reproducible in the package's build system.
>
> Will investigate and if relevant share the problem here. This is in
> salsa using the standard pipeline. Last time that I run the pipeline I
> didn't have this problem. It was months ago.
>
> If you have any ideas let me know, of course!
>
> Thanks!
>
> --
> Carles Pina i Estany
> https://carles.pina.cat
--
Carles Pina i Estany
https://carles.pina.cat
signature.asc
Description: PGP signature
Re: dh_python3: file in /usr/lib/python3.12/dist-packages ?
Hi, On 25 Feb 2024 at 01:35:41, stefa...@debian.org wrote: > Hi Carles (2024.02.25_00:14:46_+) > > It generates a .deb file with a directory: > > > > /usr/lib/python3.12/dist-packages/ping3-4.0.4.dist-info/ > > > > With two files there: > > -INSTALLER > > -top_level.txt > > If there are files in /usr/lib/python3.*/dist-packages/ after running > dh_python3, it means they differed between python 3.x versions. yes... > So to investigate, diff the files against the ones in > /usr/lib/python3/dist-packages/ and see if you can spot why they are not > matching. $ ls python3.12/dist-packages/ping3-4.0.4.dist-info/ INSTALLER top_level.txt $ ls python3/dist-packages/ping3-4.0.4.dist-info/ entry_points.txt INSTALLER METADATA top_level.txt WHEEL The file INSTALLER is the same: $ diff -u python3.12/dist-packages/ping3-4.0.4.dist-info/INSTALLER python3/dist-packages/ping3-4.0.4.dist-info/INSTALLER The file top_level.txt is different: $ diff -u python3.12/dist-packages/ping3-4.0.4.dist-info/top_level.txt python3/dist-packages/ping3-4.0.4.dist-info/top_level.txt --- python3.12/dist-packages/ping3-4.0.4.dist-info/top_level.txt 2023-11-06 22:53:00.0 + +++ python3/dist-packages/ping3-4.0.4.dist-info/top_level.txt 2023-11-06 22:53:00.0 + @@ -1,2 +1,3 @@ +build debian ping3 > It's probably something non-reproducible in the package's build system. Will investigate and if relevant share the problem here. This is in salsa using the standard pipeline. Last time that I run the pipeline I didn't have this problem. It was months ago. If you have any ideas let me know, of course! Thanks! -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
dh_python3: file in /usr/lib/python3.12/dist-packages ?
Hi, CCed Stefano, since I've found a commit that might be relevant that he wrote. Via salsa and with dh_python3: D: dh_python3 dh_python3:179: version: 6.20231223 It generates a .deb file with a directory: /usr/lib/python3.12/dist-packages/ping3-4.0.4.dist-info/ With two files there: -INSTALLER -top_level.txt Causing lintian to complain via: - W: python3-ping3: python-module-in-wrong-location usr/lib/python3.12/dist-packages/ping3-4.0.4.dist-info -> usr/lib/python3/dist-packages/ping3-4.0.4.dist-info I: python3-ping3: package-contains-documentation-outside-usr-share-doc [usr/lib/python3.12/dist-packages/ping3-4.0.4.dist-info/top_level.txt] - I think that this was supposed to be not happening via the next changelog: https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/debian/changelog?ref_type=heads#L20 fix: https://salsa.debian.org/python-team/tools/dh-python/-/commit/c4bfdac892371c7fc3c1cc6a4c28e21b2dd384d9 I'm not familiar with dh_python3 internals at all. Has anyone had the same problem? Is it a bug in dh_python3, or should I do something else in the package that I'm packaging to avoid it? (this might be a red herring of something else that I've missed!). Thanks very much, -- Carles Pina i Estany https://carles.pina.cat signature.asc Description: PGP signature
