Re: Contacting DPT

[Andrey Rakhmatullin]

On Thu, Jun 06, 2024 at 05:40:15PM +0200, Andreas Tille wrote:
> I'd like to officially contact all our teams to learn about potential
> issues that might affect your work.  I would love to learn how you
> organise / share your workload.
I don't think we do, I think it's all ad-hoc. There are some bug lists
in the IRC channel topic.

> If you do some regular meetings - be it on IRC, video conference or
> whatever 
We do not.

> I have some specific questions to the Debian Python team.  I'm really
> interested into individual answers from single contributors either here
> on this list or in private.
> 
>   - Do you feel good when doing your work in Debian Python team?
Yes.

>   - Do you consider the workload of your team equally shared amongst its
> members?
No idea if this question makes sense. I think most people just work on
their own packages.

>   - Do you have some strategy to gather new contributors for your team?
No.

>   - Can you give some individual estimation how many hours per week you
> are working on your tasks in youre team?  Does this fit the amount of
> time you can really afford for this task?
<1

>   - In the beginning of this year there was a change in the policy of DPT.
> I'd like to hear your opinion about:
>  * The process how it went (possibly with suggestions to do better)
>  * The final result after a couple of months
> (I'm specifically interested also in the opinion of people who were
>  not happy about the change.)
It showed yet again that with any change, and equally lack of change, in
Debian there is a possibility that some people will resign.

>   - Since a long time we try to migrate from Python3.11 to Python3.12.
>  * What are your thoughts about the transition process?
It's not transparent. 

>  * Can you identify some blockers?
I have no idea.
I know many RC bugs are open but I don't know if any of them are blockers
and not things to be removed from testing.

>  * Do you have some suggestions for enhancements of this process?
If DPT is expected to be involved besides looking at the RC bug list at
the IRC channel topic, I would like to know how.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: python devs complaining about debian packaging

[Andrey Rakhmatullin]

I'll just say that the equal amount of FUD and ranting can be easily
generated abotu Debian or even just about Debian Python packaging.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: zlib missing issue - bookworm

[Andrey Rakhmatullin]

On Fri, May 10, 2024 at 11:59:32AM -0400, mike wrote:
> Anyone know how to fix this.  I have tried purging and reinstalling python3, 
> no luck with that.
This isn't a good place for asking about locally-built Python
installations.

>   File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
  ^^

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Request to join the Debian Python Team

[Andrey Rakhmatullin]

On Wed, Apr 03, 2024 at 05:21:15PM +0200, Paul Boddie wrote:
> I was just looking at the Wiki documentation for this and I think there might 
> need to be some updates and clarifications. For example, on this page:
> 
> https://wiki.debian.org/Teams/PythonTeam/HowToJoin
> 
> Here, it mentions the Maintainers and Uploaders fields and a more 
> comprehensive policy than in the Salsa-hosted policy document. I assume that 
> this policy is still applicable, but I wonder how it interacts with people 
> like me who do not have Debian Developer status and presumably cannot be an 
> uploader.
The Maintainer/Uploaders fields list maintainers, not people who literally
upload this package. See
https://www.debian.org/doc/debian-policy/ch-controlfields.html#maintainer

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Request to join the Debian Python Team

[Andrey Rakhmatullin]

On Wed, Apr 03, 2024 at 04:48:44PM +0200, Paul Boddie wrote:
> On Wednesday, 3 April 2024 16:25:10 CEST Andrey Rakhmatullin wrote:
> > On Wed, Apr 03, 2024 at 04:21:05PM +0200, Paul Boddie wrote:
> > > 
> > > Many thanks for giving me access! Would it make sense to move the existing
> > > project into the Python Team's packages collection on Salsa, or is that
> > > only permitted for packages that are actually adopted by the team?
> > 
> > If you are going to maintain this package in the team then it can and
> > should be in the team namespace, or are you asking about something else?
> 
> My aim is to maintain it in the team, yes. 
Then putting the repo in the team namespace makes sense.

> However, I did not presume that I could just set the Maintainer field in 
> debian/control before applying to join the team, nor did I set any particular 
> headers or metadata in the ITP for shedskin to reference the team, since I 
> thought that this might be impolite. I imagine that I would change the 
> Maintainer as noted in the team policy if the package were to be adopted 
> within the team.
There is no "adopted within the team" process for packages, so you just
put the repo there and put the team into d/control.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Request to join the Debian Python Team

[Andrey Rakhmatullin]

On Wed, Apr 03, 2024 at 04:21:05PM +0200, Paul Boddie wrote:
> On Wednesday, 3 April 2024 15:56:48 CEST Pierre-Elliott Bécue wrote:
> > 
> > I've granted you developer access to the package subgroup. You can
> > create a shedskin project here if you want.
> 
> Many thanks for giving me access! Would it make sense to move the existing 
> project into the Python Team's packages collection on Salsa, or is that only 
> permitted for packages that are actually adopted by the team?
If you are going to maintain this package in the team then it can and
should be in the team namespace, or are you asking about something else?

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Uscan: watch and changelog

[Andrey Rakhmatullin]

On Fri, Mar 29, 2024 at 02:05:41PM +, c.bu...@posteo.jp wrote:
> The Wiki page  is not up-2-date
What's wrong with it?

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Uscan: watch and changelog

[Andrey Rakhmatullin]

On Fri, Mar 29, 2024 at 07:21:48PM +0500, Andrey Rakhmatullin wrote:
> > I assume I have not yet understood the purpose of the changelog in
> > context of uscan. What do I miss?
> It needs debian/changelog to know the current upstream version of the
> package.
(also this is answered explicitly in uscan(1))



-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Uscan: watch and changelog

[Andrey Rakhmatullin]

On Fri, Mar 29, 2024 at 02:05:41PM +, c.bu...@posteo.jp wrote:
> X-Post: 
> 
> Hi,
> 
> try to get uscan working with an upstream repo hosted at Codeberg.org
> (Forgejo based).
> 
> The Wiki page  is not up-2-date
> and I try to figuring out how it works to update that page. Uscan don't
> work without having a changelog. So I created one. But uscan can not
> find my tarball.
> 
> I assume I have not yet understood the purpose of the changelog in
> context of uscan. What do I miss?
It needs debian/changelog to know the current upstream version of the
package.

> version=4
> # RegEx in Perl dialect
> https://codeberg.org/buhtz/hyperorg/archive/v(\d+).(\d+).(\d+).tar.gz
[...]
> uscan info: Requesting URL:
>https://codeberg.org/buhtz/hyperorg/archive/
> uscan warn: In watchfile debian/watch, reading webpage
>   https://codeberg.org/buhtz/hyperorg/archive/ failed: 404 Not Found
> uscan info: Scan finished
I can confirm it's 404.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Docu: Need help to understand section about package creation

[Andrey Rakhmatullin]

On Wed, Mar 20, 2024 at 09:32:34PM +, c.bu...@posteo.jp wrote:
> Hello,
> 
> in order to improve the documentation I need to understand some parts.
> But I am stuck. I don't get it what the authors trying to explain.
> 
> It is about "Creating a new package" section:
> https://wiki.debian.org/Python/GitPackaging#Creating_a_new_package
> 
> The second paragraph points to dsc-file creation described here:
> https://wiki.debian.org/PackagingWithGit/GitDpm/Initialize
As I said before, we no longer use git-dpm and git-dpm-specific
documentation is no longer relevant. One should use git-buildpackage which
is described on https://wiki.debian.org/PackagingWithGit.

> The third paragraph is a list of shell commands that do not work in my
> case. For example "uscan" don't work because it does not know what and
> where to download. Of course I understand that.
Yes, uscan doesn't make sense in that context.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: matplotlib

[Andrey Rakhmatullin]

On Wed, Mar 20, 2024 at 10:32:04AM +0900, ciel wrote:
> Alexandre-san,
> 
> Sorry, it seems like I cannot contribute this because (at least) I'm
> pretty not sure if I can transplant this line properly:
> 
> ```
> ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
> echo "backend  : TkAgg" > matplotlibrc
> ```
This doesn't need to be translated, but can be used as is.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: morph's abandoned packages (list)

[Andrey Rakhmatullin]

I'm interested in helping with cryptography and pyopenssl, though I
haven't looked at these packages before.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Feature Request (docu): Define "dsc-file"

[Andrey Rakhmatullin]

On Fri, Mar 15, 2024 at 07:39:39AM +, c.bu...@posteo.jp wrote:
> Hi,
> 
> this is a feature request to someone who has access and the knowledge to
> improve.
> 
> Description of the problem:
> 
> The second paragraph in
>  tell me
> about importing a "dsc-file" without explaining what a dsc-file is or why I
> have to import it. The knowledge background is missing.
Most of the Debian packaging docs assume at least basic knowledge about
Debian source and binary packages.

> Suggested solution:
> Create or locate the dsc-file definition in the wiki. 
I'm not sure if it makes sense to cross-link basic stuff on all wiki
pages, to be honest. But that's
https://wiki.debian.org/Packaging/SourcePackage


-- 
WBR, wRAR


signature.asc
Description: PGP signature

RFS: dateparser [RC]

[Andrey Rakhmatullin]

Hello! My key in the keyring is temporarily expired so can please someone
upload dateparser 1.2.0-2? It's tagged in salsa.

Re: cython 3.x (for Python 3.12)

[Andrey Rakhmatullin]

On Sun, Dec 10, 2023 at 09:30:03PM +0100, Andrey Rakhmatullin wrote:
> > I find that there's also a significant issue with relying on
> > cython3-legacy: it conflicts with cython3, meaning that it will be
> > impossible to simultaneously install packages depending on cython3 and
> > cython3-legacy.  Once cython 3.x moves from experimental to unstable
> > to testing and packages start depending on it, this will become a
> > significant issue.  I assume that the aim will be for everything to be
> > ported to cython 3.x and for cython3-legacy to be dropped from testing
> > before the trixie freeze?
> I wonder how many packages actually need a runtime dep on cython. I
> quickly checked three packages from `reverse-depends cython3` and while
> python3-pysph probably uses cython to generate code at the run time (not
> sure), python3-pyzoltan seems to only use it at the build time and
> python3-epimodels doesn't seem to use it at all.
(I may be terribly wrong, I just thought cython is almost always a build
time only dep and there are around 30 packages depending on it which was
surprising)

Re: cython 3.x (for Python 3.12)

[Andrey Rakhmatullin]

On Sun, Dec 10, 2023 at 08:12:40PM +, Julian Gilbey wrote:
> On Sat, Nov 25, 2023 at 04:23:46PM +, Stefano Rivera wrote:
> > As part of preparing for Python 3.12 in Debian, I've uploaded cython 3
> > to experimental.
> > [...]
> > 
> > So, that's 71 regressions with cython3. dd-list below. Please help us
> > port to cython 3. If this isn't possible, Graham is preparing a
> > cython-legacy package, to help the stragglers. But we're expecting that
> > this won't have great Python 3.12 support...
> > https://ftp-master.debian.org/new/cython-legacy_0.29.36-1~exp1.html
> 
> I find that there's also a significant issue with relying on
> cython3-legacy: it conflicts with cython3, meaning that it will be
> impossible to simultaneously install packages depending on cython3 and
> cython3-legacy.  Once cython 3.x moves from experimental to unstable
> to testing and packages start depending on it, this will become a
> significant issue.  I assume that the aim will be for everything to be
> ported to cython 3.x and for cython3-legacy to be dropped from testing
> before the trixie freeze?
I wonder how many packages actually need a runtime dep on cython. I
quickly checked three packages from `reverse-depends cython3` and while
python3-pysph probably uses cython to generate code at the run time (not
sure), python3-pyzoltan seems to only use it at the build time and
python3-epimodels doesn't seem to use it at all.

Re: Recommended way of installing system-wide python application and libraries

[Andrey Rakhmatullin]

On Tue, Dec 05, 2023 at 02:10:01AM -0800, Ivan Perez wrote:
> Hi everyone!
> 
> I'm currently trying to bring a tool we have at NASA Ames up to speed:
> https://github.com/NASA-SW-VnV/ikos
> 
> IKOS is a static analyzer for C. I'm really hoping that IKOS can be
> included in Debian in the near future.
> 
> IKOS is implemented as a  C++ library, and a number of python
> tools/wrappers. The tools call mains in modules defined in a python library
> `ikos`.
> 
> As of right now, our CMakeFiles attempt to install everything (by default)
> under /opt/ikos/.
> 
> I'm having lots of issues getting the python portions installed
> system-wide. I initially upgraded distutils to setuptools, but a recent
> update is now asking that I use a venv. More details and a link to a
> dockerfile can be found here:
> https://github.com/NASA-SW-VnV/ikos/discussions/241.
> 
> While I can hack a solution that "works" (either by making a venv under the
> target dir or by means of break-system-packages), I'd prefer to use
> recommended practices, and also conform to the way that things are done in
> Debian/Ubuntu.
> 
> What would be the recommended way of installing IKOS system-wide in Debian?
This answer may not be useful for you but the only recommended way of
installing Python modules system-wide is by making a proper Debian package
for them using proper Debian Python packaging helpers, and the only other
recommended way of installing Python modules is venvs.
If I needed to install this I would use either of these two, depending on
whether I actually need to have it system-wide and whether I want an
official package uploaded to Debian.
Alternatively, as you mentioned /opt, if the software supports installing
*everything* in /opt (maybe it adds that to sys.path when it runs), then
doing that is also fine (as it doesn't touch things outside /opt and
cannot break those).

Re: Wiki: How to discuss problems and improvements

[Andrey Rakhmatullin]

On Thu, Nov 02, 2023 at 03:02:48AM +, c.bu...@posteo.jp wrote:
> to me it is not clear how to discuss problems and improvements of the wiki.
> There is no dedicated mailing list to that topic.
I don't think it makes sense to bundle discussions of all wiki pages
together. Or do you mean problems and improvements of the wiki itself, not
pages?

> I assume this mailing list here should not polluted with stuff like this. 
It should be fine to discuss Python-related content here.
https://wiki.debian.org/Python/LibraryStyleGuide even asks to do this
explicitly.

> In the footer of the wiki (e.g. https://wiki.debian.org/Python/FAQ) is a
> ling to a wiki-related pseudo package. So I assume I can open bug
> tickets against this package. But I also assume that the "wiki-team"
> reading such tickets are not related to content related discussions and
> problems.
Yes, you shouldn't file bugs to discuss page content.

Re: Wiki "LibraryStyleGuide": pyproject.toml not mentioned

[Andrey Rakhmatullin]

On Thu, Nov 02, 2023 at 03:10:15AM +, c.bu...@posteo.jp wrote:
> Hello,
> 
> related to https://wiki.debian.org/Python/LibraryStyleGuide
> 
> The "pyproject.toml" file is not mentioned here. But it is current
> recommended way (and central file) to do python packaging. If it is not
> possible to use that file in context of debian packaging this also should be
> mentioned in that wiki page.
It's possible to package projects that use pyproject.toml if you mean
that. That wiki page lists the assumptions, having setup.py is not the
only assumption there that is not always true for actual packages.

Re: Do unit tests on Debian Servers/Build system?

[Andrey Rakhmatullin]

On Thu, Nov 02, 2023 at 02:53:48AM +, c.bu...@posteo.jp wrote:
> Hello,
> 
> I try to understand some basics about debian packaging Python applications
> and the upload and test process.
> My question in short: When (at which time point in the process of packaging
> python applications for Debian) do the Python unit tests run?
As a part of the build process they run when the package is built.
As a part of autopkgtests then run when autopkgtests for the package are
triggered, so after it's uploaded and when its deps/revdeps (?) are
uploaded.

> I know about the Salsa instance. Maintainers do create deb files out of it
> and then upload them to the official package repository.
Maintainers don't upload debs, debs are built on buildds.

> Starting from tracker.debian.org I can link to a lot of status packages
> indicating success or problems; e.g. Linitian, CI, ...
> No where I was able to find an indicator about if unit tests are successful
> or not.
Tests that run as a part of the build process fail the build ("buildd:
logs" on the tracker page).
Tests that run as a part of autopkgtests fail the autopkgtest (debci on
the tracker page, but also will have its own block if something fails).

> Do such tests to run at the Debian Servers? Or do the maintainers only start
> them at their own local machines? Is this a manual step in the process?
> If they run automatically when do they run? Before or after the deb files is
> created? If it is the latter then the test_*.py files need to be contained
> in the deb file I assume.
Autopkgtests have access to the source package so they don't need tests in
the binary packages.

Re: non-working debian/watch: create new version of package or not

[Andrey Rakhmatullin]

On Wed, Oct 25, 2023 at 12:10:43PM +0100, Carles Pina i Estany wrote:
> I recently uploaded (via sponsor) python-ping3.
> 
> The file debian/watch works locally but not in:
> https://qa.debian.org/cgi-bin/watch?pkg=python-ping3
> 
> The reason seems to be that the uscan version in the server is different
> to my local one:
> https://lists.debian.org/debian-qa/2023/10/msg00010.html
> 
> When I've fixed it: should I upload the package to mentors.debian.net
> and request a re-upload (with a new version).
> 
> Or just fix it, and no need to re-upload, until some other change in the
> package? (new upstream version, etc.).
There is no general answer to this. It's up to you and your sponsor and it
depends on when you would expect to make a new upload if you didn't need
this change.

Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)

[Andrey Rakhmatullin]

On Wed, Oct 04, 2023 at 07:58:11AM +0100, Carles Pina i Estany wrote:
> Recap: pytest executed from "pybuild-autopkgtest", in the
> python-cloudscraper package, would use the src cloudscrapper instead of
> the installed one.
> 
> So, to make sure that pytest uses the installed one, I added in
> debian/rules:
> 
> -
> before-pybuild-autopkgtest:
>   mv cloudscraper $(AUTOPKGTEST_TMP)
> 
> after-pybuild-autopkgtest:
>   mv $(AUTOPKGTEST_TMP) cloudscraper
This looks very wrong, assuming it even runs.

Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)

[Andrey Rakhmatullin]

On Tue, Oct 03, 2023 at 11:52:58PM +0100, Carles Pina i Estany wrote:
> > upstream tests. Though I think it won't see your explicit `pytest -k` and
> > you should replace the override with a PYBUILD_TEST_ARGS var.
> 
> Done this way:
> https://salsa.debian.org/python-team/packages/python-cloudscraper/-/commit/78a83fbb0fe5fdfba78136921b919a11c8c9bc43
Now you don't need to override dh_auto_test, it should call pytest with
correct args automatically.

> pytest, as run automatically when having "Testsuite:
> autopkgtest-pkg-pybuild": runs in the directory
> "/tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build" doing
> "python3.11 -m pytest -k ...".
> 
> The contents of the directory via "ls -l" can be seen here:
> https://salsa.debian.org/carlespina/python-cloudscraper/-/jobs/4766353#L357)
This gives 404.

> Because there is the sub-directory
> /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper
It shouldn't be there.

> : for what I can tell, pytest is running the tests with the code from
> /tmp/autopkgtest-lxc.u3g8w1m_/downtmp/autopkgtest_tmp/build/cloudscraper
> and not the installed package in
> /usr/lib/python3/dist-packages/cloudscraper/ . Generally speaking, I
> prefer to use the installed code (as done via the basic "import").
Sure, autopkgtests need to use the installed code.

Re: first package questions (salsa repo in personal or team, debian/control maintainers, expected failing unit tests)

[Andrey Rakhmatullin]

On Mon, Oct 02, 2023 at 11:32:31PM +0100, Carles Pina i Estany wrote:
> I will create a new version of the package and upload it into
> mentors.debian.net when I finish this email. For reference: it will be
> the version 1.2.69-3.
Note that the usual practice is not bumping Debian versions for packages
that were not uploaded yet and always uploading -1.

> In my first version (not published) I wrote a simple autopkgtest with an
> "import cloudscraper" (I know that this is not fully testing everything
> but at least something!). Then I realised that it's not needed...
Yeah.
And you should try "Testsuite: autopkgtest-pkg-pybuild" instead, to run
upstream tests. Though I think it won't see your explicit `pytest -k` and
you should replace the override with a PYBUILD_TEST_ARGS var.

> without debian/tests it's just doing it:
> 
> https://salsa.debian.org/python-team/packages/python-cloudscraper/-/jobs/4762103#L180
> """
> autopkgtest [22:24:36]: test autodep8-python3: set -e ; for py in 
> $(py3versions -r 2>/dev/null) ; do cd "$AUTOPKGTEST_TMP" ; echo "Testing with 
> $py:" ; $py -c "import cloudscraper; print(cloudscraper)" ; done
> """
> 
> I don't see this mentioned in "man autodep8":
It says "each supported package type is tried against a set of heuristics,
based on packages names, build dependencies. specific files under debian/,
or a combination of those", or do you mean something else?

Re: Bug#1052028: please update to pydantic 2.x

[Andrey Rakhmatullin]

On Sun, Sep 24, 2023 at 11:25:09AM +0200, Salvo Tomaselli wrote:
> Wouldn't it be better to package it as python3-pydantic2 directly? To avoid 
> breaking every software that uses it.
> 
> In that way they could just coexist and packages depending on the old one 
> wouldn't all suddenly break.
We don't usually do such things for Python libraries when they have the
same import name.

Re: debian/watch: How to watch version tags and what to capture?

[Andrey Rakhmatullin]

On Sun, Sep 17, 2023 at 06:56:58AM +, c.bu...@posteo.jp wrote:
> I assume I'm not the first one encountered it. Is there a bug tracker I
> can check or report that Issue?
The footer of every tracker page says "Report problems to the
tracker.debian.org pseudo-package in the Debian BTS."

Re: debian/watch: How to watch version tags and what to capture?

[Andrey Rakhmatullin]

On Sun, Sep 17, 2023 at 11:32:31AM +, c.bu...@posteo.jp wrote:
> > i get your point that you want the information fast, but it seems you 
> > are just using some arbitrary constraint that fits your personal need.
> > it appears that for "most" Debian maintainers a lag of "1 or 2 days"
> 
> I'm not a Debian maintainer.

[...]

> Because of my watch-file problem. 
The tracker uses watch files provided by Debian maintainers.

Re: Package testing with Salsa CI for new packages

[Andrey Rakhmatullin]

On Fri, Aug 18, 2023 at 07:19:18PM +0200, Paul Boddie wrote:
> > > Here, it would seem that the most prudent approach is to use the Salsa CI
> > > service instead of trying to get the test suite to run during the package
> > > build process.
> > 
> > You should do both if possible, assuming that by "Salsa CI service" you
> > mean autopkgtests which you can, and IMO should, also run locally.
> 
> I'm not really clear on what autopkgtest really is, other than a tool that 
> uses some kind of test suite description that may reside in debian/test. I'm 
> also not completely clear on what is supposed to invoke it, other than either 
> the Salsa CI mechanism or dh_auto_test.
The maintainer is supposed to invoke it before uploading the package, and
the Debian infra invokes it on all packages that are uploaded.
Notably, dh_auto_test is unrelated.

> In the Debian Wiki documentation...
> 
> https://wiki.debian.org/Python/LibraryStyleGuide
> 
> ...it mentions a field in debian/control:
> 
> Testsuite: autopkgtest-pkg-python
> 
> My impression is that this calls autodep8 to generate some kind of test suite 
Yes.
Though note that it generates a trivial test that only checks a top-level
import (just like the wiki page says).

> description which is then invoked by dh_auto_test.
It's not invoked by dh_auto_test. autopkgtests are not a part of the
package build process and they operate on built binary packages.

> It doesn't help that there 
> is an alternative to this that resembles it but behaves differently:
> 
> Testsuite: autopkgtest-pkg-pybuild
It just generates a different (better) test.

> > > I have also found it difficult to persuade the tests to run successfully
> > > during the build process. A few of these attempt to invoke the moin
> > > program, but this cannot be located since it is not installed in the
> > > build environment.
> >
> > This should also be fine, unless it's completely impossible to run it
> > without installing into /.
> 
> The moin program is made available in setup.py using an entry point. Maybe if 
> there were some kind of script instead, it would work.
AFAIK there should be ways to work with this.
Ideally, of course, the upstream test suite should support this, but I
understand that not all upstreams use common practices.

> > > However, one conclusion is that testing a system, as some of the
> > > test cases appear to do, and as opposed to testing library functionality,
> > > is not necessarily appropriate when directed by something like
> > > dh_auto_test.
> >
> > If there are tests that can't be run at build time you can skip those. You
> > can even ask the upstream to provide tool arguments to simplify that.
> 
> I may well discuss such matters with them. One challenge that is relevant in 
> this situation is that upstream have been working in their own virtualenv (or 
> venv, or whatever it is now) world for a few years, using plenty of 
> dependencies that were not packaged in Debian. 
Which is by itself not a problem from the technical side, as you could
just package those (I understand that this requires effort and may have
other problems, but by itself it's normal).

Re: Package testing with Salsa CI for new packages

[Andrey Rakhmatullin]

On Thu, Aug 17, 2023 at 05:10:08PM +0200, Paul Boddie wrote:
> Here, it would seem that the most prudent approach is to use the Salsa CI 
> service instead of trying to get the test suite to run during the package 
> build process.
You should do both if possible, assuming that by "Salsa CI service" you
mean autopkgtests which you can, and IMO should, also run locally.

> One motivation for doing so involves not having to specify 
> build dependencies for packages only needed for test suite execution, which 
> itself requires the invocation of gbp buildpackage with --extra-package 
> arguments since some packages are completely new to Debian.
This is fine. Not to mention that the same problem exists for
autopkgtests, as you say below.

> I have also found it difficult to persuade the tests to run successfully 
> during the build process. A few of these attempt to invoke the moin program, 
> but this cannot be located since it is not installed in the build 
> environment. 
This should also be fine, unless it's completely impossible to run it
without installing into /.

> However, one conclusion is that testing a system, as some of the 
> test cases appear to do, and as opposed to testing library functionality, is 
> not necessarily appropriate when directed by something like dh_auto_test.
If there are tests that can't be run at build time you can skip those. You
can even ask the upstream to provide tool arguments to simplify that.

Re: [backintime] Switch the maintainer to "Debian Python Team (DPT)"

[Andrey Rakhmatullin]

On Fri, Jul 28, 2023 at 11:08:38AM +, c.bu...@posteo.jp wrote:
> So it is possible to have packages in the debian repo that don't run any
> tests? I wasn't expecting this.
Many packages don't have tests at all. For many of them tests are not
possible or don't make much sense.

Re: How to realize different test categories?

[Andrey Rakhmatullin]

On Fri, Jul 28, 2023 at 12:14:33PM +, c.bu...@posteo.jp wrote:
> Hello,
> 
> I'm upstream maintainer (but not founder) of "backintime".
> 
> I realized that most of the tests in my test suite are integration or system
> tests which are impossible to handle by Debians build server and testing
> system (how do you name it?).
> My package do use something like "./configure && make && make test && sudo
> make install" to test and build. Works fine on a local Debian box.
> 
> I would like to have an advice from you as DPT how I should separate my
> tests into the two categories "run by debian" and "don't run by debian".
In the context of DPT I would use pytest marks.
For poorer test systems it's possible that there is no answer except
splitting the tests into separate things and providing separate commands
for running those.

> Another approach in my mind is asking for environment variables that might
> exists on a Debian build box? I know this approach from TravisCI and
> ReadTheDocs.
You shouldn't check for Debian-specific envvars in upstream sources.

Re: how to properly split up into python3-foo and foo-util package?

[Andrey Rakhmatullin]

On Mon, Jul 17, 2023 at 10:53:58PM +0200, Christoph Anton Mitterer wrote:
> > >a) Why does it work to use just usr/... and not
> > debian/tmp/usr/... ?
> > >   Actually, both seems to work, which confuses me even more ^^
> > You can check the search logic in dh_install(1).
> 
> Well I have read that but it merely says it uses the current dir (which
> I didn't know how it's determined) and falls back to debian/tmp (with
> the right debhelper compat lvl)
Doesn't this answer "Why does it work to use just usr/... and not
debian/tmp/usr/... ?"?
And the current dir when building a package is the source root.

> > >b) What - if any - is the proper way here? Like I did, with one
> > >   argument?
> > Yes, because the files are already installed into correct
> > destinations.
> 
> How do you mean that? Cause when I look at the generated files, they're
> actually below debian/tmp/usr/bin and not in debian/usr/bin?
I mean their locations relative to debian/tmp are correct and the only
remaining thing is moving them, keeping their relative paths, into
respective debian/pkgname dirs, which is precisely what the second mode of
dh_install(1) is for.
And debian/usr/bin is a wrong path that is not used at any build step.

> > >   Or should one use the two arguments per line version?
> > >   Or perhaps (for the 2nd file) rather usr/lib/python* ?
> > >   Or rather the debian/tmp/usr/ path versions?
> > >   Or using something completely different than dh_install?
> > No.
> 
> Just out of curiosity, why is the "rather usr/lib/python*" thingy
> considered bad practise?
I don't think it's bad practice, it will work just as well.

> > No, and the upstream source shouldn't contain debian/ anyway, as the
> > life
> > cycles of packaging and upstream sources should be separate even if
> > the
> > person doing both is the same.
> 
> Well, yes... but so far this is only Debian packaging for private use.
> Not sure if the project is of enough general use to submit to Debian.
You can do anything you want for packages not aimed for inclusion into
Debian. This applies even to packaging.

Re: how to properly split up into python3-foo and foo-util package?

[Andrey Rakhmatullin]

On Wed, Jul 12, 2023 at 11:56:05AM +, c.bu...@posteo.jp wrote:
> > > You build two Debian packages (deb-files) out of one source tarball?
> > > Interesting to know that this is possible.
> > It's definitely possible and I would expect any good guide to mention
> > this.
> 
> I really need to see the full repo to better understand whats going on here.
If you want to learn about Debian source packages that build several
binary packages you can look at any such package in the archive.

> One pyproject.toml indicates one and only "Python Distribution Package". I
> can not imagine why someone would split it up into two "Debian packages"?
This is answered in the original email. One package for the module and
another one for the executable.

> Isn't it easier (from Debian Package Maintainers view) to have one "Python
> Distribution Package" per "Debian package". So when I want two Debian
> packages I would create to Python Distro packages in my upstream repo.
No, and it makes no sense to do that.

Re: how to properly split up into python3-foo and foo-util package?

[Andrey Rakhmatullin]

On Wed, Jul 12, 2023 at 11:16:28AM +0100, Simon McVittie wrote:
> On Wed, 12 Jul 2023 at 11:19:07 +0200, Andrey Rakhmatullin wrote:
> > I don't think "usr/bin stuff should likely go
> > in the other". Many Python module packages ship executables, especially
> > now that you no longer have Python 2 subpackages.
> 
> I would personally say that if the executables are significant, and
> particularly if we expect that users will use them without knowing or
> caring whether they are implemented in Python, then they should be in
> a package with a name and Section that make it look like an executable
> program and not a Python library: if nothing else, that will make them
> a lot more discoverable. So I think Christoph is probably correct to be
> thinking about shipping them as foo-util or similar.
Oh yeah, I just tried to say that packaging them separately is not the
only option and probably not even the main option.

> If nothing else, making executables part of the interface of the
> python3-foo package is going to come back to bite us when Python 4 happens
> (hopefully not soon, but if there have been 3 major versions then there
> will probably be a 4th eventually).
> 
> If the Python library is considered to be a public API, then it should
> be in a python3-foo library. src:binwalk and src:tap.py are examples
> of separating out executable + library like this.
There is a popular concern about having binary packages that consist of a
1 kB file + changelog + copyright. It also somewhat complicates packaging,
unfortunately.

> If the Python library is considered to be a private implementation detail
> of the executables, then it doesn't need to be packaged separately
> (for example bmap-tools, dput, meson and offlineimap all contain
> private Python libraries that are not a stable API), and ideally it
> would be in a location that is not on the default import search path,
> like /usr/share/foo or /usr/lib/foo (dput and offlineimap do this,
> although bmap-tools and meson don't).
Yup.

Re: how to properly split up into python3-foo and foo-util package?

[Andrey Rakhmatullin]

On Wed, Jul 12, 2023 at 12:16:51PM +0200, Gregor Riepl wrote:
> > > 5) Not really 100% Debian related, but in the Python sdist,... should
> > > that contain the debian/*?
> > No, and the upstream source shouldn't contain debian/ anyway, as the life
> > cycles of packaging and upstream sources should be separate even if the
> > person doing both is the same.
> 
> This would then be a "native" package, and it's not recommended to use this
> package format in most cases:
Not necessarily, some people make non-native packages and store debian/ in
the VCS. It's not a good idea, though.

Re: how to properly split up into python3-foo and foo-util package?

[Andrey Rakhmatullin]

On Wed, Jul 12, 2023 at 11:05:57AM +, c.bu...@posteo.jp wrote:
> What do you mean by the terms "simple Python package" and "two packages"?
> These terms do not exists in Python context.
These are Debian terms.

> Python do differentiate between
> "Distribution Package" (the name you would find e.g. on PyPI) and "Import
> Packages" (the name you use with your "import" statement). And there is also
> a difference with "Debian Package" (a deb-file). Of course all three type of
> packages can be the same but don't have to.
> 
> Am 12.07.2023 02:21 schrieb Christoph Anton Mitterer:
> > debian/control (all boring fields removed):
> >   Source: foo
> 
> I assume that "foo" is the "Distribution Package" ?
It's the Debian source package as the context is debian/control. Or what
do you mean?

> > 1) Is there some mechanism in dh_python that would automatically split
> >(respectively install) the files to the two packages, and I'm just
> >to dumb to use it?
> 
> I don't understand why there are two packages? Why two? I can not find that
> in your setup.
  Package: python3-foo
  Package: foo-util

> > debian/control (all boring fields removed):
> >   Source: foo
> >   Package: python3-foo
> >   Package: foo-util
> 
> You build two Debian packages (deb-files) out of one source tarball?
> Interesting to know that this is possible.
It's definitely possible and I would expect any good guide to mention
this.

Re: how to properly split up into python3-foo and foo-util package?

[Andrey Rakhmatullin]

On Wed, Jul 12, 2023 at 02:21:48AM +0200, Christoph Anton Mitterer wrote:


> When I run debuild -us -uc on that, it generates:
>   debian/tmp/...
>   debian/tmp/usr/bin/
>   debian/tmp/usr/lib/python3.11/dist-packages/foo
>   debian/tmp/usr/lib/python3.11/dist-packages/foo-1.0.0.dist-info
>   debian/tmp/usr/lib/python3.11/dist-packages/foo-1.0.0.dist-info/LICENCE
> 
> But then complains that the files aren't installed by anyone.
Yes, you usually need to use dh_install when having several subpackages.
This is not Python-specific.

> 1) Is there some mechanism in dh_python that would automatically split
>(respectively install) the files to the two packages, and I'm just
>to dumb to use it?
>Like that it knows, the Python package should likely go into the
>python3-* Debian package, and usr/bin stuff should likely go in the
>other?
I don't think there is, and I don't think "usr/bin stuff should likely go
in the other". Many Python module packages ship executables, especially
now that you no longer have Python 2 subpackages.

> 2) I then tried with such package.install files like those:
>foo-util.install:
>  usr/bin
> 
>python3-foo.install:
>  usr/lib
> 
>a) Why does it work to use just usr/... and not debian/tmp/usr/... ?
>   Actually, both seems to work, which confuses me even more ^^
You can check the search logic in dh_install(1).

>b) What - if any - is the proper way here? Like I did, with one
>   argument?
Yes, because the files are already installed into correct destinations.

>   Or should one use the two arguments per line version?
>   Or perhaps (for the 2nd file) rather usr/lib/python* ?
>   Or rather the debian/tmp/usr/ path versions?
>   Or using something completely different than dh_install?
No.

> 3) In debian/tmp, the subdir was /python3.11/ but in the final .deb
>file it's actually /python3/ (as I think it should be).
>Is it expected, that first it's /python3.11/ or am I doing anything
>wrong?
Yes, it's expected.

> 4) Are there way to have the Dependencies in control even more
>autodetected?
>a) That foo-util's dependency on python3-foo is somehow auto-filled
>   by dh_python?
No, as there is no data to use here.

>b) Or the Build-Deps? I mean dh_python should at least be able to
>   find out about python3-setuptools, python3-setuptools-scm from my
>   pyproject.toml, shouldn't it?
You cannot autodetect build dependencies at the build time. That would be
too late.

> 5) Not really 100% Debian related, but in the Python sdist,... should
>that contain the debian/*?
No, and the upstream source shouldn't contain debian/ anyway, as the life
cycles of packaging and upstream sources should be separate even if the
person doing both is the same.

Re: python2:any dependency

[Andrey Rakhmatullin]

On Sun, Jul 02, 2023 at 08:10:15PM +0200, PICCA Frederic-Emmanuel wrote:
> I: dh_python3 tools:114: replacing shebang in 
> debian/pymca/usr/bin/edfviewerD: dh_python3 fs:400: package pymca details = 
> {'requires.txt': set(), 'egg-info': set(), 'dist-info': set(), 'nsp.txt': 
> set(), 'shebangs': {/usr/bin/python2, /usr/bin/python2, /usr/bin/python2, 
> /usr/bin/python2, /usr/bin/python2, /usr/bin/python2, /usr/bin/python2, 
> /usr/bin/python2, /usr/bin/python2}, 'public_vers': set(), 'private_dirs': 
> {}, 'compile': False, 'ext_vers': set(), 'ext_no_version': set()}
If your package ships scripts with python2 shebangs it makes sense that it
requires python2.

Re: Siging up for the Contribution on Python Packaging for Debian

[Andrey Rakhmatullin]

On Thu, May 25, 2023 at 12:19:00PM +, TopologicRose wrote:
> Hi,
> want to contribute a Python package which called pyvis 
> (https://pyvis.readthedocs.io/en/latest/index.html) for APT.
> A friend of mine works with it and wants it as a Debian package instead of 
> pip.
Start at https://mentors.debian.net/intro-maintainers/

Re: Unittests writing to HOME (backintime)

[Andrey Rakhmatullin]

On Wed, Mar 29, 2023 at 10:03:31AM +, c.bu...@posteo.jp wrote:
> > The usual solution is AFAIK to set a temporary $HOME inside d/rules
> > though.
> 
> I was looking into
> https://salsa.debian.org/jmw/pkg-backintime/-/blob/debian/debian/rules
> 
> I don't see creation of a HOME or deactivation of a test.
This means you've missed my explanation.

Re: Unittests writing to HOME (backintime)

[Andrey Rakhmatullin]

On Wed, Mar 29, 2023 at 07:52:35AM +, c.bu...@posteo.jp wrote:
> I assume this topic is not specific to one package but to the whole python
> packaging universe.
> 
> There is "backintime" which unittests do write to $HOME. I'm one of the new
> upstream maintainers and know that this isn't a good idea. It will take time
> to fix this.
> About that problem there is a debian bug report nearly 4 years old
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=940319
> 
> It tells that this is a problem because writing to HOME during build "is
> disabled on Debian auto-builders".
> 
> My question now is why newer version of this package are uploaded then? I
> couldn't find that the tests where deactivated.
As the software doesn't use a discoverabe build system the package needs
to have all build steps explicitly in d/rules, and it just doesn't have
the test step there.

> Maybe this "disabled on
> Debian auto-builders" is outdated and today it is possible to write to HOME
> during build?
No.
The usual solution is AFAIK to set a temporary $HOME inside d/rules though.

Re: BTS: What are Unclassified bugs?

[Andrey Rakhmatullin]

On Wed, Mar 29, 2023 at 08:02:56AM +, c.bu...@posteo.jp wrote:
> Hello,
> 
> I couldn't find a mailing list specific to the Bug Tracking System.
Then please use debian-mentors@ or debian-devel@, not debian-python@.

> In the bugs summary list for a specific package I can find "Unclassified"
> bugs. But what is "Unclassified"?
I assume it just means "not having any other classification such as
wontfix or confirmed".

> Looking into other (not "Unclassified") bugs all have a "severity"
(all bugs have a severity)

> One example is the backintime package
> 
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no=backintime
(I don't see the word "Unclassified" on that page)

Re: BTS: Merging bugs not working because of "forwarded" to upstream bug

[Andrey Rakhmatullin]

On Wed, Mar 29, 2023 at 08:42:02AM +, c.bu...@posteo.jp wrote:
> > Do, what it says: the forwarded value must be equal for both bugs.
> 
> The system should do what I say: merge. :D
That's not what the docs for the command say, please read them before
using the command.
And the command for "do what I say" is called forcemerge.

Re: Reviewer + Uploader for python-box

[Andrey Rakhmatullin]

On Mon, Feb 27, 2023 at 08:47:58PM +0100, Malik wrote:
> Hello Team,
> 
> I updated python-box package [1] and created MRs which update upstream and
> run autopkg tests to close #974492 [2]
Please check https://release.debian.org/testing/freeze_policy.html .

Re: sphinxext-opengraph upstream update (v 0.8.1)

[Andrey Rakhmatullin]

On Mon, Feb 27, 2023 at 10:43:04AM -1000, Chiara Marmo wrote:
> Dear list,
> 
> sphinxext-opengraph v0.8.1 is on salsa [1] ready for review and upload.
> Thanks!
Please check https://release.debian.org/testing/freeze_policy.html .

Re: can pip be made using local Debian packages for any dependencies

[Andrey Rakhmatullin]

On Fri, Feb 17, 2023 at 03:17:49AM +0100, Philippe Cerfon wrote:
> But shouldn't that use case also be interesting for Debian
> Maintainers? Whenever their pip would need to download something from
> PyPI, it would mean that some dependency is likely not fulfilled in
> Debian (unless of course that Debian package is simply not installed).
Do you mean the scenario with packaging some new software? In that case
a packager would most likely review the deps, not run pip.

Re: can one change the path of generated entry point console_scripts

[Andrey Rakhmatullin]

On Fri, Feb 17, 2023 at 02:49:49AM +0100, Philippe Cerfon wrote:
> Hey Stefano
> 
> On Wed, Feb 15, 2023 at 5:37 PM Stefano Rivera  wrote:
> > Just move it somewhere else later in the build? e.g. after dh_install.
> 
> I had tried that before, with a debian/mypackage.install file but got
That's not what "move it [...] after dh_install" means, it usually means a
manual mv call.

> an error that it doesn't find the file.
> Then I realized that I cannot use e.g.:
>usr/bin/scriptusr/sbin/
Yes, .install is not for moving files between directories inside a
package. Please check its two use cases in dh_install(1).

> but have to use:
>debian/mypackage/usr/bin/scriptusr/sbin/
I don't think this is a good idea.

Re: can pip be made using local Debian packages for any dependencies

[Andrey Rakhmatullin]

On Thu, Feb 16, 2023 at 01:12:49AM +, Ian Norton wrote:
> I agree that is "easiest" but what I was after was the ability to restrict
> myself to the curated and signed packages from debian, pypi is just as bad
> as old CPAN when it comes to packages disappearing or being broken or
> depending on totally random versions
These, or comparable, problems also happen in Debian. For example, you
cannot expect any given module to be packaged (or not disappear in the
next release), sometimes the version in the repo is several years old and
of course packages in Debian can be broken, even if it's rare.

Re: Python 3.10 in bookworm

[Andrey Rakhmatullin]

On Sun, Feb 05, 2023 at 01:50:34PM +, Julian Gilbey wrote:
> Our social contract #4 says "Our priorities are our users and free
> software".  What benefits would having the python3.10 base packages in
> bookworm bring for our users (as I point out, for some users, this is
> a necessity) and what disadvantages would it bring (none that I can
> think of)?  Why would we tell a whole bunch of our users: "Don't
> upgrade to Debian 12 until all of the critical packages you use from
> PyPI are upgraded to support Python 3.11, or fix those packages
> yourself"?
The next obvious step for these use cases is to just install 3.10 with
pyenv.