Bug#1041538: dh-python: Unhelpful inclusion of optional packages in Depends
On Friday, July 28, 2023 3:42:50 PM EDT Emmanuel Arias wrote: > Control: reassign -1 poetry-core > > Hi, > > Sorry for this delay. Yes as you mentioned Scott, you need to add the > optional dependency in the extras section, not only marked it as > optional=true. > > Poetry documentation[0] is not very explicit with that. > > There's an issue open [1]. I can work in a patch to at least raise an > exception o message when a dependency is marked as optional and is not > added to extras. But I agree that mark a deps as optional and then is > not optional is confusing. > > [0] https://python-poetry.org/docs/pyproject/#extras > [1] https://github.com/python-poetry/poetry/issues/2357 Thanks, For this particular package I pointed this out to upstream and it's fixed in their latest bug fix update. They were quite surprised that marking a dependency optional didn't actually do anything, so I think it would be good to get poetry to handle this better. Scott K signature.asc Description: This is a digitally signed message part.
Bug#1041538: dh-python: Unhelpful inclusion of optional packages in Depends
Control: reassign -1 poetry-core Hi, Sorry for this delay. Yes as you mentioned Scott, you need to add the optional dependency in the extras section, not only marked it as optional=true. Poetry documentation[0] is not very explicit with that. There's an issue open [1]. I can work in a patch to at least raise an exception o message when a dependency is marked as optional and is not added to extras. But I agree that mark a deps as optional and then is not optional is confusing. [0] https://python-poetry.org/docs/pyproject/#extras [1] https://github.com/python-poetry/poetry/issues/2357 Cheers, Emmanuel signature.asc Description: PGP signature
Bug#1041538: dh-python: Unhelpful inclusion of optional packages in Depends
Control: reassign -1 poetry Control: affects -1 dh-python > I'd call this an upstream poetry issue. > > It shouldn't declare optional dependencies as required. It should rather > create an extra to encapsulate any optional dependencies that aren't > part of a defined extra. Reassigning the bug. Stefano -- Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272
Bug#1041538: dh-python: Unhelpful inclusion of optional packages in Depends
Hi Piotr (2023.07.20_14:58:03_+) > can you show me how poetry / pyproject / whatever parsed this file > interpreted it? I.e. what's in installed requires.txt > > (I suspect it's a hard dependency there) Yep, that's the issue, METEDATA contains: Requires-Dist: httpcore (>=0.17.3) ; python_version >= "3.8" Requires-Dist: sniffio (>=1.1,<2.0) I'd call this an upstream poetry issue. It shouldn't declare optional dependencies as required. It should rather create an extra to encapsulate any optional dependencies that aren't part of a defined extra. Stefano -- Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272
Bug#1041538: dh-python: Unhelpful inclusion of optional packages in Depends
On Thursday, July 20, 2023 10:58:03 AM EDT Piotr Ożarowski wrote:
> [Scott Kitterman, 2023-07-20]
>
> > I had to override dh_python3 to add --no-guessing-deps in the latest
>
> FTR: you can override detection with debian/py3dist-overrides (see
> dh_python3's manpage or /usr/share/doc/dh-python/README.PyDist for more
> details)
>
> > dnspython upload because it was getting things wrong. Here's what it
> > generated:
> >
> > python3-httpcore | python3 (<< 3.8), python3-sniffio, python3:any
> >
> > The correct answer here is actually use python3:any. Here's what I
> > think is going on:
> >
> > From the pyproject.toml file:
> >
> > [tool.poetry.dependencies]
> > python = "^3.8"
> > httpx = {version=">=0.24.1", optional=true, python=">=3.8"}
> > httpcore = {version=">=0.17.3", optional=true, python=">=3.8"}
> > h2 = {version=">=4.1.0", optional=true, python=">=3.8"}
> > idna = {version=">=2.1,<4.0", optional=true}
> > cryptography = {version=">=2.6,<42.0", optional=true}
> > trio = {version=">=0.14,<0.23", optional=true}
> > sniffio = {version="^1.1", optional=true}
> > wmi = {version="^1.5.1", optional=true}
> > aioquic = {version=">=0.9.20", optional=true}
> >
> > ...
> >
> > [tool.poetry.extras]
> > doh = ['httpx', 'h2']
> > idna = ['idna']
> > dnssec = ['cryptography']
> > trio = ['trio']
> > wmi = ['wmi']
> > doq = ['aioquic']
> >
> > There are two issues:
> >
> > 1. httpcore and sniffio shouldn't be listed at all. They are optional.
> > I suspect that either poetry or dh-python is looking at the extras
> > section and since those packages aren't listed for one of the extras, it
> > assumes the package is required, despite the optional flag. They
> > probably should be listed somewhere (upstream bug), but I think if it
> > says optional, it shouldn't be added to Depends.
>
> dh_python3 doesn't look at source files. It generates dependencies by
> parsing installed requires.txt file so IMHO it's an upstream / poetry bug
>
> > 2. Generating an optional depends on python3 << 3.8 isn't helpful. It
> > looks to me like {version=">=0.17.3", optional=true, python=">=3.8"} is
> > being mis-interpreted. I believe the intent here is to say that the
> > dependency is optional when python3 > 3.8, not you need it if python3 >
>
> can you show me how poetry / pyproject / whatever parsed this file
> interpreted it? I.e. what's in installed requires.txt
>
> (I suspect it's a hard dependency there)
>
> > 3.8. There's a larger question of why the interpreter version is there
> > at all, given that's now the minimum python3 version supported, but
> > that's an upstream issue, we ought to get it right.
>
> I agree
I have so far managed to avoid learning anything about how poetry works
internally (learning about flit fully exhausted my curiosity in this area).
Emmanuel,
Can you help out with this?
Thanks,
Scott K
signature.asc
Description: This is a digitally signed message part.
Bug#1041538: dh-python: Unhelpful inclusion of optional packages in Depends
[Scott Kitterman, 2023-07-20]
> I had to override dh_python3 to add --no-guessing-deps in the latest
FTR: you can override detection with debian/py3dist-overrides (see
dh_python3's manpage or /usr/share/doc/dh-python/README.PyDist for more
details)
> dnspython upload because it was getting things wrong. Here's what it
> generated:
>
> python3-httpcore | python3 (<< 3.8), python3-sniffio, python3:any
>
> The correct answer here is actually use python3:any. Here's what I
> think is going on:
>
> From the pyproject.toml file:
>
> [tool.poetry.dependencies]
> python = "^3.8"
> httpx = {version=">=0.24.1", optional=true, python=">=3.8"}
> httpcore = {version=">=0.17.3", optional=true, python=">=3.8"}
> h2 = {version=">=4.1.0", optional=true, python=">=3.8"}
> idna = {version=">=2.1,<4.0", optional=true}
> cryptography = {version=">=2.6,<42.0", optional=true}
> trio = {version=">=0.14,<0.23", optional=true}
> sniffio = {version="^1.1", optional=true}
> wmi = {version="^1.5.1", optional=true}
> aioquic = {version=">=0.9.20", optional=true}
>
> ...
>
> [tool.poetry.extras]
> doh = ['httpx', 'h2']
> idna = ['idna']
> dnssec = ['cryptography']
> trio = ['trio']
> wmi = ['wmi']
> doq = ['aioquic']
>
> There are two issues:
>
> 1. httpcore and sniffio shouldn't be listed at all. They are optional.
> I suspect that either poetry or dh-python is looking at the extras
> section and since those packages aren't listed for one of the extras, it
> assumes the package is required, despite the optional flag. They
> probably should be listed somewhere (upstream bug), but I think if it
> says optional, it shouldn't be added to Depends.
dh_python3 doesn't look at source files. It generates dependencies by
parsing installed requires.txt file so IMHO it's an upstream / poetry bug
> 2. Generating an optional depends on python3 << 3.8 isn't helpful. It
> looks to me like {version=">=0.17.3", optional=true, python=">=3.8"} is
> being mis-interpreted. I believe the intent here is to say that the
> dependency is optional when python3 > 3.8, not you need it if python3 >
can you show me how poetry / pyproject / whatever parsed this file
interpreted it? I.e. what's in installed requires.txt
(I suspect it's a hard dependency there)
> 3.8. There's a larger question of why the interpreter version is there
> at all, given that's now the minimum python3 version supported, but
> that's an upstream issue, we ought to get it right.
I agree
Bug#1041538: dh-python: Unhelpful inclusion of optional packages in Depends
Package: dh-python
Version: 5.20230603
Severity: normal
I had to override dh_python3 to add --no-guessing-deps in the latest
dnspython upload because it was getting things wrong. Here's what it
generated:
python3-httpcore | python3 (<< 3.8), python3-sniffio, python3:any
The correct answer here is actually use python3:any. Here's what I
think is going on:
>From the pyproject.toml file:
[tool.poetry.dependencies]
python = "^3.8"
httpx = {version=">=0.24.1", optional=true, python=">=3.8"}
httpcore = {version=">=0.17.3", optional=true, python=">=3.8"}
h2 = {version=">=4.1.0", optional=true, python=">=3.8"}
idna = {version=">=2.1,<4.0", optional=true}
cryptography = {version=">=2.6,<42.0", optional=true}
trio = {version=">=0.14,<0.23", optional=true}
sniffio = {version="^1.1", optional=true}
wmi = {version="^1.5.1", optional=true}
aioquic = {version=">=0.9.20", optional=true}
...
[tool.poetry.extras]
doh = ['httpx', 'h2']
idna = ['idna']
dnssec = ['cryptography']
trio = ['trio']
wmi = ['wmi']
doq = ['aioquic']
There are two issues:
1. httpcore and sniffio shouldn't be listed at all. They are optional.
I suspect that either poetry or dh-python is looking at the extras
section and since those packages aren't listed for one of the extras, it
assumes the package is required, despite the optional flag. They
probably should be listed somewhere (upstream bug), but I think if it
says optional, it shouldn't be added to Depends.
2. Generating an optional depends on python3 << 3.8 isn't helpful. It
looks to me like {version=">=0.17.3", optional=true, python=">=3.8"} is
being mis-interpreted. I believe the intent here is to say that the
dependency is optional when python3 > 3.8, not you need it if python3 >
3.8. There's a larger question of why the interpreter version is there
at all, given that's now the minimum python3 version supported, but
that's an upstream issue, we ought to get it right.
Scott K
