Bug#1003567: Please make a separate package for mistune 2.x
> I'd like other python team member's opinion on this, and I'm not eager > to maintain that legacy package, as I tend to not want to maintain > obsolete software. Still, I can do the initial work of creating it. i wouldnt expect much maintenance needed tho: 0.8.4 is essentially dead upstream, so it will only need to be kept around until its rdeps are ported to mistune 2.x the proposal is "okay", it has the downside of requiring the current rdeps to be updated to point to the new binary package name for the old mistune, but it leaves the namespace open for mistune 2 to take over python3-mistune bin pkg -- Sandro "morph" Tosi My website: http://sandrotosi.me/ Me at Debian: http://wiki.debian.org/SandroTosi Twitter: https://twitter.com/sandrotosi
Bug#1003567: Please make a separate package for mistune 2.x
Nilesh Patra wrote on 05/02/2022 at 20:23:44+0100:
> [[PGP Signed Part:No public key for 00BAE74B343369F1 created at
> 2022-02-05T20:23:44+0100 using RSA]]
> On 2/6/22 12:20 AM, Julian Gilbey wrote:
>> On Fri, Feb 04, 2022 at 09:27:59PM +0530, Nilesh Patra wrote:
>>> On 2/4/22 9:18 PM, Julian Gilbey wrote:
[...]
_mistune.py within the Debian package,
and have nbconvert do "import nbconvert.filters._mistune as mistune"
(see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py).
That seems like an eminently sensible solution to this problem.
>>>
>>> But that'd lead to a number of mistune's embedded copies in a huge number
>>> of packages; since majority of
>>> the rev-deps (when I last checked) haven't adapted to this new version.
>>> When they do,
>>> and it becomes a overhead to fix each one later.
>>> Even worse, if we discover a security problem sometime later, then all such
>>> packages would be
>>> effected, and that honestly does not look like a good idea to me.
>> I have just had another idea, which might solve all of the problems:
>> create a new Debian package called mistune0 (or mistune1), which
>> contains the legacy version of mistune, but with the Python module
>> called "mistune0" instead of "mistune". Then it will be
>> co-installable with mistune 2.x, and the packages which still depend
>> on mistune 0.8.4 could be patched to say "import mistune0 as mistune"
>> until they are updated upstream. This will also avoid having multiple
>> copies of the legacy code in the archive, which addresses the security
>> issue, and allow those packages which have migrated to mistune 2.x to
>> still say "import mistune".
>
> Ah, looks like we just had to think in the reverse direction from the initial
> proposal (mistune-{0,1} instead of mistune-{1,2}).
> Indeed, that sounds like a much better plan.
> I hope PEB agrees with this as well, would like to hear from them :)
>
> Thanks for the discussion, Julian :)
>
> Regards,
> Nilesh
I'd like other python team member's opinion on this, and I'm not eager
to maintain that legacy package, as I tend to not want to maintain
obsolete software. Still, I can do the initial work of creating it.
--
PEB
signature.asc
Description: PGP signature
Bug#1003567: Please make a separate package for mistune 2.x
On 2/6/22 12:20 AM, Julian Gilbey wrote:
On Fri, Feb 04, 2022 at 09:27:59PM +0530, Nilesh Patra wrote:
On 2/4/22 9:18 PM, Julian Gilbey wrote:
[...]
_mistune.py within the Debian package,
and have nbconvert do "import nbconvert.filters._mistune as mistune"
(see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py).
That seems like an eminently sensible solution to this problem.
But that'd lead to a number of mistune's embedded copies in a huge number of
packages; since majority of
the rev-deps (when I last checked) haven't adapted to this new version. When
they do,
and it becomes a overhead to fix each one later.
Even worse, if we discover a security problem sometime later, then all such
packages would be
effected, and that honestly does not look like a good idea to me.
I have just had another idea, which might solve all of the problems:
create a new Debian package called mistune0 (or mistune1), which
contains the legacy version of mistune, but with the Python module
called "mistune0" instead of "mistune". Then it will be
co-installable with mistune 2.x, and the packages which still depend
on mistune 0.8.4 could be patched to say "import mistune0 as mistune"
until they are updated upstream. This will also avoid having multiple
copies of the legacy code in the archive, which addresses the security
issue, and allow those packages which have migrated to mistune 2.x to
still say "import mistune".
Ah, looks like we just had to think in the reverse direction from the initial
proposal (mistune-{0,1} instead of mistune-{1,2}).
Indeed, that sounds like a much better plan.
I hope PEB agrees with this as well, would like to hear from them :)
Thanks for the discussion, Julian :)
Regards,
Nilesh
OpenPGP_signature
Description: OpenPGP digital signature
Bug#1003567: Please make a separate package for mistune 2.x
On Fri, Feb 04, 2022 at 09:27:59PM +0530, Nilesh Patra wrote: > On 2/4/22 9:18 PM, Julian Gilbey wrote: > > [...] > > _mistune.py within the Debian package, > > and have nbconvert do "import nbconvert.filters._mistune as mistune" > > (see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py). > > That seems like an eminently sensible solution to this problem. > > But that'd lead to a number of mistune's embedded copies in a huge number of > packages; since majority of > the rev-deps (when I last checked) haven't adapted to this new version. When > they do, > and it becomes a overhead to fix each one later. > Even worse, if we discover a security problem sometime later, then all such > packages would be > effected, and that honestly does not look like a good idea to me. I have just had another idea, which might solve all of the problems: create a new Debian package called mistune0 (or mistune1), which contains the legacy version of mistune, but with the Python module called "mistune0" instead of "mistune". Then it will be co-installable with mistune 2.x, and the packages which still depend on mistune 0.8.4 could be patched to say "import mistune0 as mistune" until they are updated upstream. This will also avoid having multiple copies of the legacy code in the archive, which addresses the security issue, and allow those packages which have migrated to mistune 2.x to still say "import mistune". Best wishes, Julian
Bug#1003567: Please make a separate package for mistune 2.x
Le 2022-02-04 à 11 h 24, Nilesh Patra a écrit : On 2/4/22 9:33 PM, Julian Gilbey wrote: _mistune.py within the Debian package, and have nbconvert do "import nbconvert.filters._mistune as mistune" (see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py). That seems like an eminently sensible solution to this problem. But that'd lead to a number of mistune's embedded copies in a huge number of packages; since majority of the rev-deps (when I last checked) haven't adapted to this new version. When they do, and it becomes a overhead to fix each one later. Even worse, if we discover a security problem sometime later, then all such packages would be effected, and that honestly does not look like a good idea to me. This is true, though there are only 7 reverse dependencies currently in testing. Yeah, in total the reverse-deps are 8 and there is one different reverse-build-dep (pasted at end of this mail) But the problem is if these fall through the cracks, and close to the release we notice some package is not looking nice. Also, if you suppose that the upstreams are very slow, and we get to the new debian release with these things still embedded, it'd be a real mess to upload fixes to stable, right... I somehow do not understand the urgency of uploading this newer version, as the maintainer said: | I intend to upload src:mistune 2.0.0 to unstable between March the | 15th and April the 15th (depending on the progress of its | reverse-dependencies). We could simply wait a little more for the dust to settle, IMHO. That would be a reasonable approach, but how long will it take for the dust to settle? With this major change, and no guidance from upstream on how to migrate, and at least a number of upstream authors happy to rely on setup.py having "mistune <1.0.0" in the install_requires field, it might be several months or longer before things are fixed upstream. I think we could do this transition even a month or two before the soft freeze starts, which is roughly an year from now (considering general trend timings of starting in feb in release year). Situation might improve by then, I suppose. If it does not, we could still go ahead with the older python3-mistune in the archive, I do not see a huge problem there, except for maybe a few mistune uploads to stable if desired. And what do we do when some packages have converted and some haven't? In such a case, we atleast will have a few examples to see how to go about doing the API changes the right way. We could patch out at our end and send the changes upstream for review provided we are stuck in an unfortunate situation. FWIW, there's a recent patch and PR for the lektor upstream which add mistune 2.x support, so for this particular project I don't think a separate package is necessary. -- Jerome
Bug#1003567: Please make a separate package for mistune 2.x
Le 4 février 2022 16:57:59 GMT+01:00, Nilesh Patra a écrit : >On 2/4/22 9:18 PM, Julian Gilbey wrote: >> Basically, the mistune upstream author has completely messed up on >> this by making what is essentially a completely different package with >> superficially similar functionality but the same name. > >True. > >> [...] >> _mistune.py within the Debian package, >> and have nbconvert do "import nbconvert.filters._mistune as mistune" >> (see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py). >> That seems like an eminently sensible solution to this problem. > >But that'd lead to a number of mistune's embedded copies in a huge number of >packages; since majority of >the rev-deps (when I last checked) haven't adapted to this new version. When >they do, >and it becomes a overhead to fix each one later. >Even worse, if we discover a security problem sometime later, then all such >packages would be >effected, and that honestly does not look like a good idea to me. > >I somehow do not understand the urgency of uploading this newer version, as >the maintainer said: > >| I intend to upload src:mistune 2.0.0 to unstable between March the >| 15th and April the 15th (depending on the progress of its >| reverse-dependencies). > >We could simply wait a little more for the dust to settle, IMHO. > >Regards, >Nilesh > Because some packages I maintain depend on mistune 2.x and mistune 0.8.4 is, whether we like it or not, obsolete. I can't really afford to wait for a year to try having mailman3 in testing again... -- Pierre-Elliott Bécue From my phone
Bug#1003567: Please make a separate package for mistune 2.x
On 2/4/22 9:33 PM, Julian Gilbey wrote: _mistune.py within the Debian package, and have nbconvert do "import nbconvert.filters._mistune as mistune" (see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py). That seems like an eminently sensible solution to this problem. But that'd lead to a number of mistune's embedded copies in a huge number of packages; since majority of the rev-deps (when I last checked) haven't adapted to this new version. When they do, and it becomes a overhead to fix each one later. Even worse, if we discover a security problem sometime later, then all such packages would be effected, and that honestly does not look like a good idea to me. This is true, though there are only 7 reverse dependencies currently in testing. Yeah, in total the reverse-deps are 8 and there is one different reverse-build-dep (pasted at end of this mail) But the problem is if these fall through the cracks, and close to the release we notice some package is not looking nice. Also, if you suppose that the upstreams are very slow, and we get to the new debian release with these things still embedded, it'd be a real mess to upload fixes to stable, right... I somehow do not understand the urgency of uploading this newer version, as the maintainer said: | I intend to upload src:mistune 2.0.0 to unstable between March the | 15th and April the 15th (depending on the progress of its | reverse-dependencies). We could simply wait a little more for the dust to settle, IMHO. That would be a reasonable approach, but how long will it take for the dust to settle? With this major change, and no guidance from upstream on how to migrate, and at least a number of upstream authors happy to rely on setup.py having "mistune <1.0.0" in the install_requires field, it might be several months or longer before things are fixed upstream. I think we could do this transition even a month or two before the soft freeze starts, which is roughly an year from now (considering general trend timings of starting in feb in release year). Situation might improve by then, I suppose. If it does not, we could still go ahead with the older python3-mistune in the archive, I do not see a huge problem there, except for maybe a few mistune uploads to stable if desired. And what do we do when some packages have converted and some haven't? In such a case, we atleast will have a few examples to see how to go about doing the API changes the right way. We could patch out at our end and send the changes upstream for review provided we are stuck in an unfortunate situation. Let me know what you'd think about this? Regards, Nilesh Reverse-Depends * giara [amd64 arm64 armel armhf i386 mips64el mipsel ppc64el s390x] * iredis * lektor * lookatme * matrix-mirage [amd64 arm64 armel armhf i386 mips64el mipsel ppc64el s390x] * python3-flasgger * python3-m2r * python3-schema-salad Reverse-Testsuite-Triggers * markdown-it-py Reverse-Build-Depends * lektor * lookatme * markdown-it-py * python-flasgger * python-m2r * python-schema-salad OpenPGP_signature Description: OpenPGP digital signature
Bug#1003567: Please make a separate package for mistune 2.x
On 2/4/22 9:18 PM, Julian Gilbey wrote: Basically, the mistune upstream author has completely messed up on this by making what is essentially a completely different package with superficially similar functionality but the same name. True. [...] _mistune.py within the Debian package, and have nbconvert do "import nbconvert.filters._mistune as mistune" (see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py). That seems like an eminently sensible solution to this problem. But that'd lead to a number of mistune's embedded copies in a huge number of packages; since majority of the rev-deps (when I last checked) haven't adapted to this new version. When they do, and it becomes a overhead to fix each one later. Even worse, if we discover a security problem sometime later, then all such packages would be effected, and that honestly does not look like a good idea to me. I somehow do not understand the urgency of uploading this newer version, as the maintainer said: | I intend to upload src:mistune 2.0.0 to unstable between March the | 15th and April the 15th (depending on the progress of its | reverse-dependencies). We could simply wait a little more for the dust to settle, IMHO. Regards, Nilesh OpenPGP_signature Description: OpenPGP digital signature
Bug#1003567: Please make a separate package for mistune 2.x
On Fri, Feb 04, 2022 at 09:27:59PM +0530, Nilesh Patra wrote: > On 2/4/22 9:18 PM, Julian Gilbey wrote: > > Basically, the mistune upstream author has completely messed up on > > this by making what is essentially a completely different package with > > superficially similar functionality but the same name. > > True. > > [...] > > _mistune.py within the Debian package, > > and have nbconvert do "import nbconvert.filters._mistune as mistune" > > (see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py). > > That seems like an eminently sensible solution to this problem. > > But that'd lead to a number of mistune's embedded copies in a huge number of > packages; since majority of > the rev-deps (when I last checked) haven't adapted to this new version. When > they do, > and it becomes a overhead to fix each one later. > Even worse, if we discover a security problem sometime later, then all such > packages would be > effected, and that honestly does not look like a good idea to me. This is true, though there are only 7 reverse dependencies currently in testing. > I somehow do not understand the urgency of uploading this newer version, as > the maintainer said: > > | I intend to upload src:mistune 2.0.0 to unstable between March the > | 15th and April the 15th (depending on the progress of its > | reverse-dependencies). > > We could simply wait a little more for the dust to settle, IMHO. That would be a reasonable approach, but how long will it take for the dust to settle? With this major change, and no guidance from upstream on how to migrate, and at least a number of upstream authors happy to rely on setup.py having "mistune <1.0.0" in the install_requires field, it might be several months or longer before things are fixed upstream. And what do we do when some packages have converted and some haven't? Best wishes, Julian
Bug#1003567: Please make a separate package for mistune 2.x
On Thu, Feb 03, 2022 at 11:34:26PM +0100, Pierre-Elliott Bécue wrote: > Hi Michael, > > > Since Mistune 2.0.0 regresses its support for standard markdown, I ask > > that a separate package be made for mistune 2.x to give more time for > > mistune 0.8.x users to migrate to mistune 2.x or another library entirely. > > > > Cheers, > > I'm not formally against it, but it's not really standard in my > opinion. It'd lead to maintenance of two packages, probably on some long > term (as it'd relieve the pressure to migrate for maintainers of > reverse-deps of mistune). > > Besides, having a source package named "mistune2" while upstream's > package is named "mistune" adds also another layer of complexity. > > I'd like to have some python team members' opinion on this, and I am not > sure to be eager to do it as of now. I agree that this sounds like a terrible idea. What would the Python module be called in a separate python3-mistune2 package be called? If it were called mistune, then python3-mistune2 would have to conflict with python3-mistune (or "python3-mistune0.84" or similar), and there would be little benefit. If the module itself were renamed to mistune2, then the Debian mistune package would be incompatible with any other software requiring mistune. Basically, the mistune upstream author has completely messed up on this by making what is essentially a completely different package with superficially similar functionality but the same name. What the Debian nbconvert maintainers have done is to vendor mistune 0.8.4: they now include it as _mistune.py within the Debian package, and have nbconvert do "import nbconvert.filters._mistune as mistune" (see /usr/lib/python3/dist-packages/nbconvert/filters/markdown_mistune.py). That seems like an eminently sensible solution to this problem. Maybe not ideal, but it will work until the upstream maintainers find a way to work with mistune 2.0.x. Best wishes, Julian
Bug#1003567: Please make a separate package for mistune 2.x
Hi Michael, "Michael R. Crusoe" wrote on 24/01/2022 at 11:39:23+0100: > [[PGP Signed Part:No public key for 3C26763F6C67E6E2 created at > 2022-01-24T11:39:23+0100 using RSA]] > On Tue, 11 Jan 2022 22:38:44 +0100 p...@debian.org wrote: >> Package: python3-schema-salad >> Severity: wishlist >> >> Dear maintainer, >> >> Your package python3-schema-salad depends on python3-mistune 0.8.4 >> which is no longer maintained and deprecated in favour of version >> 2.0.0. >> >> As python3-mistune 2.0.0 is not backward compatible, I reverted the >> upload of it that I did in unstable and python3-mistune 0.8.4 will >> stay around a bit longer. >> >> In the meantime, please try to see if upstream of >> python3-schema-salad either released a version that is compatible >> with python3-mistune 2.0.0 or if python3-schema-salad can easily be >> fixed to support such a version. As soon as you're ready to upload >> your package please update this bug report and we'll try to >> coordinate so that I release python3-mistune at a time that is fit >> for you to also release python3-schema-salad. > > Dear Pierre-Elliott Bécue, > > Thank you for trying to find a way forward in this mess. I am also the > upstream maintainer for schema-salad. We have yet to find a way to > support mistune 2.x > > Here is our attempt so far: > > https://github.com/common-workflow-language/schema_salad/pull/496/files?short_path=5c6a130#diff-5c6a1301c6b59b30a040d747d065e861d3dd98bde0e5a4356d92d594e9835986 > > And an issue I opened with mistune directly about a regression in > multi-line list handling that goes against multiple markdown specs: > https://github.com/lepture/mistune/issues/296 > > We have not decided if we will switch to another library, wait for a > fixed mistune 2.x release, or keep with the older version. > >> I intend to upload src:mistune 2.0.0 to unstable between March the > >> 15th and April the 15th (depending on the progress of its >> reverse-dependencies). I'll raise the severity of this bug >> approximately a month before making such an upload. > > Since Mistune 2.0.0 regresses its support for standard markdown, I ask > that a separate package be made for mistune 2.x to give more time for > mistune 0.8.x users to migrate to mistune 2.x or another library entirely. > > Cheers, I'm not formally against it, but it's not really standard in my opinion. It'd lead to maintenance of two packages, probably on some long term (as it'd relieve the pressure to migrate for maintainers of reverse-deps of mistune). Besides, having a source package named "mistune2" while upstream's package is named "mistune" adds also another layer of complexity. I'd like to have some python team members' opinion on this, and I am not sure to be eager to do it as of now. Cheers! -- PEB signature.asc Description: PGP signature
Bug#1003567: Please make a separate package for mistune 2.x
On Tue, 11 Jan 2022 22:38:44 +0100 p...@debian.org wrote: > Package: python3-schema-salad > Severity: wishlist > > Dear maintainer, > > Your package python3-schema-salad depends on python3-mistune 0.8.4 > which is no longer maintained and deprecated in favour of version > 2.0.0. > > As python3-mistune 2.0.0 is not backward compatible, I reverted the > upload of it that I did in unstable and python3-mistune 0.8.4 will > stay around a bit longer. > > In the meantime, please try to see if upstream of > python3-schema-salad either released a version that is compatible > with python3-mistune 2.0.0 or if python3-schema-salad can easily be > fixed to support such a version. As soon as you're ready to upload > your package please update this bug report and we'll try to > coordinate so that I release python3-mistune at a time that is fit > for you to also release python3-schema-salad. Dear Pierre-Elliott Bécue, Thank you for trying to find a way forward in this mess. I am also the upstream maintainer for schema-salad. We have yet to find a way to support mistune 2.x Here is our attempt so far: https://github.com/common-workflow-language/schema_salad/pull/496/files?short_path=5c6a130#diff-5c6a1301c6b59b30a040d747d065e861d3dd98bde0e5a4356d92d594e9835986 And an issue I opened with mistune directly about a regression in multi-line list handling that goes against multiple markdown specs: https://github.com/lepture/mistune/issues/296 We have not decided if we will switch to another library, wait for a fixed mistune 2.x release, or keep with the older version. > I intend to upload src:mistune 2.0.0 to unstable between March the > 15th and April the 15th (depending on the progress of its > reverse-dependencies). I'll raise the severity of this bug > approximately a month before making such an upload. Since Mistune 2.0.0 regresses its support for standard markdown, I ask that a separate package be made for mistune 2.x to give more time for mistune 0.8.x users to migrate to mistune 2.x or another library entirely. Cheers, -- Michael R. Crusoe OpenPGP_signature Description: OpenPGP digital signature
