Re: Python 3.11, bytecode and new internals
On Wed, Nov 23, 2022 at 12:21:38AM +0100, Thomas Goirand wrote: > On 11/22/22 17:59, Julian Gilbey wrote: > > > > Or should we mark them as X-Python3-Version: << 3.11 so they can stay in > > > > testing as long as Python 3.10 is the default? > > > > > > I don't think this is the way. > > > > I'm sorry, I don't understand - which is not the way? > > I don't think you should "mark them as X-Python3-Version: << 3.11", because > either we use 3.10 or 3.11 in Bookworm, I don't think that there's a plan > for having both interpreters available. It's more about the present time: if I mark them as << 3.11, then they will build fine and be able to stay in testing. If I don't, they'll be pulled from testing in the next few weeks because of FTBFS bugs. Obviously, once they do build OK with 3.11, then I can remove that clause. Best wishes, Julian
Re: Python 3.11, bytecode and new internals
On 11/22/22 17:59, Julian Gilbey wrote: Or should we mark them as X-Python3-Version: << 3.11 so they can stay in testing as long as Python 3.10 is the default? I don't think this is the way. I'm sorry, I don't understand - which is not the way? I don't think you should "mark them as X-Python3-Version: << 3.11", because either we use 3.10 or 3.11 in Bookworm, I don't think that there's a plan for having both interpreters available. But I still don't know what to do in the meantime with the spyder ecosystem besides either wait for upstream to sort bytecode and pydevd and Piotr (and possibly upstream) to sort parso, or to mark them as Python 3.10 only. Well, hopefully for you, you'll get it fixed before next January, or we go back to 3.10 only (or both?). Cheers, Thomas Goirand (zigo)
Re: Python 3.11, bytecode and new internals
On Tue, Nov 22, 2022 at 05:01:03PM +0100, Thomas Goirand wrote: > > If there are people with the expertise to help upstream update > > bytecode and parso (and probably several other low-level packages) for > > 3.11 so that the software that depends on them works with 3.11, then > > fine. (And it is a non-trivial update, AFAICT.) But until then, I'd > > be very reluctant to make 3.11 the default. > > Have you tried this PR? > https://github.com/MatthieuDartiailh/bytecode/pull/107 As you can see by reading it, there is still at least one blocking point (to use Matthieu's language), and Matthieu is the core bytecode developer. Once that is sorted, then pydevd needs to be updated to use it. But this is far outside my expertise, and I'm not going to apply a huge patch that I don't understand and that is known to still be buggy. So I'm just going to have to wait. I don't know the current state of parso; it would be interesting to see whether parso 0.8.3 successfully works under 3.11, but that is Piotr Ożarowski's package (not under the Python team); he is active, but has not yet uploaded this newer version. > > I haven't decided what to do with packages which now FTBFS under 3.11 > > because of this. Should we just let them fall out of testing (that > > certainly includes spyder, and quite possibly ipython as well)? > > Or should we mark them as X-Python3-Version: << 3.11 so they can stay in > > testing as long as Python 3.10 is the default? > > I don't think this is the way. I'm sorry, I don't understand - which is not the way? > > If we make 3.11 the > > default, these packages will likely not be in bookworm, which might > > upset our users even more. > > We're not there yet. We have until January to fix, and we haven't decided > yet if 3.11 will be the default. Fair enough. But I still don't know what to do in the meantime with the spyder ecosystem besides either wait for upstream to sort bytecode and pydevd and Piotr (and possibly upstream) to sort parso, or to mark them as Python 3.10 only. Best wishes, Julian
Re: Python 3.11, bytecode and new internals
On 11/22/22 10:59, Julian Gilbey wrote: On Tue, Nov 22, 2022 at 09:22:05AM +0100, Thomas Goirand wrote: this, 100 times I very much don't agree. I think it's going pretty well, and the number of breakage isn't high. We just need a little bit of effort to make it in good enough shape. [...] Now, out of *many* of my packages, only a very few broke. Complicated packages like Eventlet for example, just worked. Others had upstream patches I applied. And I am in the opinion that we should go ahead and make 3.11 the default. If there are people with the expertise to help upstream update bytecode and parso (and probably several other low-level packages) for 3.11 so that the software that depends on them works with 3.11, then fine. (And it is a non-trivial update, AFAICT.) But until then, I'd be very reluctant to make 3.11 the default. Have you tried this PR? https://github.com/MatthieuDartiailh/bytecode/pull/107 I haven't decided what to do with packages which now FTBFS under 3.11 because of this. Should we just let them fall out of testing (that certainly includes spyder, and quite possibly ipython as well)? Or should we mark them as X-Python3-Version: << 3.11 so they can stay in testing as long as Python 3.10 is the default? I don't think this is the way. If we make 3.11 the default, these packages will likely not be in bookworm, which might upset our users even more. We're not there yet. We have until January to fix, and we haven't decided yet if 3.11 will be the default. Cheers, Thomas Goirand (zigo)
Re: Python 3.11, bytecode and new internals
On Monday, November 21, 2022 12:25:05 PM EST Scott Kitterman wrote: > On November 21, 2022 5:02:57 PM UTC, "Louis-Philippe Véronneau" wrote: > >On 2022-11-21 02 h 08, Julian Gilbey wrote: > >> I'm just flagging this up here, with a question about how we should > >> proceed. Certainly we are not ready to make Python 3.11 the default > >> Python version!! > > > >This is a concern I share and I think I've been pretty vocal about it. > > > >I feel the state of python packages for Bookworm with 3.10 was pretty good > >and it seemed reasonable to prioritize stability for our next stable > >release :) > > > >It's very frustrating to work on packaging python libraries and apps for a > >whole release cycle, just to see all that work put in the bin at the last > >minute because upstream doesn't support 3.11... > > > >I've been told the current 3.11 transition was a test, and if it was clear > >too many important things were broken and couldn't be fixed, we would roll > >back and release using 3.10. > Looks like you can add #1024521 to the list. This one is fixed. Scott K signature.asc Description: This is a digitally signed message part.
Re: Python 3.11, bytecode and new internals
On Tue, Nov 22, 2022 at 09:22:05AM +0100, Thomas Goirand wrote: > > this, 100 times > > I very much don't agree. I think it's going pretty well, and the number of > breakage isn't high. We just need a little bit of effort to make it in good > enough shape. > [...] > Now, out of *many* of my packages, only a very few broke. Complicated > packages like Eventlet for example, just worked. Others had upstream patches > I applied. And I am in the opinion that we should go ahead and make 3.11 the > default. If there are people with the expertise to help upstream update bytecode and parso (and probably several other low-level packages) for 3.11 so that the software that depends on them works with 3.11, then fine. (And it is a non-trivial update, AFAICT.) But until then, I'd be very reluctant to make 3.11 the default. I haven't decided what to do with packages which now FTBFS under 3.11 because of this. Should we just let them fall out of testing (that certainly includes spyder, and quite possibly ipython as well)? Or should we mark them as X-Python3-Version: << 3.11 so they can stay in testing as long as Python 3.10 is the default? If we make 3.11 the default, these packages will likely not be in bookworm, which might upset our users even more. Best wishes, Julian
Re: Python 3.11, bytecode and new internals
On 11/21/22 18:30, Sandro Tosi wrote: On Mon, Nov 21, 2022 at 12:03 PM Louis-Philippe Véronneau wrote: On 2022-11-21 02 h 08, Julian Gilbey wrote: I'm just flagging this up here, with a question about how we should proceed. Certainly we are not ready to make Python 3.11 the default Python version!! This is a concern I share and I think I've been pretty vocal about it. I feel the state of python packages for Bookworm with 3.10 was pretty good and it seemed reasonable to prioritize stability for our next stable release :) It's very frustrating to work on packaging python libraries and apps for a whole release cycle, just to see all that work put in the bin at the last minute because upstream doesn't support 3.11... this, 100 times I very much don't agree. I think it's going pretty well, and the number of breakage isn't high. We just need a little bit of effort to make it in good enough shape. I've been told the current 3.11 transition was a test, and if it was clear too many important things were broken and couldn't be fixed, we would roll back and release using 3.10. Though we're not there yet, as a point were we can say it's a lost battle. why are we running a "test" this close to the release? Let me fix the above sentence for you: s/release/freeze/ Because Python 3.11 brings many nice feature, one of them being that it's 15 to 50% faster, very often 20% faster. Also, releasing the last Debian with an already obsolete interpreter version doesn't feel good. *who* are we running this test for? who made this decision (i figure RT gave the go ahead, but still)? is there any searchable source for this claim? This was discussed during Debconf in Prizren. You are always free to: - join us during debconf - join on IRC if you can't go to Debconf - read the video and voice your concerns on the list So the decision was made collectively. We also discussed this on IRC, and in this very list, if I'm not mistaking (sorry, I will not search for it). Currently, Python 3.11 is only an "available interpreter version", not the default. So it's kind of easy to revert (we would "only" need to remove the 3.11 interpreter, and rebuild the packages that produce 3.11 so files). It would be a lot harder to revert having 3.11 as default, Mathias said/wrote. So we're good... I very much thank Stefano for the many fixes he already uploaded. Now, out of *many* of my packages, only a very few broke. Complicated packages like Eventlet for example, just worked. Others had upstream patches I applied. And I am in the opinion that we should go ahead and make 3.11 the default. I'd be happy to have the opinion of the rest of the team, especially Doko and Stefano. Cheers, Thomas Goirand (zigo)
