Hi Gyala and Gabor, Pyflink has provided arm64 wheel packages for Apple silicon since 1.16[1]. The use of Miniconda is only related to ci testing and packaging on linux platform, and building mac platform wheels are dependent on cibuildwheel[2]. So I guess you want to run ci tests on the m1 environment, but the current version of miniconda cannot meet this requirement, so there is a pre-step that must drop python 3.7?
[1] https://pypi.org/project/apache-flink/1.16.2/#files [2] https://github.com/apache/flink/blob/master/tools/azure-pipelines/build-python-wheels.yml#L30 Best, Xingbo Gabor Somogyi <gabor.g.somo...@gmail.com> 于2023年9月6日周三 14:36写道： > Hi Xingbo, > > *Constraint:* > I personally not found any miniconda version which provides arm64 support > together with python 3.7. > [image: image.png] > > At the moment I think new platform support means 3.7 drop. > > I fully to agree with Gyula, if we start now maybe we can release it in > half a year however *3.7 active support already ended in 27 Jun 2020*. > At the moment any python development/test execution on MacOS M1 is just > not working as-is just like any kind of python test execution on any ARM > CPU. > > Gains: > * We can release a working version in half a year hopefully and not > shifting support to 1+ year > * MacOS M1 local development would work finally which is essential for > user engagement > * It would be possible to execute python tests on ARM64 machines > * We can shake up the python development story because it's not the most > loved area > > BR, > G > > > On Wed, Sep 6, 2023 at 8:06 AM Gyula Fóra <gyula.f...@gmail.com> wrote: > >> Hi Xingbo! >> >> I think we have to analyze what we gain by dropping 3.7 and upgrading to a >> miniconda version with a multiarch support. >> >> If this is what we need to get Apple silicon support then I think it's >> worth doing it already in 1.19. Keep in mind that 1.18 is not even >> released >> yet so if we delay this to 1.20 that is basically 1 year from now. >> Making this change can increase the adoption instantly if we enable new >> platforms. >> >> Cheers, >> Gyula >> >> On Wed, Sep 6, 2023 at 4:46 AM Xingbo Huang <hxbks...@gmail.com> wrote: >> >> > Hi Gabor, >> > >> > Thanks for bringing this up. In my opinion, it is a bit aggressive to >> > directly drop Python 3.7 in 1.19. Python 3.7 is still used a lot[1], >> and as >> > far as I know, many Pyflink users are still using python 3.7 as their >> > default interpreter. I prefer to deprecate Python 3.7 in 1.19 just like >> we >> > deprecated Python 3.6 in 1.16[2] and dropped Python 3.6 in 1.17[3]. >> > >> > For the support of Python 3.11, I am very supportive of the >> implementation >> > in 1.19 (many users have this appeal, and I originally wanted to >> support it >> > in 1.18). >> > >> > Regarding the miniconda upgrade, I tend to upgrade miniconda to the >> latest >> > version that can support python 3.7 to 3.11 at the same time. >> > >> > [1] https://w3techs.com/technologies/history_details/pl-python/3 >> > [2] https://issues.apache.org/jira/browse/FLINK-28195 >> > [3] https://issues.apache.org/jira/browse/FLINK-27929 >> > >> > Best, >> > Xingbo >> > >> > Jing Ge <j...@ververica.com.invalid> 于2023年9月5日周二 04:10写道： >> > >> > > +1 >> > > >> > > @Dian should we add support of python 3.11 >> > > >> > > Best regards, >> > > Jing >> > > >> > > On Mon, Sep 4, 2023 at 3:39 PM Gabor Somogyi < >> gabor.g.somo...@gmail.com> >> > > wrote: >> > > >> > > > Thanks for all the responses! >> > > > >> > > > Based on the suggestions I've created the following jiras and >> started >> > to >> > > > work on them: >> > > > * https://issues.apache.org/jira/browse/FLINK-33029 >> > > > * https://issues.apache.org/jira/browse/FLINK-33030 >> > > > >> > > > The reason why I've split them is to separate the concerns and >> reduce >> > the >> > > > amount of code in a PR to help reviewers. >> > > > >> > > > BR, >> > > > G >> > > > >> > > > >> > > > On Mon, Sep 4, 2023 at 12:57 PM Sergey Nuyanzin < >> snuyan...@gmail.com> >> > > > wrote: >> > > > >> > > > > +1, >> > > > > Thanks for looking into this. >> > > > > >> > > > > On Mon, Sep 4, 2023 at 8:38 AM Gyula Fóra <gyula.f...@gmail.com> >> > > wrote: >> > > > > >> > > > > > +1 >> > > > > > Thanks for looking into this. >> > > > > > >> > > > > > Gyula >> > > > > > >> > > > > > On Mon, Sep 4, 2023 at 8:26 AM Matthias Pohl < >> > matthias.p...@aiven.io >> > > > > > .invalid> >> > > > > > wrote: >> > > > > > >> > > > > > > Thanks Gabor for looking into it. It sounds reasonable to me >> as >> > > well. >> > > > > > > >> > > > > > > +1 >> > > > > > > >> > > > > > > On Sun, Sep 3, 2023 at 5:44 PM Márton Balassi < >> > > > > balassi.mar...@gmail.com> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > Hi Gabor, >> > > > > > > > >> > > > > > > > Thanks for bringing this up. Similarly to when we dropped >> > Python >> > > > 3.6 >> > > > > > due >> > > > > > > to >> > > > > > > > its end of life (and added 3.10) in Flink 1.17 [1,2], it >> makes >> > > > sense >> > > > > to >> > > > > > > > proceed to remove 3.7 and add 3.11 instead. >> > > > > > > > >> > > > > > > > +1. >> > > > > > > > >> > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-27929 >> > > > > > > > [2] https://github.com/apache/flink/pull/21699 >> > > > > > > > >> > > > > > > > Best, >> > > > > > > > Marton >> > > > > > > > >> > > > > > > > On Fri, Sep 1, 2023 at 10:39 AM Gabor Somogyi < >> > > > > > gabor.g.somo...@gmail.com >> > > > > > > > >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > Hi All, >> > > > > > > > > >> > > > > > > > > I've analyzed through part of the pyflink code and found >> some >> > > > > > > improvement >> > > > > > > > > possibilities. >> > > > > > > > > I would like to hear voices on the idea. >> > > > > > > > > >> > > > > > > > > Intention: >> > > > > > > > > * upgrade several python related versions to eliminate >> > > > end-of-life >> > > > > > > issues >> > > > > > > > > and keep up with bugfixes >> > > > > > > > > * start to add python arm64 support >> > > > > > > > > >> > > > > > > > > Actual situation: >> > > > > > > > > * Flink supports the following python versions: 3.7, 3.8, >> > 3.9, >> > > > 3.10 >> > > > > > > > > * We use miniconda 4.7.10 (python package management >> system >> > and >> > > > > > > > environment >> > > > > > > > > management system) which supports the following python >> > > versions: >> > > > > 3.7, >> > > > > > > > 3.8, >> > > > > > > > > 3.9, 3.10 >> > > > > > > > > * Our python framework is not supporting anything but >> x86_64 >> > > > > > > > > >> > > > > > > > > Issues: >> > > > > > > > > * Python 3.7.17 is the latest security patch of the 3.7 >> line. >> > > > This >> > > > > > > > version >> > > > > > > > > is end-of-life and is no longer supported: >> > > > > > > > > https://www.python.org/downloads/release/python-3717/ >> > > > > > > > > * Miniconda 4.7.10 is released on 2019-07-29 which is 4 >> years >> > > old >> > > > > > > already >> > > > > > > > > and not supporting too many architectures (x86_64 and >> > ppc64le) >> > > > > > > > > * The latest miniconda which has real multi-arch feature >> set >> > > > > supports >> > > > > > > the >> > > > > > > > > following python versions: 3.8, 3.9, 3.10, 3.11 and no 3.7 >> > > > support >> > > > > > > > > >> > > > > > > > > Suggestion to solve the issues: >> > > > > > > > > * In 1.19 drop python 3.7 support and upgrade miniconda to >> > the >> > > > > latest >> > > > > > > > > version which opens the door to other platform + python >> 3.11 >> > > > > support >> > > > > > > > > >> > > > > > > > > Please note python 3.11 support is not initiated/discussed >> > > here. >> > > > > > > > > >> > > > > > > > > BR, >> > > > > > > > > G >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > Best regards, >> > > > > Sergey >> > > > > >> > > > >> > > >> > >> >
