+Matthias Feys <matth...@ml6.eu> +Valentyn Tymofieiev <valen...@google.com> +Mark Liu <mark...@google.com> could add more details here since they are working on Python 3 for a while now.
Hopeful state is that we might have python 3 working with DirectRunner in the release after this one (2.12). Mark is also working on getting python 3 working on a cluster. He has been able to run WordCount on Dataflow service with some hacks but it was not yet ready to run out of the box. I would like to note that we are targeting python 3 support only for portable runners so running on Dataflow and Flink should happen at the same time. Matthias and Valentyn are still working on converting the SDK to be Python 3 compatible. They are now mostly dealing with harder to convert parts of the SDK. (e.g. Parts were python 2/3 differences results in performance regressions or subtle changes in the behavior). To the folks working on this, it would be really helpful if you could update BEAM-1251 regularly. We have shared this issue with many people and not all of them will read this thread. Ahmet On Mon, Jan 7, 2019 at 8:15 AM Maximilian Michels <m...@apache.org> wrote: > Also curious because I see Python 3 requests quite often. I always say, > we're > close, but how close are we? :) > > Thanks, > Max > > On 05.01.19 00:03, Manu Zhang wrote: > > Guys, > > > > Happy New Year !!! > > I haven't got much time to contribute to Python 3 support. What is the > progress > > now ? It seems there are quite a few open issues under > > https://issues.apache.org/jira/browse/BEAM-1251. People have kept > asking about > > Python 3 support in tf.transform > > (https://github.com/tensorflow/transform/issues/1) which is blocked by > BEAM-1251. > > > > Thanks, > > Manu Zhang > > > > > > On Fri, Oct 12, 2018 at 3:17 AM Valentyn Tymofieiev <valen...@google.com > > <mailto:valen...@google.com>> wrote: > > > > I cc'ed a few folks who are familiar with Jenkins setup on > > https://issues.apache.org/jira/browse/BEAM-5663, I think we can > continue the > > discussion there or start a separate thread. > > > > On Wed, Oct 10, 2018 at 8:54 PM Manu Zhang <owenzhang1...@gmail.com > > <mailto:owenzhang1...@gmail.com>> wrote: > > > > Does anyone know how to set up python version on Jenkins ? It’s > Python > > 3.5.2 now. > > > > Thanks, > > Manu Zhang > > On Oct 5, 2018, 9:24 AM +0800, Valentyn Tymofieiev < > valen...@google.com > > <mailto:valen...@google.com>>, wrote: > >> I have put together a guide [1] to help get started with > investigating > >> Python 3-related test failures that may be helpful for new folks > >> joining the effort. > >> > >> Comments and improvements welcome! > >> > >> Thanks, > >> Valentyn > >> [1] > >> > https://docs.google.com/document/d/1s1BJVCY65LB_SYK1SU1u7NbZiFANoq-nEYaEvzRbYlA > >> > >> > >> On Thu, Oct 4, 2018 at 11:26 AM Valentyn Tymofieiev > >> <valen...@google.com <mailto:valen...@google.com>> wrote: > >> > >> I agree there is some overlap between JIRAs that track > individual > >> failures and module-level JIRAs. We originally wanted to do > the > >> conversion on a module-by-module basis, however we learned > that > >> test failures in some modules require changes in other > modules, > >> and it may be a little easier to slice the problem if we > focus on > >> classes of failures. > >> > >> Module-level JIRAs can still be useful for tracking the end > >> result: tox suites cover all tests in the module in Py3 > >> environment, and there are no disabled tests in the module > that > >> don't have individual JIRAs tracking them. > >> > >> I suggest that folks who are working on module-level JIRAs > assign > >> to themselves the JIRAs that track individual failures > if/when > >> they are actively addressing them. This way, unassigned > >> problem-specific JIRAs can use help from the community. > >> > >> Thanks, > >> Valentyn > >> > >> > >> On Wed, Oct 3, 2018 at 8:14 PM Manu Zhang < > owenzhang1...@gmail.com > >> <mailto:owenzhang1...@gmail.com>> wrote: > >> > >> Thanks Valentyn. Note some test failing issues are > covered by > >> “Finish Python 3 porting for *** module”, e.g. > >> https://issues.apache.org/jira/browse/BEAM-5315. > >> > >> Manu > >> 在 2018年10月3日 +0800 PM4:18，Valentyn Tymofieiev > >> <valen...@google.com <mailto:valen...@google.com>>，写道： > >>> Hi Rakesh and Manu, > >>> > >>> Thanks to both of you for offering help (in different > >>> threads). It's great to see that more and more people > get > >>> involved with helping to make Beam Python 3 compatible! > >>> > >>> There are a few PRs in flight, and several people in > the > >>> community actively work on Python 3 support now. I > would be > >>> happy to coordinate the work so that we don't step at > each > >>> others toes and avoid duplication of effort. > >>> > >>> I recently looked at unit tests that are still failing > in > >>> Python 3 environment and filed a few issues (within > range > >>> BEAM-5615 - BEAM-5629), to track similar classes of > errors. > >>> You can also find them on Kanban board [1]. > >>> In particular, BEAM-5620 and BEAM-5627 should be easy > issues > >>> to get started. > >>> > >>> There are multiple ways you can help: > >>> - Helping to rootcause errors. Even a comment why a > test is > >>> failing and a suggestion how to fix it, will be > helpful for > >>> others when you don't have time to do the fix. > >>> - Helping with code reviews. > >>> - Reporting new issues (as subtasks to BEAM-1251), > >>> deduplicating or splitting the existing issues. We > probably > >>> don't want to file a Jira for each of 250+ currently > failing > >>> tests at this point, but it may make sense to track the > >>> errors that occur repeatedly share the root cause. > >>> - Fixing the issues. Feel free to assign an issue to > yourself > >>> if you have a fix in mind and plan to actively work on > it. > >>> Due to the nature of the problem it may occasionally > happen > >>> that two issues share the rootcause, or fixing one > issue is a > >>> prerequisite for fixing another issue, so sync to > master > >>> often to make sure the issue you are working on is not > >>> already fixed. > >>> > >>> I'll also keep an eye on the PRs and will try to keep > the > >>> list of open issues up to date. > >>> > >>> Thanks, > >>> Valentyn > >>> > >>> [1]: > >>> > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245&view=detail > >>> > >>> > >>> On Tue, Oct 2, 2018 at 9:38 AM Pablo Estrada > >>> <pabl...@google.com <mailto:pabl...@google.com>> > wrote: > >>> > >>> Very cool : ) I'm also available to review / merge > if you > >>> need help from my side. > >>> Best > >>> -P. > >>> > >>> On Tue, Oct 2, 2018 at 7:45 AM Rakesh Kumar > >>> <rakeshku...@lyft.com <mailto:rakeshku...@lyft.com>> > wrote: > >>> > >>> Hi Rob, > >>> > >>> I am, Rakesh Kumar, using Beam SDK for one of > my > >>> projects at Lyft. I have been working closely > with > >>> Thomas Weise. I have already met a couple of > Python > >>> SDK developers in person. > >>> I am interested to help migrate to Python 3. > You can > >>> assign me PRs for review. I am also more than > happy > >>> to take a simple ticket to begin development > work on > >>> Beam. > >>> > >>> Thank you, > >>> Rakesh > >>> > >>> On Wed, Sep 5, 2018 at 9:12 AM Robbe Sneyders > >>> <robbe.sneyd...@ml6.eu > >>> <mailto:robbe.sneyd...@ml6.eu>> wrote: > >>> > >>> Hi everyone, > >>> > >>> With the merging of [1], we now have > Python 3 > >>> tests running on Jenkins, which allows us > to move > >>> forward with the last step of the Python 3 > porting. > >>> > >>> You can follow the progress on the Jira > Kanban > >>> Board [2]. If you're interested in helping > by > >>> porting a module, you can assign one of the > >>> issues to yourself and start coding. You > can find > >>> the different steps outlined in the design > >>> document [3]. > >>> > >>> We could also use some extra reviewers. If > you're > >>> interested, let us know, and we'll tag you > in our > >>> PRs. > >>> > >>> [1] > https://github.com/apache/beam/pull/6266 > >>> [2] > >>> > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245 > >>> [3] https://s.apache.org/beam-python-3 > >>> > >>> kind regards, > >>> Robbe > >>> -- > >>> > >>> https://ml6.eu <https://ml6.eu/> > >>> > >>> > >>> > >>> > >>> *Robbe Sneyders* > >>> > >>> ML6 Gent > >>> < > https://www.google.be/maps/place/ML6/@51.037408,3.7044893,17z/data=!3m1!4b1!4m5!3m4!1s0x47c37161feeca14b:0xb8f72585fdd21c90!8m2!3d51.037408!4d3.706678?hl=nl > > > >>> > >>> M: +32 474 71 31 08 > <tel:+32%20474%2071%2031%2008> > >>> > >>> -- > >>> Rakesh Kumar > >>> Software Engineer > >>> 510-761-1364 <tel:(510)%20761-1364> | > >>> <https://www.lyft.com/> > >>> >
