Thanks for bringing this up. I've actually been thinking about the same thing (specifically with regards to 3.5 and 3.8).
I think it would makes sense to add support for 3.8 right away (or at least get a good sense of what work needs to be done and what our dependency situation is like), and to drop support for 3.5 soonish (in particular, Python 3.5 is EOL 2020-09-13 [1], and also it'd be good if users porting from Python 2.7 don't port to it). As for testing and infrastructure, we probably don't need to test all permutations on all releases (though exactly what the correct subsetting should be is TBD). Likely we could run complete suites for the oldest and newest supported version, slimmed down, cheaper versions for the middle ones, and maybe full post-commits (which would give us signals into what our slimmed-down coverage is). [1] https://devguide.python.org/#status-of-python-branches On Wed, Feb 26, 2020 at 3:00 PM Valentyn Tymofieiev <valen...@google.com> wrote: > > I would like to start a discussion about identifying a guideline for > answering questions like: > > 1. When will Beam support a new Python version (say, Python 3.8)? > 2. When will Beam drop support for an old Python version (say, Python 3.5)? > 3. How many Python versions should we aim to support concurrently > (investigate issues, have continuous integration tests)? > 4. What comes first: adding support for a new version (3.8) or deprecating > older one (3.5)? This may affect the max load our test infrastructure needs > to sustain. > > We are already getting requests for supporting Python 3.8 and there were some > good reasons[1] to drop support for Python 3.5 (at least, early versions of > 3.5). Answering these questions would help set expectations in Beam user > community, Beam dev community, and may help us establish resource > requirements for test infrastructure and plan efforts. > > PEP-0602 [2] establishes a yearly release cycle for Python versions starting > from 3.9. Each release is a long-term support release and is supported for 5 > years: first 1.5 years allow for general bug fix support, remaining 3.5 years > have security fix support. > > At every point, there may be up to 5 Python minor versions that did not yet > reach EOL, see "Release overlap with 12 month diagram" [3]. We can try to > support all of them, but that may come at a cost of velocity: we will have > more tests to maintain, and we will have to develop Beam against a lower > version for a longer period. Supporting less versions will have implications > for user experience. It also may be difficult to ensure support of the most > recent version early, since our dependencies (e.g. picklers) may not be > supporting them yet. > > Currently we support 4 Python versions (2.7, 3.5, 3.6, 3.7). > > Is 4 versions a sweet spot? Too much? Too little? What do you think? > > [1] https://github.com/apache/beam/pull/10821#issuecomment-590167711 > [2] https://www.python.org/dev/peps/pep-0602/ > [3] https://www.python.org/dev/peps/pep-0602/#id17
