We actually do try to maintain state (and pipeline shape) compatibility between Beam versions (e.g. this is why we have multiple distinct sketch implementations rather than "fix" the one). It's true that this is easier said than done, and some people are more vigilant about that than others). State-upgrading, and a better upgrade story in general, has been discussed from time to time, and it can get quite tricky but that doesn't mean we can't do better here. I know that Reuven has been thinking about these ideas.
As for why people don't upgrade, I think the largest reason is that which is common to all aversion to software upgrades: trading what works (and has a long track record) for something that merely should work is always a risk and even when things go well involves a certain amount of toil (testing, deployment, ...) On Tue, Apr 6, 2021 at 2:43 PM Jan Lukavský <je...@seznam.cz> wrote: > Hi, > do we know what is the reason users stay on an older version of Beam? My > guess would be that it is not related to API changes, but more likely to > state incompatibility. Maybe if we could figure out a way which would > enable a smooth migration of state (and timers) between Beam versions, that > might help? The migration would probably have to be runner-dependent, but > Beam might offer some tooling to make this easier. One example would be > coder evolution, where we currently do not have the option of "reading old > way, writing new way" with some "coder-version-registry". I suppose there > might have been a discussion about this in the past, does anyone know of > any conclusion? > > Jan > On 4/6/21 10:54 PM, Robert Bradshaw wrote: > > I do think there's value in having an LTS release, if there's sufficient > interest to fund it (specifically, figuring out who would be backporting > fixes and cutting the new releases). > > On Mon, Apr 5, 2021 at 1:14 PM Elliotte Rusty Harold <elh...@ibiblio.org> > wrote: > >> Hi, >> >> I'd like to return to the discussion around a long term support >> release that first came up here in 2018: >> >> >> https://lists.apache.org/thread.html/6ec572d8edfe93225edebec18792cbcf44ef447ffe54ea35549cdafe%40%3Cdev.beam.apache.org%3E >> >> This is important to some Google Cloud Dataflow Java customers, and >> likely others as well. >> >> Specifically, I'd like to propose cutting an LTS release off a branch >> and maintaining it with critical bug fixes and security updates for 18 >> months. Right now we're finding that the current one year support >> period and six week release cycle is a tad fast for some customers. >> >> There's some wiggle room in terms of what's "critical", but in >> that category I include security fixes and data integrity issues. >> Essentially, this is any bug so bad that, if found in a new release, >> we'd recommend customers wait for the fix before upgrading to the >> latest and greatest. The difference is we'd backport the patch to the >> not-latest-and-greatest release. >> >> To run something up the flagpole, I propose: >> >> 1. 2.28.0 becomes the first LTS release. >> 2. New patch versions are released as 2.28.1, 2.28.2, etc. >> 3. Patch releases do not change API, at all, except in the unlikely >> event this is absolutely required for a security fix. >> 4. Dependencies are not upgraded in patch releases unless required to >> fix a critical bug or security issue. >> 5. In a year, cut a new LTS release from whatever is then current so >> there's some overlap to give customers time to switch over. >> >> I picked 2.28.0 since it's the most recent release, and I prefer to >> stay off the bleeding edge for longterm support. This would also >> enable customers to develop on top of it sooner. However I understand >> others may well prefer to pick a different release such as 2.29.0 or >> 2.30.0. I'm OK with whatever recent version the community picks. >> >> Thoughts? >> >> -- >> Elliotte Rusty Harold >> elh...@ibiblio.org >> >
