I agree with everything that you said. However by 'source release', I was merely referring to a Python/PyPI packaging technicality, where we distribute the package in the form that is generated by *python setup.py sdist*, as opposed to the pre-built binary egg or wheel formats.
But since you brought this up, do we need to create actual zip/tarballs of the source during release, or is creating a git tag enough ? And if we do, do we package the full queryserver repo, or just the python subdirectory ? The docs in the package assume that the source is built from the full repo. On Tue, Jun 16, 2020 at 6:24 PM Josh Elser <[email protected]> wrote: > Yup, let's just draw a number (whatever we'd like, as long as it follows > the standard PEP for versioning). When we release it, we can advertise > whatever level of stability we expect it to have (i.e. "This is an > alpha, you can try it if you want"). > > ASF is very strict when it comes to releases in source form. In the > legalese for the foundation, _only_ source releases are actually > considered a release. That means, all of the foundation documentation > about release policies applies to just the source release, further, > that's really all we have to vote on. > > My current interpretation of the policy is that we should also try to > validate any binaries we build to make sure that they still follow > proper licensing guidelines for the foundation (but this is a "should" > and not a "must" because they are not technically real releases). > > All that to say, when we do a vote in the ASF, it must be on source. If > we have an egg to also use for testing, great. But, the pushing an > approved ASF release to PyPi is a post-vote step. > > Shout if this doesn't make sense. It reads like you're on the right path :) > > On 6/16/20 3:06 AM, Istvan Toth wrote: > > The Beam guide looks interesting, we could certainly borrow from it. > > > > I'm fine with your suggested process. Here's my interpretation of it: > > > > - No dev releases on PyPI > > - Decoupling PhoenixDB versioning from PQS versioning > > - Making a proper minor/patch release when important features/fixes > land > > - Testing the release process /making RC releases on TestPyPI > > > > One more detail that we haven't discussed yet: > > The previous PyPI releases were source packages. > > I suggest that we can keep releasing PhoenixDB as a source package, as it > > is pure python. > > > > Istvan > > > > On Tue, Jun 16, 2020 at 3:32 AM Josh Elser <[email protected]> wrote: > > > >> ASF releases do not have to be on the order of years. As long as we have > >> three people to vote, we can do releases as often as we'd like. The > >> lower-bound on release cadence is probably on the order of "weeks" (as > >> opposed to days) but that's primarily limited by bandwidth of our > >> volunteers :) > >> > >> Anyways, if that's the major worry, let's just push to doing proper > >> votes. We should not have the problem in having people turn out to vote. > >> > >> I'll put L&N review to the top of my list to do (my) tomorrow morning. > >> Sorry I haven't gotten to it yet. > >> > >> On 6/15/20 1:42 PM, Istvan Toth wrote: > >>> TestPyPi certainly seems useful to practice the release process, and > >> avoid > >>> mistakes with it. > >>> > >>> However, I don't think that it is a substitute for a .devN release, > which > >>> (in my mind) is meant to give the users something to use until we get > all > >>> our ducks in a row to do a proper 1.0 release. The primary user in this > >>> case would be Hue, which needs the new features, and is currently > >> mirroring > >>> our dev sources into their own repo. (TBH, that's their normal modus > >>> operandi anyway, but for their Python 3 version, they'd like to switch > to > >>> PyPI modules) > >>> > >>> To use a dev package, the users have to explicitly specify the exact > >>> development version, (or go all-out, and use development versions of > all > >>> packages), so there is no chance of them accidentally upgrading from a > >>> stable release. Making them specify another repo as well to install a > dev > >>> release sounds like too much to me. > >>> > >>> If on the other hand we plan on releasing the current state of > PhoenixDB > >>> as 1.0 soon (like in month), and then do frequent-ish releases as new > >>> features/fixes are (hopefully) added, then we can skip the dev releases > >>> altogether. The Phoenix norm so far is more like yearly releases (if we > >> are > >>> lucky). > >>> > >>> regards > >>> István > >>> > >>> On Mon, Jun 15, 2020 at 7:08 PM Josh Elser <[email protected]> wrote: > >>> > >>>> Did a little searching.. > >>>> > >>>> * Found a 2011 blog post from sqlalchemy which said (as a project) > they > >>>> would not post devN releases to pypi > >>>> * There's a TestPyPi [1] instead which seems to be for staging work. > >>>> > >>>> Could we play with staging there? And then push to pypi (real) after > we > >>>> do a normal vote? > >>>> > >>>> I think we can keep our python release super-low friction :) > >>>> > >>>> [1] https://packaging.python.org/guides/using-testpypi/ > >>>> > >>>> On 6/15/20 1:02 PM, Josh Elser wrote: > >>>>> Hey Istvan, > >>>>> > >>>>> Great of you to drive this work! > >>>>> > >>>>> I do have one concern about pushing the dev releases to PyPi (I'm > >>>>> assuming that's what you mean). I understand that in the Python > world a > >>>>> "dev" release indicates that this isn't an "official" release [1]. > >>>>> > >>>>> At the ASF, you're correct that we, developers, are empowered to make > >>>>> "builds" of our product(s) for the sake of our development. There is > a > >>>>> clear line when that "build" is published in a location to which a > user > >>>>> may find it and begin to use it. There has been at least one example > in > >>>>> recent memory of a project which made "developer only releases" > without > >>>>> proper voting, but published them in a high-visibility location and > >>>>> (inadvertently or intentionally) circumvented the ASF release > >>>> requirements. > >>>>> > >>>>> My biggest concern is: would a user who ran a `pip install phoenixdb` > >>>>> after we make a devN release get the last-stable release (0.7) or the > >>>>> new devN release? If they would get the dev release, does PyPi give > us > >>>>> any flexibility to prevent this from happening? I believe that we > >> should > >>>>> consider publishing to the "official" location should be treated as a > >>>>> release if it means that a user could begin to use it with "low > >>>> friction". > >>>>> > >>>>> - Josh > >>>>> > >>>>> > >>>>> [1] https://www.python.org/dev/peps/pep-0440/#id24 > >>>>> > >>>>> On 6/12/20 7:11 AM, Istvan Toth wrote: > >>>>>> Hi! > >>>>>> > >>>>>> Even though we have adopted the PhoenixDB driver in 2018, there > hasn't > >>>>>> been > >>>>>> much activity on it, and the version available on PyPI is still the > >>>>>> old 0.7 > >>>>>> release by Lukas. > >>>>>> > >>>>>> Recently I have worked on it quite a bit, adding fixes and new > >> features, > >>>>>> and adopting the partial SQLAlchemy driver from pyPhoenix, thus > >> enabling > >>>>>> Hue support. > >>>>>> > >>>>>> I plan to start releasing the driver publicly on PyPI. Lukas has > >> kindly > >>>>>> shared control of the PyPI phoenixdb project with us, so we are good > >>>>>> to go. > >>>>>> > >>>>>> The short-term plan is to release 1.0.0.dev0 and later 1.0.0.devN > >>>>>> releases > >>>>>> from the current HEAD of phoenix-queryserver. As these will be dev > >>>>>> releases, I am not planning to follow a formal release process for > >>>> these. > >>>>>> > >>>>>> When and how to release 1.0.0 final, and the versioning > scheme/process > >>>> to > >>>>>> use after that are still not finalized. > >>>>>> > >>>>>> Please join the discussion here, or in > >>>>>> https://issues.apache.org/jira/browse/PHOENIX-5939 if you have any > >>>>>> questions or suggestions! > >>>>>> > >>>>>> regards > >>>>>> Istvan > >>>>>> > >>>> > >>> > >>> > >> > > > -- *István Tóth* | Staff Software Engineer [email protected] <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> <https://www.cloudera.com/> ------------------------------
