Ash, There seem to be some solutions, but there are pretty hacky and poorly documented :
https://stackoverflow.com/questions/18026980/python-setuptools-how-can-i-list-a-private-repository-under-install-requires https://github.com/pypa/pip/issues/2124 That said, we should be able to figure out a path :) Maybe we can get the incubator to reconsider for this particular case. Best, Arthur On Fri, Nov 10, 2017 at 1:25 AM, Ash Berlin-Taylor < [email protected]> wrote: > The other difference is that if you depend upon airflow in a module where > you want to put it in the install_requires section of a setup.py (not an > application which has a requirements.txt, say) you can't use a git tag. Or > at least I couldn't get it working. > > It doesn't make a difference a lot of time, but it is occasionally useful. > > -ash > > > On 9 Nov 2017, at 23:08, Alek Storm <[email protected]> wrote: > > > > It’s not a major difference, but installing from a git repo via pip > > requires a completely different syntax, which complicates our tooling, > e.g.: > > > > $ pip install 'apache-airflow[postgres,celery,rabbitmq]=={{version}}' > > > > $ pip install 'git+git://github.com/apache/ > incubator-airflow@{{version}}#egg=apache-airflow[postgres,celery,rabbitmq] > ' > > > > Alek > > > > > > On Thu, Nov 9, 2017 at 3:53 PM, Arthur Wiedmer <[email protected] > > > > wrote: > > > >> I agree with Bolke that it would be better to provide dev releases in > PyPI, > >> but my understanding was that, while not an official release channel, it > >> still has the apache branding and we should be careful nonetheless. > >> > >> I am still confused as to why installing from a git tag or the like is > not > >> OK for testing, provided our release artifact creation process is > >> consistent. > >> > >> Best, > >> Arthur > >> > >> On Thu, Nov 9, 2017 at 12:09 PM, Daniel Huang <[email protected]> > wrote: > >> > >>> This is how pip handles RC/beta versions: > >>> > >>> > >>>> Pre-release Versions > >>>> Starting with v1.4, pip will only install stable versions as specified > >> by > >>>> PEP426 by default. If a version cannot be parsed as a compliant PEP426 > >>>> version then it is assumed to be a pre-release. > >>>> If a Requirement specifier includes a pre-release or development > >> version > >>>> (e.g. >=0.0.dev0) then pip will allow pre-release and development > >>> versions > >>>> for that requirement. This does not include the != flag. > >>>> The pip install command also supports a --pre flag that will enable > >>>> installing pre-releases and development releases. > >>> > >>> > >>> Source: > >>> https://pip.pypa.io/en/stable/reference/pip_install/#pre- > >> release-versions > >>> <https://pip.pypa.io/en/stable/reference/pip_install/# > >> pre-release-versions > >>>> > >>> > >>> On Thu, Nov 9, 2017 at 11:54 AM, Bolke de Bruin <[email protected]> > >> wrote: > >>> > >>>> I think we should put this up for discussion. PyPi is not an official > >>>> apache channel, so in theory we could put anything on PyPI. I also > >> think > >>>> (didn’t confirm) pip doesn’t upgrade to RC/beta etc. > >>>> > >>>> Any thoughts? > >>>> > >>>> Bolke. > >>>> > >>>>> On 9 Nov 2017, at 15:53, Arthur Wiedmer <[email protected]> wrote: > >>>>> > >>>>> Hi Alek, > >>>>> > >>>>> Technically, we cannot release a distribution on PyPI until we have > >>> voted > >>>>> on a release. And here usually a release artifact. It is a little > >>>>> convoluted in the case of Python, but we are getting the hang of it. > >>>>> > >>>>> That said, installing from a git reference is a possibility too if > >> you > >>>> want > >>>>> the fastest path to install. > >>>>> > >>>>> Best, > >>>>> Arthur > >>>>> > >>>>> On Nov 9, 2017 06:34, "Alek Storm" <[email protected]> wrote: > >>>>> > >>>>> I think this has been mentioned before, but it would be much easier > >> for > >>>> us > >>>>> (my team) to test RCs if they were published to PyPI. Or is that > >>> against > >>>>> Apache guidelines? > >>>>> > >>>>> Alek > >>>>> > >>>>> On Thu, Nov 9, 2017 at 8:29 AM, Michael Crawford < > >>>>> [email protected]> wrote: > >>>>> > >>>>>> Thanks. Yes I understand it isn’t released yet. > >>>>>> > >>>>>> > >>>>>>> On Nov 9, 2017, at 9:09 AM, Driesprong, Fokko <[email protected] > >>> > >>>>>> wrote: > >>>>>>> > >>>>>>> Hi Michael, > >>>>>>> > >>>>>>> You have to install it from the tar.gz: > >>>>>>> > >>>>>>> wget > >>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1. > >>>>>> 9.0rc1/apache-airflow-1.9.0rc1+incubating-bin.tar.gz > >>>>>>> pip install /tmp/apache-airflow.tar.gz > >>>>>>> > >>>>>>> The steps of updating, are in the UPDATING.md: > >>>>>>> https://github.com/apache/incubator-airflow/blob/master/ > >> UPDATING.md > >>>>>>> > >>>>>>> Please note that 1.9 is not released yet, but you are welcome to > >> try > >>>> out > >>>>>>> RC1. > >>>>>>> > >>>>>>> Cheers, Fokko > >>>>>> > >>>>>> > >>>> > >>>> > >>> > >> > >
