The dependency of clean on setupVirtualenv has been removed. I use gradle for daily development on python, but only to run the "lint" task. There are a lot of python build artifacts that "clean" doesn't delete.
On Wed, Feb 20, 2019 at 1:17 AM Michael Luckey <[email protected]> wrote: > Thats said, removing this dependency on setupVirtualEnv is definitely > a good idea. > > On Wed, Feb 20, 2019 at 10:14 AM Michael Luckey <[email protected]> > wrote: > >> Is anyone using gradle for daily development of python parts? Or is this >> just used as integration for Jenkins build? In case of the latter, clean >> isn't called anyway, right? >> >> If the former, its probably worth to put more effort into python setup >> anyway, e.g. declaring proper input/outputs... If done correctly (if thats >> possible at all) clean will work out of the box anyway. >> >> On Wed, Feb 20, 2019 at 1:01 AM Udi Meiri <[email protected]> wrote: >> >>> I think I can solve this issue by removing the dependency and adding a >>> check to see if the virtualenv was created. >>> Otherwise, there shouldn't be anything to cleanup anyway. >>> >>> On Sat, Feb 16, 2019 at 8:04 PM Ryan Williams <[email protected]> >>> wrote: >>> >>>> Thanks Michael, your assessment was correct. I needed python3.5 on my >>>> $PATH. >>>> >>>> For completeness, I needed this >>>> <https://github.com/pyenv/pyenv/wiki/Common-build-problems#build-failed-error-the-python-zlib-extension-was-not-compiled-missing-the-zlib> >>>> to get pyenv to install python 3.5.6 on macOS: >>>> >>>> ``` >>>> CPPFLAGS="-I$(brew --prefix zlib)/include" pyenv install -v 3.5.6 >>>> ``` >>>> >>>> `./gradlew clean` worked after that. >>>> >>>> >>>> On Sat, Feb 16, 2019 at 7:28 PM Michael Luckey <[email protected]> >>>> wrote: >>>> >>>>> As far as I understand, the build got bound to 3.5 [1]. >>>>> >>>>> Could it be that you do not have python3.5 on your path? e.g. try >>>>> python3.5 >>>>> --version >>>>> >>>>> If that is missing, you will not be able to run any py3 task, I >>>>> guess... >>>>> >>>>> So, to get out of this state, you have to get python3.5 command >>>>> working. >>>>> >>>>> At least on my machine, './gradlew clean' is working iff python3.5 is >>>>> on my path. >>>>> >>>>> michel >>>>> >>>>> [1] >>>>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1591 >>>>> >>>>> >>>>> On Sat, Feb 16, 2019 at 9:54 PM Ryan Williams <[email protected]> >>>>> wrote: >>>>> >>>>>> I'm seeing the same thing as Thomas above: `./gradlew clean` fails on >>>>>> two py3 setupVirtualEnv tasks, >>>>>> :beam-sdks-python-test-suites-direct-py3:setupVirtualenv and >>>>>> :beam-sdks-python-test-suites-dataflow-py3:setupVirtualenv. Here's >>>>>> full output >>>>>> <https://gist.github.com/ryan-williams/402e20131d23905163de3a4e2b178f39> >>>>>> . >>>>>> >>>>>> Any tips how to get out of this state, or what is causing it? >>>>>> >>>>>> On Fri, Feb 8, 2019 at 7:17 PM Kenneth Knowles <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Maybe add that case to >>>>>>> https://issues.apache.org/jira/browse/BEAM-6459. >>>>>>> >>>>>>> Kenn >>>>>>> >>>>>>> On Fri, Feb 8, 2019 at 9:09 AM Thomas Weise <[email protected]> wrote: >>>>>>> >>>>>>>> Probably related, a top level ./gradlew clean fails with the >>>>>>>> following: >>>>>>>> >>>>>>>> > Task :beam-sdks-python-precommit-direct-py3:setupVirtualenv FAILED >>>>>>>> The path python3.5 (from --python=python3.5) does not exist >>>>>>>> >>>>>>>> Can we just limit clean to do cleanup?! >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Jan 21, 2019 at 7:58 AM Robert Bradshaw < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Just some background, grpcio-tools is what's used to do the proto >>>>>>>>> generation. Unfortunately it's expensive to compile and doesn't >>>>>>>>> provide very many wheels, so we want to install it once, not every >>>>>>>>> time. (It's also used in more than just tests; one needs it every >>>>>>>>> time >>>>>>>>> the .proto files change.) >>>>>>>>> >>>>>>>>> That being said, we could probably do a much cheaper clean. >>>>>>>>> >>>>>>>>> On Fri, Jan 18, 2019 at 8:56 PM Udi Meiri <[email protected]> >>>>>>>>> wrote: >>>>>>>>> > >>>>>>>>> > grpcio-tools could probably be moved under the "test" tag in >>>>>>>>> setup.py. Not sure why it has to be specified in gradle configs. >>>>>>>>> > >>>>>>>>> > On Fri, Jan 18, 2019 at 11:43 AM Kenneth Knowles <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >> >>>>>>>>> >> Can you `setupVirtualEnv` just enough to run `setup.py clean` >>>>>>>>> without installing gcpio-tools, etc? >>>>>>>>> >> >>>>>>>>> >> Kenn >>>>>>>>> >> >>>>>>>>> >> On Fri, Jan 18, 2019 at 11:20 AM Udi Meiri <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>> >>>>>>>>> >>> setup.py has requirements like setuptools, which are installed >>>>>>>>> in the virtual environment. >>>>>>>>> >>> So even running the clean command requires the virtualenv to >>>>>>>>> be set up. >>>>>>>>> >>> >>>>>>>>> >>> A possible fix could be to skip :beam-sdks-python:cleanPython >>>>>>>>> if setupVirtualenv has not been run. (perhaps by checking for the >>>>>>>>> existence >>>>>>>>> of its output directory) >>>>>>>>> >>> >>>>>>>>> >>> On Wed, Jan 16, 2019 at 7:03 PM Kenneth Knowles < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>> >>>>>>>>> >>>> Filed https://issues.apache.org/jira/browse/BEAM-6459 to >>>>>>>>> record the conclusion. Doesn't require Beam knowledge so I labeled >>>>>>>>> "starter". >>>>>>>>> >>>> >>>>>>>>> >>>> Kenn >>>>>>>>> >>>> >>>>>>>>> >>>> On Wed, Jan 16, 2019 at 12:14 AM Michael Luckey < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>> >>>>>>>>> >>>>> This seems to be on purpose [1] >>>>>>>>> >>>>> >>>>>>>>> >>>>> AFAIU setup is done to be able to call into setup.py clean. >>>>>>>>> We probably should work around that. >>>>>>>>> >>>>> >>>>>>>>> >>>>> [1] >>>>>>>>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1600-L1610 >>>>>>>>> >>>>> >>>>>>>>> >>>>> On Wed, Jan 16, 2019 at 7:01 AM Manu Zhang < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> I have the same question. Sometimes even `./gradlew clean` >>>>>>>>> fails due to failure of `setupVirtualEnv` tasks. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> Manu Zhang >>>>>>>>> >>>>>> On Jan 16, 2019, 12:22 PM +0800, Kenneth Knowles < >>>>>>>>> [email protected]>, wrote: >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> A global `./gradlew clean` runs various `setupVirtualEnv` >>>>>>>>> tasks that invoke things such as `setup.py bdist_wheel for >>>>>>>>> grpcio-tools`. >>>>>>>>> Overall it took 4 minutes. Is this intended? >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> Kenn >>>>>>>>> >>>>>>>>
smime.p7s
Description: S/MIME Cryptographic Signature
