"rm -r ~/.gradle/go/repo/" worked for me (there was more than one package with issues). My ~/.bashrc has export GOPATH=$HOME/go so maybe that's making the difference in my setup.
On Tue, Mar 26, 2019 at 11:28 AM Thomas Weise <t...@apache.org> wrote: > Can this be addressed by having "clean" remove all state that gogradle > leaves behind? This staleness issue has bitten me a few times also and it > would be good to have a reliable way to deal with it, even if it involves > an extra clean. > > > On Tue, Mar 26, 2019 at 11:14 AM Michael Luckey <adude3...@gmail.com> > wrote: > >> @Udi >> Did you try to just delete the >> '/usr/local/google/home/ehudm/.gradle/go/repo/cloud.google.com' folder? >> >> @Robert >> As said before, I am a bit scared about the implications. Shelling out is >> done by python, and from build perspective, this does not work very well, >> unfortunately. I.e. no caching, up-to-date checks etc... >> >> But of course, we need to play with this a bit more. >> >> On Tue, Mar 26, 2019 at 6:24 PM Robert Burke <rob...@frantil.com> wrote: >> >>> Reading the error from the gradle scan, it largely looks like some part >>> of the GCP dependencies for the build depends on a package, where the >>> commit version is no longer around. The main issue with gogradle is that >>> it's entirely distinct from the usual Go workflow, which means deps users >>> use are likely to be different to what's in the lock file. >>> >>> This work will be tracked in >>> https://issues.apache.org/jira/browse/BEAM-5379 >>> GoGradle hasn't moved to support the new-go way of handling deps, so my >>> inclination is to simplify to simple scripts for Gradle that shell out the >>> to Go tool for handling Go dep management, over trying to fix GoGradle. >>> >>> On Tue, 26 Mar 2019 at 09:43, Udi Meiri <eh...@google.com> wrote: >>> >>>> Robert, from what I recall it's not flaky for me - it consistently >>>> fails. Let me know if there's a way to get more logging about this error. >>>> >>>> On Mon, Mar 25, 2019, 19:50 Robert Burke <rob...@frantil.com> wrote: >>>> >>>>> It's concerning to me that 1) the Go dependency resolution via >>>>> gogradle is flaky, and 2) that it can block other languages. >>>>> >>>>> I suppose 2) makes sense since it's part of the container >>>>> bootstrapping code, but that makes 1) a serious problem, of which I wasn't >>>>> aware. >>>>> I should have time to investigate this in the next two weeks. >>>>> >>>>> On Mon, 25 Mar 2019 at 18:08, Michael Luckey <adude3...@gmail.com> >>>>> wrote: >>>>> >>>>>> Just for the record, >>>>>> >>>>>> using a vm here, because did not yet get all task running on my mac, >>>>>> and did not want to mess with my setup. >>>>>> >>>>>> So installed vanilla ubuntu-18.04 LTS on virtual box, 26GB ram, 6 >>>>>> cores and further >>>>>> >>>>>> sudo apt update >>>>>> >>>>>> sudo apt install gcc >>>>>> >>>>>> sudo apt install make >>>>>> >>>>>> sudo apt install perl >>>>>> >>>>>> sudo apt install curl >>>>>> >>>>>> sudo apt install openjdk-8-jdk >>>>>> >>>>>> sudo apt install python >>>>>> >>>>>> sudo apt install -y software-properties-common >>>>>> >>>>>> sudo add-apt-repository ppa:deadsnakes/ppa >>>>>> >>>>>> sudo apt update >>>>>> >>>>>> sudo apt install python3.5 >>>>>> >>>>>> sudo apt-get install apt-transport-https ca-certificates curl >>>>>> gnupg-agent software-properties-common >>>>>> >>>>>> curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo >>>>>> apt-key add - >>>>>> >>>>>> sudo apt-key fingerprint 0EBFCD88 >>>>>> >>>>>> sudo add-apt-repository "deb [arch=amd64] >>>>>> https://download.docker.com/linux/ubuntu \ >>>>>> >>>>>> $(lsb_release -cs) \ >>>>>> >>>>>> stable" >>>>>> >>>>>> sudo apt-get update >>>>>> >>>>>> sudo apt-get install docker-ce docker-ce-cli containerd.io >>>>>> >>>>>> sudo groupadd docker >>>>>> >>>>>> sudo usermod -aG docker $USER >>>>>> >>>>>> git config --global user.email "d...@spam.me" >>>>>> >>>>>> git config --global user.name "Some Guy" >>>>>> >>>>>> curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py >>>>>> >>>>>> sudo python get-pip.py >>>>>> >>>>>> rm get-pip.py >>>>>> >>>>>> sudo pip install --upgrade virtualenv >>>>>> >>>>>> sudo pip install cython >>>>>> >>>>>> sudo apt-get install python-dev >>>>>> >>>>>> sudo apt-get install python3-distutils >>>>>> >>>>>> sudo apt-get install python3-dev # for python3.x installs >>>>>> >>>>>> >>>>>> git clone https://github.com/apache/beam.git cd beam/ ./gradlew >>>>>> build >>>>>> >>>>>> Nothing else changed/added. (hopefully, need to reassure myself here) >>>>>> >>>>>> Unfortunately, this is failing. Need to exclude those python tests >>>>>> (and of course website, which usually fails on lira links) >>>>>> >>>>>> So I might be missing some env settings for gap, dunno. Probably >>>>>> missed some docs. >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 26, 2019 at 1:46 AM Michael Luckey <adude3...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks Udi for trying that! >>>>>>> >>>>>>> In fact, the go dependency resolution is flaky. Did not look into >>>>>>> that, but just rerunning usually works. Of course, less than optimal, >>>>>>> but, >>>>>>> well... >>>>>>> >>>>>>> Running build target is of course just an aggregation of task to >>>>>>> run. And unfortunately just running that >>>>>>> >>>>>>> ./gradlew :beam-sdks-python:testPy2Gcp >>>>>>> >>>>>>> stalls on my (virtual) machine. >>>>>>> >>>>>>> On Tue, Mar 26, 2019 at 1:35 AM Udi Meiri <eh...@google.com> wrote: >>>>>>> >>>>>>>> Okay, `./gradlew build` failed pretty quickly for me: >>>>>>>> >>>>>>>> > Task :beam-sdks-go:resolveBuildDependencies FAILED >>>>>>>> cloud.google.com/go: >>>>>>>> commit='4f6c921ec566a33844f4e7879b31cd8575a6982d', urls=[ >>>>>>>> https://code.googlesource.com/gocloud] does not exist in >>>>>>>> /usr/local/google/home/ehudm/.gradle/go/repo/ >>>>>>>> cloud.google.com/go/625660c387d9403fde4d73cacaf2d2ac, updating >>>>>>>> will be performed. >>>>>>>> >>>>>>>> https://gradle.com/s/x5zqbc5zwd3bg >>>>>>>> >>>>>>>> (Now I remember why I stopped using `build` :/) >>>>>>>> >>>>>>>> On Mon, Mar 25, 2019 at 5:30 PM Udi Meiri <eh...@google.com> wrote: >>>>>>>> >>>>>>>>> It shouldn't stall. That's a bug. >>>>>>>>> OTOH, I never use the `build` target. >>>>>>>>> I'll try running that myself. >>>>>>>>> >>>>>>>>> On Mon, Mar 25, 2019, 07:24 Michael Luckey <adude3...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> trying to run './gradlew build' on vanilla setup, my build >>>>>>>>>> consistently stalls during execution of python gcp tests, e.g. on >>>>>>>>>> both of >>>>>>>>>> - > :beam-sdks-python:testPy2Gcp >>>>>>>>>> - > :beam-sdks-python-test-suites-tox-py35:testPy35Gcp >>>>>>>>>> >>>>>>>>>> Console output: >>>>>>>>>> #### snip #### >>>>>>>>>> test_big_query_standard_sql >>>>>>>>>> (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) >>>>>>>>>> ... SKIP: IT is skipped because --test-pipeline-options is not >>>>>>>>>> specified >>>>>>>>>> test_big_query_standard_sql_kms_key >>>>>>>>>> (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) >>>>>>>>>> ... SKIP: This test requires BQ Dataflow native source support for >>>>>>>>>> KMS, >>>>>>>>>> which is not available yet. >>>>>>>>>> test_multiple_destinations_transform >>>>>>>>>> (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) >>>>>>>>>> ... SKIP: >>>>>>>>>> IT is skipped because --test-pipeline-options is not specified >>>>>>>>>> test_one_job_fails_all_jobs_fail >>>>>>>>>> (apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT) >>>>>>>>>> ... SKIP: >>>>>>>>>> IT is skipped because --test-pipeline-options is not specified >>>>>>>>>> test_records_traverse_transform_with_mocks >>>>>>>>>> (apache_beam.io.gcp.bigquery_file_loads_test.TestBigQueryFileLoads) >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>>> output ends here, would expect a failed or ok here. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Afterwards no progress - even waiting for hours. Any idea, what >>>>>>>>>> might be causing this? Do I need to add some GCP properties for this >>>>>>>>>> task ? >>>>>>>>>> >>>>>>>>>> Any ideas, what I am doing wrong? >>>>>>>>>> >>>>>>>>>> best, >>>>>>>>>> >>>>>>>>>> michel >>>>>>>>>> >>>>>>>>>>
smime.p7s
Description: S/MIME Cryptographic Signature
