On Mon, Feb 17, 2020 at 5:28 PM Wes McKinney <[email protected]> wrote: > > On Mon, Feb 17, 2020 at 10:19 AM Neal Richardson > <[email protected]> wrote: > > > > Ok, I made https://issues.apache.org/jira/browse/ARROW-7870 to look into > > this further. > > > > If the flaky nightly builds persist, maybe we should suspend the uploading > > to GitHub releases until we sort it out? The noise will block out real > > failures in the nightlies that we need to address. > > Yeah, that seems reasonable. These are the exact same Crossbow tasks > that are used for releases, so we'll need some option to toggle off > the artifact uploads in the nightly job.
In order to test that the artifact upload is working we should upload the assets from nightly jobs as well. I find it pretty strange that the uploads are rejected, because it used to work much more stable than other alternatives (like the builtin travis deployer). Perhaps github has changed something because this issue was around previously https://github.com/sigmavirus24/github3.py/issues/779#issuecomment-379470626 Created a blocker jira to track it https://issues.apache.org/jira/browse/ARROW-7877 > > > Neal > > > > On Mon, Feb 17, 2020 at 8:08 AM Wes McKinney <[email protected]> wrote: > > > > > On Mon, Feb 17, 2020 at 9:57 AM Neal Richardson > > > <[email protected]> wrote: > > > > > > > > Maybe the new nightly wheel GitHub Page is triggering rate limits, as > > > > Krisztián suggested it might: https://github.com/apache/arrow/pull/6366. > > > > The increase in timeouts seems correlated with when that PR merged. > > > > > > > > I wonder if there's a better solution to making nightly wheels > > > > available. > > > > GitHub releases (on a crossbow repo separate from apache/arrow, no less) > > > > seem like an unnatural fit for the problem. > > > > > > Agreed. I think we can make nightly development builds available in a > > > more formalized way as long as there are appropriate warning signs > > > that they should not be depended on in production, see e.g. another > > > Apache project giving links to nightlies > > > > > > https://jmeter.apache.org/nightly.html > > > > > > > Neal > > > > > > > > > > > > On Mon, Feb 17, 2020 at 3:35 AM Wes McKinney <[email protected]> > > > wrote: > > > > > > > > > It seems like GitHub release uploads continue to be a bit flaky. Is > > > > > there a JIRA about adding some retry logic with delay (e.g. waiting > > > > > 30-60 seconds in between tries) to attempt to reduce the flakiness? > > > > > > > > > > > > > https://github.com/apache/arrow/blob/master/dev/tasks/crossbow.py#L1517 > > > > > > > > > > On Sun, Feb 16, 2020 at 7:31 AM Crossbow <[email protected]> > > > wrote: > > > > > > > > > > > > > > > > > > Arrow Build Report for Job nightly-2020-02-16-0 > > > > > > > > > > > > All tasks: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0 > > > > > > > > > > > > Failed Tasks: > > > > > > - centos-7: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-7 > > > > > > - conda-linux-gcc-py27: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py27 > > > > > > - conda-osx-clang-py27: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py27 > > > > > > - debian-stretch: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-debian-stretch > > > > > > - gandiva-jar-trusty: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-gandiva-jar-trusty > > > > > > - test-conda-python-2.7-pandas-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-2.7-pandas-latest > > > > > > - test-conda-python-2.7: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-2.7 > > > > > > - test-conda-python-3.7-pandas-master: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-pandas-master > > > > > > - test-conda-python-3.7-spark-master: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-spark-master > > > > > > - test-conda-python-3.7-turbodbc-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-turbodbc-latest > > > > > > - test-conda-python-3.7-turbodbc-master: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-turbodbc-master > > > > > > - test-ubuntu-18.04-docs: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-docs > > > > > > - ubuntu-bionic: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-bionic > > > > > > - ubuntu-disco: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-disco > > > > > > - wheel-manylinux1-cp35m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp35m > > > > > > - wheel-manylinux1-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp36m > > > > > > > > > > > > Succeeded Tasks: > > > > > > - centos-6: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-6 > > > > > > - centos-8: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-centos-8 > > > > > > - conda-linux-gcc-py36: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py36 > > > > > > - conda-linux-gcc-py37: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py37 > > > > > > - conda-linux-gcc-py38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-linux-gcc-py38 > > > > > > - conda-osx-clang-py36: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py36 > > > > > > - conda-osx-clang-py37: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py37 > > > > > > - conda-osx-clang-py38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-osx-clang-py38 > > > > > > - conda-win-vs2015-py36: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py36 > > > > > > - conda-win-vs2015-py37: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py37 > > > > > > - conda-win-vs2015-py38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-conda-win-vs2015-py38 > > > > > > - debian-buster: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-debian-buster > > > > > > - gandiva-jar-osx: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-gandiva-jar-osx > > > > > > - homebrew-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-homebrew-cpp > > > > > > - macos-r-autobrew: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-macos-r-autobrew > > > > > > - test-conda-cpp-valgrind: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-cpp-valgrind > > > > > > - test-conda-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-cpp > > > > > > - test-conda-python-3.6: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.6 > > > > > > - test-conda-python-3.7-dask-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-dask-latest > > > > > > - test-conda-python-3.7-hdfs-2.9.2: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-hdfs-2.9.2 > > > > > > - test-conda-python-3.7-pandas-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7-pandas-latest > > > > > > - test-conda-python-3.7: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.7 > > > > > > - test-conda-python-3.8-dask-master: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.8-dask-master > > > > > > - test-conda-python-3.8-pandas-latest: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-python-3.8-pandas-latest > > > > > > - test-conda-r-3.6: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-conda-r-3.6 > > > > > > - test-debian-10-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-cpp > > > > > > - test-debian-10-go-1.12: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-go-1.12 > > > > > > - test-debian-10-python-3: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-10-python-3 > > > > > > - test-debian-c-glib: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-c-glib > > > > > > - test-debian-ruby: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-debian-ruby > > > > > > - test-fedora-29-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-fedora-29-cpp > > > > > > - test-fedora-29-python-3: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-fedora-29-python-3 > > > > > > - test-r-rhub-debian-gcc-devel: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rhub-debian-gcc-devel > > > > > > - test-r-rhub-ubuntu-gcc-release: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rhub-ubuntu-gcc-release > > > > > > - test-r-rstudio-r-base-3.6-bionic: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-bionic > > > > > > - test-r-rstudio-r-base-3.6-centos6: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-centos6 > > > > > > - test-r-rstudio-r-base-3.6-opensuse15: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-opensuse15 > > > > > > - test-r-rstudio-r-base-3.6-opensuse42: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-test-r-rstudio-r-base-3.6-opensuse42 > > > > > > - test-ubuntu-16.04-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-16.04-cpp > > > > > > - test-ubuntu-18.04-cpp-cmake32: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-cmake32 > > > > > > - test-ubuntu-18.04-cpp-release: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-release > > > > > > - test-ubuntu-18.04-cpp-static: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp-static > > > > > > - test-ubuntu-18.04-cpp: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-cpp > > > > > > - test-ubuntu-18.04-python-3: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-python-3 > > > > > > - test-ubuntu-18.04-r-sanitizer: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-18.04-r-sanitizer > > > > > > - test-ubuntu-c-glib: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-c-glib > > > > > > - test-ubuntu-ruby: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-circle-test-ubuntu-ruby > > > > > > - ubuntu-eoan: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-eoan > > > > > > - ubuntu-xenial: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-ubuntu-xenial > > > > > > - wheel-manylinux1-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp37m > > > > > > - wheel-manylinux1-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux1-cp38 > > > > > > - wheel-manylinux2010-cp35m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp35m > > > > > > - wheel-manylinux2010-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp36m > > > > > > - wheel-manylinux2010-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp37m > > > > > > - wheel-manylinux2010-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2010-cp38 > > > > > > - wheel-manylinux2014-cp35m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp35m > > > > > > - wheel-manylinux2014-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp36m > > > > > > - wheel-manylinux2014-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp37m > > > > > > - wheel-manylinux2014-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-azure-wheel-manylinux2014-cp38 > > > > > > - wheel-osx-cp35m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp35m > > > > > > - wheel-osx-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp36m > > > > > > - wheel-osx-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp37m > > > > > > - wheel-osx-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-travis-wheel-osx-cp38 > > > > > > - wheel-win-cp36m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp36m > > > > > > - wheel-win-cp37m: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp37m > > > > > > - wheel-win-cp38: > > > > > > URL: > > > > > > > > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-02-16-0-appveyor-wheel-win-cp38 > > > > > > > >
