[jira] [Created] (ARROW-8523) [C++] Optimize BitmapReader
Yibo Cai created ARROW-8523: --- Summary: [C++] Optimize BitmapReader Key: ARROW-8523 URL: https://issues.apache.org/jira/browse/ARROW-8523 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Yibo Cai Assignee: Yibo Cai -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: 0.17 release blog post: help needed
I've edited Ruby and C GLib parts. Kou and Shiro will check them later. 2020年4月20日(月) 11:09 Wes McKinney : > > I made a pass through the changelog and added a bunch of TODOs related > to C++. In general, as a reminder, in these blog posts since the > releases are growing large we should try to present as compact a high > level summary as possible to convey some of the highlights of our > labors (so likely not needed to write out any JIRA numbers, people can > look at the changelog for that). I'll spend some more time on the blog > post after others have had a chance to take a pass through > > On Sat, Apr 18, 2020 at 12:13 PM Neal Richardson > wrote: > > > > Hi all, > > Since it looks like we're close to releasing 0.17, we need to fill in the > > details for our blog post announcement. I've started a document here: > > https://docs.google.com/document/d/16UKZtvL49o8nCDN8JU3Ut6y76Y9d8-4qXv5vFv7aNvs/edit#heading=h.kqqacbm2lpv8 > > > > Please fill in the details for the parts of the project you're close to. > > I'll handle wrapping this up in the usual boilerplate when we're done. > > > > Thanks, > > Neal -- Regards, Kenta Murata
Re: 0.17 release blog post: help needed
I made a pass through the changelog and added a bunch of TODOs related to C++. In general, as a reminder, in these blog posts since the releases are growing large we should try to present as compact a high level summary as possible to convey some of the highlights of our labors (so likely not needed to write out any JIRA numbers, people can look at the changelog for that). I'll spend some more time on the blog post after others have had a chance to take a pass through On Sat, Apr 18, 2020 at 12:13 PM Neal Richardson wrote: > > Hi all, > Since it looks like we're close to releasing 0.17, we need to fill in the > details for our blog post announcement. I've started a document here: > https://docs.google.com/document/d/16UKZtvL49o8nCDN8JU3Ut6y76Y9d8-4qXv5vFv7aNvs/edit#heading=h.kqqacbm2lpv8 > > Please fill in the details for the parts of the project you're close to. > I'll handle wrapping this up in the usual boilerplate when we're done. > > Thanks, > Neal
[jira] [Created] (ARROW-8522) [Developer] Add environment variable option to toggle whether ephemeral NodeJS is installed in release verification script
Wes McKinney created ARROW-8522: --- Summary: [Developer] Add environment variable option to toggle whether ephemeral NodeJS is installed in release verification script Key: ARROW-8522 URL: https://issues.apache.org/jira/browse/ARROW-8522 Project: Apache Arrow Issue Type: Improvement Components: Developer Tools Reporter: Wes McKinney Fix For: 1.0.0 Currently the script fails if the user does not have a new enough NodeJS and associated utilities (npm/npx) on their system. The code to install NodeJS exists already but it was disabled in https://github.com/apache/arrow/commit/e570db9c45ca97f77c5633e5525c02f55dbb6c4b#diff-8cc7fa3ae5de30b356c17d7a4b59fe09 because it caused problems when run in GitHub Actions -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-8521) [Developer] Group Sub-task, Task, Test, and Wish issue types as "Improvement" in Changelog
Wes McKinney created ARROW-8521: --- Summary: [Developer] Group Sub-task, Task, Test, and Wish issue types as "Improvement" in Changelog Key: ARROW-8521 URL: https://issues.apache.org/jira/browse/ARROW-8521 Project: Apache Arrow Issue Type: Improvement Components: Developer Tools Reporter: Wes McKinney In my opinion this makes the changelog more readable. The "Wish" type in particular might cause people to not see improvements -- This message was sent by Atlassian Jira (v8.3.4#803005)
[NIGHTLY] Arrow Build Report for Job nightly-2020-04-19-2
Arrow Build Report for Job nightly-2020-04-19-2 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2 Failed Tasks: - centos-8-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-centos-8-amd64 - debian-buster-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-debian-buster-amd64 - gandiva-jar-osx: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-travis-gandiva-jar-osx - homebrew-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-travis-homebrew-cpp - test-conda-cpp-valgrind: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-test-conda-cpp-valgrind - test-conda-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-test-conda-cpp - test-conda-python-3.7-pandas-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-circle-test-conda-python-3.7-pandas-latest - test-ubuntu-18.04-cpp-static: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-test-ubuntu-18.04-cpp-static - ubuntu-bionic-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-ubuntu-bionic-amd64 - ubuntu-eoan-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-ubuntu-eoan-amd64 - ubuntu-focal-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-ubuntu-focal-amd64 - wheel-osx-cp35m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-travis-wheel-osx-cp35m - wheel-osx-cp36m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-travis-wheel-osx-cp36m - wheel-osx-cp37m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-travis-wheel-osx-cp37m - wheel-osx-cp38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-travis-wheel-osx-cp38 Succeeded Tasks: - centos-6-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-centos-6-amd64 - centos-7-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-centos-7-amd64 - conda-linux-gcc-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-linux-gcc-py36 - conda-linux-gcc-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-linux-gcc-py37 - conda-linux-gcc-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-linux-gcc-py38 - conda-osx-clang-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-osx-clang-py36 - conda-osx-clang-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-osx-clang-py37 - conda-osx-clang-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-osx-clang-py38 - conda-win-vs2015-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-win-vs2015-py36 - conda-win-vs2015-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-win-vs2015-py37 - conda-win-vs2015-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-conda-win-vs2015-py38 - debian-stretch-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-github-debian-stretch-amd64 - gandiva-jar-xenial: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-travis-gandiva-jar-xenial - homebrew-r-autobrew: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-travis-homebrew-r-autobrew - test-conda-python-3.6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-azure-test-conda-python-3.6 - test-conda-python-3.7-dask-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-circle-test-conda-python-3.7-dask-latest - test-conda-python-3.7-hdfs-2.9.2: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-circle-test-conda-python-3.7-hdfs-2.9.2 - test-conda-python-3.7-kartothek-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-circle-test-conda-python-3.7-kartothek-latest - test-conda-python-3.7-kartothek-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-2-circle-test-conda-python-3.7-kartothek-master - test-conda-python-3.7-pandas-master: URL:
Re: GitBox e-mails to dev@arrow
Alright, I think I've fixed it. If we need to alter the notification scheme the details are now in .asf.yaml in the root of the repository. See ARROW-8520 and the INFRA ticket On Sun, Apr 19, 2020 at 5:19 PM Wes McKinney wrote: > > These e-mails should not be coming here, I don't know what's going on. > I just opened > > https://issues.apache.org/jira/browse/INFRA-20149
[GitHub] [arrow] pitrou commented on issue #6984: ARROW-8520: [Developer] Use .asf.yaml to direct GitHub notifications to JIRA and mailing lists
pitrou commented on issue #6984: URL: https://github.com/apache/arrow/pull/6984#issuecomment-616237297 Wow, thank you for solving this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] wesm commented on issue #6984: ARROW-8520: [Developer] Use .asf.yaml to direct GitHub notifications to JIRA and mailing lists
wesm commented on issue #6984: URL: https://github.com/apache/arrow/pull/6984#issuecomment-616237030 +1. This can only be "tested in production" so I'm merging this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] wesm opened a new pull request #6984: ARROW-8520: [Developer] Use .asf.yaml to direct GitHub notifications to JIRA and mailing lists
wesm opened a new pull request #6984: URL: https://github.com/apache/arrow/pull/6984 Context: INFRA-20149 Following instructions per https://s.apache.org/asfyaml-notify This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] wesm commented on issue #6984: ARROW-8520: [Developer] Use .asf.yaml to direct GitHub notifications to JIRA and mailing lists
wesm commented on issue #6984: URL: https://github.com/apache/arrow/pull/6984#issuecomment-616236838 I just requested the creation of github@ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (ARROW-8520) [Developer] Use .asf.yaml to direct GitHub notifications to e-mail lists and JIRA
Wes McKinney created ARROW-8520: --- Summary: [Developer] Use .asf.yaml to direct GitHub notifications to e-mail lists and JIRA Key: ARROW-8520 URL: https://issues.apache.org/jira/browse/ARROW-8520 Project: Apache Arrow Issue Type: New Feature Components: Developer Tools Reporter: Wes McKinney Fix For: 1.0.0 Context: INFRA-20149 -- This message was sent by Atlassian Jira (v8.3.4#803005)
GitBox e-mails to dev@arrow
These e-mails should not be coming here, I don't know what's going on. I just opened https://issues.apache.org/jira/browse/INFRA-20149
[GitHub] [arrow] github-actions[bot] commented on issue #6983: ARROW-8519: [C++][Packaging] Reduce disk usage for external projects
github-actions[bot] commented on issue #6983: URL: https://github.com/apache/arrow/pull/6983#issuecomment-616234088 https://issues.apache.org/jira/browse/ARROW-8519 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on issue #6983: ARROW-8519: [C++][Packaging] Reduce disk usage for external projects
github-actions[bot] commented on issue #6983: URL: https://github.com/apache/arrow/pull/6983#issuecomment-616233760 Revision: ac4a31458b83fa39e659c2a0551a07f4f645bcbb Submitted crossbow builds: [ursa-labs/crossbow @ actions-148](https://github.com/ursa-labs/crossbow/branches/all?query=actions-148) |Task|Status| ||--| |centos-6-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-centos-6-amd64)| |centos-7-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-centos-7-amd64)| |centos-8-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-centos-8-amd64)| |debian-buster-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-debian-buster-amd64)| |debian-stretch-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-debian-stretch-amd64)| |ubuntu-bionic-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-ubuntu-bionic-amd64)| |ubuntu-eoan-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-ubuntu-eoan-amd64)| |ubuntu-focal-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-ubuntu-focal-amd64)| |ubuntu-xenial-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-ubuntu-xenial-amd64)| This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] kou commented on issue #6983: ARROW-8519: [C++][Packaging] Reduce disk usage for external projects
kou commented on issue #6983: URL: https://github.com/apache/arrow/pull/6983#issuecomment-616233577 @github-actions crossbow submit -g linux This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] kou opened a new pull request #6983: ARROW-8519: [C++][Packaging] Reduce disk usage for external projects
kou opened a new pull request #6983: URL: https://github.com/apache/arrow/pull/6983 This change removes built files after installing built files for external projects. It reduces disk usage for external projects. This may resolve "No space left on device" caused by building Linux packages on GitHub Actions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (ARROW-8519) [C++][Packaging][deb][RPM] Linux package builds are failed by "No space left on device" on GitHub Actions
Kouhei Sutou created ARROW-8519:
---
Summary: [C++][Packaging][deb][RPM] Linux package builds are
failed by "No space left on device" on GitHub Actions
Key: ARROW-8519
URL: https://issues.apache.org/jira/browse/ARROW-8519
Project: Apache Arrow
Issue Type: Improvement
Components: C++, Packaging
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou
For example: https://github.com/ursa-labs/crossbow/runs/597759127
{noformat}
/usr/bin/ranlib: release/libarrow.a: No space left on device
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-8518) [Python] Create tools to enable optional components (like Gandiva, Flight) to be built and deployed as separate Python packages
Wes McKinney created ARROW-8518:
---
Summary: [Python] Create tools to enable optional components (like
Gandiva, Flight) to be built and deployed as separate Python packages
Key: ARROW-8518
URL: https://issues.apache.org/jira/browse/ARROW-8518
Project: Apache Arrow
Issue Type: Improvement
Components: Packaging, Python
Reporter: Wes McKinney
Fix For: 1.0.0
Our current monolithic approach to Python packaging isn't likely to be
sustainable long-term.
At a high level, I would propose a structure like this:
{code}
pip install pyarrow # core package containing libarrow, libarrow_python, and
any other common bundled C++ library dependencies
pip install pyarrow-flight # installs pyarrow, pyarrow_flight
pip install pyarrow-gandiva # installs pyarrow, pyarrow_gandiva
{code}
We can maintain the semantic appearance of a single {{pyarrow}} package by
having thin API modules that would look like
{code}
CONTENTS OF pyarrow/flight.py
from pyarrow_flight import *
{code}
Obviously, this is more difficult to build and package:
* CMake and setup.py files must be refactored a bit so that we can reuse code
between the parent and child packages
* Separate conda and wheel packages must be produced. With conda this seems
more straightforward but since the child wheels depend on the parent core
wheel, the build process seems more complicated
In any case, I don't think these challenges are insurmountable. This will have
several benefits:
* Smaller installation footprint for simple use cases (though note we are STILL
duplicating shared libraries in the wheels, which is quite bad)
* Less developer anxiety about expanding the scope of what Python code is
shipped from apache/arrow. If in 5 years we are shipping 5 different Python
wheels with each Apache Arrow release, that sounds completely fine to me.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [VOTE] Release Apache Arrow 0.17.0 - RC0
+1 (binding) I was able to run the verification of source and wheels on Windows after addressing the issues I ran into. The failure reported in ARROW-8510 is worked around by using the UNITY build so we'll need to investigate that separately On Sat, Apr 18, 2020 at 4:01 PM Wes McKinney wrote: > > Right, sorry about that. > > So the difference it turns out is the UNITY build. The compiler error > in ARROW-8510 does not occur for some reason when > CMAKE_UNITY_BUILD=ON. > > On Sat, Apr 18, 2020 at 3:36 PM Antoine Pitrou wrote: > > > > > > We build with both VS2015 and VS2017 on AppVeyor. > > > > Regards > > > > Antoine. > > > > > > Le 18/04/2020 à 22:34, Wes McKinney a écrit : > > > The release verification script (which uses the VS 2017 compiler mode) > > > fails for me on Windows > > > > > > https://issues.apache.org/jira/browse/ARROW-8510 > > > > > > It looks like we are building with VS 2019 in CI. We may want to be > > > more conservative and build with VS 2015 or VS 2017 in CI. This is > > > peculiar though because the wheels are using VS 2015. So it's possible > > > there's a compiler bug with VS 2017 that doesn't present on VS 2015. > > > > > > I don't think this should block the release however. I'll vote after I > > > check some more things > > > > > > On Sat, Apr 18, 2020 at 3:32 PM Neal Richardson > > > wrote: > > >> > > >> +1 (binding) > > >> > > >> I ran verification on Ubuntu, macOS, and Windows using crossbow. Thanks > > >> to > > >> all who helped debug. A few things are still failing: see > > >> https://github.com/apache/arrow/pull/6961#issuecomment-615940240 for > > >> details. Since Python folks are unconcerned by the wheel issues I > > >> encountered, I won't object. > > >> > > >> In addition to the usual verifications, I worked with Jeroen to build > > >> the R > > >> Windows binaries for the 0.17 rc0 and confirmed that the R package passes > > >> all checks with them. > > >> > > >> Neal > > >> > > >> On Sat, Apr 18, 2020 at 12:13 AM Sutou Kouhei > > >> wrote: > > >> > > >>> +1 (binding) > > >>> > > >>> I ran the followings on Debian GNU/Linux sid: > > >>> > > >>> * TEST_CSHARP=0 \ > > >>> JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 \ > > >>> CUDA_TOOLKIT_ROOT=/usr \ > > >>> ARROW_CMAKE_OPTIONS=-DgRPC_SOURCE=BUNDLED \ > > >>> dev/release/verify-release-candidate.sh source 0.17.0 0 > > >>> * dev/release/verify-release-candidate.sh binaries 0.17.0 0 > > >>> * dev/release/verify-release-candidate.sh wheels 0.17.0 0 > > >>> > > >>> with: > > >>> > > >>> * gcc (Debian 9.3.0-10) 9.3.0 > > >>> * openjdk version "1.8.0_252" > > >>> * Node.JS v12.1.0 > > >>> * nvidia-cuda-dev 10.1.168-8 > > >>> > > >>> Notes: > > >>> > > >>> * C# sourcelink is failed as usual. > > >>> https://issues.apache.org/jira/browse/ARROW-8505 > > >>> > > >>> > > >>> Thanks, > > >>> -- > > >>> kou > > >>> > > >>> In > > >>> "[VOTE] Release Apache Arrow 0.17.0 - RC0" on Fri, 17 Apr 2020 > > >>> 02:26:34 > > >>> +0200, > > >>> Krisztián Szűcs wrote: > > >>> > > Hi, > > > > I would like to propose the following release candidate (RC0) of Apache > > Arrow version 0.17.0. This is a release consisting of 582 > > resolved JIRA issues[1]. > > > > This release candidate is based on commit: > > 3cbcb7b62c2f2d02851bff837758637eb592a64b [2] > > > > The source release rc0 is hosted at [3]. > > The binary artifacts are hosted at [4][5][6][7]. > > The changelog is located at [8]. > > > > Please download, verify checksums and signatures, run the unit tests, > > and vote on the release. See [9] for how to validate a release > > candidate. > > > > The vote will be open for at least 72 hours. > > > > [ ] +1 Release this as Apache Arrow 0.17.0 > > [ ] +0 > > [ ] -1 Do not release this as Apache Arrow 0.17.0 because... > > > > [1]: > > >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20%28Resolved%2C%20Closed%29%20AND%20fixVersion%20%3D%200.17.0 > > [2]: > > >>> https://github.com/apache/arrow/tree/3cbcb7b62c2f2d02851bff837758637eb592a64b > > [3]: > > >>> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-0.17.0-rc0 > > [4]: https://bintray.com/apache/arrow/centos-rc/0.17.0-rc0 > > [5]: https://bintray.com/apache/arrow/debian-rc/0.17.0-rc0 > > [6]: https://bintray.com/apache/arrow/python-rc/0.17.0-rc0 > > [7]: https://bintray.com/apache/arrow/ubuntu-rc/0.17.0-rc0 > > [8]: > > >>> https://github.com/apache/arrow/blob/3cbcb7b62c2f2d02851bff837758637eb592a64b/CHANGELOG.md > > [9]: > > >>> https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates > > >>>
[jira] [Created] (ARROW-8517) [Developer][Release] Update Crossbow RC verification setup for changes since 0.16.0
Wes McKinney created ARROW-8517: --- Summary: [Developer][Release] Update Crossbow RC verification setup for changes since 0.16.0 Key: ARROW-8517 URL: https://issues.apache.org/jira/browse/ARROW-8517 Project: Apache Arrow Issue Type: Improvement Components: Developer Tools Reporter: Wes McKinney Assignee: Neal Richardson Fix For: 0.17.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[NIGHTLY] Arrow Build Report for Job nightly-2020-04-19-1
Arrow Build Report for Job nightly-2020-04-19-1 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1 Failed Tasks: - centos-8-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-centos-8-amd64 - debian-buster-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-debian-buster-amd64 - gandiva-jar-osx: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-travis-gandiva-jar-osx - test-conda-cpp-valgrind: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-test-conda-cpp-valgrind - test-conda-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-test-conda-cpp - test-r-linux-as-cran: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-test-r-linux-as-cran - test-ubuntu-18.04-cpp-static: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-test-ubuntu-18.04-cpp-static - test-ubuntu-18.04-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-test-ubuntu-18.04-cpp - ubuntu-bionic-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-ubuntu-bionic-amd64 - ubuntu-eoan-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-ubuntu-eoan-amd64 - ubuntu-focal-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-ubuntu-focal-amd64 - wheel-osx-cp35m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-travis-wheel-osx-cp35m - wheel-osx-cp36m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-travis-wheel-osx-cp36m - wheel-osx-cp37m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-travis-wheel-osx-cp37m - wheel-osx-cp38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-travis-wheel-osx-cp38 Succeeded Tasks: - centos-6-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-centos-6-amd64 - centos-7-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-centos-7-amd64 - conda-linux-gcc-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-linux-gcc-py36 - conda-linux-gcc-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-linux-gcc-py37 - conda-linux-gcc-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-linux-gcc-py38 - conda-osx-clang-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-osx-clang-py36 - conda-osx-clang-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-osx-clang-py37 - conda-osx-clang-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-osx-clang-py38 - conda-win-vs2015-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-win-vs2015-py36 - conda-win-vs2015-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-win-vs2015-py37 - conda-win-vs2015-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-conda-win-vs2015-py38 - debian-stretch-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-github-debian-stretch-amd64 - gandiva-jar-xenial: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-travis-gandiva-jar-xenial - homebrew-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-travis-homebrew-cpp - homebrew-r-autobrew: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-travis-homebrew-r-autobrew - test-conda-python-3.6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-azure-test-conda-python-3.6 - test-conda-python-3.7-dask-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-circle-test-conda-python-3.7-dask-latest - test-conda-python-3.7-hdfs-2.9.2: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-circle-test-conda-python-3.7-hdfs-2.9.2 - test-conda-python-3.7-kartothek-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-1-circle-test-conda-python-3.7-kartothek-latest - test-conda-python-3.7-kartothek-master: URL:
[jira] [Created] (ARROW-8516) Slow BufferBuilder inserts
Raphael Taylor-Davies created ARROW-8516:
Summary: Slow BufferBuilder inserts
Key: ARROW-8516
URL: https://issues.apache.org/jira/browse/ARROW-8516
Project: Apache Arrow
Issue Type: Improvement
Reporter: Raphael Taylor-Davies
{color:#00}BufferBuilder{color}{color:#0073bf}<{color}{color:#00}BooleanType>::append_slice
is called by ArrayBuilder::append_slice with a constructed vector of true
values. {color}
{color:#00}Even in release builds the associated allocations and iterations
are not optimised out, resulting in a third of the time to parse a parquet file
containing single integers being spent in PrimitiveBuilder::append_slice.{color}
{color:#00}This PR adds an append_n method to the BufferBuilderTrait that
allows this to be handled more efficiently. My rather unscientific testing
shows it to halve the amount of time spent in this method yielding an ~20%
speedup for my particular workload.
{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-8515) [C++] Bitmap ToString should have an option of grouping by bytes
Micah Kornfield created ARROW-8515: -- Summary: [C++] Bitmap ToString should have an option of grouping by bytes Key: ARROW-8515 URL: https://issues.apache.org/jira/browse/ARROW-8515 Project: Apache Arrow Issue Type: New Feature Components: C++ Reporter: Micah Kornfield Assignee: Micah Kornfield ToString for anything longer than really sort bitmaps can be unreadable. Grouping by bytes makes it more useful. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[NIGHTLY] Arrow Build Report for Job nightly-2020-04-19-0
Arrow Build Report for Job nightly-2020-04-19-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0 Failed Tasks: - centos-8-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-centos-8-amd64 - debian-buster-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-debian-buster-amd64 - gandiva-jar-osx: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-travis-gandiva-jar-osx - test-conda-cpp-valgrind: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-test-conda-cpp-valgrind - test-conda-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-test-conda-cpp - test-conda-python-3.7-turbodbc-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-circle-test-conda-python-3.7-turbodbc-master - test-debian-10-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-test-debian-10-cpp - test-ubuntu-18.04-cpp-cmake32: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-test-ubuntu-18.04-cpp-cmake32 - test-ubuntu-18.04-cpp-static: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-test-ubuntu-18.04-cpp-static - ubuntu-bionic-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-ubuntu-bionic-amd64 - ubuntu-eoan-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-ubuntu-eoan-amd64 - ubuntu-focal-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-ubuntu-focal-amd64 - wheel-osx-cp35m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-travis-wheel-osx-cp35m - wheel-osx-cp36m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-travis-wheel-osx-cp36m - wheel-osx-cp37m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-travis-wheel-osx-cp37m - wheel-osx-cp38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-travis-wheel-osx-cp38 Succeeded Tasks: - centos-6-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-centos-6-amd64 - centos-7-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-centos-7-amd64 - conda-linux-gcc-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-linux-gcc-py36 - conda-linux-gcc-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-linux-gcc-py37 - conda-linux-gcc-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-linux-gcc-py38 - conda-osx-clang-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-osx-clang-py36 - conda-osx-clang-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-osx-clang-py37 - conda-osx-clang-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-osx-clang-py38 - conda-win-vs2015-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-win-vs2015-py36 - conda-win-vs2015-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-win-vs2015-py37 - conda-win-vs2015-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-conda-win-vs2015-py38 - debian-stretch-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-github-debian-stretch-amd64 - gandiva-jar-xenial: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-travis-gandiva-jar-xenial - homebrew-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-travis-homebrew-cpp - homebrew-r-autobrew: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-travis-homebrew-r-autobrew - test-conda-python-3.6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-azure-test-conda-python-3.6 - test-conda-python-3.7-dask-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-circle-test-conda-python-3.7-dask-latest - test-conda-python-3.7-hdfs-2.9.2: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-04-19-0-circle-test-conda-python-3.7-hdfs-2.9.2 - test-conda-python-3.7-kartothek-latest: URL:
