[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705348517 I reverted some of the OS X changes. I'm not sure I actually needed to change Python 3.8 there since it looks like that travis file is templated out to all OS X wheels and

[GitHub] [arrow] terencehonles commented on a change in pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on a change in pull request #8386: URL: https://github.com/apache/arrow/pull/8386#discussion_r501464330 ## File path: python/pyproject.toml ## @@ -19,7 +19,9 @@ requires = [ "cython >= 0.29", "numpy==1.14.5; python_version<'3.7'", -

[GitHub] [arrow] terencehonles commented on a change in pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on a change in pull request #8386: URL: https://github.com/apache/arrow/pull/8386#discussion_r501463131 ## File path: dev/tasks/python-wheels/travis.osx.yml ## @@ -31,7 +31,7 @@ addons: - git - openssl@1.1 - protobuf - -

[GitHub] [arrow] jduo commented on pull request #8325: ARROW-10206: [C++][Python][FlightRPC] Allow disabling server validation

2020-10-07 Thread GitBox
jduo commented on pull request #8325: URL: https://github.com/apache/arrow/pull/8325#issuecomment-705342986 I've implemented auto-detection of the namespace of TlsCredentialsOptions now. This should let the code work across multiple gRPC versions. If TlsCredentialsOptions doesn't exist,

[GitHub] [arrow] jduo commented on a change in pull request #8325: ARROW-10206: [C++][Python][FlightRPC] Allow disabling server validation

2020-10-07 Thread GitBox
jduo commented on a change in pull request #8325: URL: https://github.com/apache/arrow/pull/8325#discussion_r501460070 ## File path: python/manylinux1/scripts/build_re2.sh ## @@ -31,5 +31,7 @@ make prefix=/usr/local -j${NCORES} install popd -# Need to remove shared

[GitHub] [arrow] terencehonles removed a comment on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles removed a comment on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705332360 Not sure if that will work for me, but might as well give it a shot 路 This is an automated message

[GitHub] [arrow] jduo commented on a change in pull request #8325: ARROW-10206: [C++][Python][FlightRPC] Allow disabling server validation

2020-10-07 Thread GitBox
jduo commented on a change in pull request #8325: URL: https://github.com/apache/arrow/pull/8325#discussion_r501459974 ## File path: python/manylinux1/scripts/build_grpc.sh ## @@ -16,8 +16,61 @@ # specific language governing permissions and limitations # under the License.

[GitHub] [arrow] terencehonles removed a comment on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles removed a comment on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705332230 @github-actions crossbow submit -g conda -g wheel This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8393: ARROW-10228: Contribute Julia implementation

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8393: URL: https://github.com/apache/arrow/pull/8393#issuecomment-705339836 https://issues.apache.org/jira/browse/ARROW-10228 This is an automated message from the Apache Git

[GitHub] [arrow] kou commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
kou commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705334155 (We need to wait a while until our comments are processed.) This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705333263 Revision: 59496cc5698070f4ddf7fb071d73239eff833a4c Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705332791 Revision: 59496cc5698070f4ddf7fb071d73239eff833a4c Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] kou commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
kou commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705332674 @github-actions crossbow submit -g conda -g wheel This is an automated message from the Apache Git Service. To

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705332360 Not sure if that will work for me, but might as well give it a shot 路 This is an automated message from the

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705332230 @github-actions crossbow submit -g conda -g wheel This is an automated message from the Apache Git Service.

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705332107 It looks like cmake was already part of the quay.io/pypa/manylinux2014_x86_64:2020-10-06-b2ca7a1 and `scripts/install_cmake.sh` needed to be updated.

[GitHub] [arrow] kiszk commented on a change in pull request #8374: ARROW-10203: Give guidance on big-endian support in the contributors docs

2020-10-07 Thread GitBox
kiszk commented on a change in pull request #8374: URL: https://github.com/apache/arrow/pull/8374#discussion_r501448247 ## File path: docs/source/developers/contributing.rst ## @@ -304,3 +304,40 @@ to your branch, which they sometimes do to help move a pull request along. In

[GitHub] [arrow] emkornfield commented on a change in pull request #8374: ARROW-10203: Give guidance on big-endian support in the contributors docs

2020-10-07 Thread GitBox
emkornfield commented on a change in pull request #8374: URL: https://github.com/apache/arrow/pull/8374#discussion_r501447504 ## File path: docs/source/developers/contributing.rst ## @@ -304,3 +304,40 @@ to your branch, which they sometimes do to help move a pull request

[GitHub] [arrow] emkornfield commented on a change in pull request #8374: ARROW-10203: Give guidance on big-endian support in the contributors docs

2020-10-07 Thread GitBox
emkornfield commented on a change in pull request #8374: URL: https://github.com/apache/arrow/pull/8374#discussion_r501447270 ## File path: docs/source/developers/contributing.rst ## @@ -304,3 +304,40 @@ to your branch, which they sometimes do to help move a pull request

[GitHub] [arrow] emkornfield commented on a change in pull request #8366: ARROW-9943: [C++] Recursively apply Arrow metadata when reading from Parquet

2020-10-07 Thread GitBox
emkornfield commented on a change in pull request #8366: URL: https://github.com/apache/arrow/pull/8366#discussion_r501446625 ## File path: cpp/src/parquet/arrow/schema.h ## @@ -91,7 +91,6 @@ struct PARQUET_EXPORT SchemaField { std::shared_ptr<::arrow::Field> field; //

[GitHub] [arrow] github-actions[bot] commented on pull request #8392: ARROW-10229: remove errant log line

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8392: URL: https://github.com/apache/arrow/pull/8392#issuecomment-705326448 https://issues.apache.org/jira/browse/ARROW-10229 This is an automated message from the Apache Git

[GitHub] [arrow] emkornfield opened a new pull request #8392: ARROW-10229: remove errant log line

2020-10-07 Thread GitBox
emkornfield opened a new pull request #8392: URL: https://github.com/apache/arrow/pull/8392 noticed this on rereviewing the merged code. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] emkornfield commented on pull request #6979: ARROW-7800 [Python] implement iter_batches() method for ParquetFile and ParquetReader

2020-10-07 Thread GitBox
emkornfield commented on pull request #6979: URL: https://github.com/apache/arrow/pull/6979#issuecomment-705321792 @wesm does your +1 still hold? Can this be merged now? This is an automated message from the Apache Git

[GitHub] [arrow] jorgecarleitao commented on pull request #8287: ARROW-10111: [Rust] Added new crate with code that consumes C Data interface

2020-10-07 Thread GitBox
jorgecarleitao commented on pull request #8287: URL: https://github.com/apache/arrow/pull/8287#issuecomment-705321509 Prob. the best place to see that is in [this PR](https://github.com/jorgecarleitao/arrow/pull/13) on my repo. It has the latest version.

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8222: ARROW-10043: [Rust][DataFusion] Implement COUNT(DISTINCT col)

2020-10-07 Thread GitBox
jorgecarleitao commented on a change in pull request #8222: URL: https://github.com/apache/arrow/pull/8222#discussion_r501440700 ## File path: rust/datafusion/src/test/mod.rs ## @@ -135,6 +135,13 @@ pub fn format_batch(batch: ) -> Vec { } let array =

[GitHub] [arrow] nevi-me commented on pull request #8287: ARROW-10111: [Rust] Added new crate with code that consumes C Data interface

2020-10-07 Thread GitBox
nevi-me commented on pull request #8287: URL: https://github.com/apache/arrow/pull/8287#issuecomment-705319947 No worries, I'll still have a look at this so I can see the approach that you're taking This is an automated

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8222: ARROW-10043: [Rust][DataFusion] Implement COUNT(DISTINCT col)

2020-10-07 Thread GitBox
jorgecarleitao commented on a change in pull request #8222: URL: https://github.com/apache/arrow/pull/8222#discussion_r501436730 ## File path: rust/datafusion/src/physical_plan/distinct_expressions.rs ## @@ -0,0 +1,203 @@ +// Licensed to the Apache Software Foundation (ASF)

[GitHub] [arrow] jorgecarleitao closed pull request #8287: ARROW-10111: [Rust] Added new crate with code that consumes C Data interface

2020-10-07 Thread GitBox
jorgecarleitao closed pull request #8287: URL: https://github.com/apache/arrow/pull/8287 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] jorgecarleitao commented on pull request #8287: ARROW-10111: [Rust] Added new crate with code that consumes C Data interface

2020-10-07 Thread GitBox
jorgecarleitao commented on pull request #8287: URL: https://github.com/apache/arrow/pull/8287#issuecomment-705314475 No need, @nevi-me I am still working on this on another place. I will close it as this won't fly. I will PR separately.

[GitHub] [arrow] kou closed pull request #8391: ARROW-10227: [Ruby] Use a table size as the default for parquet chunk_size

2020-10-07 Thread GitBox
kou closed pull request #8391: URL: https://github.com/apache/arrow/pull/8391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] nealrichardson commented on pull request #7807: ARROW-6537 [R]: Pass column_types to CSV reader

2020-10-07 Thread GitBox
nealrichardson commented on pull request #7807: URL: https://github.com/apache/arrow/pull/7807#issuecomment-705301068 @romainfrancois PTAL, I adjusted a few things, wrote docs, and then adjusted a little more based on what made sense to document. I think the new section in the docs for

[GitHub] [arrow] kiszk commented on a change in pull request #8374: ARROW-10203: Give guidance on big-endian support in the contributors docs

2020-10-07 Thread GitBox
kiszk commented on a change in pull request #8374: URL: https://github.com/apache/arrow/pull/8374#discussion_r500762410 ## File path: docs/source/developers/contributing.rst ## @@ -304,3 +304,40 @@ to your branch, which they sometimes do to help move a pull request along. In

[GitHub] [arrow] github-actions[bot] commented on pull request #8391: ARROW-10227: [Ruby] Use a table size as the default for parquet chunk_size

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8391: URL: https://github.com/apache/arrow/pull/8391#issuecomment-705297244 https://issues.apache.org/jira/browse/ARROW-10227 This is an automated message from the Apache Git

[GitHub] [arrow] kanga333 opened a new pull request #8391: ARROW-10227: [Ruby] Use a table size as the default for parquet chunk_size

2020-10-07 Thread GitBox
kanga333 opened a new pull request #8391: URL: https://github.com/apache/arrow/pull/8391 A chunk_size that is too small will cause metadata bloat in the parquet file, leading to poor read performance. Set the chunk_size to be the same value as the table size so that one file becomes one

[GitHub] [arrow] github-actions[bot] commented on pull request #8390: ARROW-8518: [Python] Setup tools for building an optional pyarrow_gandiva package [WIP]

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8390: URL: https://github.com/apache/arrow/pull/8390#issuecomment-705291500 https://issues.apache.org/jira/browse/ARROW-8518 This is an automated message from the Apache Git

[GitHub] [arrow] wjones1 opened a new pull request #8390: ARROW-8518: [Python] Setup tools for building an optional pyarrow_gandiva package [WIP]

2020-10-07 Thread GitBox
wjones1 opened a new pull request #8390: URL: https://github.com/apache/arrow/pull/8390 Goal is to create a separate module for gandiva pyarrow, which hopefully will work as a model for other optional components such as flight. The setup.py arrangement is inspired by the dynd_python

[GitHub] [arrow] terencehonles commented on a change in pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on a change in pull request #8386: URL: https://github.com/apache/arrow/pull/8386#discussion_r501409865 ## File path: dev/tasks/python-wheels/win-build.bat ## @@ -110,6 +110,6 @@ set ARROW_TEST_DATA=%ARROW_SRC%\testing\data %PYTHON_INTERPRETER% -c

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705273300 Revision: da52256400bb1cdde72fb6b0cdcf92b9142a5eeb Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] kou commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
kou commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705272660 @github-actions crossbow submit -g conda -g wheel This is an automated message from the Apache Git Service. To

[GitHub] [arrow] kou commented on a change in pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
kou commented on a change in pull request #8386: URL: https://github.com/apache/arrow/pull/8386#discussion_r501393657 ## File path: python/pyproject.toml ## @@ -19,7 +19,9 @@ requires = [ "cython >= 0.29", "numpy==1.14.5; python_version<'3.7'", -"numpy==1.16.0;

[GitHub] [arrow] github-actions[bot] commented on pull request #8389: ARROW-8296: [C++][Dataset] Add IpcFileWriteOptions

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8389: URL: https://github.com/apache/arrow/pull/8389#issuecomment-705269694 https://issues.apache.org/jira/browse/ARROW-8296 This is an automated message from the Apache Git

[GitHub] [arrow] bkietz commented on pull request #8317: ARROW-10134: [Python][Dataset] Add ParquetFileFragment.num_row_groups

2020-10-07 Thread GitBox
bkietz commented on pull request #8317: URL: https://github.com/apache/arrow/pull/8317#issuecomment-705261226 CI failure is unrelated. Merging This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] bkietz closed pull request #8317: ARROW-10134: [Python][Dataset] Add ParquetFileFragment.num_row_groups

2020-10-07 Thread GitBox
bkietz closed pull request #8317: URL: https://github.com/apache/arrow/pull/8317 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] bkietz opened a new pull request #8389: ARROW-8296: [C++][Dataset] Add IpcFileWriteOptions

2020-10-07 Thread GitBox
bkietz opened a new pull request #8389: URL: https://github.com/apache/arrow/pull/8389 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] bkietz closed pull request #8305: ARROW-9782: [C++][Dataset] More configurable Dataset writing

2020-10-07 Thread GitBox
bkietz closed pull request #8305: URL: https://github.com/apache/arrow/pull/8305 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] bkietz commented on pull request #8305: ARROW-9782: [C++][Dataset] More configurable Dataset writing

2020-10-07 Thread GitBox
bkietz commented on pull request #8305: URL: https://github.com/apache/arrow/pull/8305#issuecomment-705259779 Merging This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow] drusso commented on a change in pull request #8222: ARROW-10043: [Rust][DataFusion] Implement COUNT(DISTINCT col)

2020-10-07 Thread GitBox
drusso commented on a change in pull request #8222: URL: https://github.com/apache/arrow/pull/8222#discussion_r501375560 ## File path: rust/datafusion/src/physical_plan/distinct_expressions.rs ## @@ -0,0 +1,203 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [arrow] github-actions[bot] commented on pull request #8145: ARROW-9967: [Python] Add compute module docs + expose more option classes

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8145: URL: https://github.com/apache/arrow/pull/8145#issuecomment-705240530 Revision: 73272bf38f2d8406b60a6ebaa20c5f4e58dff075 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] kszucs commented on pull request #8145: ARROW-9967: [Python] Add compute module docs + expose more option classes

2020-10-07 Thread GitBox
kszucs commented on pull request #8145: URL: https://github.com/apache/arrow/pull/8145#issuecomment-705240057 @github-actions crossbow submit test-ubuntu-18.04-docs This is an automated message from the Apache Git Service.

[GitHub] [arrow] kszucs commented on a change in pull request #8374: ARROW-10203: Give guidance on big-endian support in the contributors docs

2020-10-07 Thread GitBox
kszucs commented on a change in pull request #8374: URL: https://github.com/apache/arrow/pull/8374#discussion_r501356855 ## File path: docs/source/developers/contributing.rst ## @@ -304,3 +304,40 @@ to your branch, which they sometimes do to help move a pull request along.

[GitHub] [arrow] github-actions[bot] commented on pull request #8388: ARROW-10225: [Rust] [Parquet] Fix null comparison in roundtrip

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8388: URL: https://github.com/apache/arrow/pull/8388#issuecomment-705232831 https://issues.apache.org/jira/browse/ARROW-10225 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8387: ARROW-9843: [C++] Between Kernel

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8387: URL: https://github.com/apache/arrow/pull/8387#issuecomment-705232830 https://issues.apache.org/jira/browse/ARROW-9843 This is an automated message from the Apache Git

[GitHub] [arrow] nevi-me commented on a change in pull request #8262: ARROW-10040: [Rust] Iterate over and combine boolean buffers with arbitrary offsets

2020-10-07 Thread GitBox
nevi-me commented on a change in pull request #8262: URL: https://github.com/apache/arrow/pull/8262#discussion_r501349134 ## File path: rust/arrow/src/buffer.rs ## @@ -371,118 +388,165 @@ where fn bitwise_bin_op_helper( left: , -left_offset: usize, +

[GitHub] [arrow] nevi-me commented on pull request #8388: ARROW-10225: [Rust] [Parquet] Fix null comparison in roundtrip

2020-10-07 Thread GitBox
nevi-me commented on pull request #8388: URL: https://github.com/apache/arrow/pull/8388#issuecomment-705228998 @carols10cents this fixes the string test roundtrips. Binary still doesn't work This is an automated message

[GitHub] [arrow] nevi-me opened a new pull request #8388: ARROW-10225: [Rust] [Parquet] Fix null comparison in roundtrip

2020-10-07 Thread GitBox
nevi-me opened a new pull request #8388: URL: https://github.com/apache/arrow/pull/8388 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] github-actions[bot] commented on pull request #8387: Arrow-9843 Between kernel

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8387: URL: https://github.com/apache/arrow/pull/8387#issuecomment-705228286 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] david1437 opened a new pull request #8387: Arrow-9843 Between kernel

2020-10-07 Thread GitBox
david1437 opened a new pull request #8387: URL: https://github.com/apache/arrow/pull/8387 Added between kernel as per the Jira specs. As part of this issue added ScalarTernary struct in order to support ternary functions more easily.

[GitHub] [arrow] nevi-me commented on pull request #8354: ARROW-10168: [Rust] [Parquet] Schema roundtrip - use Arrow schema from Parquet metadata when available

2020-10-07 Thread GitBox
nevi-me commented on pull request #8354: URL: https://github.com/apache/arrow/pull/8354#issuecomment-705222763 Merged This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow] nevi-me closed pull request #8354: ARROW-10168: [Rust] [Parquet] Schema roundtrip - use Arrow schema from Parquet metadata when available

2020-10-07 Thread GitBox
nevi-me closed pull request #8354: URL: https://github.com/apache/arrow/pull/8354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705219863 It looks like I "missed" the github workflows, but that might need to be done after the wheels are built and published as I haven't looked at what exactly are doing, and I

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-705218612 https://issues.apache.org/jira/browse/ARROW-10224 This is an automated message from the Apache Git

[GitHub] [arrow] terencehonles commented on a change in pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on a change in pull request #8386: URL: https://github.com/apache/arrow/pull/8386#discussion_r501333833 ## File path: python/pyproject.toml ## @@ -19,7 +19,9 @@ requires = [ "cython >= 0.29", "numpy==1.14.5; python_version<'3.7'", -

[GitHub] [arrow] terencehonles commented on a change in pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on a change in pull request #8386: URL: https://github.com/apache/arrow/pull/8386#discussion_r501333833 ## File path: python/pyproject.toml ## @@ -19,7 +19,9 @@ requires = [ "cython >= 0.29", "numpy==1.14.5; python_version<'3.7'", -

[GitHub] [arrow] terencehonles commented on a change in pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on a change in pull request #8386: URL: https://github.com/apache/arrow/pull/8386#discussion_r501332755 ## File path: dev/tasks/python-wheels/win-build.bat ## @@ -110,6 +110,6 @@ set ARROW_TEST_DATA=%ARROW_SRC%\testing\data %PYTHON_INTERPRETER% -c

[GitHub] [arrow] terencehonles commented on a change in pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles commented on a change in pull request #8386: URL: https://github.com/apache/arrow/pull/8386#discussion_r501332638 ## File path: dev/tasks/homebrew-formulae/apache-arrow.rb ## @@ -16,7 +16,7 @@ class ApacheArrow < Formula depends_on "numpy" depends_on

[GitHub] [arrow] terencehonles opened a new pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-07 Thread GitBox
terencehonles opened a new pull request #8386: URL: https://github.com/apache/arrow/pull/8386 Adds support and testing for Python 3.9. I am looking for review as this change may have touched too many things, but I'm also looking to get the CI to test all the different environments.

[GitHub] [arrow] nevi-me commented on pull request #8287: ARROW-10111: [Rust] Added new crate with code that consumes C Data interface

2020-10-07 Thread GitBox
nevi-me commented on pull request #8287: URL: https://github.com/apache/arrow/pull/8287#issuecomment-705212143 Hey @jorgecarleitao, I'll only be able to look at this either over the weekend or during the coming week This is

[GitHub] [arrow] kszucs commented on a change in pull request #8072: ARROW-9879: [Python] Add support for numpy scalars to ChunkedArray.__getitem__

2020-10-07 Thread GitBox
kszucs commented on a change in pull request #8072: URL: https://github.com/apache/arrow/pull/8072#discussion_r501312248 ## File path: python/pyarrow/table.pxi ## @@ -160,6 +160,8 @@ cdef class ChunkedArray(_PandasConvertible): return _normalize_slice(self, key)

[GitHub] [arrow] kou closed pull request #8356: ARROW-10139: [C++] Add support for building arrow_testing without building tests

2020-10-07 Thread GitBox
kou closed pull request #8356: URL: https://github.com/apache/arrow/pull/8356 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] kou commented on pull request #8356: ARROW-10139: [C++] Add support for building arrow_testing without building tests

2020-10-07 Thread GitBox
kou commented on pull request #8356: URL: https://github.com/apache/arrow/pull/8356#issuecomment-705194501 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [arrow] kszucs commented on a change in pull request #8325: ARROW-10206: [C++][Python][FlightRPC] Allow disabling server validation

2020-10-07 Thread GitBox
kszucs commented on a change in pull request #8325: URL: https://github.com/apache/arrow/pull/8325#discussion_r501307054 ## File path: python/manylinux1/scripts/build_grpc.sh ## @@ -16,8 +16,61 @@ # specific language governing permissions and limitations # under the License.

[GitHub] [arrow] bkietz closed pull request #8367: ARROW-10099: [C++][Dataset] Simplify type inference for partition columns

2020-10-07 Thread GitBox
bkietz closed pull request #8367: URL: https://github.com/apache/arrow/pull/8367 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] bkietz commented on a change in pull request #8351: ARROW-9870: [R] Friendly interface for filesystems (S3)

2020-10-07 Thread GitBox
bkietz commented on a change in pull request #8351: URL: https://github.com/apache/arrow/pull/8351#discussion_r501234407 ## File path: r/vignettes/fs.Rmd ## @@ -44,41 +44,46 @@ the cost of reading the data over the network should be much lower. ## Creating a FileSystem

[GitHub] [arrow] github-actions[bot] commented on pull request #8349: ARROW-3080: [Python] Unify Arrow to Python object conversion paths

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8349: URL: https://github.com/apache/arrow/pull/8349#issuecomment-705178406 Revision: f359b3d27a52b705de30e363801cdaa72c2c2b8f Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] kszucs removed a comment on pull request #8349: ARROW-3080: [Python] Unify Arrow to Python object conversion paths

2020-10-07 Thread GitBox
kszucs removed a comment on pull request #8349: URL: https://github.com/apache/arrow/pull/8349#issuecomment-704914556 @github-actions crossbow submit test-conda-python-3.8-hypothesis This is an automated message from the

[GitHub] [arrow] github-actions[bot] removed a comment on pull request #8349: ARROW-3080: [Python] Unify Arrow to Python object conversion paths

2020-10-07 Thread GitBox
github-actions[bot] removed a comment on pull request #8349: URL: https://github.com/apache/arrow/pull/8349#issuecomment-704918186 Revision: f359b3d27a52b705de30e363801cdaa72c2c2b8f Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] kszucs commented on pull request #8349: ARROW-3080: [Python] Unify Arrow to Python object conversion paths

2020-10-07 Thread GitBox
kszucs commented on pull request #8349: URL: https://github.com/apache/arrow/pull/8349#issuecomment-705177619 @github-actions crossbow submit test-conda-python-3.8-hypothesis This is an automated message from the Apache Git

[GitHub] [arrow] kszucs closed pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels

2020-10-07 Thread GitBox
kszucs closed pull request #8315: URL: https://github.com/apache/arrow/pull/8315 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] jhorstmann commented on pull request #8262: ARROW-10040: [Rust] Iterate over and combine boolean buffers with arbitrary offsets

2020-10-07 Thread GitBox
jhorstmann commented on pull request #8262: URL: https://github.com/apache/arrow/pull/8262#issuecomment-705176180 I ran some benchmarks and it seems the simd versions of `bitwise_bin_op` etc are only about 20% to 50% faster than the iterator based ones. I think it would be an acceptable

[GitHub] [arrow] kszucs edited a comment on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels

2020-10-07 Thread GitBox
kszucs edited a comment on pull request #8315: URL: https://github.com/apache/arrow/pull/8315#issuecomment-705168356 > > Sadly we can't utilize travis' build cache since crossbow always creates new branches and the fallback default branch won't have any cache entries. > > We _could_

[GitHub] [arrow] kszucs commented on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels

2020-10-07 Thread GitBox
kszucs commented on pull request #8315: URL: https://github.com/apache/arrow/pull/8315#issuecomment-705169726 > Does this explain why some of the nightly builds seem to be intermittently missing certain macOS wheels? E.g., sometimes for example there's no cp38 wheel, only for that wheel

[GitHub] [arrow] kszucs edited a comment on pull request #8315: ARROW-9266: [Python][Packaging] Enable S3 support in macOS wheels

2020-10-07 Thread GitBox
kszucs edited a comment on pull request #8315: URL: https://github.com/apache/arrow/pull/8315#issuecomment-705168356 > > Sadly we can't utilize travis' build cache since crossbow always creates new branches and the fallback default branch won't have any cache entries. > > We _could_

[GitHub] [arrow] alamb commented on pull request #8346: ARROW-10164: [Rust] Add support for DictionaryArray to cast kernel

2020-10-07 Thread GitBox
alamb commented on pull request #8346: URL: https://github.com/apache/arrow/pull/8346#issuecomment-705163232 Rebased as I had some CI failures -- hoping to get a green run This is an automated message from the Apache Git

[GitHub] [arrow] jduo commented on a change in pull request #8325: ARROW-10206: [C++][Python][FlightRPC] Allow disabling server validation

2020-10-07 Thread GitBox
jduo commented on a change in pull request #8325: URL: https://github.com/apache/arrow/pull/8325#discussion_r501274245 ## File path: cpp/cmake_modules/ThirdpartyToolchain.cmake ## @@ -2463,7 +2463,7 @@ macro(build_grpc) endmacro() if(ARROW_WITH_GRPC) -

[GitHub] [arrow] jhorstmann commented on a change in pull request #8262: ARROW-10040: [Rust] Iterate over and combine boolean buffers with arbitrary offsets

2020-10-07 Thread GitBox
jhorstmann commented on a change in pull request #8262: URL: https://github.com/apache/arrow/pull/8262#discussion_r501273876 ## File path: rust/arrow/src/buffer.rs ## @@ -371,118 +388,165 @@ where fn bitwise_bin_op_helper( left: , -left_offset: usize, +

[GitHub] [arrow] jduo commented on pull request #8325: ARROW-10206: [C++][Python][FlightRPC] Allow disabling server validation

2020-10-07 Thread GitBox
jduo commented on pull request #8325: URL: https://github.com/apache/arrow/pull/8325#issuecomment-705155690 FYI @nealrichardson @pitrou @wesm This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] alamb commented on a change in pull request #8340: ARROW-10165: [Rust] [DataFusion]: Remove special case DataFusion casting checks in favor of Arrow cast kernel

2020-10-07 Thread GitBox
alamb commented on a change in pull request #8340: URL: https://github.com/apache/arrow/pull/8340#discussion_r501262082 ## File path: rust/datafusion/src/logical_plan/mod.rs ## @@ -323,21 +322,19 @@ impl Expr { /// /// # Errors /// -/// This function errors

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8340: ARROW-10165: [Rust] [DataFusion]: Remove special case DataFusion casting checks in favor of Arrow cast kernel

2020-10-07 Thread GitBox
jorgecarleitao commented on a change in pull request #8340: URL: https://github.com/apache/arrow/pull/8340#discussion_r501257096 ## File path: rust/datafusion/src/logical_plan/mod.rs ## @@ -323,21 +322,19 @@ impl Expr { /// /// # Errors /// -/// This

[GitHub] [arrow] github-actions[bot] commented on pull request #8385: fix toArray() to handle null values

2020-10-07 Thread GitBox
github-actions[bot] commented on pull request #8385: URL: https://github.com/apache/arrow/pull/8385#issuecomment-705144305 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] nevi-me commented on a change in pull request #8262: ARROW-10040: [Rust] Iterate over and combine boolean buffers with arbitrary offsets

2020-10-07 Thread GitBox
nevi-me commented on a change in pull request #8262: URL: https://github.com/apache/arrow/pull/8262#discussion_r501255235 ## File path: rust/arrow/src/util/bit_chunk_iterator.rs ## @@ -0,0 +1,223 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] nevi-me closed pull request #8378: ARROW-10204: [Rust] Filter kernel should only count bits in valid range

2020-10-07 Thread GitBox
nevi-me closed pull request #8378: URL: https://github.com/apache/arrow/pull/8378 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] bmschmidt opened a new pull request #8385: fix toArray() to handle null values

2020-10-07 Thread GitBox
bmschmidt opened a new pull request #8385: URL: https://github.com/apache/arrow/pull/8385 Remove fast method that ignores null values. JIRA issued filed. A better version might use the old method if the vector is guaranteed not to be nullable.

[GitHub] [arrow] alamb commented on a change in pull request #8340: ARROW-10165: [Rust] [DataFusion]: Remove special case DataFusion casting checks in favor of Arrow cast kernel

2020-10-07 Thread GitBox
alamb commented on a change in pull request #8340: URL: https://github.com/apache/arrow/pull/8340#discussion_r501246915 ## File path: rust/datafusion/src/logical_plan/mod.rs ## @@ -323,21 +322,19 @@ impl Expr { /// /// # Errors /// -/// This function errors

[GitHub] [arrow] arw2019 commented on a change in pull request #8244: ARROW-8355: [Python] Reduce the number of pandas dependent test cases in test_feather

2020-10-07 Thread GitBox
arw2019 commented on a change in pull request #8244: URL: https://github.com/apache/arrow/pull/8244#discussion_r501246193 ## File path: python/pyarrow/tests/test_feather.py ## @@ -128,19 +128,22 @@ def test_dataset(version): num_values = (100, 100) Review comment:

[GitHub] [arrow] carols10cents commented on pull request #8354: ARROW-10168: [Rust] [Parquet] Schema roundtrip - use Arrow schema from Parquet metadata when available

2020-10-07 Thread GitBox
carols10cents commented on pull request #8354: URL: https://github.com/apache/arrow/pull/8354#issuecomment-705131362 @nevi-me I added your commit onto this branch! This is an automated message from the Apache Git Service. To

[GitHub] [arrow] BryanCutler commented on a change in pull request #8337: ARROW-10151: [Python] Add support for MapArray conversion to Pandas

2020-10-07 Thread GitBox
BryanCutler commented on a change in pull request #8337: URL: https://github.com/apache/arrow/pull/8337#discussion_r501237857 ## File path: cpp/src/arrow/python/arrow_to_pandas.cc ## @@ -791,6 +791,111 @@ Status ConvertListsLike(PandasOptions options, const ChunkedArray&

[GitHub] [arrow] BryanCutler commented on pull request #8337: ARROW-10151: [Python] Add support for MapArray conversion to Pandas

2020-10-07 Thread GitBox
BryanCutler commented on pull request #8337: URL: https://github.com/apache/arrow/pull/8337#issuecomment-705128360 Thanks for all the help @pitrou ! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] randomgambit opened a new issue #8384: how to test whether arrow works correctly in R?

2020-10-07 Thread GitBox
randomgambit opened a new issue #8384: URL: https://github.com/apache/arrow/issues/8384 Hello there! I managed to install the great `arrow` on my linux system (with Rstudio) and it seems to be working great. However, I was wondering if a test script is available (like in Pandas) to

[GitHub] [arrow] nevi-me commented on pull request #8354: ARROW-10168: [Rust] [Parquet] Schema roundtrip - use Arrow schema from Parquet metadata when available

2020-10-07 Thread GitBox
nevi-me commented on pull request #8354: URL: https://github.com/apache/arrow/pull/8354#issuecomment-705123343 > @nevi-me I saw it just after :) I'm looking at it now! I don't think there are conflicts, and I think my last commit is addressing a different issue than your last commit?

[GitHub] [arrow] carols10cents commented on pull request #8354: ARROW-10168: [Rust] [Parquet] Schema roundtrip - use Arrow schema from Parquet metadata when available

2020-10-07 Thread GitBox
carols10cents commented on pull request #8354: URL: https://github.com/apache/arrow/pull/8354#issuecomment-705119984 @nevi-me I saw it just after :) I'm looking at it now! I don't think there are conflicts, and I think my last commit is addressing a different issue than your last commit?

  1   2   3   4   >