[GitHub] [arrow] jorgecarleitao commented on pull request #8503: ARROW-10366: [Rust] [DataFusion] Remove collect from within threads on merge

2020-10-21 Thread GitBox
jorgecarleitao commented on pull request #8503: URL: https://github.com/apache/arrow/pull/8503#issuecomment-714243113 This is not working with aggregates and I am trying to understand why, thus moved it to draft. This is an

[GitHub] [arrow] github-actions[bot] commented on pull request #8503: ARROW-10366: [Rust] [DataFusion] Remove collect from within threads on merge

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8503: URL: https://github.com/apache/arrow/pull/8503#issuecomment-714238006 https://issues.apache.org/jira/browse/ARROW-10366 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-714235628 Revision: a3076ebe2a33e7a7a3b7b3aec5afd725046bc0cf Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] pprudhvi commented on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
pprudhvi commented on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-714234917 @github-actions crossbow submit -g gandiva This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8473: ARROW-10320 [Rust] [DataFusion] Migrated from batch iterators to batch streams.

2020-10-21 Thread GitBox
jorgecarleitao commented on a change in pull request #8473: URL: https://github.com/apache/arrow/pull/8473#discussion_r509884815 ## File path: rust/datafusion/src/physical_plan/merge.rs ## @@ -100,27 +103,29 @@ impl ExecutionPlan for MergeExec {

[GitHub] [arrow] jorgecarleitao opened a new pull request #8503: ARROW-10366: [Rust] [DataFusion] Remove collect from within threads on merge

2020-10-21 Thread GitBox
jorgecarleitao opened a new pull request #8503: URL: https://github.com/apache/arrow/pull/8503 This makes `merge` send batches to a receiver stream as they arrive and, thereby removing the need to wait for each thread to finish collecting all its batches. on micro-benchmarks,

[GitHub] [arrow] emkornfield commented on pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
emkornfield commented on pull request #8475: URL: https://github.com/apache/arrow/pull/8475#issuecomment-714233095 i Believe the archery test is unrelated i opened https://issues.apache.org/jira/browse/ARROW-10367 to track.

[GitHub] [arrow] liyafan82 commented on pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
liyafan82 commented on pull request #8475: URL: https://github.com/apache/arrow/pull/8475#issuecomment-714226000 > @liyafan82 thanks for the typo catches. I addressed the comments. Any other concerns? Sorry for my delayed review. Hopefully, I will finish the second pass today.

[GitHub] [arrow] emkornfield commented on pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
emkornfield commented on pull request #8475: URL: https://github.com/apache/arrow/pull/8475#issuecomment-714223728 @liyafan82 thanks for the typo catches. I addressed the comments. Any other concerns? This is an

[GitHub] [arrow] emkornfield commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
emkornfield commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509436836 ## File path: java/vector/src/main/java/org/apache/arrow/vector/util/DecimalUtility.java ## @@ -119,34 +121,47 @@ public static boolean

[GitHub] [arrow] liyafan82 commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
liyafan82 commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509872357 ## File path: java/vector/src/main/codegen/templates/UnionListWriter.java ## @@ -224,6 +249,27 @@ public void writeBigEndianBytesToDecimal(byte[] value,

[GitHub] [arrow] liyafan82 commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
liyafan82 commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509870508 ## File path: java/vector/src/main/codegen/templates/ArrowType.java ## @@ -165,7 +165,20 @@ public final T visit(${type.name?remove_ending("_")} type) {

[GitHub] [arrow] emkornfield commented on pull request #7887: ARROW-9304: [C++] Add "AppendEmpty" builder APIs for use inside StructBuilder::AppendNull

2020-10-21 Thread GitBox
emkornfield commented on pull request #7887: URL: https://github.com/apache/arrow/pull/7887#issuecomment-714212972 @pitrou any objections to merging now (I would expect this might conflict with Decimal256 PR). This is an

[GitHub] [arrow] emkornfield commented on pull request #8417: WIP: [C++] Get rid of code duplication in Decimal##bit_width

2020-10-21 Thread GitBox
emkornfield commented on pull request #8417: URL: https://github.com/apache/arrow/pull/8417#issuecomment-714209176 > Moving forward, I think that it could be useful if we'll declare some interface for BasicDecimal and Decimal classes, which will be implemented by BasicDecimal/Decimal128,

[GitHub] [arrow] liyafan82 commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
liyafan82 commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509864656 ## File path: java/vector/src/main/codegen/templates/AbstractPromotableFieldWriter.java ## @@ -75,7 +75,7 @@ public void endList() { <#list

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714200074 Revision: 27bfc7034db054b1226c15bf513abe059e0e431f Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714195699 @github-actions crossbow submit wheel-osx-* This is an automated message from the Apache Git Service. To

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714191764 > Could you try the command lines that are executed in CMake on the environment? > > *

[GitHub] [arrow] terencehonles edited a comment on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles edited a comment on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714174255 > > ``` > > * For the windows wheel it looks like the issue is the VS runtime is pinned

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714179077 Revision: 94d3091b093b41f0ea25df2fd11ad889c2a56c96 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714178549 @github-actions crossbow submit wheel-win-* This is an automated message from the Apache Git Service. To

[GitHub] [arrow] terencehonles edited a comment on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles edited a comment on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714174255 > > ``` > > * For the windows wheel it looks like the issue is the VS runtime is pinned

[GitHub] [arrow] wesm commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
wesm commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714177844 We need to add 1.74.0 to https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L770

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714178008 > We need to add 1.74.0 to https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L770 Thanks! I was literally just looking at that

[GitHub] [arrow] terencehonles edited a comment on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles edited a comment on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714174255 > > ``` > > * For the windows wheel it looks like the issue is the VS runtime is pinned

[GitHub] [arrow] cyb70289 edited a comment on pull request #8466: ARROW-10304: [C++][Compute] Optimize variance kernel for integers

2020-10-21 Thread GitBox
cyb70289 edited a comment on pull request #8466: URL: https://github.com/apache/arrow/pull/8466#issuecomment-714173837 > I'm curious why Int64 would be faster than Double. Aren't they using the same algorithm? (and Int64 goes through an additional int-to-float conversion for each value)

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714174255 > > ``` > > * For the windows wheel it looks like the issue is the VS runtime is pinned

[GitHub] [arrow] cyb70289 commented on pull request #8466: ARROW-10304: [C++][Compute] Optimize variance kernel for integers

2020-10-21 Thread GitBox
cyb70289 commented on pull request #8466: URL: https://github.com/apache/arrow/pull/8466#issuecomment-714173837 > I'm curious why Int64 would be faster than Double. Aren't they using the same algorithm? (and Int64 goes through an additional int-to-float conversion for each value)

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714173334 Revision: 6862ba3e18e48a74f307da7c0ed1fddc98364592 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714172270 @github-actions crossbow submit wheel-osx-* This is an automated message from the Apache Git Service. To

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714168120 Revision: 6862ba3e18e48a74f307da7c0ed1fddc98364592 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714167236 @github-actions crossbow submit wheel-osx-* This is an automated message from the Apache Git Service. To

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714166626 @github-actions crossbow submit wheel-win-* This is an automated message from the Apache Git Service. To

[GitHub] [arrow] kou commented on pull request #8492: Allow LLVM 11

2020-10-21 Thread GitBox
kou commented on pull request #8492: URL: https://github.com/apache/arrow/pull/8492#issuecomment-714147660 > will we be able to build gandiva on centos with llvm 8? I think so. This is an automated message from the

[GitHub] [arrow] kou closed pull request #8500: ARROW-10363: [Python] Remove CMake bug workaround in manylinux

2020-10-21 Thread GitBox
kou closed pull request #8500: URL: https://github.com/apache/arrow/pull/8500 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] wesm commented on a change in pull request #8311: ARROW-10008: [C++][Dataset] Fix filtering/row group statistics of dict columns

2020-10-21 Thread GitBox
wesm commented on a change in pull request #8311: URL: https://github.com/apache/arrow/pull/8311#discussion_r509816304 ## File path: cpp/src/arrow/dataset/filter.cc ## @@ -1255,14 +1255,42 @@ struct TreeEvaluator::Impl { } Result operator()(const ComparisonExpression&

[GitHub] [arrow] kou commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
kou commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-714049064 > * For the windows wheel it looks like the issue is the VS runtime is pinned

[GitHub] [arrow] jduo commented on a change in pull request #7994: ARROW-9804: [FlightRPC] Flight auth redesign

2020-10-21 Thread GitBox
jduo commented on a change in pull request #7994: URL: https://github.com/apache/arrow/pull/7994#discussion_r509802432 ## File path: java/flight/flight-core/src/main/java/org/apache/arrow/flight/FlightClient.java ## @@ -156,23 +150,12 @@ } /** - * Authenticates with

[GitHub] [arrow] github-actions[bot] commented on pull request #8502: ARROW-10365: [R] Remove duplicate setting of S3 flag on macOS

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8502: URL: https://github.com/apache/arrow/pull/8502#issuecomment-713904620 https://issues.apache.org/jira/browse/ARROW-10365 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8502: ARROW-10365: [R] Remove duplicate setting of S3 flag on macOS

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8502: URL: https://github.com/apache/arrow/pull/8502#issuecomment-713898187 Revision: b7b5729781fa33240be77342f19ef0b3a2a962a9 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] nealrichardson opened a new pull request #8502: ARROW-10365: [R] Remove duplicate setting of S3 flag on macOS

2020-10-21 Thread GitBox
nealrichardson opened a new pull request #8502: URL: https://github.com/apache/arrow/pull/8502 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] nealrichardson commented on pull request #8502: ARROW-10365: [R] Remove duplicate setting of S3 flag on macOS

2020-10-21 Thread GitBox
nealrichardson commented on pull request #8502: URL: https://github.com/apache/arrow/pull/8502#issuecomment-713897036 @github-actions crossbow submit homebrew-r-autobrew This is an automated message from the Apache Git

[GitHub] [arrow] wesm commented on pull request #8498: ARROW-10207: [C++] Allow precomputing output string/list offsets in kernels

2020-10-21 Thread GitBox
wesm commented on pull request #8498: URL: https://github.com/apache/arrow/pull/8498#issuecomment-713892264 Hm, I am not sure whether the contiguous allocation feature is worth the trouble. I'll review this This is an

[GitHub] [arrow] github-actions[bot] commented on pull request #8501: ARROW-10364: [Dev][Archery] Add support for semver 2.13.0

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8501: URL: https://github.com/apache/arrow/pull/8501#issuecomment-713864020 https://issues.apache.org/jira/browse/ARROW-10364 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8500: ARROW-10363: [Python] Remove CMake bug workaround in manylinux

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8500: URL: https://github.com/apache/arrow/pull/8500#issuecomment-713863530 Revision: 05450f4ed5579efa9be5f0b0e928e3e68f925ff2 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] kou commented on pull request #8500: ARROW-10363: [Python] Remove CMake bug workaround in manylinux

2020-10-21 Thread GitBox
kou commented on pull request #8500: URL: https://github.com/apache/arrow/pull/8500#issuecomment-713862043 @github-actions crossbow submit wheel-manylinux* This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] kou opened a new pull request #8501: ARROW-10364: [Dev][Archery] Add support for semver 2.13.0

2020-10-21 Thread GitBox
kou opened a new pull request #8501: URL: https://github.com/apache/arrow/pull/8501 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] github-actions[bot] commented on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-713838435 Revision: a3076ebe2a33e7a7a3b7b3aec5afd725046bc0cf Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-713838185 @kou not sure if you have any thoughts on the following: - For the windows wheel it looks like the issue is the VS runtime is pinned

[GitHub] [arrow] kou commented on pull request #8497: removing llvm@10 test branch

2020-10-21 Thread GitBox
kou commented on pull request #8497: URL: https://github.com/apache/arrow/pull/8497#issuecomment-713837677 Can we close this because we have #8499 ? This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] kou commented on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
kou commented on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-713837276 @github-actions crossbow submit -g gandiva This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] kou commented on a change in pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
kou commented on a change in pull request #8499: URL: https://github.com/apache/arrow/pull/8499#discussion_r509629578 ## File path: cpp/Brewfile ## @@ -25,8 +25,7 @@ brew "flatbuffers" brew "git" brew "glog" brew "grpc" -brew "llvm" -brew "llvm@8" Review comment: We

[GitHub] [arrow] pitrou commented on a change in pull request #8401: ARROW-10109: [Rust] Add support to the C data interface for primitive types and utf8

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8401: URL: https://github.com/apache/arrow/pull/8401#discussion_r509592855 ## File path: rust/arrow/src/memory.rs ## @@ -135,6 +135,10 @@ const FALLBACK_ALIGNMENT: usize = 1 << 6; /// If you use allocation methods shown here you

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8401: ARROW-10109: [Rust] Add support to the C data interface for primitive types and utf8

2020-10-21 Thread GitBox
jorgecarleitao commented on a change in pull request #8401: URL: https://github.com/apache/arrow/pull/8401#discussion_r509588774 ## File path: rust/arrow/src/memory.rs ## @@ -135,6 +135,10 @@ const FALLBACK_ALIGNMENT: usize = 1 << 6; /// If you use allocation methods shown

[GitHub] [arrow] github-actions[bot] commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-713758654 Revision: fa456de6d03c4fa5bac104b5b31f72d7be92f450 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-713754384 I can't tell what's actually the issue with the OS X Python 3.9 builds. I see https://travis-ci.com/github/ursa-labs/crossbow/builds/191336854 and my guess is that CMake

[GitHub] [arrow] terencehonles commented on pull request #8386: ARROW-10224: [Python] Build, test, and support Python 3.9

2020-10-21 Thread GitBox
terencehonles commented on pull request #8386: URL: https://github.com/apache/arrow/pull/8386#issuecomment-713745132 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow] github-actions[bot] commented on pull request #8500: ARROW-10363: [Python] Remove CMake bug workaround in manylinux

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8500: URL: https://github.com/apache/arrow/pull/8500#issuecomment-713734601 https://issues.apache.org/jira/browse/ARROW-10363 This is an automated message from the Apache Git

[GitHub] [arrow] pitrou closed pull request #8494: ARROW-10328: [C++] Vendor fast_float number parsing library

2020-10-21 Thread GitBox
pitrou closed pull request #8494: URL: https://github.com/apache/arrow/pull/8494 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] pitrou opened a new pull request #8500: ARROW-10363: [Python] Remove CMake bug workaround in manylinux

2020-10-21 Thread GitBox
pitrou opened a new pull request #8500: URL: https://github.com/apache/arrow/pull/8500 The bug was fixed in 3.18.0: https://gitlab.kitware.com/cmake/cmake/-/issues/20425 This is an automated message from the Apache Git

[GitHub] [arrow] naman1996 commented on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
naman1996 commented on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-713697528 @github-actions crossbow submit gandiva* This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] pprudhvi edited a comment on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
pprudhvi edited a comment on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-713690127 @kszucs, @naman1996 tried to build gandiva with this change and it is failing when it tries to install llvm 10 as part of make. (see cpp/CmakeLists.txt) p.s., is

[GitHub] [arrow] emkornfield commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
emkornfield commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509419492 ## File path: cpp/src/arrow/type.cc ## @@ -131,6 +133,7 @@ std::string ToString(Type::type id) { TO_STRING_CASE(FLOAT) TO_STRING_CASE(DOUBLE)

[GitHub] [arrow] pprudhvi commented on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
pprudhvi commented on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-713690127 @kszucs, @naman1996 tried to build gandiva with this change and it is failing when it tries to install llvm 10 as part of make. (see cpp/CmakeLists.txt) p.s., is there a

[GitHub] [arrow] pprudhvi commented on pull request #8492: Allow LLVM 11

2020-10-21 Thread GitBox
pprudhvi commented on pull request #8492: URL: https://github.com/apache/arrow/pull/8492#issuecomment-713686157 @kou looks like this requires code changes, will we be able to build gandiva on centos with llvm 8? This is an

[GitHub] [arrow] pprudhvi commented on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
pprudhvi commented on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-713680213 @ursabot crossbow submit -g gandiva This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] pitrou commented on a change in pull request #8401: ARROW-10109: [Rust] Add support to the C data interface for primitive types and utf8

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8401: URL: https://github.com/apache/arrow/pull/8401#discussion_r509396279 ## File path: rust/arrow/src/memory.rs ## @@ -135,6 +135,10 @@ const FALLBACK_ALIGNMENT: usize = 1 << 6; /// If you use allocation methods shown here you

[GitHub] [arrow] lemire commented on a change in pull request #8494: ARROW-10328: [C++] Vendor fast_float number parsing library

2020-10-21 Thread GitBox
lemire commented on a change in pull request #8494: URL: https://github.com/apache/arrow/pull/8494#discussion_r509392967 ## File path: LICENSE.txt ## @@ -2223,3 +2223,11 @@ exception of some code pulled in from other repositories (such as public domain, released using the

[GitHub] [arrow] pitrou commented on a change in pull request #8401: ARROW-10109: [Rust] Add support to the C data interface for primitive types and utf8

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8401: URL: https://github.com/apache/arrow/pull/8401#discussion_r509392394 ## File path: rust/arrow-c-integration/Cargo.toml ## @@ -0,0 +1,38 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [arrow] pitrou commented on a change in pull request #8494: ARROW-10328: [C++] Vendor fast_float number parsing library

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8494: URL: https://github.com/apache/arrow/pull/8494#discussion_r509391243 ## File path: LICENSE.txt ## @@ -2223,3 +2223,11 @@ exception of some code pulled in from other repositories (such as public domain, released using the

[GitHub] [arrow] lemire commented on a change in pull request #8494: ARROW-10328: [C++] Vendor fast_float number parsing library

2020-10-21 Thread GitBox
lemire commented on a change in pull request #8494: URL: https://github.com/apache/arrow/pull/8494#discussion_r509388406 ## File path: LICENSE.txt ## @@ -2223,3 +2223,11 @@ exception of some code pulled in from other repositories (such as public domain, released using the

[GitHub] [arrow] pitrou commented on pull request #8466: ARROW-10304: [C++][Compute] Optimize variance kernel for integers

2020-10-21 Thread GitBox
pitrou commented on pull request #8466: URL: https://github.com/apache/arrow/pull/8466#issuecomment-71366 Results on an AMD Zen 2 CPU: ``` VarianceKernelInt32/1048576/1 140 us 140 us 5030 bytes_per_second=6.98658G/s null_percent=0.01 size=1048.58k

[GitHub] [arrow] pitrou commented on a change in pull request #8466: ARROW-10304: [C++][Compute] Optimize variance kernel for integers

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8466: URL: https://github.com/apache/arrow/pull/8466#discussion_r509381901 ## File path: cpp/src/arrow/compute/kernels/aggregate_var_std.cc ## @@ -57,6 +63,54 @@ struct VarStdState { this->m2 = m2; } + // int32/16/8:

[GitHub] [arrow] pitrou commented on pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
pitrou commented on pull request #8475: URL: https://github.com/apache/arrow/pull/8475#issuecomment-713654633 I posted two follow-up comments, but generally the C++ changes look good to me. I see that the CSV reader hasn't been updated, feel free to open a JIRA and I can do it later.

[GitHub] [arrow] pitrou commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509370051 ## File path: cpp/src/arrow/util/basic_decimal.cc ## @@ -254,67 +254,125 @@ BasicDecimal128& BasicDecimal128::operator>>=(uint32_t bits) { namespace {

[GitHub] [arrow] pitrou commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509364544 ## File path: cpp/src/arrow/array/validate.cc ## @@ -64,6 +64,13 @@ struct ValidateArrayVisitor { return Status::OK(); } + Status Visit(const

[GitHub] [arrow] pitrou commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509362542 ## File path: cpp/src/arrow/util/basic_decimal.cc ## @@ -775,4 +833,119 @@ int32_t BasicDecimal128::CountLeadingBinaryZeros() const { } } +#if

[GitHub] [arrow] pitrou commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509362379 ## File path: cpp/src/arrow/type.cc ## @@ -131,6 +133,7 @@ std::string ToString(Type::type id) { TO_STRING_CASE(FLOAT) TO_STRING_CASE(DOUBLE)

[GitHub] [arrow] pitrou closed pull request #8489: ARROW-9963: [Python] Recognize datetime.timezone.utc as UTC on conversion python->pyarrow

2020-10-21 Thread GitBox
pitrou closed pull request #8489: URL: https://github.com/apache/arrow/pull/8489 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] github-actions[bot] commented on pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8499: URL: https://github.com/apache/arrow/pull/8499#issuecomment-713633366 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] github-actions[bot] commented on pull request #8498: ARROW-10207: [C++] Allow precomputing output string/list offsets in kernels

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8498: URL: https://github.com/apache/arrow/pull/8498#issuecomment-713632620 https://issues.apache.org/jira/browse/ARROW-10207 This is an automated message from the Apache Git

[GitHub] [arrow] kszucs opened a new pull request #8499: [CI] Pin homebrew LLVM@9

2020-10-21 Thread GitBox
kszucs opened a new pull request #8499: URL: https://github.com/apache/arrow/pull/8499 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] pitrou commented on pull request #8498: ARROW-10207: [C++] Allow precomputing output string/list offsets in kernels

2020-10-21 Thread GitBox
pitrou commented on pull request #8498: URL: https://github.com/apache/arrow/pull/8498#issuecomment-713627106 The main complication here is with the "contiguous preallocation" feature. I'm not sure how much of it is desirable, since if you have a chunked input, you may want it to remain

[GitHub] [arrow] pitrou opened a new pull request #8498: ARROW-10207: [C++] Allow precomputing output string/list offsets in kernels

2020-10-21 Thread GitBox
pitrou opened a new pull request #8498: URL: https://github.com/apache/arrow/pull/8498 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] github-actions[bot] commented on pull request #8496: ARROW-10208: [C++] Fix split string kernels on sliced input

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8496: URL: https://github.com/apache/arrow/pull/8496#issuecomment-713601413 https://issues.apache.org/jira/browse/ARROW-10208 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8497: removing llvm@10 test branch

2020-10-21 Thread GitBox
github-actions[bot] commented on pull request #8497: URL: https://github.com/apache/arrow/pull/8497#issuecomment-713601761 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] naman1996 opened a new pull request #8497: Add llvm@9 during brew install for macOS

2020-10-21 Thread GitBox
naman1996 opened a new pull request #8497: URL: https://github.com/apache/arrow/pull/8497 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] pitrou opened a new pull request #8496: ARROW-10208: [C++] Fix split string kernels on sliced input

2020-10-21 Thread GitBox
pitrou opened a new pull request #8496: URL: https://github.com/apache/arrow/pull/8496 Nulls were propagated incorrectly. We can simply let the kernel machinery do this for us. This is an automated message from the Apache

[GitHub] [arrow] jorisvandenbossche commented on pull request #8489: ARROW-9963: [Python] Recognize datetime.timezone.utc as UTC on conversion python->pyarrow

2020-10-21 Thread GitBox
jorisvandenbossche commented on pull request #8489: URL: https://github.com/apache/arrow/pull/8489#issuecomment-713535131 I updated the expected return value for Python 3.5 to still be the former "+00:00" (instead of adding further special cases to the C++ code to "correct" this for

[GitHub] [arrow] pitrou commented on a change in pull request #8494: ARROW-10328: [C++] Vendor fast_float number parsing library

2020-10-21 Thread GitBox
pitrou commented on a change in pull request #8494: URL: https://github.com/apache/arrow/pull/8494#discussion_r509234886 ## File path: LICENSE.txt ## @@ -2223,3 +2223,11 @@ exception of some code pulled in from other repositories (such as public domain, released using the

[GitHub] [arrow] rdettai edited a comment on pull request #8300: ARROW-10135: [Rust] [Parquet] Refactor file module to help adding sources

2020-10-21 Thread GitBox
rdettai edited a comment on pull request #8300: URL: https://github.com/apache/arrow/pull/8300#issuecomment-713467151 I agree that this PR is hanging, but as this is an API change, I guess its better to think things through before moving forward!  This should maybe have been prepared in

[GitHub] [arrow] rdettai commented on pull request #8300: ARROW-10135: [Rust] [Parquet] Refactor file module to help adding sources

2020-10-21 Thread GitBox
rdettai commented on pull request #8300: URL: https://github.com/apache/arrow/pull/8300#issuecomment-713467151 I agree that this PR is hanging, but as this is an API change, I guess its better to think things through before moving forward!  This should maybe have been thought through in

[GitHub] [arrow] alamb commented on pull request #8300: ARROW-10135: [Rust] [Parquet] Refactor file module to help adding sources

2020-10-21 Thread GitBox
alamb commented on pull request #8300: URL: https://github.com/apache/arrow/pull/8300#issuecomment-713455867 > I'm having second thoughts about the ChunkReader interface. I personally think the `ChunkReader` interface is good enough as is and this PR has been hanging out for quite a

[GitHub] [arrow] rdettai commented on pull request #8300: ARROW-10135: [Rust] [Parquet] Refactor file module to help adding sources

2020-10-21 Thread GitBox
rdettai commented on pull request #8300: URL: https://github.com/apache/arrow/pull/8300#issuecomment-713398263 I'm having second thoughts about the `ChunkReader` interface. It has a length, but what we really want is the capability to "read from end" in order to get the parquet footer. So

[GitHub] [arrow] rdettai commented on a change in pull request #8300: ARROW-10135: [Rust] [Parquet] Refactor file module to help adding sources

2020-10-21 Thread GitBox
rdettai commented on a change in pull request #8300: URL: https://github.com/apache/arrow/pull/8300#discussion_r509056820 ## File path: rust/parquet/src/util/cursor.rs ## @@ -0,0 +1,203 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] rdettai commented on a change in pull request #8300: ARROW-10135: [Rust] [Parquet] Refactor file module to help adding sources

2020-10-21 Thread GitBox
rdettai commented on a change in pull request #8300: URL: https://github.com/apache/arrow/pull/8300#discussion_r509056820 ## File path: rust/parquet/src/util/cursor.rs ## @@ -0,0 +1,203 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] rdettai commented on a change in pull request #8300: ARROW-10135: [Rust] [Parquet] Refactor file module to help adding sources

2020-10-21 Thread GitBox
rdettai commented on a change in pull request #8300: URL: https://github.com/apache/arrow/pull/8300#discussion_r509071560 ## File path: rust/parquet/src/util/cursor.rs ## @@ -0,0 +1,203 @@ +// Licensed to the Apache Software Foundation (ASF) under one Review comment:

[GitHub] [arrow] liyafan82 commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
liyafan82 commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509040668 ## File path: java/vector/src/main/java/org/apache/arrow/vector/util/DecimalUtility.java ## @@ -119,34 +121,47 @@ public static boolean

[GitHub] [arrow] emkornfield commented on pull request #8417: WIP: [C++] Get rid of code duplication in Decimal##bit_width

2020-10-21 Thread GitBox
emkornfield commented on pull request #8417: URL: https://github.com/apache/arrow/pull/8417#issuecomment-713343383 took a little more of a look, this mostly seems reasonable. I've opened PR to merge decimal256 https://github.com/apache/arrow/pull/8475 and we should reopen this against

[GitHub] [arrow] emkornfield commented on a change in pull request #8475: ARROW-9747: [Java][C++] Initial Support for 256-bit Decimals

2020-10-21 Thread GitBox
emkornfield commented on a change in pull request #8475: URL: https://github.com/apache/arrow/pull/8475#discussion_r509021509 ## File path: cpp/src/arrow/array/validate.cc ## @@ -64,6 +64,13 @@ struct ValidateArrayVisitor { return Status::OK(); } + Status

  1   2   >