[GitHub] [arrow] emkornfield commented on a change in pull request #7885: ARROW-9640: [C++][Gandiva] Implement round() for integers and long integers

2020-08-18 Thread GitBox
emkornfield commented on a change in pull request #7885: URL: https://github.com/apache/arrow/pull/7885#discussion_r472734667 ## File path: cpp/src/gandiva/precompiled/extended_math_ops.cc ## @@ -111,6 +112,76 @@ LOG_WITH_BASE(float64, float64, float64) POWER(float64,

[GitHub] [arrow] kou closed pull request #7986: ARROW-9781: [C++] Fix valgrind uninitialized value warnings

2020-08-18 Thread GitBox
kou closed pull request #7986: URL: https://github.com/apache/arrow/pull/7986 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] kou commented on pull request #7986: ARROW-9781: [C++] Fix valgrind uninitialized value warnings

2020-08-18 Thread GitBox
kou commented on pull request #7986: URL: https://github.com/apache/arrow/pull/7986#issuecomment-675866692 Thanks. We can work on the issue as a separated task. (We will disable the test with valgrind or disable AVX512 for valgrind build.) I'll merge this.

[GitHub] [arrow] emkornfield commented on a change in pull request #7885: ARROW-9640: [C++][Gandiva] Implement round() for integers and long integers

2020-08-18 Thread GitBox
emkornfield commented on a change in pull request #7885: URL: https://github.com/apache/arrow/pull/7885#discussion_r472717430 ## File path: cpp/src/gandiva/precompiled/extended_math_ops.cc ## @@ -111,6 +112,76 @@ LOG_WITH_BASE(float64, float64, float64) POWER(float64,

[GitHub] [arrow] emkornfield commented on a change in pull request #7885: ARROW-9640: [C++][Gandiva] Implement round() for integers and long integers

2020-08-18 Thread GitBox
emkornfield commented on a change in pull request #7885: URL: https://github.com/apache/arrow/pull/7885#discussion_r472724359 ## File path: cpp/src/gandiva/precompiled/extended_math_ops.cc ## @@ -111,6 +112,76 @@ LOG_WITH_BASE(float64, float64, float64) POWER(float64,

[GitHub] [arrow] emkornfield commented on a change in pull request #7885: ARROW-9640: [C++][Gandiva] Implement round() for integers and long integers

2020-08-18 Thread GitBox
emkornfield commented on a change in pull request #7885: URL: https://github.com/apache/arrow/pull/7885#discussion_r472717430 ## File path: cpp/src/gandiva/precompiled/extended_math_ops.cc ## @@ -111,6 +112,76 @@ LOG_WITH_BASE(float64, float64, float64) POWER(float64,

[GitHub] [arrow] github-actions[bot] commented on pull request #8001: ARROW-9795: [C++][Gandiva] Implement castTIMESTAMP(int64) in Gandiva

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #8001: URL: https://github.com/apache/arrow/pull/8001#issuecomment-675853337 https://issues.apache.org/jira/browse/ARROW-9795 This is an automated message from the Apache Git

[GitHub] [arrow] sagnikc-dremio opened a new pull request #8001: ARROW-9795: [C++][Gandiva] Implement castTIMESTAMP(int64) in Gandiva

2020-08-18 Thread GitBox
sagnikc-dremio opened a new pull request #8001: URL: https://github.com/apache/arrow/pull/8001 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] emkornfield commented on pull request #7979: ARROW-9766: [C++][Parquet] Add EngineVersion for toggling between new and old logic

2020-08-18 Thread GitBox
emkornfield commented on pull request #7979: URL: https://github.com/apache/arrow/pull/7979#issuecomment-675842493 actually, change might be invasive enough that this probably won't mitigate risk, we can hold off until I get a little further.

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #7993: ARROW-9760: [Rust] [DataFusion] Added DataFrame::explain

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #7993: URL: https://github.com/apache/arrow/pull/7993#discussion_r472638005 ## File path: rust/datafusion/src/dataframe.rs ## @@ -174,4 +174,18 @@ pub trait DataFrame { /// Return the logical plan represented by this

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #7993: ARROW-9760: [Rust] [DataFusion] Added DataFrame::explain

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #7993: URL: https://github.com/apache/arrow/pull/7993#discussion_r472638005 ## File path: rust/datafusion/src/dataframe.rs ## @@ -174,4 +174,18 @@ pub trait DataFrame { /// Return the logical plan represented by this

[GitHub] [arrow] ryannicholson commented on a change in pull request #7994: Flight auth redesign

2020-08-18 Thread GitBox
ryannicholson commented on a change in pull request #7994: URL: https://github.com/apache/arrow/pull/7994#discussion_r472617468 ## File path: java/flight/flight-core/src/main/java/org/apache/arrow/flight/auth/ClientBearerTokenMiddleware.java ## @@ -0,0 +1,116 @@ +/* + *

[GitHub] [arrow] andygrove commented on pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
andygrove commented on pull request #7998: URL: https://github.com/apache/arrow/pull/7998#issuecomment-675827332 > Shouldn't we also align `LogicalPlan::Selection`? Thanks. I missed that. This is an automated message

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #7998: URL: https://github.com/apache/arrow/pull/7998#discussion_r472628286 ## File path: rust/datafusion/src/execution/physical_plan/expressions.rs ## @@ -966,11 +966,32 @@ impl fmt::Display for BinaryExpr { impl

[GitHub] [arrow] github-actions[bot] commented on pull request #8000: ARROW-9793: [Rust] [DataFusion] Fixed unit tests

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #8000: URL: https://github.com/apache/arrow/pull/8000#issuecomment-675823978 https://issues.apache.org/jira/browse/ARROW-9793 This is an automated message from the Apache Git

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #7998: URL: https://github.com/apache/arrow/pull/7998#discussion_r472623554 ## File path: rust/datafusion/src/execution/physical_plan/filter.rs ## @@ -80,31 +88,38 @@ impl ExecutionPlan for SelectionExec { /// Represents

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #7998: URL: https://github.com/apache/arrow/pull/7998#discussion_r472623200 ## File path: rust/datafusion/src/execution/physical_plan/filter.rs ## @@ -15,41 +15,49 @@ // specific language governing permissions and

[GitHub] [arrow] jorgecarleitao commented on pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
jorgecarleitao commented on pull request #7998: URL: https://github.com/apache/arrow/pull/7998#issuecomment-675822143 Shouldn't we also align `LogicalPlan::Selection`? This is an automated message from the Apache Git

[GitHub] [arrow] emkornfield commented on a change in pull request #7883: ARROW-9620: [C++] Added split function to Gandiva

2020-08-18 Thread GitBox
emkornfield commented on a change in pull request #7883: URL: https://github.com/apache/arrow/pull/7883#discussion_r472621421 ## File path: cpp/src/gandiva/precompiled/string_ops.cc ## @@ -567,6 +567,54 @@ const char* castVARCHAR_utf8_int64(gdv_int64 context, const char*

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8000: ARROW-9793: [Rust] [DataFusion] Fixed unit tests

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #8000: URL: https://github.com/apache/arrow/pull/8000#discussion_r472617010 ## File path: rust/datafusion/src/optimizer/filter_push_down.rs ## @@ -370,10 +368,9 @@ mod tests { let plan =

[GitHub] [arrow] jorgecarleitao opened a new pull request #8000: ARROW-9793: [Rust] [DataFusion] Fixed unit tests

2020-08-18 Thread GitBox
jorgecarleitao opened a new pull request #8000: URL: https://github.com/apache/arrow/pull/8000 FYI @andygrove We did not rebased against master and run the tests, which caused some API incompatibilities. This is an

[GitHub] [arrow] emkornfield commented on pull request #7979: ARROW-9766: [C++][Parquet] Add EngineVersion for toggling between new and old logic

2020-08-18 Thread GitBox
emkornfield commented on pull request #7979: URL: https://github.com/apache/arrow/pull/7979#issuecomment-675817406 @pitrou that is the plan, but since there will be a decent amount of new code, I think it is worth-while to provide the option. Once the new engine is implemented that will

[GitHub] [arrow] jianxind commented on pull request #7986: ARROW-9781: [C++] Fix valgrind uninitialized value warnings

2020-08-18 Thread GitBox
jianxind commented on pull request #7986: URL: https://github.com/apache/arrow/pull/7986#issuecomment-675806083 The failed compute case is running with SimdLevel::AVX512(4), probably Valgrind doesn't support AVX512? ``` ==5971== Your program just tried to execute an instruction

[GitHub] [arrow] andygrove closed pull request #7880: ARROW-9619: [Rust] [DataFusion] Add predicate push-down

2020-08-18 Thread GitBox
andygrove closed pull request #7880: URL: https://github.com/apache/arrow/pull/7880 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] andygrove commented on a change in pull request #7993: ARROW-9760: [Rust] [DataFusion] Added DataFrame::explain

2020-08-18 Thread GitBox
andygrove commented on a change in pull request #7993: URL: https://github.com/apache/arrow/pull/7993#discussion_r472583526 ## File path: rust/datafusion/src/dataframe.rs ## @@ -174,4 +174,18 @@ pub trait DataFrame { /// Return the logical plan represented by this

[GitHub] [arrow] github-actions[bot] commented on pull request #7999: ARROW-9792: [Rust] [DataFusion] Aggregate expression functions should not return result

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7999: URL: https://github.com/apache/arrow/pull/7999#issuecomment-675797627 https://issues.apache.org/jira/browse/ARROW-9792 This is an automated message from the Apache Git

[GitHub] [arrow] andygrove opened a new pull request #7999: ARROW-9792: [Rust] [DataFusion] Aggregate expression functions should not return result

2020-08-18 Thread GitBox
andygrove opened a new pull request #7999: URL: https://github.com/apache/arrow/pull/7999 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] github-actions[bot] commented on pull request #7997: ARROW-9266: [Python][Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7997: URL: https://github.com/apache/arrow/pull/7997#issuecomment-675789705 Revision: ad77b8fdacb942cee7556e20b09fc8dcf82c5e1a Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] github-actions[bot] commented on pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7998: URL: https://github.com/apache/arrow/pull/7998#issuecomment-675789488 https://issues.apache.org/jira/browse/ARROW-9788 This is an automated message from the Apache Git

[GitHub] [arrow] kou commented on a change in pull request #7997: ARROW-9266: [Python/Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
kou commented on a change in pull request #7997: URL: https://github.com/apache/arrow/pull/7997#discussion_r472571654 ## File path: dev/tasks/python-wheels/travis.osx.yml ## @@ -21,6 +21,7 @@ language: generic addons: homebrew: packages: + - aws-sdk-cpp Review

[GitHub] [arrow] kou commented on pull request #7997: ARROW-9266: [Python/Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
kou commented on pull request #7997: URL: https://github.com/apache/arrow/pull/7997#issuecomment-675789132 @github-actions crossbow submit wheel-osx-* This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] kou closed pull request #7995: ARROW-9789: [C++] Don't install jemalloc in parallel

2020-08-18 Thread GitBox
kou closed pull request #7995: URL: https://github.com/apache/arrow/pull/7995 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] kou closed pull request #7982: ARROW-9744: [Python] Fix build failure on aarch64

2020-08-18 Thread GitBox
kou closed pull request #7982: URL: https://github.com/apache/arrow/pull/7982 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] kou commented on pull request #7982: ARROW-9744: [Python] Fix build failure on aarch64

2020-08-18 Thread GitBox
kou commented on pull request #7982: URL: https://github.com/apache/arrow/pull/7982#issuecomment-675783743 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [arrow] andygrove commented on pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
andygrove commented on pull request #7998: URL: https://github.com/apache/arrow/pull/7998#issuecomment-675782864 Spark provides some synonyms, such as `where` as a synonym for `filter`, and `orderBy` as a synonym for `sort`. I'm not sure if we want to do the same thing or not?

[GitHub] [arrow] andygrove commented on pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
andygrove commented on pull request #7998: URL: https://github.com/apache/arrow/pull/7998#issuecomment-675782137 @jorgecarleitao @alamb This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] andygrove commented on a change in pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
andygrove commented on a change in pull request #7998: URL: https://github.com/apache/arrow/pull/7998#discussion_r472562916 ## File path: rust/datafusion/src/execution/physical_plan/expressions.rs ## @@ -966,11 +966,32 @@ impl fmt::Display for BinaryExpr { impl PhysicalExpr

[GitHub] [arrow] andygrove opened a new pull request #7998: ARROW-9788: [Rust] [DataFusion] Rename SelectionExec to FilterExec

2020-08-18 Thread GitBox
andygrove opened a new pull request #7998: URL: https://github.com/apache/arrow/pull/7998 The use of the term "selection" rather than "filter" was the main inconsistency that I am seeing in our APIs and was also different to the terminology used in Spark and Pandas. I had used the

[GitHub] [arrow] github-actions[bot] commented on pull request #7997: ARROW-9266: [Python/Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7997: URL: https://github.com/apache/arrow/pull/7997#issuecomment-675756491 https://issues.apache.org/jira/browse/ARROW-9266 This is an automated message from the Apache Git

[GitHub] [arrow] corleyma opened a new pull request #7997: ARROW-9266: [Python/Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
corleyma opened a new pull request #7997: URL: https://github.com/apache/arrow/pull/7997 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] corleyma commented on pull request #7996: ARROW-9266: [Python/Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
corleyma commented on pull request #7996: URL: https://github.com/apache/arrow/pull/7996#issuecomment-675754738 accidentally had the wrong ticket number, which confused your jirabot. trying again. This is an automated

[GitHub] [arrow] corleyma closed pull request #7996: ARROW-9266: [Python/Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
corleyma closed pull request #7996: URL: https://github.com/apache/arrow/pull/7996 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] github-actions[bot] commented on pull request #7996: ARROW-9517: [Python/Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7996: URL: https://github.com/apache/arrow/pull/7996#issuecomment-675747987 https://issues.apache.org/jira/browse/ARROW-9517 This is an automated message from the Apache Git

[GitHub] [arrow] corleyma opened a new pull request #7996: ARROW-9517: [Python/Packaging] enable C++ S3FS in macOS wheels

2020-08-18 Thread GitBox
corleyma opened a new pull request #7996: URL: https://github.com/apache/arrow/pull/7996 This PR updates the `osx-build.sh` script to enable S3 when building the C++ dependencies and the python wheels, and updates the corresponding travis job definition to include the requisite

[GitHub] [arrow] nealrichardson commented on pull request #7875: ARROW-3757: [R] R bindings for Flight RPC client

2020-08-18 Thread GitBox
nealrichardson commented on pull request #7875: URL: https://github.com/apache/arrow/pull/7875#issuecomment-675727617 @lidavidm do you have any opinions about this? This is an automated message from the Apache Git Service.

[GitHub] [arrow] github-actions[bot] commented on pull request #7995: ARROW-9789: [C++] Don't install jemalloc in parallel

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7995: URL: https://github.com/apache/arrow/pull/7995#issuecomment-675726294 https://issues.apache.org/jira/browse/ARROW-9789 This is an automated message from the Apache Git

[GitHub] [arrow] nealrichardson commented on a change in pull request #7928: ARROW-6437: [R] Add AWS SDK to system dependencies for macOS and Windows

2020-08-18 Thread GitBox
nealrichardson commented on a change in pull request #7928: URL: https://github.com/apache/arrow/pull/7928#discussion_r472493966 ## File path: ci/scripts/r_windows_build.sh ## @@ -34,6 +34,12 @@ if [ "$RTOOLS_VERSION" = "35" ]; then # lib-4.9.3 is for libraries compiled

[GitHub] [arrow] nealrichardson edited a comment on pull request #7928: ARROW-6437: [R] Add AWS SDK to system dependencies for macOS and Windows

2020-08-18 Thread GitBox
nealrichardson edited a comment on pull request #7928: URL: https://github.com/apache/arrow/pull/7928#issuecomment-675712973 Googling the error pointed me back to our own JIRA. Looks like #456 fixed this issue previously, but #2779 undid the fix. Unclear why we're seeing this race

[GitHub] [arrow] nealrichardson opened a new pull request #7995: ARROW-9789: [C++] Don't install jemalloc in parallel

2020-08-18 Thread GitBox
nealrichardson opened a new pull request #7995: URL: https://github.com/apache/arrow/pull/7995 On ARROW-6437 (#7928) we saw occasional "File exists" errors on `jemalloc_ep` on macOS. Googling the error message led back to ARROW-739 (#456), which fixed this before by forcing install with

[GitHub] [arrow] nealrichardson commented on pull request #7928: ARROW-6437: [R] Add AWS SDK to system dependencies for macOS and Windows

2020-08-18 Thread GitBox
nealrichardson commented on pull request #7928: URL: https://github.com/apache/arrow/pull/7928#issuecomment-675712973 Googling the error pointed me back to our own JIRA. Looks like #456 fixed this issue previously, but #2779 undid the fix. Unclear why we're seeing this race condition only

[GitHub] [arrow] alamb commented on a change in pull request #7993: ARROW-9760: [Rust] [DataFusion] Added DataFrame::explain

2020-08-18 Thread GitBox
alamb commented on a change in pull request #7993: URL: https://github.com/apache/arrow/pull/7993#discussion_r472482939 ## File path: rust/datafusion/src/dataframe.rs ## @@ -174,4 +174,18 @@ pub trait DataFrame { /// Return the logical plan represented by this

[GitHub] [arrow] andygrove commented on a change in pull request #7993: ARROW-9760: [Rust] [DataFusion] Added DataFrame::explain

2020-08-18 Thread GitBox
andygrove commented on a change in pull request #7993: URL: https://github.com/apache/arrow/pull/7993#discussion_r472425993 ## File path: rust/datafusion/src/dataframe.rs ## @@ -174,4 +174,18 @@ pub trait DataFrame { /// Return the logical plan represented by this

[GitHub] [arrow] andygrove commented on a change in pull request #7993: ARROW-9760: [Rust] [DataFusion] Added DataFrame::explain

2020-08-18 Thread GitBox
andygrove commented on a change in pull request #7993: URL: https://github.com/apache/arrow/pull/7993#discussion_r472425993 ## File path: rust/datafusion/src/dataframe.rs ## @@ -174,4 +174,18 @@ pub trait DataFrame { /// Return the logical plan represented by this

[GitHub] [arrow] wesm commented on pull request #7994: Flight auth redesign

2020-08-18 Thread GitBox
wesm commented on pull request #7994: URL: https://github.com/apache/arrow/pull/7994#issuecomment-675665511 This PR includes protocol changes. These will have to be discussed on the mailing list to be accepted This is an

[GitHub] [arrow] github-actions[bot] commented on pull request #7994: Flight auth redesign

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7994: URL: https://github.com/apache/arrow/pull/7994#issuecomment-675665124 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] github-actions[bot] commented on pull request #7993: ARROW-9760: [Rust] [DataFusion] Added DataFrame::explain

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7993: URL: https://github.com/apache/arrow/pull/7993#issuecomment-675665151 https://issues.apache.org/jira/browse/ARROW-9760 This is an automated message from the Apache Git

[GitHub] [arrow] jduo opened a new pull request #7994: Flight auth redesign

2020-08-18 Thread GitBox
jduo opened a new pull request #7994: URL: https://github.com/apache/arrow/pull/7994 - Remove authentication payloads from the handshake protobuf messages - Add server middleware which utilizes ServerAuthHandler instead of an interceptor - Change FlightClient builder to taken

[GitHub] [arrow] jorgecarleitao opened a new pull request #7993: ARROW-9760: [Rust] [DataFusion] Added DataFrame::explain

2020-08-18 Thread GitBox
jorgecarleitao opened a new pull request #7993: URL: https://github.com/apache/arrow/pull/7993 FYI @andygrove and @alamb I admit I find this API a bit counter-intuitive: coming from spark, I would be expect a string when I call `df.explain()?`. However, I am following the

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #7984: ARROW-9779: [Rust] [DataFusion] Increase stability of average accumulator

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #7984: URL: https://github.com/apache/arrow/pull/7984#discussion_r472386086 ## File path: rust/datafusion/src/execution/physical_plan/expressions.rs ## @@ -337,40 +336,86 @@ impl AggregateExpr for Avg { } }

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #7984: ARROW-9779: [Rust] [DataFusion] Increase stability of average accumulator

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #7984: URL: https://github.com/apache/arrow/pull/7984#discussion_r472386086 ## File path: rust/datafusion/src/execution/physical_plan/expressions.rs ## @@ -337,40 +336,86 @@ impl AggregateExpr for Avg { } }

[GitHub] [arrow] bkietz commented on a change in pull request #7921: ARROW-9658: [Python] Python bindings for dataset writing

2020-08-18 Thread GitBox
bkietz commented on a change in pull request #7921: URL: https://github.com/apache/arrow/pull/7921#discussion_r472368169 ## File path: python/pyarrow/dataset.py ## @@ -682,3 +683,70 @@ def dataset(source, schema=None, format=None, filesystem=None, 'Expected a

[GitHub] [arrow] bkietz commented on a change in pull request #7921: ARROW-9658: [Python] Python bindings for dataset writing

2020-08-18 Thread GitBox
bkietz commented on a change in pull request #7921: URL: https://github.com/apache/arrow/pull/7921#discussion_r472368545 ## File path: python/pyarrow/dataset.py ## @@ -682,3 +683,70 @@ def dataset(source, schema=None, format=None, filesystem=None, 'Expected a

[GitHub] [arrow] pitrou commented on a change in pull request #7992: ARROW-9660: [C++] IPC - dictionaries in maps

2020-08-18 Thread GitBox
pitrou commented on a change in pull request #7992: URL: https://github.com/apache/arrow/pull/7992#discussion_r472355484 ## File path: cpp/src/arrow/ipc/dictionary.h ## @@ -21,34 +21,75 @@ #include #include -#include #include #include -#include

[GitHub] [arrow] pitrou closed pull request #7990: ARROW-9785: [Python] Fix excessively slow S3 options test

2020-08-18 Thread GitBox
pitrou closed pull request #7990: URL: https://github.com/apache/arrow/pull/7990 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] pitrou commented on pull request #7990: ARROW-9785: [Python] Fix excessively slow S3 options test

2020-08-18 Thread GitBox
pitrou commented on pull request #7990: URL: https://github.com/apache/arrow/pull/7990#issuecomment-675604502 +1. This is a trivial change, will merge. This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] nevi-me commented on pull request #7917: ARROW-8423: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-18 Thread GitBox
nevi-me commented on pull request #7917: URL: https://github.com/apache/arrow/pull/7917#issuecomment-675595547 merged in the parquet branch This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] nevi-me closed pull request #7917: ARROW-8423: [Rust] [Parquet] Serialize Arrow schema metadata

2020-08-18 Thread GitBox
nevi-me closed pull request #7917: URL: https://github.com/apache/arrow/pull/7917 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] kiszk commented on pull request #7986: ARROW-9781: [C++] Fix valgrind uninitialized value warnings

2020-08-18 Thread GitBox
kiszk commented on pull request #7986: URL: https://github.com/apache/arrow/pull/7986#issuecomment-675593712 sounds good to me This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [arrow] emkornfield commented on pull request #7973: ARROW-8493: [C++][Parquet] Start populating repeated ancestor defintion

2020-08-18 Thread GitBox
emkornfield commented on pull request #7973: URL: https://github.com/apache/arrow/pull/7973#issuecomment-675585382 > Is it better to create an issue in PARQUET instead of ARROW? It is a fine line. Generally, I use PARQUET for issues related to core parquet reading (i.e. not arrow

[GitHub] [arrow] emkornfield commented on pull request #7030: ARROW-7808: [Java][Dataset] Implement Datasets Java API by JNI to C++

2020-08-18 Thread GitBox
emkornfield commented on pull request #7030: URL: https://github.com/apache/arrow/pull/7030#issuecomment-675576763 @zhztheplayer based on ML discussion I think the consensus is to use Bits.java to limit allocation (ideally with some sort of pre-reservation to limit JNI calls). Do you

[GitHub] [arrow] pitrou commented on pull request #7928: ARROW-6437: [R] Add AWS SDK to system dependencies for macOS and Windows

2020-08-18 Thread GitBox
pitrou commented on pull request #7928: URL: https://github.com/apache/arrow/pull/7928#issuecomment-675573289 No idea. Perhaps you can find someone with CMake and macOS expertise... :-S This is an automated message from the

[GitHub] [arrow] nealrichardson commented on pull request #7928: ARROW-6437: [R] Add AWS SDK to system dependencies for macOS and Windows

2020-08-18 Thread GitBox
nealrichardson commented on pull request #7928: URL: https://github.com/apache/arrow/pull/7928#issuecomment-675565571 All green here but I got the file exists again on one of the autobrew upstream builds: https://travis-ci.org/github/autobrew/homebrew-core/jobs/718998685#L767 I

[GitHub] [arrow] pitrou opened a new pull request #7992: ARROW-9660: [C++] IPC - dictionaries in maps

2020-08-18 Thread GitBox
pitrou opened a new pull request #7992: URL: https://github.com/apache/arrow/pull/7992 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] github-actions[bot] commented on pull request #7992: ARROW-9660: [C++] IPC - dictionaries in maps

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7992: URL: https://github.com/apache/arrow/pull/7992#issuecomment-675559545 https://issues.apache.org/jira/browse/ARROW-9660 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #7990: ARROW-9785: [Python] Fix excessively slow S3 options test

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7990: URL: https://github.com/apache/arrow/pull/7990#issuecomment-675551252 https://issues.apache.org/jira/browse/ARROW-9785 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #7991: ARROW-9718: [Python] ParquetWriter to work with new FileSystem API

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7991: URL: https://github.com/apache/arrow/pull/7991#issuecomment-675551251 https://issues.apache.org/jira/browse/ARROW-9718 This is an automated message from the Apache Git

[GitHub] [arrow] nealrichardson commented on a change in pull request #7819: ARROW-9405: [R] Switch to cpp11

2020-08-18 Thread GitBox
nealrichardson commented on a change in pull request #7819: URL: https://github.com/apache/arrow/pull/7819#discussion_r472278210 ## File path: r/tests/testthat/test-buffer.R ## @@ -39,6 +39,7 @@ test_that("Buffer can be created from numeric vector", { }) test_that("Buffer

[GitHub] [arrow] pitrou opened a new pull request #7990: ARROW-9785: [Python] Fix excessively slow S3 options test

2020-08-18 Thread GitBox
pitrou opened a new pull request #7990: URL: https://github.com/apache/arrow/pull/7990 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] pitrou commented on a change in pull request #7748: ARROW-9388: [C++] Division kernels

2020-08-18 Thread GitBox
pitrou commented on a change in pull request #7748: URL: https://github.com/apache/arrow/pull/7748#discussion_r472277809 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -186,6 +187,42 @@ struct MultiplyChecked { } }; +struct Divide { + template +

[GitHub] [arrow] bkietz commented on a change in pull request #7921: ARROW-9658: [Python] Python bindings for dataset writing

2020-08-18 Thread GitBox
bkietz commented on a change in pull request #7921: URL: https://github.com/apache/arrow/pull/7921#discussion_r472262132 ## File path: python/pyarrow/dataset.py ## @@ -682,3 +683,68 @@ def dataset(source, schema=None, format=None, filesystem=None, 'Expected a

[GitHub] [arrow] bkietz commented on a change in pull request #7921: ARROW-9658: [Python] Python bindings for dataset writing

2020-08-18 Thread GitBox
bkietz commented on a change in pull request #7921: URL: https://github.com/apache/arrow/pull/7921#discussion_r472259523 ## File path: python/pyarrow/_dataset.pyx ## @@ -2022,3 +2022,66 @@ def _get_partition_keys(Expression partition_expression):

[GitHub] [arrow] jorgecarleitao closed pull request #7974: ARROW-9756: [Rust] [DataFusion] Added support for scalar UDFs of arbitrary return types

2020-08-18 Thread GitBox
jorgecarleitao closed pull request #7974: URL: https://github.com/apache/arrow/pull/7974 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] jorgecarleitao edited a comment on pull request #7971: ARROW-9752: [Rust] [DataFusion] Add support for aggregate UDFs

2020-08-18 Thread GitBox
jorgecarleitao edited a comment on pull request #7971: URL: https://github.com/apache/arrow/pull/7971#issuecomment-675520454 I am fine with either option we offer to our users. My concern at the moment is not the public API, but the internal one: after spending some time working on this

[GitHub] [arrow] jorgecarleitao commented on pull request #7971: ARROW-9752: [Rust] [DataFusion] Add support for aggregate UDFs

2020-08-18 Thread GitBox
jorgecarleitao commented on pull request #7971: URL: https://github.com/apache/arrow/pull/7971#issuecomment-675520454 I am fine with either option we offer to our users. My concern at the moment is not the public API, but the internal one: after spending some time working on this code

[GitHub] [arrow] jorgecarleitao closed pull request #7971: ARROW-9752: [Rust] [DataFusion] Add support for aggregate UDFs

2020-08-18 Thread GitBox
jorgecarleitao closed pull request #7971: URL: https://github.com/apache/arrow/pull/7971 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] github-actions[bot] commented on pull request #7928: ARROW-6437: [R] Add AWS SDK to system dependencies for macOS and Windows

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7928: URL: https://github.com/apache/arrow/pull/7928#issuecomment-675513283 Revision: 45705a93b94797dfb96f31bc5d83f08b61549a18 Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] bkietz commented on a change in pull request #7748: ARROW-9388: [C++] Division kernels

2020-08-18 Thread GitBox
bkietz commented on a change in pull request #7748: URL: https://github.com/apache/arrow/pull/7748#discussion_r472240168 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -186,6 +187,42 @@ struct MultiplyChecked { } }; +struct Divide { + template +

[GitHub] [arrow] nealrichardson commented on pull request #7928: ARROW-6437: [R] Add AWS SDK to system dependencies for macOS and Windows

2020-08-18 Thread GitBox
nealrichardson commented on pull request #7928: URL: https://github.com/apache/arrow/pull/7928#issuecomment-675511987 @github-actions crossbow submit homebrew-r-autobrew This is an automated message from the Apache Git

[GitHub] [arrow] andygrove closed pull request #7988: ARROW-9783: [Rust] [DataFusion] Remove aggregate expression data type

2020-08-18 Thread GitBox
andygrove closed pull request #7988: URL: https://github.com/apache/arrow/pull/7988 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #7988: ARROW-9783: [Rust] [DataFusion] Remove aggregate expression data type

2020-08-18 Thread GitBox
jorgecarleitao commented on a change in pull request #7988: URL: https://github.com/apache/arrow/pull/7988#discussion_r472221795 ## File path: rust/datafusion/src/logicalplan.rs ## @@ -363,8 +335,6 @@ pub enum Expr { name: String, /// List of expressions to

[GitHub] [arrow] andygrove commented on a change in pull request #7988: ARROW-9783: [Rust] [DataFusion] Remove aggregate expression data type

2020-08-18 Thread GitBox
andygrove commented on a change in pull request #7988: URL: https://github.com/apache/arrow/pull/7988#discussion_r472217735 ## File path: rust/datafusion/src/logicalplan.rs ## @@ -379,7 +349,48 @@ impl Expr { Expr::Literal(l) => l.get_datatype(),

[GitHub] [arrow] alamb commented on a change in pull request #7988: ARROW-9783: [Rust] [DataFusion] Remove aggregate expression data type

2020-08-18 Thread GitBox
alamb commented on a change in pull request #7988: URL: https://github.com/apache/arrow/pull/7988#discussion_r472213122 ## File path: rust/datafusion/src/logicalplan.rs ## @@ -363,8 +335,6 @@ pub enum Expr { name: String, /// List of expressions to feed to

[GitHub] [arrow] github-actions[bot] commented on pull request #7989: ARROW-9784: Make running TPCH benchmark repeatable

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7989: URL: https://github.com/apache/arrow/pull/7989#issuecomment-675488435 https://issues.apache.org/jira/browse/ARROW-9784 This is an automated message from the Apache Git

[GitHub] [arrow] alamb opened a new pull request #7989: ARROW-9784: Make running TPCH benchmark repeatable

2020-08-18 Thread GitBox
alamb opened a new pull request #7989: URL: https://github.com/apache/arrow/pull/7989 While trying to ru the TPCH benchmark introduced in https://github.com/apache/arrow/pull/7946/files, I found that the referenced `tpch-dbgen` program did not produce files in the way that the benchmark

[GitHub] [arrow] andygrove commented on pull request #7988: ARROW-9783: [Rust] [DataFusion] Remove aggregate expression data type

2020-08-18 Thread GitBox
andygrove commented on pull request #7988: URL: https://github.com/apache/arrow/pull/7988#issuecomment-675480952 @jorgecarleitao @alamb Could I get a review please This is an automated message from the Apache Git Service. To

[GitHub] [arrow] github-actions[bot] commented on pull request #7988: ARROW-9783: [Rust] [DataFusion] Remove aggregate expression data type

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7988: URL: https://github.com/apache/arrow/pull/7988#issuecomment-675480647 https://issues.apache.org/jira/browse/ARROW-9783 This is an automated message from the Apache Git

[GitHub] [arrow] andygrove opened a new pull request #7988: ARROW-9783: [Rust] [DataFusion] Remove aggregate expression data type

2020-08-18 Thread GitBox
andygrove opened a new pull request #7988: URL: https://github.com/apache/arrow/pull/7988 This is a step towards cleaning up the handling of aggregate expressions and acheives the following: - It is now possible to construct a logical aggregate expression without specifying a

[GitHub] [arrow] github-actions[bot] commented on pull request #7987: ARROW-9768: [Python] Check overflow in conversion of datetime objects to nanosecond timestamps

2020-08-18 Thread GitBox
github-actions[bot] commented on pull request #7987: URL: https://github.com/apache/arrow/pull/7987#issuecomment-675473829 https://issues.apache.org/jira/browse/ARROW-9768 This is an automated message from the Apache Git

[GitHub] [arrow] jorisvandenbossche opened a new pull request #7987: ARROW-9768: [Python] Check overflow in conversion of datetime objects to nanosecond timestamps

2020-08-18 Thread GitBox
jorisvandenbossche opened a new pull request #7987: URL: https://github.com/apache/arrow/pull/7987 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] alamb commented on a change in pull request #7984: ARROW-9779: [Rust] [DataFusion] Increase stability of average accumulator

2020-08-18 Thread GitBox
alamb commented on a change in pull request #7984: URL: https://github.com/apache/arrow/pull/7984#discussion_r472141713 ## File path: rust/datafusion/src/execution/physical_plan/expressions.rs ## @@ -337,40 +336,86 @@ impl AggregateExpr for Avg { } } -macro_rules!

  1   2   >