[GitHub] [arrow] kiszk commented on pull request #8674: ARROW-10610: [C++] Support exponential float nontion on big-endian in fast_float

2020-11-18 Thread GitBox
kiszk commented on pull request #8674: URL: https://github.com/apache/arrow/pull/8674#issuecomment-729580682 @pitrou Sounds good, I will close this PR. This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] Ulimo commented on a change in pull request #8694: ARROW-10542: [C#][Flight] Add beginning on flight code for net core

2020-11-18 Thread GitBox
Ulimo commented on a change in pull request #8694: URL: https://github.com/apache/arrow/pull/8694#discussion_r526016969 ## File path: csharp/src/Apache.Arrow.Flight/Reader/RecordBatcReaderImplementation.cs ## @@ -0,0 +1,110 @@ +// Licensed to the Apache Software Foundation

[GitHub] [arrow] github-actions[bot] commented on pull request #8702: ARROW-10634: [C#][CI] Change the build version from 2.2 to 3.1 in CI

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8702: URL: https://github.com/apache/arrow/pull/8702#issuecomment-729562831 https://issues.apache.org/jira/browse/ARROW-10634 This is an automated message from the Apache Git

[GitHub] [arrow] jorisvandenbossche commented on pull request #8677: ARROW-10617: [Python] Fix RecordBatchStreamReader iteration with Python 3.8

2020-11-18 Thread GitBox
jorisvandenbossche commented on pull request #8677: URL: https://github.com/apache/arrow/pull/8677#issuecomment-729632072 Indeed, it seems the full `test_ipc.py` file is being skipped on the "Python 3.8 Without Pandas" build

[GitHub] [arrow] pitrou opened a new pull request #8703: ARROW-10143: [C++] Rewrite Array(Range)Equals

2020-11-18 Thread GitBox
pitrou opened a new pull request #8703: URL: https://github.com/apache/arrow/pull/8703 ArrayEquals now defers to ArrayRangeEquals under the hood. ArrayRangeEquals now allows passing an EqualOptions argument. Also add ArrayRangeApproxEquals. Comparison speed is massively

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r525984128 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] Ulimo opened a new pull request #8702: [ARROW-10634][C#][CI] Change the build version from 2.2 to 3.1 in CI

2020-11-18 Thread GitBox
Ulimo opened a new pull request #8702: URL: https://github.com/apache/arrow/pull/8702 Upgraded build version to 3.1 and C# version to 8. Also added EmbedUntrackedSources so auto generated code will work with sourcelink.

[GitHub] [arrow] pitrou commented on pull request #8674: ARROW-10610: [C++] Support exponential float nontion on big-endian in fast_float

2020-11-18 Thread GitBox
pitrou commented on pull request #8674: URL: https://github.com/apache/arrow/pull/8674#issuecomment-729570863 @kiszk I will, unless you beat me to it. This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] github-actions[bot] commented on pull request #8703: ARROW-10143: [C++] Rewrite Array(Range)Equals

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8703: URL: https://github.com/apache/arrow/pull/8703#issuecomment-729584943 https://issues.apache.org/jira/browse/ARROW-10143 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8702: [ARROW-10634][C#][CI] Change the build version from 2.2 to 3.1 in CI

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8702: URL: https://github.com/apache/arrow/pull/8702#issuecomment-729554537 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r525984998 ## File path: rust/arrow/src/compute/kernels/comparison.rs ## @@ -570,27 +570,26 @@ where )); } -let num_bytes =

[GitHub] [arrow] Ulimo commented on pull request #8702: ARROW-10634: [C#][CI] Change the build version from 2.2 to 3.1 in CI

2020-11-18 Thread GitBox
Ulimo commented on pull request #8702: URL: https://github.com/apache/arrow/pull/8702#issuecomment-729559868 I did not change the version under examples, should FluentBuilderExample.csproj also be updated to 3.1? This is an

[GitHub] [arrow] vertexclique commented on a change in pull request #8701: ARROW-10639: [Rust] Added examples to is_null kernel and simplified signature.

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8701: URL: https://github.com/apache/arrow/pull/8701#discussion_r526013126 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -111,7 +111,22 @@ pub fn not(left: ) -> Result {

[GitHub] [arrow] vertexclique commented on a change in pull request #8701: ARROW-10639: [Rust] Added examples to is_null kernel and simplified signature.

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8701: URL: https://github.com/apache/arrow/pull/8701#discussion_r526013126 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -111,7 +111,22 @@ pub fn not(left: ) -> Result {

[GitHub] [arrow] sighingnow commented on pull request #8677: ARROW-10617: [Python] Fix RecordBatchStreamReader iteration with Python 3.8

2020-11-18 Thread GitBox
sighingnow commented on pull request #8677: URL: https://github.com/apache/arrow/pull/8677#issuecomment-729642276 Hi @jorgecarleitao Then could we get this one merged first ? Or should I enable pandas with python 3.8 tests on github workflow?

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r525980049 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] jorisvandenbossche commented on pull request #8677: ARROW-10617: [Python] Fix RecordBatchStreamReader iteration with Python 3.8

2020-11-18 Thread GitBox
jorisvandenbossche commented on pull request #8677: URL: https://github.com/apache/arrow/pull/8677#issuecomment-729633901 But don't we have a ursabot build with python 3.8? (they don't seem active / be triggered on this PR, though)

[GitHub] [arrow] velvia commented on a change in pull request #8688: ARROW-10330: [Rust][DataFusion] Implement NULLIF() SQL function

2020-11-18 Thread GitBox
velvia commented on a change in pull request #8688: URL: https://github.com/apache/arrow/pull/8688#discussion_r526346099 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -457,4 +517,42 @@ mod tests { assert_eq!(true, res.value(2));

[GitHub] [arrow] sunchao commented on pull request #8708: ARROW-10647: [Rust] [Parquet] Port benchmarks from from parquet-rs to arrow repo

2020-11-18 Thread GitBox
sunchao commented on pull request #8708: URL: https://github.com/apache/arrow/pull/8708#issuecomment-729911859 Unrelated: there is also the [fuzz module](https://github.com/sunchao/parquet-rs/tree/master/fuzz) which is quite useful for detecting bad crashes in the code. It probably worth

[GitHub] [arrow] yordan-pavlov commented on a change in pull request #8630: ARROW-10540 [Rust] Improve filtering

2020-11-18 Thread GitBox
yordan-pavlov commented on a change in pull request #8630: URL: https://github.com/apache/arrow/pull/8630#discussion_r526371625 ## File path: rust/arrow/src/buffer.rs ## @@ -888,6 +888,15 @@ impl MutableBuffer { } self.len += bytes.len(); } + +///

[GitHub] [arrow] alamb commented on pull request #8698: ARROW-10636: [Rust][Parquet] Remove rust specialization

2020-11-18 Thread GitBox
alamb commented on pull request #8698: URL: https://github.com/apache/arrow/pull/8698#issuecomment-729809822 Here is a potential contribution to this effort: https://github.com/apache/arrow/pull/8708 (a PR with the bench marks ported -- fyi @GregBowyer ). To run: ``` cd

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8630: ARROW-10540 [Rust] Improve filtering

2020-11-18 Thread GitBox
jorgecarleitao commented on a change in pull request #8630: URL: https://github.com/apache/arrow/pull/8630#discussion_r526252139 ## File path: rust/arrow/src/buffer.rs ## @@ -888,6 +888,15 @@ impl MutableBuffer { } self.len += bytes.len(); } + +///

[GitHub] [arrow] jhorstmann commented on pull request #8686: ARROW-10297: [Rust] Optional json output in parquet-read binary

2020-11-18 Thread GitBox
jhorstmann commented on pull request #8686: URL: https://github.com/apache/arrow/pull/8686#issuecomment-729823739 @nevi-me @vertexclique thanks for the reviews. I put the json dependency now behind a `json_output` feature gate and fixed the clippy warning.

[GitHub] [arrow] andygrove commented on pull request #8705: ARROW-10464: [Rust] [DataFusion] Add utility to convert TPC-H data from tbl to CSV and Parquet

2020-11-18 Thread GitBox
andygrove commented on pull request #8705: URL: https://github.com/apache/arrow/pull/8705#issuecomment-729838339 Build error seems unrelated to these changes. This is an automated message from the Apache Git Service. To

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526358664 ## File path: rust/arrow/src/buffer.rs ## @@ -258,39 +259,52 @@ impl Buffer { /// Returns a slice of this buffer starting at a certain bit

[GitHub] [arrow] Ulimo commented on a change in pull request #8694: ARROW-10542: [C#][Flight] Add beginning on flight code for net core

2020-11-18 Thread GitBox
Ulimo commented on a change in pull request #8694: URL: https://github.com/apache/arrow/pull/8694#discussion_r525941887 ## File path: .github/workflows/csharp.yml ## @@ -38,8 +38,12 @@ jobs: strategy: fail-fast: false matrix: -dotnet: [2.2.103] +

[GitHub] [arrow] lidavidm opened a new pull request #8707: ARROW-10646: [C++][FlightRPC] Disable flaky Flight test on Windows

2020-11-18 Thread GitBox
lidavidm opened a new pull request #8707: URL: https://github.com/apache/arrow/pull/8707 On Windows, gRPC sometimes gives us a blank peer address. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] andygrove commented on pull request #8705: ARROW-10464: [Rust] [DataFusion] Add utility to convert TPC-H data from tbl to CSV and Parquet

2020-11-18 Thread GitBox
andygrove commented on pull request #8705: URL: https://github.com/apache/arrow/pull/8705#issuecomment-729825709 @jorgecarleitao @alamb This adds the Rust-native utility to convert the TPC-H data to CSV and Parquet. I will create follow-on PRs to add support for re-partitioning the

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526309397 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] velvia commented on a change in pull request #8688: ARROW-10330: [Rust][DataFusion] Implement NULLIF() SQL function

2020-11-18 Thread GitBox
velvia commented on a change in pull request #8688: URL: https://github.com/apache/arrow/pull/8688#discussion_r526344874 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -457,4 +517,42 @@ mod tests { assert_eq!(true, res.value(2));

[GitHub] [arrow] kiszk closed pull request #8674: ARROW-10610: [C++] Support exponential float nontion on big-endian in fast_float

2020-11-18 Thread GitBox
kiszk closed pull request #8674: URL: https://github.com/apache/arrow/pull/8674 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] github-actions[bot] commented on pull request #8707: ARROW-10646: [C++][FlightRPC] Disable flaky Flight test on Windows

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8707: URL: https://github.com/apache/arrow/pull/8707#issuecomment-729801384 https://issues.apache.org/jira/browse/ARROW-10646 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8706: ARROW-10032: [Documentation] update C++ windows docs

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8706: URL: https://github.com/apache/arrow/pull/8706#issuecomment-729801385 https://issues.apache.org/jira/browse/ARROW-10032 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8705: ARROW-10464: [Rust] [DataFusion] Add utility to convert TPC-H data from tbl to CSV and Parquet [WIP]

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8705: URL: https://github.com/apache/arrow/pull/8705#issuecomment-729801386 https://issues.apache.org/jira/browse/ARROW-10464 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8704: URL: https://github.com/apache/arrow/pull/8704#issuecomment-729801389 https://issues.apache.org/jira/browse/ARROW-10644 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #8708: ARROW-10647: [Rust] [Parquet] Port benchmarks from from parquet-rs to arrow repo

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8708: URL: https://github.com/apache/arrow/pull/8708#issuecomment-729811338 https://issues.apache.org/jira/browse/ARROW-10647 This is an automated message from the Apache Git

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8630: ARROW-10540 [Rust] Improve filtering

2020-11-18 Thread GitBox
jorgecarleitao commented on a change in pull request #8630: URL: https://github.com/apache/arrow/pull/8630#discussion_r526252139 ## File path: rust/arrow/src/buffer.rs ## @@ -888,6 +888,15 @@ impl MutableBuffer { } self.len += bytes.len(); } + +///

[GitHub] [arrow] alamb commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526371716 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] alamb commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526371341 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] yordan-pavlov commented on a change in pull request #8630: ARROW-10540 [Rust] Improve filtering

2020-11-18 Thread GitBox
yordan-pavlov commented on a change in pull request #8630: URL: https://github.com/apache/arrow/pull/8630#discussion_r526371625 ## File path: rust/arrow/src/buffer.rs ## @@ -888,6 +888,15 @@ impl MutableBuffer { } self.len += bytes.len(); } + +///

[GitHub] [arrow] nevi-me commented on pull request #8708: ARROW-10647: [Rust] [Parquet] Port benchmarks from from parquet-rs to arrow repo

2020-11-18 Thread GitBox
nevi-me commented on pull request #8708: URL: https://github.com/apache/arrow/pull/8708#issuecomment-729813794 This will also close https://issues.apache.org/jira/browse/ARROW-4063 This is an automated message from the

[GitHub] [arrow] sunchao commented on a change in pull request #8708: ARROW-10647: [Rust] [Parquet] Port benchmarks from from parquet-rs to arrow repo

2020-11-18 Thread GitBox
sunchao commented on a change in pull request #8708: URL: https://github.com/apache/arrow/pull/8708#discussion_r526370843 ## File path: rust/parquet/benches/codec.rs ## @@ -0,0 +1,196 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] lidavidm opened a new pull request #8706: ARROW-10032: [Documentation] update C++ windows docs

2020-11-18 Thread GitBox
lidavidm opened a new pull request #8706: URL: https://github.com/apache/arrow/pull/8706 Update the docs on replicating the AppVeyor build. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] aibaars commented on pull request #7926: ARROW-9628: [Rust] Disable artifact caching for Mac OSX builds

2020-11-18 Thread GitBox
aibaars commented on pull request #7926: URL: https://github.com/apache/arrow/pull/7926#issuecomment-729804800 @alamb @kszucs I ran into the same error on one of our repositories and were pointed to https://github.com/actions/cache/issues/403#issuecomment-678348989 by a colleague .

[GitHub] [arrow] alamb opened a new pull request #8708: ARROW-10647: [Rust] [Parquet] Port benchmarks from from parquet-rs to arrow repo

2020-11-18 Thread GitBox
alamb opened a new pull request #8708: URL: https://github.com/apache/arrow/pull/8708 This PR ports the parquet benchmarks from the original [parquet-rs](https://github.com/sunchao/parquet-rs/tree/master/benches) repo in service of helping to get https://github.com/apache/arrow/pull/8698

[GitHub] [arrow] alamb commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526364599 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] jorisvandenbossche commented on pull request #8677: ARROW-10617: [Python] Fix RecordBatchStreamReader iteration with Python 3.8

2020-11-18 Thread GitBox
jorisvandenbossche commented on pull request #8677: URL: https://github.com/apache/arrow/pull/8677#issuecomment-729705678 > Then could we get this one merged first ? Ideally we would first ensure we have such a build, so we can know it is failing before applying this patch (so we

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526137607 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] Fonsan commented on a change in pull request #8661: ARROW-10581: [website] IPC dictionary reference to relevant section

2020-11-18 Thread GitBox
Fonsan commented on a change in pull request #8661: URL: https://github.com/apache/arrow/pull/8661#discussion_r526204115 ## File path: docs/source/format/Columnar.rst ## @@ -1053,7 +1053,7 @@ the dictionaries can be properly interpreted: :: The dictionary ``id`` in the

[GitHub] [arrow] andygrove opened a new pull request #8705: ARROW-10464: Add utility to convert TPC-H data from tbl to CSV and Parquet [WIP]

2020-11-18 Thread GitBox
andygrove opened a new pull request #8705: URL: https://github.com/apache/arrow/pull/8705 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] Ulimo commented on a change in pull request #8694: ARROW-10542: [C#][Flight] Add beginning on flight code for net core

2020-11-18 Thread GitBox
Ulimo commented on a change in pull request #8694: URL: https://github.com/apache/arrow/pull/8694#discussion_r526102674 ## File path: csharp/test/Apache.Arrow.Flight.TestWeb/Apache.Arrow.Flight.TestWeb.csproj ## @@ -0,0 +1,16 @@ + + + +netcoreapp3.0 Review comment:

[GitHub] [arrow] jorisvandenbossche opened a new pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs

2020-11-18 Thread GitBox
jorisvandenbossche opened a new pull request #8704: URL: https://github.com/apache/arrow/pull/8704 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [arrow] jorisvandenbossche commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs

2020-11-18 Thread GitBox
jorisvandenbossche commented on pull request #8704: URL: https://github.com/apache/arrow/pull/8704#issuecomment-729793788 - Slight downside is that we now do a `get_file_info` twice, but only in the case of a local file path with no filesystem specified (so I think for this case this is

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526143753 ## File path: rust/arrow/src/util/utils.rs ## @@ -0,0 +1,119 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526122655 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526161079 ## File path: rust/arrow/src/compute/kernels/aggregate.rs ## @@ -219,24 +223,27 @@ where let data_chunks = data.chunks_exact(64);

[GitHub] [arrow] nevi-me closed pull request #8700: ARROW-10638: [Rust] Improved tests of boolean kernel.

2020-11-18 Thread GitBox
nevi-me closed pull request #8700: URL: https://github.com/apache/arrow/pull/8700 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] Ulimo commented on a change in pull request #8694: ARROW-10542: [C#][Flight] Add beginning on flight code for net core

2020-11-18 Thread GitBox
Ulimo commented on a change in pull request #8694: URL: https://github.com/apache/arrow/pull/8694#discussion_r526101991 ## File path: .github/workflows/csharp.yml ## @@ -38,8 +38,12 @@ jobs: strategy: fail-fast: false matrix: -dotnet: [2.2.103] +

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526122170 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526122010 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526121662 ## File path: rust/arrow/src/util/bit_ops.rs ## @@ -0,0 +1,407 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #8661: ARROW-10581: [website] IPC dictionary reference to relevant section

2020-11-18 Thread GitBox
jorisvandenbossche commented on a change in pull request #8661: URL: https://github.com/apache/arrow/pull/8661#discussion_r526136508 ## File path: docs/source/format/Columnar.rst ## @@ -1053,7 +1053,7 @@ the dictionaries can be properly interpreted: :: The dictionary ``id``

[GitHub] [arrow] vertexclique commented on a change in pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8664: URL: https://github.com/apache/arrow/pull/8664#discussion_r526145990 ## File path: rust/arrow/src/array/array_list.rs ## @@ -711,8 +722,9 @@ mod tests { assert_eq!(1, sliced_array.offset());

[GitHub] [arrow] nevi-me closed pull request #8695: ARROW-10631: [Rust] Fixed error in computing equality of fixed-sized binary.

2020-11-18 Thread GitBox
nevi-me closed pull request #8695: URL: https://github.com/apache/arrow/pull/8695 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] alamb commented on a change in pull request #8698: ARROW-10636: [Rust][Parquet] Remove rust specialization

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8698: URL: https://github.com/apache/arrow/pull/8698#discussion_r526207035 ## File path: rust/parquet/src/column/writer.rs ## @@ -955,81 +963,31 @@ impl ColumnWriterImpl { /// Trait to define default encoding for types, including

[GitHub] [arrow] Ulimo commented on a change in pull request #8694: ARROW-10542: [C#][Flight] Add beginning on flight code for net core

2020-11-18 Thread GitBox
Ulimo commented on a change in pull request #8694: URL: https://github.com/apache/arrow/pull/8694#discussion_r526101764 ## File path: csharp/src/Apache.Arrow.Flight/Apache.Arrow.Flight.csproj ## @@ -0,0 +1,28 @@ + + + +netstandard2.1 +8.0 +true Review comment:

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8630: ARROW-10540 [Rust] Improve filtering

2020-11-18 Thread GitBox
jorgecarleitao commented on a change in pull request #8630: URL: https://github.com/apache/arrow/pull/8630#discussion_r526401885 ## File path: rust/arrow/src/buffer.rs ## @@ -888,6 +888,15 @@ impl MutableBuffer { } self.len += bytes.len(); } + +///

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8630: ARROW-10540 [Rust] Improve filtering

2020-11-18 Thread GitBox
jorgecarleitao commented on a change in pull request #8630: URL: https://github.com/apache/arrow/pull/8630#discussion_r526401885 ## File path: rust/arrow/src/buffer.rs ## @@ -888,6 +888,15 @@ impl MutableBuffer { } self.len += bytes.len(); } + +///

[GitHub] [arrow] velvia commented on a change in pull request #8688: ARROW-10330: [Rust][DataFusion] Implement NULLIF() SQL function

2020-11-18 Thread GitBox
velvia commented on a change in pull request #8688: URL: https://github.com/apache/arrow/pull/8688#discussion_r526462964 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -457,4 +517,42 @@ mod tests { assert_eq!(true, res.value(2));

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8688: ARROW-10330: [Rust][DataFusion] Implement NULLIF() SQL function

2020-11-18 Thread GitBox
jorgecarleitao commented on a change in pull request #8688: URL: https://github.com/apache/arrow/pull/8688#discussion_r526473704 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -457,4 +517,42 @@ mod tests { assert_eq!(true, res.value(2));

[GitHub] [arrow] alamb commented on a change in pull request #8705: ARROW-10464: [Rust] [DataFusion] Add utility to convert TPC-H data from tbl to CSV and Parquet

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8705: URL: https://github.com/apache/arrow/pull/8705#discussion_r526473525 ## File path: rust/datafusion/src/execution/context.rs ## @@ -1187,6 +1219,47 @@ mod tests { Ok(()) } +#[tokio::test] +async fn

[GitHub] [arrow] alamb closed pull request #8697: ARROW-10633: [Rust][DataFusion] Dependency version updates

2020-11-18 Thread GitBox
alamb closed pull request #8697: URL: https://github.com/apache/arrow/pull/8697 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] alamb commented on pull request #8697: ARROW-10633: [Rust][DataFusion] Dependency version updates

2020-11-18 Thread GitBox
alamb commented on pull request #8697: URL: https://github.com/apache/arrow/pull/8697#issuecomment-730026146 Thank you @Dandandan ! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [arrow] velvia commented on a change in pull request #8688: ARROW-10330: [Rust][DataFusion] Implement NULLIF() SQL function

2020-11-18 Thread GitBox
velvia commented on a change in pull request #8688: URL: https://github.com/apache/arrow/pull/8688#discussion_r526498177 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -457,4 +517,42 @@ mod tests { assert_eq!(true, res.value(2));

[GitHub] [arrow] Dandandan commented on a change in pull request #8710: ARROW-10649: [Rust] Parse manually in infer_field_schema, remove lazy static dependency

2020-11-18 Thread GitBox
Dandandan commented on a change in pull request #8710: URL: https://github.com/apache/arrow/pull/8710#discussion_r526508067 ## File path: rust/arrow/src/csv/reader.rs ## @@ -77,15 +70,20 @@ fn infer_field_schema(string: ) -> DataType { return DataType::Utf8; }

[GitHub] [arrow] Dandandan commented on a change in pull request #8710: ARROW-10649: [Rust] Parse manually in infer_field_schema, remove lazy static dependency

2020-11-18 Thread GitBox
Dandandan commented on a change in pull request #8710: URL: https://github.com/apache/arrow/pull/8710#discussion_r526512058 ## File path: rust/arrow/src/csv/reader.rs ## @@ -77,15 +70,20 @@ fn infer_field_schema(string: ) -> DataType { return DataType::Utf8; }

[GitHub] [arrow] alamb commented on a change in pull request #8708: ARROW-10647: [Rust] [Parquet] Port benchmarks from from parquet-rs to arrow repo

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8708: URL: https://github.com/apache/arrow/pull/8708#discussion_r526411337 ## File path: rust/parquet/benches/codec.rs ## @@ -0,0 +1,196 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] alamb commented on pull request #8708: ARROW-10647: [Rust] [Parquet] Port benchmarks from from parquet-rs to arrow repo

2020-11-18 Thread GitBox
alamb commented on pull request #8708: URL: https://github.com/apache/arrow/pull/8708#issuecomment-729945589 @GregBowyer -- sounds great! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] alamb commented on a change in pull request #8682: ARROW-10620: [Rust][Parquet] move column chunk range logic to metadata.rs

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8682: URL: https://github.com/apache/arrow/pull/8682#discussion_r526488579 ## File path: rust/parquet/src/file/metadata.rs ## @@ -433,6 +433,21 @@ impl ColumnChunkMetaData { self.dictionary_page_offset } +///

[GitHub] [arrow] alamb commented on a change in pull request #8710: ARROW-10649: [Rust] Parse manually in infer_field_schema, remove lazy static dependency

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8710: URL: https://github.com/apache/arrow/pull/8710#discussion_r526502537 ## File path: rust/arrow/src/csv/reader.rs ## @@ -77,15 +70,20 @@ fn infer_field_schema(string: ) -> DataType { return DataType::Utf8; }

[GitHub] [arrow] vertexclique commented on a change in pull request #8688: ARROW-10330: [Rust][DataFusion] Implement NULLIF() SQL function

2020-11-18 Thread GitBox
vertexclique commented on a change in pull request #8688: URL: https://github.com/apache/arrow/pull/8688#discussion_r526511029 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -457,4 +517,42 @@ mod tests { assert_eq!(true, res.value(2));

[GitHub] [arrow] jhorstmann commented on a change in pull request #8685: ARROW-10216: [Rust] Simd implementation for primitive min/max kernels

2020-11-18 Thread GitBox
jhorstmann commented on a change in pull request #8685: URL: https://github.com/apache/arrow/pull/8685#discussion_r526388987 ## File path: rust/arrow/src/datatypes.rs ## @@ -618,7 +628,7 @@ macro_rules! make_numeric_type { fn mask_from_u64(mask: u64) ->

[GitHub] [arrow] alamb edited a comment on pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
alamb edited a comment on pull request #8664: URL: https://github.com/apache/arrow/pull/8664#issuecomment-729965696 I ran the filtering selectivity benchmark on my laptop again and this time it shows a ~400% slowdown... So I wonder if that unsafe code is actually giving us a performance

[GitHub] [arrow] alamb commented on a change in pull request #8705: ARROW-10464: [Rust] [DataFusion] Add utility to convert TPC-H data from tbl to CSV and Parquet

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8705: URL: https://github.com/apache/arrow/pull/8705#discussion_r526441242 ## File path: rust/benchmarks/README.md ## @@ -49,45 +49,16 @@ data. This value can be increased to generate larger data sets. The benchmark can then be

[GitHub] [arrow] alamb commented on a change in pull request #8705: ARROW-10464: [Rust] [DataFusion] Add utility to convert TPC-H data from tbl to CSV and Parquet

2020-11-18 Thread GitBox
alamb commented on a change in pull request #8705: URL: https://github.com/apache/arrow/pull/8705#discussion_r526432425 ## File path: rust/benchmarks/README.md ## @@ -49,45 +49,16 @@ data. This value can be increased to generate larger data sets. The benchmark can then be

[GitHub] [arrow] andygrove commented on a change in pull request #8705: ARROW-10464: [Rust] [DataFusion] Add utility to convert TPC-H data from tbl to CSV and Parquet

2020-11-18 Thread GitBox
andygrove commented on a change in pull request #8705: URL: https://github.com/apache/arrow/pull/8705#discussion_r526460805 ## File path: rust/benchmarks/README.md ## @@ -49,45 +49,16 @@ data. This value can be increased to generate larger data sets. The benchmark can then

[GitHub] [arrow] jorgecarleitao opened a new pull request #8709: ARROW-9555: [Rust] [DataFusion] Implement physical node for inner join

2020-11-18 Thread GitBox
jorgecarleitao opened a new pull request #8709: URL: https://github.com/apache/arrow/pull/8709 This PR is based on top of #8630 and contains a physical node to perform an inner join in DataFusion. This is still a draft, but IMO the design is here and the two tests already pass.

[GitHub] [arrow] andygrove commented on a change in pull request #8709: ARROW-9555: [Rust] [DataFusion] Implement physical node for inner join

2020-11-18 Thread GitBox
andygrove commented on a change in pull request #8709: URL: https://github.com/apache/arrow/pull/8709#discussion_r526467098 ## File path: rust/datafusion/src/physical_plan/hash_join.rs ## @@ -0,0 +1,467 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8709: ARROW-9555: [Rust] [DataFusion] Implement physical node for inner join

2020-11-18 Thread GitBox
jorgecarleitao commented on a change in pull request #8709: URL: https://github.com/apache/arrow/pull/8709#discussion_r526476310 ## File path: rust/datafusion/src/physical_plan/hash_join.rs ## @@ -0,0 +1,467 @@ +// Licensed to the Apache Software Foundation (ASF) under one +//

[GitHub] [arrow] alamb closed pull request #8705: ARROW-10464: [Rust] [DataFusion] Add utility to convert TPC-H data from tbl to CSV and Parquet

2020-11-18 Thread GitBox
alamb closed pull request #8705: URL: https://github.com/apache/arrow/pull/8705 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] alamb closed pull request #8699: ARROW-10637: [Rust] Added examples to some boolean kernels.

2020-11-18 Thread GitBox
alamb closed pull request #8699: URL: https://github.com/apache/arrow/pull/8699 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] alamb edited a comment on pull request #8701: ARROW-10639: [Rust] Added examples to is_null kernel and simplified signature.

2020-11-18 Thread GitBox
alamb edited a comment on pull request #8701: URL: https://github.com/apache/arrow/pull/8701#issuecomment-730029379 CI failure looks unrelated .https://github.com/apache/arrow/pull/8701/checks?check_run_id=1416672486 ``` Post job cleanup. /bin/tar -cz -f

[GitHub] [arrow] alamb commented on pull request #8701: ARROW-10639: [Rust] Added examples to is_null kernel and simplified signature.

2020-11-18 Thread GitBox
alamb commented on pull request #8701: URL: https://github.com/apache/arrow/pull/8701#issuecomment-730029379 CI failure looks unrelated . I am restarting it https://github.com/apache/arrow/pull/8701/checks?check_run_id=1416672486 ``` Post job cleanup. /bin/tar -cz -f

[GitHub] [arrow] velvia commented on a change in pull request #8688: ARROW-10330: [Rust][DataFusion] Implement NULLIF() SQL function

2020-11-18 Thread GitBox
velvia commented on a change in pull request #8688: URL: https://github.com/apache/arrow/pull/8688#discussion_r526498177 ## File path: rust/arrow/src/compute/kernels/boolean.rs ## @@ -457,4 +517,42 @@ mod tests { assert_eq!(true, res.value(2));

[GitHub] [arrow] Dandandan commented on a change in pull request #8710: ARROW-10649: [Rust] Parse manually in infer_field_schema, remove lazy static dependency

2020-11-18 Thread GitBox
Dandandan commented on a change in pull request #8710: URL: https://github.com/apache/arrow/pull/8710#discussion_r526506545 ## File path: rust/arrow/src/csv/reader.rs ## @@ -77,15 +70,20 @@ fn infer_field_schema(string: ) -> DataType { return DataType::Utf8; }

[GitHub] [arrow] Dandandan commented on a change in pull request #8710: ARROW-10649: [Rust] Parse manually in infer_field_schema, remove lazy static dependency

2020-11-18 Thread GitBox
Dandandan commented on a change in pull request #8710: URL: https://github.com/apache/arrow/pull/8710#discussion_r526520746 ## File path: rust/arrow/src/csv/reader.rs ## @@ -77,15 +70,20 @@ fn infer_field_schema(string: ) -> DataType { return DataType::Utf8; }

[GitHub] [arrow] alamb commented on pull request #8708: ARROW-10647: [Rust] [Parquet] Port benchmarks from from parquet-rs to arrow repo

2020-11-18 Thread GitBox
alamb commented on pull request #8708: URL: https://github.com/apache/arrow/pull/8708#issuecomment-729945761 ![image](https://user-images.githubusercontent.com/490673/99586507-d7b95980-29b5-11eb-9f3f-4c51884eb32f.png)

[GitHub] [arrow] vertexclique commented on pull request #8664: ARROW-10588: [Rust] Safe bit operations for Arrow

2020-11-18 Thread GitBox
vertexclique commented on pull request #8664: URL: https://github.com/apache/arrow/pull/8664#issuecomment-729991100 Weird, on my machine when I pushed the initial implementation of this PR I got the numbers above. Seems like it is regressed for me too.

[GitHub] [arrow] github-actions[bot] commented on pull request #8709: ARROW-9555: [Rust] [DataFusion] Implement physical node for inner join

2020-11-18 Thread GitBox
github-actions[bot] commented on pull request #8709: URL: https://github.com/apache/arrow/pull/8709#issuecomment-729995084 https://issues.apache.org/jira/browse/ARROW-9555 This is an automated message from the Apache Git

[GitHub] [arrow] andygrove commented on a change in pull request #8709: ARROW-9555: [Rust] [DataFusion] Implement physical node for inner join

2020-11-18 Thread GitBox
andygrove commented on a change in pull request #8709: URL: https://github.com/apache/arrow/pull/8709#discussion_r526466282 ## File path: rust/datafusion/src/physical_plan/hash_utils.rs ## @@ -0,0 +1,145 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

  1   2   >