[GitHub] [arrow] github-actions[bot] commented on pull request #9701: ARROW-11962: [Rust][DataFusion] Improve DataFusion docs

2021-03-14 Thread GitBox
github-actions[bot] commented on pull request #9701: URL: https://github.com/apache/arrow/pull/9701#issuecomment-798897258 https://issues.apache.org/jira/browse/ARROW-11962 This is an automated message from the Apache Git

[GitHub] [arrow] jorgecarleitao edited a comment on pull request #9692: ARROW-11790: [Rust][DataFusion] RFC: Change builder signatures to take Vec rather than &[Expr]

2021-03-14 Thread GitBox
jorgecarleitao edited a comment on pull request #9692: URL: https://github.com/apache/arrow/pull/9692#issuecomment-798977003 Another option is to use `T: AsRef<[Expr]>`, which works for any type that can be represented as a reference to an array (e.g. `vec` and &[]`) @houqp , AFAIK

[GitHub] [arrow] Dandandan commented on a change in pull request #9645: ARROW-11894: [Rust][DataFusion] Change flight server example to use DataFrame API

2021-03-14 Thread GitBox
Dandandan commented on a change in pull request #9645: URL: https://github.com/apache/arrow/pull/9645#discussion_r593884833 ## File path: rust/datafusion/examples/flight_server.rs ## @@ -94,28 +94,26 @@ impl FlightService for FlightServiceImpl {

[GitHub] [arrow] codecov-io edited a comment on pull request #9695: ARROW-11955: [Rust][DataFusion] Support Union

2021-03-14 Thread GitBox
codecov-io edited a comment on pull request #9695: URL: https://github.com/apache/arrow/pull/9695#issuecomment-798734537 # [Codecov](https://codecov.io/gh/apache/arrow/pull/9695?src=pr=h1) Report > Merging [#9695](https://codecov.io/gh/apache/arrow/pull/9695?src=pr=desc) (ac89542) into

[GitHub] [arrow] Crystrix opened a new pull request #9700: ARROW-11960: [Gandiva][C++]Support escape in LIKE

2021-03-14 Thread GitBox
Crystrix opened a new pull request #9700: URL: https://github.com/apache/arrow/pull/9700 Add gdv_fn_like_utf8_utf8_int8 function in Gandiva to support escape char in LIKE. An escape char is stored in an int8 type which is compatible with char type in C++.

[GitHub] [arrow] github-actions[bot] commented on pull request #9700: ARROW-11960: [Gandiva][C++]Support escape in LIKE

2021-03-14 Thread GitBox
github-actions[bot] commented on pull request #9700: URL: https://github.com/apache/arrow/pull/9700#issuecomment-798895560 https://issues.apache.org/jira/browse/ARROW-11960 This is an automated message from the Apache Git

[GitHub] [arrow] sundy-li commented on a change in pull request #9602: ARROW-11630: [Rust] Introduce limit option for sort kernel

2021-03-14 Thread GitBox
sundy-li commented on a change in pull request #9602: URL: https://github.com/apache/arrow/pull/9602#discussion_r593899532 ## File path: rust/arrow/src/compute/kernels/sort.rs ## @@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) -> Result { };

[GitHub] [arrow] sundy-li commented on a change in pull request #9602: ARROW-11630: [Rust] Introduce limit option for sort kernel

2021-03-14 Thread GitBox
sundy-li commented on a change in pull request #9602: URL: https://github.com/apache/arrow/pull/9602#discussion_r593899532 ## File path: rust/arrow/src/compute/kernels/sort.rs ## @@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) -> Result { };

[GitHub] [arrow] alamb commented on a change in pull request #9703: ARROW-11790: [Rust][DataFusion] Change builder signatures to take impl Interator rather than &[Expr]

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9703: URL: https://github.com/apache/arrow/pull/9703#discussion_r593901579 ## File path: rust/datafusion/src/logical_plan/builder.rs ## @@ -39,6 +39,43 @@ use crate::logical_plan::{DFField, DFSchema, DFSchemaRef, Partitioning};

[GitHub] [arrow] Dandandan commented on a change in pull request #9602: ARROW-11630: [Rust] Introduce limit option for sort kernel

2021-03-14 Thread GitBox
Dandandan commented on a change in pull request #9602: URL: https://github.com/apache/arrow/pull/9602#discussion_r593904547 ## File path: rust/arrow/src/compute/kernels/sort.rs ## @@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) -> Result { };

[GitHub] [arrow] returnString commented on pull request #9703: ARROW-11790: [Rust][DataFusion] Change builder signatures to take impl Interator rather than &[Expr]

2021-03-14 Thread GitBox
returnString commented on pull request #9703: URL: https://github.com/apache/arrow/pull/9703#issuecomment-798918286 > I couldn't figure out how to allow the Dataframe API (which is a Trait) to take an impl Iterator Yeah, it's a pain but `impl trait` (in both return and argument

[GitHub] [arrow] github-actions[bot] commented on pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
github-actions[bot] commented on pull request #9704: URL: https://github.com/apache/arrow/pull/9704#issuecomment-798937300 https://issues.apache.org/jira/browse/ARROW-11964 This is an automated message from the Apache Git

[GitHub] [arrow] Dandandan opened a new pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
Dandandan opened a new pull request #9704: URL: https://github.com/apache/arrow/pull/9704 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] kou closed pull request #8717: ARROW-10659: [Ruby] Refactor Table#initialize

2021-03-14 Thread GitBox
kou closed pull request #8717: URL: https://github.com/apache/arrow/pull/8717 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] kou commented on pull request #8717: ARROW-10659: [Ruby] Refactor Table#initialize

2021-03-14 Thread GitBox
kou commented on pull request #8717: URL: https://github.com/apache/arrow/pull/8717#issuecomment-798973718 I close this because this isn't active in 4 months. This is an automated message from the Apache Git Service. To

[GitHub] [arrow] houqp commented on pull request #9692: ARROW-11790: [Rust][DataFusion] RFC: Change builder signatures to take Vec rather than &[Expr]

2021-03-14 Thread GitBox
houqp commented on pull request #9692: URL: https://github.com/apache/arrow/pull/9692#issuecomment-798975788 Yes, I think if the function only requires sequential access, iterator would have been even better. Although I would prefer to write it as a generic function rather than using

[GitHub] [arrow] yordan-pavlov commented on pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
yordan-pavlov commented on pull request #9704: URL: https://github.com/apache/arrow/pull/9704#issuecomment-798984352 @Dandandan I have mixed feelings about this change - I get the performance improvement side, but in my opinion it makes the explain output less readable by replacing the

[GitHub] [arrow] Dandandan commented on a change in pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
Dandandan commented on a change in pull request #9704: URL: https://github.com/apache/arrow/pull/9704#discussion_r593964624 ## File path: rust/datafusion/src/physical_plan/parquet.rs ## @@ -645,8 +654,29 @@ fn build_predicate_expression( use crate::logical_plan; //

[GitHub] [arrow] alamb commented on a change in pull request #9701: ARROW-11962: [Rust][DataFusion] Improve DataFusion docs

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9701: URL: https://github.com/apache/arrow/pull/9701#discussion_r593891472 ## File path: rust/datafusion/DEVELOPERS.md ## @@ -0,0 +1,79 @@ +# Developer's guide + +This section describes how you can get started at developing

[GitHub] [arrow] alamb commented on a change in pull request #9701: ARROW-11962: [Rust][DataFusion] Improve DataFusion docs

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9701: URL: https://github.com/apache/arrow/pull/9701#discussion_r593891392 ## File path: rust/datafusion/DEVELOPERS.md ## @@ -0,0 +1,79 @@ +# Developer's guide Review comment: I pulled this file into its own separate file so

[GitHub] [arrow] alamb commented on pull request #9692: ARROW-11790: [Rust][DataFusion] RFC: Change builder signatures to take Vec rather than &[Expr]

2021-03-14 Thread GitBox
alamb commented on pull request #9692: URL: https://github.com/apache/arrow/pull/9692#issuecomment-798900651 @Dandandan and @houqp What do you think about something like this (which is even more Idiomatic (TM) I think), though harder for beginner Rust programmers to grok?

[GitHub] [arrow] kou closed pull request #9687: ARROW-11949: [Ruby] Accept raw Ruby objects as sort key and options

2021-03-14 Thread GitBox
kou closed pull request #9687: URL: https://github.com/apache/arrow/pull/9687 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] Dandandan commented on a change in pull request #9602: ARROW-11630: [Rust] Introduce limit option for sort kernel

2021-03-14 Thread GitBox
Dandandan commented on a change in pull request #9602: URL: https://github.com/apache/arrow/pull/9602#discussion_r593904547 ## File path: rust/arrow/src/compute/kernels/sort.rs ## @@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) -> Result { };

[GitHub] [arrow] jorgecarleitao commented on pull request #9692: ARROW-11790: [Rust][DataFusion] RFC: Change builder signatures to take Vec rather than &[Expr]

2021-03-14 Thread GitBox
jorgecarleitao commented on pull request #9692: URL: https://github.com/apache/arrow/pull/9692#issuecomment-798977003 Another option is to use `T: AsRef<[Expr]>`, which works for any type that can be represented as a reference to an array (e.g. `vec` and &[]`) @houqp , AFAIK `impl`

[GitHub] [arrow] alamb commented on a change in pull request #9701: ARROW-11962: [Rust][DataFusion] Improve DataFusion docs

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9701: URL: https://github.com/apache/arrow/pull/9701#discussion_r593891961 ## File path: rust/datafusion/README.md ## @@ -19,11 +19,48 @@ # DataFusion -DataFusion is an in-memory query engine that uses Apache Arrow as the

[GitHub] [arrow] Dandandan commented on a change in pull request #9701: ARROW-11962: [Rust][DataFusion] Improve DataFusion docs

2021-03-14 Thread GitBox
Dandandan commented on a change in pull request #9701: URL: https://github.com/apache/arrow/pull/9701#discussion_r593902991 ## File path: rust/datafusion/README.md ## @@ -19,11 +19,48 @@ # DataFusion -DataFusion is an in-memory query engine that uses Apache Arrow as the

[GitHub] [arrow] Dandandan removed a comment on pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
Dandandan removed a comment on pull request #9704: URL: https://github.com/apache/arrow/pull/9704#issuecomment-798943827 @yordan-pavlov I am trying to use the filtering in parquet - however on any query the row groups being filtered seems to be 0. Do you know what could be happening here?

[GitHub] [arrow] kou commented on pull request #9687: ARROW-11949: [Ruby] Accept raw Ruby objects as sort key and options

2021-03-14 Thread GitBox
kou commented on pull request #9687: URL: https://github.com/apache/arrow/pull/9687#issuecomment-798972719 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [arrow] alamb commented on a change in pull request #9645: ARROW-11894: [Rust][DataFusion] Change flight server example to use DataFrame API

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9645: URL: https://github.com/apache/arrow/pull/9645#discussion_r593885249 ## File path: rust/datafusion/examples/flight_server.rs ## @@ -94,28 +94,22 @@ impl FlightService for FlightServiceImpl {

[GitHub] [arrow] Dandandan commented on a change in pull request #9602: ARROW-11630: [Rust] Introduce limit option for sort kernel

2021-03-14 Thread GitBox
Dandandan commented on a change in pull request #9602: URL: https://github.com/apache/arrow/pull/9602#discussion_r593885246 ## File path: rust/arrow/src/compute/kernels/sort.rs ## @@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) -> Result { };

[GitHub] [arrow] alamb opened a new pull request #9701: ARROW-11962: [Rust][DataFusion] Improve DataFusion docs

2021-03-14 Thread GitBox
alamb opened a new pull request #9701: URL: https://github.com/apache/arrow/pull/9701 # Rationale 1. It would be nice to market / explain DataFusion a bit more and explain what it is good for # Changes 1. Describe usecases for DataFusion (Add some marketing "spin"??) 2. Add

[GitHub] [arrow] Dandandan commented on a change in pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
Dandandan commented on a change in pull request #9704: URL: https://github.com/apache/arrow/pull/9704#discussion_r593962933 ## File path: rust/datafusion/src/physical_plan/parquet.rs ## @@ -645,8 +654,29 @@ fn build_predicate_expression( use crate::logical_plan; //

[GitHub] [arrow] sundy-li commented on a change in pull request #9602: ARROW-11630: [Rust] Introduce limit option for sort kernel

2021-03-14 Thread GitBox
sundy-li commented on a change in pull request #9602: URL: https://github.com/apache/arrow/pull/9602#discussion_r593899532 ## File path: rust/arrow/src/compute/kernels/sort.rs ## @@ -686,90 +815,124 @@ pub fn lexsort_to_indices(columns: &[SortColumn]) -> Result { };

[GitHub] [arrow] alamb opened a new pull request #9703: ARROW-11790: [Rust][DataFusion] Change builder signatures to take impl Interator rather than &[Expr]

2021-03-14 Thread GitBox
alamb opened a new pull request #9703: URL: https://github.com/apache/arrow/pull/9703 # NOTE: Since is a fairly major backwards incompatible change (many callsites need to be updated, though mostly mechanically); I gathered some feedback on this approach in

[GitHub] [arrow] alamb commented on a change in pull request #9703: ARROW-11790: [Rust][DataFusion] Change builder signatures to take impl Interator rather than &[Expr]

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9703: URL: https://github.com/apache/arrow/pull/9703#discussion_r593901365 ## File path: rust/benchmarks/src/bin/tpch.rs ## @@ -1636,7 +1636,7 @@ mod tests { .file_extension(".out"); let df =

[GitHub] [arrow] github-actions[bot] commented on pull request #9703: ARROW-11790: [Rust][DataFusion] Change builder signatures to take impl Interator rather than &[Expr]

2021-03-14 Thread GitBox
github-actions[bot] commented on pull request #9703: URL: https://github.com/apache/arrow/pull/9703#issuecomment-798907390 https://issues.apache.org/jira/browse/ARROW-11790 This is an automated message from the Apache Git

[GitHub] [arrow] houqp commented on a change in pull request #9701: ARROW-11962: [Rust][DataFusion] Improve DataFusion docs

2021-03-14 Thread GitBox
houqp commented on a change in pull request #9701: URL: https://github.com/apache/arrow/pull/9701#discussion_r593941886 ## File path: rust/datafusion/README.md ## @@ -19,11 +19,48 @@ # DataFusion -DataFusion is an in-memory query engine that uses Apache Arrow as the

[GitHub] [arrow] mathyingzhou commented on pull request #8648: ARROW-7906: [C++] [Python] Add ORC write support

2021-03-14 Thread GitBox
mathyingzhou commented on pull request #8648: URL: https://github.com/apache/arrow/pull/8648#issuecomment-798971411 @pitrou Could you please check again? My next PR is going to be ready soon and it is dependent on this one merging to be clean.

[GitHub] [arrow] alamb commented on a change in pull request #9701: ARROW-11962: [Rust][DataFusion] Improve DataFusion docs

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9701: URL: https://github.com/apache/arrow/pull/9701#discussion_r593891533 ## File path: rust/datafusion/README.md ## @@ -19,11 +19,48 @@ # DataFusion -DataFusion is an in-memory query engine that uses Apache Arrow as the

[GitHub] [arrow] github-actions[bot] commented on pull request #9702: ARROW-11297: [C++][Python] Add ORC writer options

2021-03-14 Thread GitBox
github-actions[bot] commented on pull request #9702: URL: https://github.com/apache/arrow/pull/9702#issuecomment-798904875 https://issues.apache.org/jira/browse/ARROW-11297 This is an automated message from the Apache Git

[GitHub] [arrow] mathyingzhou opened a new pull request #9702: ARROW-11297: [C++][Python] Add ORC writer options

2021-03-14 Thread GitBox
mathyingzhou opened a new pull request #9702: URL: https://github.com/apache/arrow/pull/9702 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [arrow] alamb commented on a change in pull request #9703: ARROW-11790: [Rust][DataFusion] Change builder signatures to take impl Interator rather than &[Expr]

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9703: URL: https://github.com/apache/arrow/pull/9703#discussion_r593902253 ## File path: rust/datafusion/src/logical_plan/builder.rs ## @@ -220,20 +260,28 @@ impl LogicalPlanBuilder { })) } -/// Apply an

[GitHub] [arrow] Dandandan commented on pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
Dandandan commented on pull request #9704: URL: https://github.com/apache/arrow/pull/9704#issuecomment-798943827 @yordan-pavlov I am trying to use the filtering in parquet - however on any query the row groups being filtered seems to be 0. Do you know what could be happening here?

[GitHub] [arrow] andygrove commented on a change in pull request #9695: ARROW-11955: [Rust][DataFusion] Support Union

2021-03-14 Thread GitBox
andygrove commented on a change in pull request #9695: URL: https://github.com/apache/arrow/pull/9695#discussion_r593934133 ## File path: rust/datafusion/src/physical_plan/union.rs ## @@ -0,0 +1,76 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] yordan-pavlov commented on a change in pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
yordan-pavlov commented on a change in pull request #9704: URL: https://github.com/apache/arrow/pull/9704#discussion_r593961778 ## File path: rust/datafusion/src/physical_plan/parquet.rs ## @@ -645,8 +654,29 @@ fn build_predicate_expression( use crate::logical_plan;

[GitHub] [arrow] nevi-me commented on a change in pull request #9232: ARROW-10818: [Rust] Initial implementation for Decimal (Decimal128Type only)

2021-03-14 Thread GitBox
nevi-me commented on a change in pull request #9232: URL: https://github.com/apache/arrow/pull/9232#discussion_r593875606 ## File path: rust/arrow/src/compute/kernels/cast.rs ## @@ -1511,6 +1641,65 @@ mod tests { assert!(9.0 - c.value(4) < f64::EPSILON); } +

[GitHub] [arrow] alamb closed pull request #9690: ARROW-11951: [Rust] Remove OffsetSize::prefix

2021-03-14 Thread GitBox
alamb closed pull request #9690: URL: https://github.com/apache/arrow/pull/9690 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] alamb commented on pull request #9600: ARROW-11822: [Rust][Datafusion] Support case sensitive for function

2021-03-14 Thread GitBox
alamb commented on pull request #9600: URL: https://github.com/apache/arrow/pull/9600#issuecomment-798885671 > The above is the primary reason why I did not introduce this idea before; it will lead to either no one changing that parameter once the system is running Yes -- this is

[GitHub] [arrow] Dandandan commented on a change in pull request #9695: ARROW-11955: [Rust][DataFusion] Support Union

2021-03-14 Thread GitBox
Dandandan commented on a change in pull request #9695: URL: https://github.com/apache/arrow/pull/9695#discussion_r593883069 ## File path: rust/datafusion/src/physical_plan/union.rs ## @@ -0,0 +1,76 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] alamb closed pull request #9699: ARROW-11959: [Rust][DataFusion] Fix log line

2021-03-14 Thread GitBox
alamb closed pull request #9699: URL: https://github.com/apache/arrow/pull/9699 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] alamb commented on pull request #9691: ARROW-11952: [Rust] Make ArrayData --> GenericListArray fallable instead of `panic!`

2021-03-14 Thread GitBox
alamb commented on pull request #9691: URL: https://github.com/apache/arrow/pull/9691#issuecomment-79368 I am not sure how much of an improvement this is and I would be happy to simply close the PR, but I didn't want to leave it unresolved outstanding

[GitHub] [arrow] alamb commented on a change in pull request #9639: ARROW-11879 [Rust][DataFusion] Make ExecutionContext::sql return dataframe with optimized plan

2021-03-14 Thread GitBox
alamb commented on a change in pull request #9639: URL: https://github.com/apache/arrow/pull/9639#discussion_r593883317 ## File path: rust/datafusion/src/execution/context.rs ## @@ -1702,6 +1702,23 @@ mod tests { } Ok(()) } +#[test] +fn

[GitHub] [arrow] alamb closed pull request #9639: ARROW-11879 [Rust][DataFusion] Make ExecutionContext::sql return dataframe with optimized plan

2021-03-14 Thread GitBox
alamb closed pull request #9639: URL: https://github.com/apache/arrow/pull/9639 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] cyb70289 commented on pull request #9706: ARROW-11941: [Dev] Don't update Jira if run "DEBUG=1 merge_arrow_pr.py"

2021-03-14 Thread GitBox
cyb70289 commented on pull request #9706: URL: https://github.com/apache/arrow/pull/9706#issuecomment-799050981 Please note **this patch is not tested**. I only checked the added lines manually that there is no syntax error.

[GitHub] [arrow] nevi-me opened a new pull request #9705: ARROW-11365: [Rust] [Parquet] Logical type printer and parser

2021-03-14 Thread GitBox
nevi-me opened a new pull request #9705: URL: https://github.com/apache/arrow/pull/9705 This implements the parser and printer for logical types, allowing us to read and generate the schema in the form `REQUIRED INT32 field_name (INTEGER(16,false))`.

[GitHub] [arrow] github-actions[bot] commented on pull request #9705: ARROW-11365: [Rust] [Parquet] Logical type printer and parser

2021-03-14 Thread GitBox
github-actions[bot] commented on pull request #9705: URL: https://github.com/apache/arrow/pull/9705#issuecomment-799030723 https://issues.apache.org/jira/browse/ARROW-11365 This is an automated message from the Apache Git

[GitHub] [arrow] yordan-pavlov commented on a change in pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
yordan-pavlov commented on a change in pull request #9704: URL: https://github.com/apache/arrow/pull/9704#discussion_r593965350 ## File path: rust/datafusion/src/physical_plan/parquet.rs ## @@ -645,8 +654,29 @@ fn build_predicate_expression( use crate::logical_plan;

[GitHub] [arrow] Dandandan commented on pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
Dandandan commented on pull request #9704: URL: https://github.com/apache/arrow/pull/9704#issuecomment-798985687 > @Dandandan I have mixed feelings about this change - I get the performance improvement side, but in my opinion it makes the explain output less readable by replacing the

[GitHub] [arrow] nevi-me commented on a change in pull request #9612: ARROW-11824: [Rust] [Parquet] Use logical types in Arrow schema conversion

2021-03-14 Thread GitBox
nevi-me commented on a change in pull request #9612: URL: https://github.com/apache/arrow/pull/9612#discussion_r593995672 ## File path: rust/parquet/src/arrow/schema.rs ## @@ -364,32 +385,51 @@ fn arrow_to_parquet_type(field: ) -> Result { DataType::Float64 =>

[GitHub] [arrow] nevi-me commented on a change in pull request #9612: ARROW-11824: [Rust] [Parquet] Use logical types in Arrow schema conversion

2021-03-14 Thread GitBox
nevi-me commented on a change in pull request #9612: URL: https://github.com/apache/arrow/pull/9612#discussion_r593995897 ## File path: rust/parquet/src/arrow/schema.rs ## @@ -583,48 +626,109 @@ impl ParquetTypeConverter<'_> { } fn from_int32() -> Result { -

[GitHub] [arrow] cyb70289 opened a new pull request #9706: [ARROW-11941]: [Dev] Don't update Jira if run "DEBUG=1 merge_arrow_pr.py"

2021-03-14 Thread GitBox
cyb70289 opened a new pull request #9706: URL: https://github.com/apache/arrow/pull/9706 When environment variable DEBUG=1, merge_arrow_pr.py should only try the merge steps without updating anything. Though PR status is not changed, Jira issue is updated even if DEBUG=1. This patch

[GitHub] [arrow] houqp commented on pull request #9692: ARROW-11790: [Rust][DataFusion] RFC: Change builder signatures to take Vec rather than &[Expr]

2021-03-14 Thread GitBox
houqp commented on pull request #9692: URL: https://github.com/apache/arrow/pull/9692#issuecomment-799038552 @jorgecarleitao yeah, you are right, sorry I got it mixed up with `dyn` :P I think `IntoIterator` won't require copy or clone because the iterator owns those values. `AsRef`

[GitHub] [arrow] github-actions[bot] commented on pull request #9706: [ARROW-11941]: [Dev] Don't update Jira if run "DEBUG=1 merge_arrow_pr.py"

2021-03-14 Thread GitBox
github-actions[bot] commented on pull request #9706: URL: https://github.com/apache/arrow/pull/9706#issuecomment-799049571 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] github-actions[bot] commented on pull request #9706: ARROW-11941: [Dev] Don't update Jira if run "DEBUG=1 merge_arrow_pr.py"

2021-03-14 Thread GitBox
github-actions[bot] commented on pull request #9706: URL: https://github.com/apache/arrow/pull/9706#issuecomment-799049802 https://issues.apache.org/jira/browse/ARROW-11941 This is an automated message from the Apache Git

[GitHub] [arrow] nevi-me commented on pull request #9705: ARROW-11365: [Rust] [Parquet] Logical type printer and parser

2021-03-14 Thread GitBox
nevi-me commented on pull request #9705: URL: https://github.com/apache/arrow/pull/9705#issuecomment-799030871 @sunchao I've created this on top of #9612, PTAL when you can. This is an automated message from the Apache Git

[GitHub] [arrow] sunchao commented on a change in pull request #9612: ARROW-11824: [Rust] [Parquet] Use logical types in Arrow schema conversion

2021-03-14 Thread GitBox
sunchao commented on a change in pull request #9612: URL: https://github.com/apache/arrow/pull/9612#discussion_r594048206 ## File path: rust/parquet/src/arrow/schema.rs ## @@ -364,32 +385,51 @@ fn arrow_to_parquet_type(field: ) -> Result { DataType::Float64 =>

[GitHub] [arrow] yordan-pavlov commented on a change in pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-14 Thread GitBox
yordan-pavlov commented on a change in pull request #9704: URL: https://github.com/apache/arrow/pull/9704#discussion_r593965090 ## File path: rust/datafusion/src/physical_plan/parquet.rs ## @@ -645,8 +654,29 @@ fn build_predicate_expression( use crate::logical_plan;

[GitHub] [arrow] liyafan82 commented on issue #9636: Is there an API to deserialize ListVector into double[] efficiently ?

2021-03-14 Thread GitBox
liyafan82 commented on issue #9636: URL: https://github.com/apache/arrow/issues/9636#issuecomment-799069748 The float8 data in the list vector and double array are both in a continuous memory region. So in theory, it is possible to populate the data in the `double[]` in a single copy.