Re: [PR] Minor: Add comments for removed protobuf nodes [datafusion]

2024-04-27 Thread via GitHub
Weijun-H commented on code in PR #10252: URL: https://github.com/apache/datafusion/pull/10252#discussion_r1581754624 ## datafusion/proto/proto/datafusion.proto: ## @@ -1310,6 +1311,7 @@ message PhysicalExprNode { PhysicalSortExprNode sort = 10; PhysicalNegativeNode neg

Re: [PR] Minor: Add comments for removed protobuf nodes [datafusion]

2024-04-27 Thread via GitHub
Weijun-H commented on code in PR #10252: URL: https://github.com/apache/datafusion/pull/10252#discussion_r1581755391 ## datafusion/proto/proto/datafusion.proto: ## @@ -1310,6 +1311,7 @@ message PhysicalExprNode { PhysicalSortExprNode sort = 10; PhysicalNegativeNode neg

[PR] chore: Update Error for Unnest Rewritter [datafusion]

2024-04-27 Thread via GitHub
Weijun-H opened a new pull request, #10263: URL: https://github.com/apache/datafusion/pull/10263 ## Which issue does this PR close? Related #10040 ## Rationale for this change ## What changes are included in this PR? ## Are these changes tes

Re: [I] Release DataFusion `38.0.0` [datafusion]

2024-04-27 Thread via GitHub
alamb closed issue #10255: Release DataFusion `38.0.0` URL: https://github.com/apache/datafusion/issues/10255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [I] Release DataFusion `38.0.0` [datafusion]

2024-04-27 Thread via GitHub
alamb commented on issue #10255: URL: https://github.com/apache/datafusion/issues/10255#issuecomment-2080446984 > This duplicated #10217 ? Yes, sorry I didn't find that one for some reason -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] DataFusion `38.0.0` Release [datafusion]

2024-04-27 Thread via GitHub
alamb commented on issue #10217: URL: https://github.com/apache/datafusion/issues/10217#issuecomment-2080447455 I added a few more items to the waiting list: https://github.com/apache/datafusion/pull/10254 https://github.com/apache/datafusion/issues/10181 https://github.com/apac

Re: [I] Support OrderBy and Sort in Expr->String [datafusion]

2024-04-27 Thread via GitHub
alamb commented on issue #10256: URL: https://github.com/apache/datafusion/issues/10256#issuecomment-2080448399 I don't think we should change SQL parser In general, `ORDER BY ...` isn't really a general Expr (it can't appear in arbitrary locations in SQL, only specific ones) F

Re: [I] [EPIC] Improved support for nested / structured types (`Struct` , `List`, `ListArray`, and other Composite types) [datafusion]

2024-04-27 Thread via GitHub
alamb commented on issue #2326: URL: https://github.com/apache/datafusion/issues/2326#issuecomment-2080448944 > Hi, i think unnest support for struct can be an item in this epic right? That would make sense to me -- is there a ticket that describes what this means? I -- This is an

Re: [I] DataFusion weekly project plan (Andrew Lamb) - April 22, 2024 [datafusion]

2024-04-27 Thread via GitHub
alamb commented on issue #10172: URL: https://github.com/apache/datafusion/issues/10172#issuecomment-2080449894 Review - [ ] https://github.com/apache/datafusion/pull/10221 - [ ] https://github.com/apache/datafusion/pull/10148 - [ ] https://github.com/apache/datafusion/pull/9593 -

Re: [PR] Minor: Add comments for removed protobuf nodes [datafusion]

2024-04-27 Thread via GitHub
alamb commented on code in PR #10252: URL: https://github.com/apache/datafusion/pull/10252#discussion_r1581800502 ## datafusion/proto/proto/datafusion.proto: ## @@ -1310,6 +1311,7 @@ message PhysicalExprNode { PhysicalSortExprNode sort = 10; PhysicalNegativeNode negati

Re: [I] Move `create_physical_expr` to `physical-expr-common` [datafusion]

2024-04-27 Thread via GitHub
alamb commented on issue #10074: URL: https://github.com/apache/datafusion/issues/10074#issuecomment-2080451607 Hmm this is a tricky refactor -- it is like a ball knot in a piece of string -- we just need to keep tugging at it and at some point it will unravel. -- This is an automated me

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
alamb commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581806680 ## datafusion/functions/src/core/arrowtypeof.rs: ## @@ -69,4 +69,8 @@ impl ScalarUDFImpl for ArrowTypeOfFunc { "{input_data_type}" }

Re: [I] [EPIC] Improved support for nested / structured types (`Struct` , `List`, `ListArray`, and other Composite types) [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on issue #2326: URL: https://github.com/apache/datafusion/issues/2326#issuecomment-2080458071 i created a ticket: https://github.com/apache/datafusion/issues/10264 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581810888 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581810996 ## datafusion/functions/src/core/arrowtypeof.rs: ## @@ -69,4 +69,8 @@ impl ScalarUDFImpl for ArrowTypeOfFunc { "{input_data_type}"

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581815100 ## datafusion/functions/src/core/getfield.rs: ## @@ -107,29 +109,55 @@ impl ScalarUDFImpl for GetFieldFunc { ); } }; +

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581815100 ## datafusion/functions/src/core/getfield.rs: ## @@ -107,29 +109,55 @@ impl ScalarUDFImpl for GetFieldFunc { ); } }; +

[PR] Improve documentation on `ColumnarValue` [datafusion]

2024-04-27 Thread via GitHub
alamb opened a new pull request, #10265: URL: https://github.com/apache/datafusion/pull/10265 ## Which issue does this PR close? Part of #7013 ## Rationale for this change While reviewing https://github.com/apache/datafusion/pull/10148 I realized that the documentat

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
alamb commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581816110 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function ma

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581818121 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
alamb commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581818211 ## datafusion/functions/src/core/getfield.rs: ## @@ -107,29 +109,55 @@ impl ScalarUDFImpl for GetFieldFunc { ); } }; +

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581818121 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581819373 ## datafusion/functions/src/core/getfield.rs: ## @@ -107,29 +109,55 @@ impl ScalarUDFImpl for GetFieldFunc { ); } }; +

Re: [PR] fix: Correct null_count in describe() [datafusion]

2024-04-27 Thread via GitHub
alamb commented on code in PR #10260: URL: https://github.com/apache/datafusion/pull/10260#discussion_r1581819093 ## datafusion/expr/src/expr_fn.rs: ## @@ -215,6 +215,18 @@ pub fn count(expr: Expr) -> Expr { )) } +/// Create an expression to represent the count_null() ag

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581819534 ## datafusion/functions/src/core/getfield.rs: ## @@ -107,29 +109,55 @@ impl ScalarUDFImpl for GetFieldFunc { ); } }; +

Re: [PR] feat: add a config param to avoid grouping partitions [datafusion]

2024-04-27 Thread via GitHub
alamb commented on code in PR #10259: URL: https://github.com/apache/datafusion/pull/10259#discussion_r1581819981 ## datafusion/core/src/physical_optimizer/enforce_distribution.rs: ## @@ -1723,21 +1727,30 @@ pub(crate) mod tests { /// * `REPARTITION_FILE_MIN_SIZE` (optional

Re: [PR] feat: add a config param to avoid grouping partitions [datafusion]

2024-04-27 Thread via GitHub
alamb commented on PR #10259: URL: https://github.com/apache/datafusion/pull/10259#issuecomment-2080500041 cc @mustafasrepo and @ozankabak for your thoughts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581821617 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581822766 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581822766 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

Re: [PR] feat: add a config param to avoid grouping partitions [datafusion]

2024-04-27 Thread via GitHub
ozankabak commented on PR #10259: URL: https://github.com/apache/datafusion/pull/10259#issuecomment-2080623778 Thanks @NGA-TRAN and @alamb, we will take a look and circle back on Monday -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] fix: Correct null_count in describe() [datafusion]

2024-04-27 Thread via GitHub
Weijun-H commented on code in PR #10260: URL: https://github.com/apache/datafusion/pull/10260#discussion_r1581828836 ## datafusion/core/src/dataframe/mod.rs: ## @@ -534,7 +534,7 @@ impl DataFrame { vec![], original_schema_fields

[I] Display of built in window functions do not work with struct elements [datafusion-python]

2024-04-27 Thread via GitHub
timsaucer opened a new issue, #647: URL: https://github.com/apache/datafusion-python/issues/647 **Describe the bug** When attempting to call `show()` on a DataFrame that contains a built in window function on a column that has struct elements, it produces the error `Compute error: concat

Re: [I] Support OrderBy and Sort in Expr->String [datafusion]

2024-04-27 Thread via GitHub
kevinmingtarja commented on issue #10256: URL: https://github.com/apache/datafusion/issues/10256#issuecomment-2080644645 Yeah I was kinda wondering about this too last time (https://github.com/apache/datafusion/issues/9726#issuecomment-2032753357). I was thinking we don't change the S

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
duongcongtoai commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581831954 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

Re: [I] Support OrderBy and Sort in Expr->String [datafusion]

2024-04-27 Thread via GitHub
yyy1000 commented on issue #10256: URL: https://github.com/apache/datafusion/issues/10256#issuecomment-2080708518 Thank you all for your reply! > when turning the expression back into an entire query Yeah, I was just a little curious why for Expr::Sort we ignore the ORDER BY in

Re: [PR] feat: Implement Spark-compatible CAST from string to timestamp types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on PR #335: URL: https://github.com/apache/datafusion-comet/pull/335#issuecomment-2080715428 @vaibhawvipul You can run `make format` to fix the linting issues that are causing some CI tests to fail -- This is an automated message from the Apache Git Service. To respond

Re: [PR] feat: Implement Spark-compatible CAST from string to timestamp types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #335: URL: https://github.com/apache/datafusion-comet/pull/335#discussion_r1581834740 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -222,3 +263,139 @@ impl PhysicalExpr for Cast { self.hash(&mut s); } } + +fn parse_t

Re: [PR] feat: Implement Spark-compatible CAST from string to timestamp types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #335: URL: https://github.com/apache/datafusion-comet/pull/335#discussion_r1581835065 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -222,3 +263,139 @@ impl PhysicalExpr for Cast { self.hash(&mut s); } } + +fn parse_t

Re: [I] Support skipping comments in `CsvReader` [datafusion]

2024-04-27 Thread via GitHub
pingsutw commented on issue #10262: URL: https://github.com/apache/datafusion/issues/10262#issuecomment-2080726223 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] feat: Implement Spark-compatible CAST from string to integral types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #307: URL: https://github.com/apache/datafusion-comet/pull/307#discussion_r1581836049 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -142,6 +226,311 @@ impl Cast { } } +fn cast_string_to_i8(str: &str, eval_mode: EvalMode) ->

Re: [PR] feat: Implement Spark-compatible CAST from string to timestamp types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #335: URL: https://github.com/apache/datafusion-comet/pull/335#discussion_r1581837448 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -222,3 +263,139 @@ impl PhysicalExpr for Cast { self.hash(&mut s); } } + +fn parse_t

Re: [PR] Fix docs.rs build for datafusion-proto (hopefully) [datafusion]

2024-04-27 Thread via GitHub
leoyvens commented on PR #10254: URL: https://github.com/apache/datafusion/pull/10254#issuecomment-2080763530 A command that tests this is: `cargo +nightly rustdoc --config 'build.rustdocflags=["--cfg", "docsrs", "--cap-lints", "warn"]'` I took this from the [docs.rs build logs](h

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581838538 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

Re: [PR] Validate ScalarUDF output rows and support nulls for `array_has` and `get_field` for `Map` [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10148: URL: https://github.com/apache/datafusion/pull/10148#discussion_r1581838702 ## datafusion/physical-expr/src/scalar_function.rs: ## @@ -146,11 +146,22 @@ impl PhysicalExpr for ScalarFunctionExpr { // evaluate the function

[PR] feat: add static_name() to ExecutionPlan [datafusion]

2024-04-27 Thread via GitHub
waynexia opened a new pull request, #10266: URL: https://github.com/apache/datafusion/pull/10266 ## Which issue does this PR close? Closes #10246 ## Rationale for this change Add `static_name()` method to make the name of an execution plan accessible

Re: [I] idea: add another `static_name()` method to `ExecutionPlan` trait [datafusion]

2024-04-27 Thread via GitHub
waynexia commented on issue #10246: URL: https://github.com/apache/datafusion/issues/10246#issuecomment-2080809948 > Thanks @waynexia could you please give an example how the static be called for different plan nodes to obtain different names? This (old) test case can be an example:

Re: [PR] feat: add static_name() to ExecutionPlan [datafusion]

2024-04-27 Thread via GitHub
waynexia commented on code in PR #10266: URL: https://github.com/apache/datafusion/pull/10266#discussion_r1581841078 ## datafusion/physical-plan/src/repartition/distributor_channels.rs: ## @@ -474,7 +474,7 @@ type SharedGate = Arc; #[cfg(test)] mod tests { -use std::sync

Re: [PR] Improve documentation on `ColumnarValue` [datafusion]

2024-04-27 Thread via GitHub
waynexia commented on code in PR #10265: URL: https://github.com/apache/datafusion/pull/10265#discussion_r1581841389 ## datafusion/expr/src/columnar_value.rs: ## @@ -25,15 +25,65 @@ use datafusion_common::format::DEFAULT_CAST_OPTIONS; use datafusion_common::{internal_err, Resul

Re: [I] select multiple columns in a single `Expr` [datafusion]

2024-04-27 Thread via GitHub
edmondop commented on issue #10102: URL: https://github.com/apache/datafusion/issues/10102#issuecomment-2080824673 > > How does the selection happens ? > > I think you can select with something like `my_struct_col['foo']` which returns the 'foo' field. > > However now re-readin

Re: [PR] Minor: Possibility to strip datafusion error name [datafusion]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #10186: URL: https://github.com/apache/datafusion/pull/10186#discussion_r1581854626 ## datafusion/common/src/error.rs: ## @@ -450,6 +399,69 @@ impl DataFusionError { #[cfg(not(feature = "backtrace"))] "".to_owned() } + +

Re: [I] Implement Spark-compatible CAST float/double to string [datafusion-comet]

2024-04-27 Thread via GitHub
viirya commented on issue #312: URL: https://github.com/apache/datafusion-comet/issues/312#issuecomment-2081104450 Of course. Thanks @mattharder91. Please let us know if you have any question. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[I] Support for filtered arrow datasets [datafusion]

2024-04-27 Thread via GitHub
adriangb opened a new issue, #10267: URL: https://github.com/apache/datafusion/issues/10267 ### Is your feature request related to a problem or challenge? ```python from pathlib import Path import shutil import polars as pl from deltalake import DeltaTable import pyarrow

Re: [PR] feat: Implement Spark-compatible CAST from string to integral types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #307: URL: https://github.com/apache/datafusion-comet/pull/307#discussion_r1581871855 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -103,10 +125,72 @@ impl Cast { (DataType::LargeUtf8, DataType::Boolean) => {

Re: [PR] feat: Implement Spark-compatible CAST from string to integral types [datafusion-comet]

2024-04-27 Thread via GitHub
viirya commented on code in PR #307: URL: https://github.com/apache/datafusion-comet/pull/307#discussion_r1581872458 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -103,10 +125,72 @@ impl Cast { (DataType::LargeUtf8, DataType::Boolean) => {

Re: [PR] feat: Implement Spark-compatible CAST from string to integral types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #307: URL: https://github.com/apache/datafusion-comet/pull/307#discussion_r1581875517 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -142,6 +226,311 @@ impl Cast { } } +fn cast_string_to_i8(str: &str, eval_mode: EvalMode) ->

Re: [PR] feat: Implement Spark-compatible CAST from string to integral types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #307: URL: https://github.com/apache/datafusion-comet/pull/307#discussion_r1581876228 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -103,10 +125,72 @@ impl Cast { (DataType::LargeUtf8, DataType::Boolean) => {

Re: [PR] feat: Implement Spark-compatible CAST from string to integral types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #307: URL: https://github.com/apache/datafusion-comet/pull/307#discussion_r1581876329 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -103,10 +125,72 @@ impl Cast { (DataType::LargeUtf8, DataType::Boolean) => {

Re: [PR] feat: Implement Spark-compatible CAST from string to integral types [datafusion-comet]

2024-04-27 Thread via GitHub
andygrove commented on code in PR #307: URL: https://github.com/apache/datafusion-comet/pull/307#discussion_r1581876400 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -142,6 +226,311 @@ impl Cast { } } +fn cast_string_to_i8(str: &str, eval_mode: EvalMode) ->

Re: [I] Setup regular meetups for Comet development [datafusion-comet]

2024-04-27 Thread via GitHub
parthchandra commented on issue #217: URL: https://github.com/apache/datafusion-comet/issues/217#issuecomment-2081124910 I'm also (still) getting the "Couldn't find the requested event" . The Meet link seems to be working. -- This is an automated message from the Apache Git Service. To

Re: [PR] feat: Implement Spark-compatible CAST from string to integral types [datafusion-comet]

2024-04-27 Thread via GitHub
viirya commented on code in PR #307: URL: https://github.com/apache/datafusion-comet/pull/307#discussion_r1581891542 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -103,10 +125,72 @@ impl Cast { (DataType::LargeUtf8, DataType::Boolean) => {

Re: [I] docs.rs build fails for datafusion-proto `37.0.0` [datafusion]

2024-04-27 Thread via GitHub
comphead closed issue #10163: docs.rs build fails for datafusion-proto `37.0.0` URL: https://github.com/apache/datafusion/issues/10163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Fix docs.rs build for datafusion-proto (hopefully) [datafusion]

2024-04-27 Thread via GitHub
comphead merged PR #10254: URL: https://github.com/apache/datafusion/pull/10254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

[PR] build(deps): bump object_store from 0.9.1 to 0.10.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #648: URL: https://github.com/apache/datafusion-python/pull/648 Bumps [object_store](https://github.com/apache/arrow-rs) from 0.9.1 to 0.10.0. Changelog Sourced from https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md";>object_

[PR] build(deps): bump datafusion-functions-array from 36.0.0 to 37.1.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #649: URL: https://github.com/apache/datafusion-python/pull/649 Bumps [datafusion-functions-array](https://github.com/apache/arrow-datafusion) from 36.0.0 to 37.1.0. Commits https://github.com/apache/datafusion/commit/aee976aa1a75514c

Re: [PR] build(deps): bump datafusion-functions-array from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] commented on PR #636: URL: https://github.com/apache/datafusion-python/pull/636#issuecomment-2081165569 Superseded by #649. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] build(deps): bump datafusion-functions-array from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] closed pull request #636: build(deps): bump datafusion-functions-array from 36.0.0 to 37.0.0 URL: https://github.com/apache/datafusion-python/pull/636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[PR] build(deps): bump datafusion-expr from 36.0.0 to 37.1.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #650: URL: https://github.com/apache/datafusion-python/pull/650 Bumps [datafusion-expr](https://github.com/apache/arrow-datafusion) from 36.0.0 to 37.1.0. Commits https://github.com/apache/datafusion/commit/aee976aa1a75514c7dbb33ef4752

Re: [PR] build(deps): bump datafusion-expr from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] closed pull request #628: build(deps): bump datafusion-expr from 36.0.0 to 37.0.0 URL: https://github.com/apache/datafusion-python/pull/628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] build(deps): bump datafusion-expr from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] commented on PR #628: URL: https://github.com/apache/datafusion-python/pull/628#issuecomment-2081165590 Superseded by #650. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] build(deps): bump datafusion-substrait from 36.0.0 to 37.1.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #651: URL: https://github.com/apache/datafusion-python/pull/651 Bumps [datafusion-substrait](https://github.com/apache/arrow-datafusion) from 36.0.0 to 37.1.0. Commits https://github.com/apache/datafusion/commit/aee976aa1a75514c7dbb33e

Re: [PR] build(deps): bump datafusion-substrait from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] closed pull request #635: build(deps): bump datafusion-substrait from 36.0.0 to 37.0.0 URL: https://github.com/apache/datafusion-python/pull/635 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] build(deps): bump datafusion-substrait from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] commented on PR #635: URL: https://github.com/apache/datafusion-python/pull/635#issuecomment-2081165631 Superseded by #651. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] build(deps): bump datafusion from 36.0.0 to 37.1.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #652: URL: https://github.com/apache/datafusion-python/pull/652 Bumps [datafusion](https://github.com/apache/arrow-datafusion) from 36.0.0 to 37.1.0. Commits https://github.com/apache/datafusion/commit/aee976aa1a75514c7dbb33ef47527b3ba

Re: [PR] build(deps): bump datafusion from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] commented on PR #629: URL: https://github.com/apache/datafusion-python/pull/629#issuecomment-2081165656 Superseded by #652. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] build(deps): bump datafusion from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] closed pull request #629: build(deps): bump datafusion from 36.0.0 to 37.0.0 URL: https://github.com/apache/datafusion-python/pull/629 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] build(deps): bump parking_lot from 0.12.1 to 0.12.2 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #653: URL: https://github.com/apache/datafusion-python/pull/653 Bumps [parking_lot](https://github.com/Amanieu/parking_lot) from 0.12.1 to 0.12.2. Changelog Sourced from https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md";>par

[PR] build(deps): bump datafusion-common from 36.0.0 to 37.1.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #654: URL: https://github.com/apache/datafusion-python/pull/654 Bumps [datafusion-common](https://github.com/apache/arrow-datafusion) from 36.0.0 to 37.1.0. Commits https://github.com/apache/datafusion/commit/aee976aa1a75514c7dbb33ef47

Re: [PR] build(deps): bump datafusion-common from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] closed pull request #626: build(deps): bump datafusion-common from 36.0.0 to 37.0.0 URL: https://github.com/apache/datafusion-python/pull/626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] build(deps): bump datafusion-common from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] commented on PR #626: URL: https://github.com/apache/datafusion-python/pull/626#issuecomment-2081165726 Superseded by #654. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] build(deps): bump datafusion-sql from 36.0.0 to 37.1.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #655: URL: https://github.com/apache/datafusion-python/pull/655 Bumps [datafusion-sql](https://github.com/apache/arrow-datafusion) from 36.0.0 to 37.1.0. Commits https://github.com/apache/datafusion/commit/aee976aa1a75514c7dbb33ef47527

Re: [PR] build(deps): bump datafusion-sql from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] closed pull request #632: build(deps): bump datafusion-sql from 36.0.0 to 37.0.0 URL: https://github.com/apache/datafusion-python/pull/632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] build(deps): bump datafusion-sql from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] commented on PR #632: URL: https://github.com/apache/datafusion-python/pull/632#issuecomment-2081165765 Superseded by #655. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] build(deps): bump datafusion-optimizer from 36.0.0 to 37.1.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #656: URL: https://github.com/apache/datafusion-python/pull/656 Bumps [datafusion-optimizer](https://github.com/apache/arrow-datafusion) from 36.0.0 to 37.1.0. Commits https://github.com/apache/datafusion/commit/aee976aa1a75514c7dbb33e

Re: [PR] build(deps): bump datafusion-optimizer from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] commented on PR #633: URL: https://github.com/apache/datafusion-python/pull/633#issuecomment-2081165802 Superseded by #656. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] build(deps): bump mimalloc from 0.1.39 to 0.1.41 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] opened a new pull request, #657: URL: https://github.com/apache/datafusion-python/pull/657 Bumps [mimalloc](https://github.com/purpleprotocol/mimalloc_rust) from 0.1.39 to 0.1.41. Release notes Sourced from https://github.com/purpleprotocol/mimalloc_rust/releases";>

Re: [PR] build(deps): bump datafusion-optimizer from 36.0.0 to 37.0.0 [datafusion-python]

2024-04-27 Thread via GitHub
dependabot[bot] closed pull request #633: build(deps): bump datafusion-optimizer from 36.0.0 to 37.0.0 URL: https://github.com/apache/datafusion-python/pull/633 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[PR] Avoid casting for `coalesce`, introduce signature VariadicEqualOrNull for `coalesce`. [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 opened a new pull request, #10268: URL: https://github.com/apache/datafusion/pull/10268 ## Which issue does this PR close? Closes #10261 Closes #10241 ## Rationale for this change ## What changes are included in this PR? ## Are

Re: [PR] Avoid casting for `coalesce`, introduce signature VariadicEqualOrNull for `coalesce`. [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10268: URL: https://github.com/apache/datafusion/pull/10268#discussion_r1581973804 ## datafusion/expr/src/signature.rs: ## @@ -92,14 +92,19 @@ pub enum TypeSignature { /// A function such as `concat` is `Variadic(vec![DataType::Utf8, Dat

Re: [PR] Avoid casting for `coalesce`, introduce signature VariadicEqualOrNull for `coalesce`. [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10268: URL: https://github.com/apache/datafusion/pull/10268#discussion_r1581973912 ## datafusion/expr/src/signature.rs: ## @@ -226,6 +234,11 @@ impl TypeSignature { _ => false, } } + +/// Skip coercion to match t

Re: [PR] Avoid casting for `coalesce`, introduce signature VariadicEqualOrNull for `coalesce`. [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10268: URL: https://github.com/apache/datafusion/pull/10268#discussion_r1581974152 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -450,19 +473,7 @@ fn coerced_from<'a>( { Some(type_into.clone()) } -

Re: [PR] Avoid casting for `coalesce`, introduce signature VariadicEqualOrNull for `coalesce`. [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 commented on code in PR #10268: URL: https://github.com/apache/datafusion/pull/10268#discussion_r1581974222 ## datafusion/sqllogictest/test_files/functions.slt: ## @@ -1159,3 +1159,247 @@ drop table uuid_table statement ok drop table t + +# Test coalesce function

Re: [PR] Minor: Possibility to strip datafusion error name [datafusion]

2024-04-27 Thread via GitHub
andygrove merged PR #10186: URL: https://github.com/apache/datafusion/pull/10186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@data

Re: [PR] feat: add static_name() to ExecutionPlan [datafusion]

2024-04-27 Thread via GitHub
andygrove commented on PR #10266: URL: https://github.com/apache/datafusion/pull/10266#issuecomment-2081254883 @waynexia This looks good, but could you add a test to demonstrate what the output looks like? -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] Avoid casting for `coalesce`, introduce signature VariadicEqualOrNull for `coalesce`. [datafusion]

2024-04-27 Thread via GitHub
jayzhan211 closed pull request #10268: Avoid casting for `coalesce`, introduce signature VariadicEqualOrNull for `coalesce`. URL: https://github.com/apache/datafusion/pull/10268 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Docs: Add governance page to contributor guide [datafusion]

2024-04-27 Thread via GitHub
andygrove merged PR #10238: URL: https://github.com/apache/datafusion/pull/10238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@data

Re: [PR] ArraySum [datafusion]

2024-04-27 Thread via GitHub
github-actions[bot] closed pull request #8325: ArraySum URL: https://github.com/apache/datafusion/pull/8325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Fix incorrect results for NOT IN subqueries with nulls [datafusion]

2024-04-27 Thread via GitHub
github-actions[bot] closed pull request #8271: Fix incorrect results for NOT IN subqueries with nulls URL: https://github.com/apache/datafusion/pull/8271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] WIP: Consolidate statistics aggregation [datafusion]

2024-04-27 Thread via GitHub
github-actions[bot] closed pull request #8254: WIP: Consolidate statistics aggregation URL: https://github.com/apache/datafusion/pull/8254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] Enable the UDF function to collect its metrics. [datafusion]

2024-04-27 Thread via GitHub
JasonLi-cn closed issue #9993: Enable the UDF function to collect its metrics. URL: https://github.com/apache/datafusion/issues/9993 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

  1   2   >