Re: [PR] Remove builtin Sum [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on code in PR #10831: URL: https://github.com/apache/datafusion/pull/10831#discussion_r1631923006 ## datafusion/sql/Cargo.toml: ## @@ -54,6 +54,7 @@ strum = { version = "0.26.1", features = ["derive"] } [dev-dependencies] ctor = { workspace = true } dataf

Re: [PR] Remove builtin Sum [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on code in PR #10831: URL: https://github.com/apache/datafusion/pull/10831#discussion_r1631923230 ## datafusion/substrait/Cargo.toml: ## @@ -43,6 +43,7 @@ prost = "0.12" substrait = { version = "0.34.0", features = ["serde"] } [dev-dependencies] +datafus

Re: [PR] Remove builtin Sum [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on code in PR #10831: URL: https://github.com/apache/datafusion/pull/10831#discussion_r1631922668 ## datafusion/expr/src/type_coercion/aggregates.rs: ## @@ -621,17 +592,6 @@ mod tests { assert_eq!(*input_type, result.unwrap());

Re: [PR] feat: Add HashJoin support for BuildRight [datafusion-comet]

2024-06-08 Thread via GitHub
viirya commented on PR #437: URL: https://github.com/apache/datafusion-comet/pull/437#issuecomment-2155848656 Merged. Thanks @kazuyukitanimura @andygrove -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] feat: Add HashJoin support for BuildRight [datafusion-comet]

2024-06-08 Thread via GitHub
viirya merged PR #437: URL: https://github.com/apache/datafusion-comet/pull/437 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [I] feat: Add support for BuildRight [datafusion-comet]

2024-06-08 Thread via GitHub
viirya closed issue #390: feat: Add support for BuildRight URL: https://github.com/apache/datafusion-comet/issues/390 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] fix: null character not permitted in chr function [datafusion-comet]

2024-06-08 Thread via GitHub
viirya commented on code in PR #513: URL: https://github.com/apache/datafusion-comet/pull/513#discussion_r1631926325 ## core/src/execution/datafusion/expressions/scalar_funcs/chr.rs: ## @@ -0,0 +1,108 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [PR] Support explicit type and name during table creation [datafusion]

2024-06-08 Thread via GitHub
duongcongtoai commented on PR #10273: URL: https://github.com/apache/datafusion/pull/10273#issuecomment-2155849917 hi @jayzhan211 the PR is ready for review https://github.com/apache/datafusion/pull/10392, but the syntax is bigquery-liked. Do you need the syntax of duckdb to complete this i

Re: [PR] fix: null character not permitted in chr function [datafusion-comet]

2024-06-08 Thread via GitHub
viirya commented on code in PR #513: URL: https://github.com/apache/datafusion-comet/pull/513#discussion_r1631927233 ## core/src/execution/datafusion/expressions/scalar_funcs/chr.rs: ## @@ -0,0 +1,108 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [PR] fix: null character not permitted in chr function [datafusion-comet]

2024-06-08 Thread via GitHub
vaibhawvipul commented on code in PR #513: URL: https://github.com/apache/datafusion-comet/pull/513#discussion_r1631946555 ## core/src/execution/datafusion/expressions/scalar_funcs/chr.rs: ## @@ -0,0 +1,108 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

[PR] Chained aliases in plans [datafusion]

2024-06-08 Thread via GitHub
MohamedAbdeen21 opened a new pull request, #10832: URL: https://github.com/apache/datafusion/pull/10832 ## Which issue does this PR close? Closes #. ## Rationale for this change Removing unnecessary aliasing in the logical plan ## What changes are i

Re: [I] Convert `stddev` to udaf [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on issue #10827: URL: https://github.com/apache/datafusion/issues/10827#issuecomment-2155938062 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Support explicit type and name during table creation [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on PR #10273: URL: https://github.com/apache/datafusion/pull/10273#issuecomment-2155984401 > hi @jayzhan211 the PR is ready for review, but the syntax is bigquery-liked. Do you need the syntax of duckdb to complete this issue? It would be nice if we support duckdb

Re: [I] `extract` doesn't accept quoted field names [datafusion]

2024-06-08 Thread via GitHub
Weijun-H commented on issue #10826: URL: https://github.com/apache/datafusion/issues/10826#issuecomment-2156002605 It should be related to the sqlparse. And I also found. ``` shell > SELECT extract("day" FROM now()); Execution error: Date part '"day"' not supported ``` -- This

[PR] fix: Support double quotes in date_part.rs [datafusion]

2024-06-08 Thread via GitHub
Weijun-H opened a new pull request, #10833: URL: https://github.com/apache/datafusion/pull/10833 ## Which issue does this PR close? Parts #10826 ## Rationale for this change ## What changes are included in this PR? ## Are these changes teste

Re: [PR] fix: Support double quotes in date_part.rs [datafusion]

2024-06-08 Thread via GitHub
Weijun-H commented on code in PR #10833: URL: https://github.com/apache/datafusion/pull/10833#discussion_r1632033037 ## datafusion/functions/src/datetime/date_part.rs: ## @@ -127,7 +127,7 @@ impl ScalarUDFImpl for DatePartFunc { ColumnarValue::Scalar(scalar) => scal

Re: [PR] Remove Built-in sum and Rename to lowercase `sum` [datafusion]

2024-06-08 Thread via GitHub
alamb commented on code in PR #10831: URL: https://github.com/apache/datafusion/pull/10831#discussion_r1632033601 ## datafusion/sql/Cargo.toml: ## @@ -54,6 +54,7 @@ strum = { version = "0.26.1", features = ["derive"] } [dev-dependencies] ctor = { workspace = true } datafusion

Re: [PR] Make Logical Plans more readable by removing extra aliases [datafusion]

2024-06-08 Thread via GitHub
peter-toth commented on code in PR #10832: URL: https://github.com/apache/datafusion/pull/10832#discussion_r1632034489 ## datafusion/optimizer/src/common_subexpr_eliminate.rs: ## @@ -696,7 +696,8 @@ impl<'n> TreeNodeVisitor<'n> for ExprIdentifierVisitor<'_> { return

Re: [PR] fix: Support double quotes in `date_part` [datafusion]

2024-06-08 Thread via GitHub
alamb commented on code in PR #10833: URL: https://github.com/apache/datafusion/pull/10833#discussion_r1632034345 ## datafusion/functions/src/datetime/date_part.rs: ## @@ -127,7 +127,7 @@ impl ScalarUDFImpl for DatePartFunc { ColumnarValue::Scalar(scalar) => scalar.

Re: [PR] Debug print as many fields as possible for `SessionState` [datafusion]

2024-06-08 Thread via GitHub
alamb merged PR #10818: URL: https://github.com/apache/datafusion/pull/10818 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Debug print as many fields as possible for `SessionState` [datafusion]

2024-06-08 Thread via GitHub
alamb commented on PR #10818: URL: https://github.com/apache/datafusion/pull/10818#issuecomment-2156015303 Thanks again @lewiszlw -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Prune Parquet RowGroup in a single call to `PruningPredicate::prune`, update StatisticsExtractor API [datafusion]

2024-06-08 Thread via GitHub
alamb commented on PR #10802: URL: https://github.com/apache/datafusion/pull/10802#issuecomment-2156016560 Thank you @NGA-TRAN @xinlifoobar and @waynexia for the reviews -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Prune Parquet RowGroup in a single call to `PruningPredicate::prune`, update StatisticsExtractor API [datafusion]

2024-06-08 Thread via GitHub
alamb merged PR #10802: URL: https://github.com/apache/datafusion/pull/10802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Minor: consider precision/length parameter for varchar/char types [datafusion]

2024-06-08 Thread via GitHub
alamb commented on code in PR #10746: URL: https://github.com/apache/datafusion/pull/10746#discussion_r1632035977 ## datafusion/sql/src/expr/mod.rs: ## @@ -274,6 +276,34 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> { if let Some(format) = format {

Re: [PR] fix: Support double quotes in `date_part` [datafusion]

2024-06-08 Thread via GitHub
Weijun-H commented on PR #10833: URL: https://github.com/apache/datafusion/pull/10833#issuecomment-2156019628 > Just curious, does this let you get away with something funky like: > > ```sql > select extract('''epoch''' from now()); > ``` It seems the invalid input @Jefffr

Re: [PR] Make Logical Plans more readable by removing extra aliases [datafusion]

2024-06-08 Thread via GitHub
MohamedAbdeen21 commented on code in PR #10832: URL: https://github.com/apache/datafusion/pull/10832#discussion_r1632036676 ## datafusion/optimizer/src/common_subexpr_eliminate.rs: ## @@ -696,7 +696,8 @@ impl<'n> TreeNodeVisitor<'n> for ExprIdentifierVisitor<'_> { r

Re: [I] Precision/length parameter of varchar/char types is ignored [datafusion]

2024-06-08 Thread via GitHub
alamb commented on issue #10743: URL: https://github.com/apache/datafusion/issues/10743#issuecomment-2156019945 Maybe we should simply error if DataFusion encouters a type of `VARCHAR` with an explcit length 🤔 Though if I worry that some people might expect it to work the way it currenty d

Re: [PR] Support user defined `ParquetAccessPlan` in `ParquetExec`, validation to `ParquetAccessPlan::select` [datafusion]

2024-06-08 Thread via GitHub
Jefffrey commented on code in PR #10813: URL: https://github.com/apache/datafusion/pull/10813#discussion_r1632036259 ## datafusion/core/src/datasource/physical_plan/parquet/opener.rs: ## @@ -212,3 +213,34 @@ impl FileOpener for ParquetOpener { })) } } + +/// Retur

Re: [PR] Make Logical Plans more readable by removing extra aliases [datafusion]

2024-06-08 Thread via GitHub
peter-toth commented on code in PR #10832: URL: https://github.com/apache/datafusion/pull/10832#discussion_r1632037636 ## datafusion/optimizer/src/common_subexpr_eliminate.rs: ## @@ -782,9 +786,11 @@ impl TreeNodeRewriter for CommonSubexprRewriter<'_> { return Ok(Tr

Re: [PR] chore: Create initial release process scripts for official ASF source release [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on PR #429: URL: https://github.com/apache/datafusion-comet/pull/429#issuecomment-2156022398 @viirya @kazuyukitanimura @huaxingao could I get a committer review/approval -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] fix: Support double quotes in `date_part` [datafusion]

2024-06-08 Thread via GitHub
Jefffrey commented on PR #10833: URL: https://github.com/apache/datafusion/pull/10833#issuecomment-2156022903 > > Just curious, does this let you get away with something funky like: > > ```sql > > select extract('''epoch''' from now()); > > ``` > > It seems the invalid input @

Re: [PR] Make Logical Plans more readable by removing extra aliases [datafusion]

2024-06-08 Thread via GitHub
MohamedAbdeen21 commented on code in PR #10832: URL: https://github.com/apache/datafusion/pull/10832#discussion_r1632038405 ## datafusion/optimizer/src/common_subexpr_eliminate.rs: ## @@ -782,9 +786,11 @@ impl TreeNodeRewriter for CommonSubexprRewriter<'_> { return

Re: [PR] Make Logical Plans more readable by removing extra aliases [datafusion]

2024-06-08 Thread via GitHub
peter-toth commented on code in PR #10832: URL: https://github.com/apache/datafusion/pull/10832#discussion_r1632037636 ## datafusion/optimizer/src/common_subexpr_eliminate.rs: ## @@ -782,9 +786,11 @@ impl TreeNodeRewriter for CommonSubexprRewriter<'_> { return Ok(Tr

Re: [PR] Make Logical Plans more readable by removing extra aliases [datafusion]

2024-06-08 Thread via GitHub
MohamedAbdeen21 commented on code in PR #10832: URL: https://github.com/apache/datafusion/pull/10832#discussion_r1632038405 ## datafusion/optimizer/src/common_subexpr_eliminate.rs: ## @@ -782,9 +786,11 @@ impl TreeNodeRewriter for CommonSubexprRewriter<'_> { return

Re: [PR] Make Logical Plans more readable by removing extra aliases [datafusion]

2024-06-08 Thread via GitHub
MohamedAbdeen21 commented on code in PR #10832: URL: https://github.com/apache/datafusion/pull/10832#discussion_r1632038405 ## datafusion/optimizer/src/common_subexpr_eliminate.rs: ## @@ -782,9 +786,11 @@ impl TreeNodeRewriter for CommonSubexprRewriter<'_> { return

Re: [PR] Make Logical Plans more readable by removing extra aliases [datafusion]

2024-06-08 Thread via GitHub
peter-toth commented on code in PR #10832: URL: https://github.com/apache/datafusion/pull/10832#discussion_r1632039885 ## datafusion/optimizer/src/common_subexpr_eliminate.rs: ## @@ -782,9 +786,11 @@ impl TreeNodeRewriter for CommonSubexprRewriter<'_> { return Ok(Tr

[I] Add support for Spark 3.5 [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove opened a new issue, #541: URL: https://github.com/apache/datafusion-comet/issues/541 ### What is the problem the feature request solves? _No response_ ### Describe the potential solution _No response_ ### Additional context _No response_ -- This

[PR] fix: Overflow when reading Timestamp from parquet file [datafusion-comet]

2024-06-08 Thread via GitHub
eejbyfeldt opened a new pull request, #542: URL: https://github.com/apache/datafusion-comet/pull/542 ## Which issue does this PR close? Closes #481. ## Rationale for this change When spark reads and writes timestamps in parquet file it using the following code:

Re: [PR] Remove Built-in sum and Rename to lowercase `sum` [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on PR #10831: URL: https://github.com/apache/datafusion/pull/10831#issuecomment-2156032850 > If we are going to go with this approach, perhaps we could remove the stubs added in https://github.com/apache/datafusion/pull/10816 🤔 It is nice if we don't need stubs 👍

Re: [PR] fix: Support double quotes in `date_part` [datafusion]

2024-06-08 Thread via GitHub
Weijun-H commented on PR #10833: URL: https://github.com/apache/datafusion/pull/10833#issuecomment-2156033118 > > > Just curious, does this let you get away with something funky like: > > > ```sql > > > select extract('''epoch''' from now()); > > > ``` > > > > > > It seems

Re: [I] shuffle write fails with out of range slice [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on issue #540: URL: https://github.com/apache/datafusion-comet/issues/540#issuecomment-2156039206 > @andygrove Is there any reproducible example? I was running TPC-H @ 100gb -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] range end index 294912 out of range for slice of length 147456 [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on issue #540: URL: https://github.com/apache/datafusion-comet/issues/540#issuecomment-2156041758 It failed on q10. I am adding debug logging to see if I can track this down. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Convert builtin Sum aggregate function to UDAF [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 closed issue #10731: Convert builtin Sum aggregate function to UDAF URL: https://github.com/apache/datafusion/issues/10731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Remove Built-in sum and Rename to lowercase `sum` [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 merged PR #10831: URL: https://github.com/apache/datafusion/pull/10831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dat

Re: [PR] Remove Built-in sum and Rename to lowercase `sum` [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on PR #10831: URL: https://github.com/apache/datafusion/pull/10831#issuecomment-2156043600 Thanks @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Remove Built-in sum and Rename to lowercase `sum` [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on PR #10831: URL: https://github.com/apache/datafusion/pull/10831#issuecomment-2156043495 I found it is not too trivial and need review, so I will merge this one first and file another PR for removing stubs -- This is an automated message from the Apache Git Service.

Re: [PR] Remove Built-in sum and Rename to lowercase `sum` [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on PR #10831: URL: https://github.com/apache/datafusion/pull/10831#issuecomment-2156053741 It seems I cant remove stub for test in `datafusion/expr/src/utils.rs` because we can't import datafusion-aggregate in dev-dependency. We should move the test to other crate befor

Re: [I] Convert `stddev` to udaf [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on issue #10827: URL: https://github.com/apache/datafusion/issues/10827#issuecomment-2156058041 Hi @jayzhan211, Although this issue only mentions `stddev`, I'm thinking about `stddev_pop`. Should I also convert it to UDAF? I found that #10713 only converted `var_sampl

Re: [I] Convert `stddev` to udaf [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on issue #10827: URL: https://github.com/apache/datafusion/issues/10827#issuecomment-2156062641 > Hi @jayzhan211, Although this issue only mentions `stddev`, I'm thinking about `stddev_pop`. Should I also convert it to UDAF? I found that #10713 only converted `var_sampl

Re: [I] Convert `stddev` to udaf [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on issue #10827: URL: https://github.com/apache/datafusion/issues/10827#issuecomment-2156063570 Got it. I think I can convert them in the same PR. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] range end index 294912 out of range for slice of length 147456 [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on issue #540: URL: https://github.com/apache/datafusion-comet/issues/540#issuecomment-2156065507 The error is happening in unsafe code in arrow-rs. Here is some debug output showing the calls leading up the the error: ``` copy_or_cast_array() len=8192, t

Re: [I] range end index 294912 out of range for slice of length 147456 [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on issue #540: URL: https://github.com/apache/datafusion-comet/issues/540#issuecomment-2156072408 I ran with a debug build and got a more detailed stack trace: ``` at core::panicking::panic_fmt(/rustc/25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04/library/cor

Re: [I] range end index 294912 out of range for slice of length 147456 [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on issue #540: URL: https://github.com/apache/datafusion-comet/issues/540#issuecomment-2156093191 Here is some more debug info, showing the size of the buffers in the array data and the value in the first element of each buffer: ``` copy_array(typeInt64

[PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
goldmedal opened a new pull request, #10834: URL: https://github.com/apache/datafusion/pull/10834 ## Which issue does this PR close? Closes #10827 . I converted `stddev_pop` to UDAF, too. ## Rationale for this change ## What changes are included in this

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on code in PR #10834: URL: https://github.com/apache/datafusion/pull/10834#discussion_r1632069853 ## datafusion/physical-expr/src/aggregate/stddev.rs: ## @@ -17,168 +17,14 @@ //! Defines physical expressions that can evaluated at runtime during query exec

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on code in PR #10834: URL: https://github.com/apache/datafusion/pull/10834#discussion_r1632070035 ## datafusion/physical-expr/src/aggregate/build_in.rs: ## @@ -939,32 +847,6 @@ mod tests { assert!(observed.is_err()); } -#[test] -fn tes

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on code in PR #10834: URL: https://github.com/apache/datafusion/pull/10834#discussion_r1632070143 ## datafusion/physical-expr/src/aggregate/build_in.rs: ## @@ -747,82 +731,6 @@ mod tests { Ok(()) } -#[test] -fn test_stddev_expr() -> Re

Re: [I] Error `NamedStructField should be rewritten in OperatorToFunction with subquery` if query is wrapped in view [datafusion]

2024-06-08 Thread via GitHub
ahirner commented on issue #10764: URL: https://github.com/apache/datafusion/issues/10764#issuecomment-2156104504 Can confirm that it's gone for our more complicated use case in `8fcb3e4b`, thus also probably in `0.39`. Was or should it be covered in a test? Otherwise feel free to close.

Re: [I] Make repartitioning in `PhysicalPlan` output less confusing [datafusion]

2024-06-08 Thread via GitHub
edmondop commented on issue #9370: URL: https://github.com/apache/datafusion/issues/9370#issuecomment-2156109681 I spent a bit of extra time on this and I have some thoughts worth sharing. `pull_from_input` is the task that pulls the record and send them to the ouput channels. However

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
comphead commented on code in PR #10834: URL: https://github.com/apache/datafusion/pull/10834#discussion_r1632080337 ## datafusion/functions-aggregate/src/stddev.rs: ## @@ -0,0 +1,380 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licen

Re: [PR] chore: Create initial release process scripts for official ASF source release [datafusion-comet]

2024-06-08 Thread via GitHub
comphead commented on code in PR #429: URL: https://github.com/apache/datafusion-comet/pull/429#discussion_r1632081084 ## dev/release/verify-release-candidate.sh: ## @@ -0,0 +1,133 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contri

Re: [PR] chore: Create initial release process scripts for official ASF source release [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove merged PR #429: URL: https://github.com/apache/datafusion-comet/pull/429 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@da

Re: [PR] chore: Create initial release process scripts for official ASF source release [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on code in PR #429: URL: https://github.com/apache/datafusion-comet/pull/429#discussion_r1632081896 ## dev/release/verify-release-candidate.sh: ## @@ -0,0 +1,133 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contr

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on code in PR #10834: URL: https://github.com/apache/datafusion/pull/10834#discussion_r1632082931 ## datafusion/functions-aggregate/src/stddev.rs: ## @@ -0,0 +1,380 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor lice

Re: [PR] Introduce expr builder for aggregate function [datafusion]

2024-06-08 Thread via GitHub
alamb commented on PR #10560: URL: https://github.com/apache/datafusion/pull/10560#issuecomment-2156133115 > > I have some ideas about additional comments / documentation that I would be happy to help add > > Sure! Thank you for your patience @jayzhan211 -- I just pushed a bun

Re: [I] Make it easier to create WindowFunctions with the Expr API [datafusion]

2024-06-08 Thread via GitHub
alamb commented on issue #6747: URL: https://github.com/apache/datafusion/issues/6747#issuecomment-2156133859 Update here is that @jayzhan211 and I have been working on a similar API for creating `Aggregate` exprs on https://github.com/apache/datafusion/pull/10560. I am quite pleased with h

Re: [I] AggregateUDF expression API design [datafusion]

2024-06-08 Thread via GitHub
alamb commented on issue #10545: URL: https://github.com/apache/datafusion/issues/10545#issuecomment-2156133971 https://github.com/apache/datafusion/pull/10560 is looking sweet to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[PR] Stop copying LogicalPlan and Exprs in `CommonSubexprEliminate` [datafusion]

2024-06-08 Thread via GitHub
alamb opened a new pull request, #10835: URL: https://github.com/apache/datafusion/pull/10835 ## Which issue does this PR close? Closes https://github.com/apache/datafusion/issues/9873 Closes https://github.com/apache/arrow-datafusion/issues/9637 -- the last one of my planned

Re: [PR] Rewrite `CommonSubexprEliminate` to avoid copies using TreeNode [datafusion]

2024-06-08 Thread via GitHub
alamb commented on PR #10067: URL: https://github.com/apache/datafusion/pull/10067#issuecomment-2156135199 Superceded by https://github.com/apache/datafusion/pull/10835, will continue work there -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Rewrite `CommonSubexprEliminate` to avoid copies using TreeNode [datafusion]

2024-06-08 Thread via GitHub
alamb closed pull request #10067: Rewrite `CommonSubexprEliminate` to avoid copies using TreeNode URL: https://github.com/apache/datafusion/pull/10067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] fix: Return error in pre_timestamp_cast instead of panic [datafusion-comet]

2024-06-08 Thread via GitHub
eejbyfeldt opened a new pull request, #543: URL: https://github.com/apache/datafusion-comet/pull/543 ## Which issue does this PR close? Resolves issue discovered in #481 . ## Rationale for this change Datafusion and coment uses `chrono::Datetime` for many of it d

[PR] Convert `VariancePopulation` to UDAF [datafusion]

2024-06-08 Thread via GitHub
mknaw opened a new pull request, #10836: URL: https://github.com/apache/datafusion/pull/10836 ## Which issue does this PR close? Closes #10668 ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested?

[PR] Fix error message [datafusion-comet]

2024-06-08 Thread via GitHub
comphead opened a new pull request, #544: URL: https://github.com/apache/datafusion-comet/pull/544 ## Which issue does this PR close? Closes #293 . ## Rationale for this change ## What changes are included in this PR? ## How are these change

Re: [PR] Convert `VariancePopulation` to UDAF [datafusion]

2024-06-08 Thread via GitHub
mknaw commented on code in PR #10836: URL: https://github.com/apache/datafusion/pull/10836#discussion_r1632099605 ## datafusion/physical-expr/src/aggregate/build_in.rs: ## @@ -709,44 +703,6 @@ mod tests { Ok(()) } -#[test] -fn test_var_pop_expr() -> Resul

Re: [I] Convert Variance Population to UDAF [datafusion]

2024-06-08 Thread via GitHub
mknaw commented on issue #10668: URL: https://github.com/apache/datafusion/issues/10668#issuecomment-2156156982 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [I] Integrate with the substrait integration test [datafusion]

2024-06-08 Thread via GitHub
Lordworms commented on issue #10710: URL: https://github.com/apache/datafusion/issues/10710#issuecomment-2156188075 While I was adding this feature, I encountered lots of issues while trying to generate LogicalPlan from substrait https://github.com/substrait-io/consumer-testing/tree/main

Re: [PR] Upgrade maturin [datafusion-python]

2024-06-08 Thread via GitHub
andygrove merged PR #725: URL: https://github.com/apache/datafusion-python/pull/725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@d

Re: [PR] feat: Fix Comet error message [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on PR #544: URL: https://github.com/apache/datafusion-comet/pull/544#issuecomment-2156211661 Thanks @comphead. LGTM. Could you also update this section in `CometCastSuite`: ``` // We have to workaround https://github.com/apache/datafusion-comet/issu

[PR] chore: Add changelog generator [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove opened a new pull request, #545: URL: https://github.com/apache/datafusion-comet/pull/545 ## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## How are these changes t

Re: [PR] chore: Add changelog generator [datafusion-comet]

2024-06-08 Thread via GitHub
andygrove commented on code in PR #545: URL: https://github.com/apache/datafusion-comet/pull/545#discussion_r1632135709 ## dev/changelog/0.1.0.md: ## @@ -0,0 +1,344 @@ + + +# DataFusion Comet 0.1.0 Changelog Review Comment: This is an example of the generated changelog. I ad

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on code in PR #10834: URL: https://github.com/apache/datafusion/pull/10834#discussion_r1632136912 ## datafusion/functions-aggregate/src/stddev.rs: ## @@ -0,0 +1,380 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor lic

[I] Convert `approx_distinct` to UDAF [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 opened a new issue, #10837: URL: https://github.com/apache/datafusion/issues/10837 ### Is your feature request related to a problem or challenge? Similar to #10713 #10836 #10834 and others in #8708 ### Describe the solution you'd like _No response_ ###

[I] Convert `approx_median` to UDAF [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 opened a new issue, #10838: URL: https://github.com/apache/datafusion/issues/10838 ### Is your feature request related to a problem or challenge? Similar to https://github.com/apache/datafusion/pull/10713 https://github.com/apache/datafusion/pull/10836 https://github.com/a

Re: [PR] Convert `VariancePopulation` to UDAF [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on PR #10836: URL: https://github.com/apache/datafusion/pull/10836#issuecomment-2156247362 I think you need to comment out `variance pop` in proto https://github.com/apache/datafusion/blob/6b7021479c956ba3ca2a04fff81487cd57f80624/datafusion/proto/proto/datafusion.

Re: [PR] feat: explicit implementation for union's required_input_ordering [datafusion]

2024-06-08 Thread via GitHub
github-actions[bot] closed pull request #9867: feat: explicit implementation for union's required_input_ordering URL: https://github.com/apache/datafusion/pull/9867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] Docs: Add `unnest` to SQL Reference [datafusion]

2024-06-08 Thread via GitHub
gloomweaver opened a new pull request, #10839: URL: https://github.com/apache/datafusion/pull/10839 ## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## Are these changes teste

Re: [I] range end index 294912 out of range for slice of length 147456 [datafusion-comet]

2024-06-08 Thread via GitHub
viirya commented on issue #540: URL: https://github.com/apache/datafusion-comet/issues/540#issuecomment-2156272695 Ok. I tried TPC-H @ 100gb and TPC-H @ 1gb. Only TPC-H @ 100gb can reproduce it. I will look into this. -- This is an automated message from the Apache Git Service. To respon

Re: [I] Apply guarantee rewriter to sql workflow [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on issue #10456: URL: https://github.com/apache/datafusion/issues/10456#issuecomment-2156277714 @dmitrybugakov are you working on #10510? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on code in PR #10834: URL: https://github.com/apache/datafusion/pull/10834#discussion_r1632154001 ## datafusion/functions-aggregate/src/stddev.rs: ## @@ -0,0 +1,380 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor lice

Re: [I] Convert `approx_median` to UDAF [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on issue #10838: URL: https://github.com/apache/datafusion/issues/10838#issuecomment-2156291208 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on PR #10834: URL: https://github.com/apache/datafusion/pull/10834#issuecomment-2156295747 Thanks @goldmedal and @comphead -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 merged PR #10834: URL: https://github.com/apache/datafusion/pull/10834 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dat

Re: [I] Convert `stddev` to udaf [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 closed issue #10827: Convert `stddev` to udaf URL: https://github.com/apache/datafusion/issues/10827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] fix: null character not permitted in chr function [datafusion-comet]

2024-06-08 Thread via GitHub
viirya commented on code in PR #513: URL: https://github.com/apache/datafusion-comet/pull/513#discussion_r1632159142 ## core/src/execution/datafusion/expressions/scalar_funcs/chr.rs: ## @@ -94,15 +94,43 @@ impl ScalarUDFImpl for ChrFunc { } fn invoke(&self, args: &[C

Re: [PR] Convert `stddev` and `stddev_pop` to UDAF [datafusion]

2024-06-08 Thread via GitHub
goldmedal commented on PR #10834: URL: https://github.com/apache/datafusion/pull/10834#issuecomment-2156299124 Thanks again @jayzhan211 @comphead -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] bug: requested character too large for encoding in chr function [datafusion-comet]

2024-06-08 Thread via GitHub
vaibhawvipul commented on issue #480: URL: https://github.com/apache/datafusion-comet/issues/480#issuecomment-2156314547 This is extension to #479 and my PR #513 . Hence, I will work on this. -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] fix: null character not permitted in chr function [datafusion-comet]

2024-06-08 Thread via GitHub
vaibhawvipul commented on code in PR #513: URL: https://github.com/apache/datafusion-comet/pull/513#discussion_r1632167007 ## core/src/execution/datafusion/expressions/scalar_funcs/chr.rs: ## @@ -94,15 +94,43 @@ impl ScalarUDFImpl for ChrFunc { } fn invoke(&self, arg

Re: [PR] Introduce expr builder for aggregate function [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 commented on PR #10560: URL: https://github.com/apache/datafusion/pull/10560#issuecomment-2156333141 It looks pretty nice now! Thanks @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Introduce expr builder for aggregate function [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 merged PR #10560: URL: https://github.com/apache/datafusion/pull/10560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dat

Re: [I] AggregateUDF expression API design [datafusion]

2024-06-08 Thread via GitHub
jayzhan211 closed issue #10545: AggregateUDF expression API design URL: https://github.com/apache/datafusion/issues/10545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

  1   2   >