Re: [I] Reorganize the GroupColumn implementations into more coherent modules [datafusion]

2024-11-10 Thread via GitHub
jayzhan211 commented on issue #13262: URL: https://github.com/apache/datafusion/issues/13262#issuecomment-2467462703 @Chen-Yuan-Lai Have you started on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
berkaysynnada commented on code in PR #13201: URL: https://github.com/apache/datafusion/pull/13201#discussion_r1836070735 ## datafusion/proto/tests/cases/roundtrip_physical_plan.rs: ## @@ -272,31 +269,34 @@ fn roundtrip_window() -> Result<()> { let field_b = Field::new("b",

[PR] Support TypeSignature::ZeroArg [datafusion]

2024-11-10 Thread via GitHub
jayzhan211 opened a new pull request, #13354: URL: https://github.com/apache/datafusion/pull/13354 ## Which issue does this PR close? Closes #. ## Rationale for this change Currently there are uniform(0), any(0), expect(0) 3 ways to describe zero argument signatu

[PR] parquet:Add file_extension for specify file_extension of ParquetReadOptions [datafusion]

2024-11-10 Thread via GitHub
glfeng318 opened a new pull request, #13353: URL: https://github.com/apache/datafusion/pull/13353 ## Which issue does this PR close? ## Rationale for this change This allows users to specify the file extendsion in ParquetReadOptions ``` let opt = ParquetReadOptions::new().fil

Re: [PR] Reorganize the GroupColumn Implemenations [datafusion]

2024-11-10 Thread via GitHub
jiashenC commented on PR #13352: URL: https://github.com/apache/datafusion/pull/13352#issuecomment-2467389211 @alamb Can you please take a look at this PR? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] support to_timestamp with optional chrono formats [datafusion]

2024-11-10 Thread via GitHub
jayzhan211 commented on PR #8886: URL: https://github.com/apache/datafusion/pull/8886#issuecomment-2467340441 Tracked in https://github.com/apache/datafusion/issues/13351 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[I] Ensure `to_timestamp` behaves consistently with PostgreSQL [datafusion]

2024-11-10 Thread via GitHub
jayzhan211 opened a new issue, #13351: URL: https://github.com/apache/datafusion/issues/13351 ### Is your feature request related to a problem or challenge? In postgres, `to_timestamp` has signature `to_timestamp ( text, text ) → timestamp with time zone` and `to_timestamp ( double pr

Re: [PR] support to_timestamp with optional chrono formats [datafusion]

2024-11-10 Thread via GitHub
jayzhan211 commented on PR #8886: URL: https://github.com/apache/datafusion/pull/8886#issuecomment-2467332555 @Omega359 Do you know why `to_timestamp` has more than 2 arguments? What behaviour does this function follow? In Postgres, there are two arguments ``` to_timestamp ( tex

Re: [PR] feat: support normalized expr in CSE [datafusion]

2024-11-10 Thread via GitHub
jayzhan211 commented on code in PR #13315: URL: https://github.com/apache/datafusion/pull/13315#discussion_r1835961051 ## datafusion/expr/src/expr.rs: ## @@ -1666,6 +1666,69 @@ impl Expr { } } +impl NormalizeNode for Expr { +fn enable_normalized(&self) -> bool { +

Re: [I] Detect stack overflow and reduce stack usage on debug build [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
Eason0729 commented on issue #1465: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1465#issuecomment-2467246247 I will try to reproduce it with https://github.com/apache/datafusion/pull/13177 next week, chances are that it's related. -- This is an automated message from t

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835721679 ## src/parser/mod.rs: ## @@ -11141,19 +11154,34 @@ impl<'a> Parser<'a> { /// FIRST_VALUE(x IGNORE NULL); /// ``` fn parse_function_arg

Re: [I] Type coercion for binary operators forces UTF8View to Utf8 [datafusion]

2024-11-10 Thread via GitHub
jayzhan211 commented on issue #13348: URL: https://github.com/apache/datafusion/issues/13348#issuecomment-2467170531 I think change the signature to TypeSignature::String can solve this issue. See #13301 -- This is an automated message from the Apache Git Service. To respond to the messa

[PR] Add Utf8View to `TypeCategory::Unknown` [datafusion]

2024-11-10 Thread via GitHub
jayzhan211 opened a new pull request, #13350: URL: https://github.com/apache/datafusion/pull/13350 ## Which issue does this PR close? Closes #13347 . ## Rationale for this change ## What changes are included in this PR? ## Are these changes

Re: [I] `cargo build --no-default-features` does not build cleanly [datafusion]

2024-11-10 Thread via GitHub
jonahgao commented on issue #8844: URL: https://github.com/apache/datafusion/issues/8844#issuecomment-2467150295 I have verified that this issue has been resolved on [main](https://github.com/apache/datafusion/commit/7ebd993fe47e78006bad33256160716f73b13cef). ```sh $ cargo build --no-d

Re: [I] `cargo build --no-default-features` does not build cleanly [datafusion]

2024-11-10 Thread via GitHub
jonahgao closed issue #8844: `cargo build --no-default-features` does not build cleanly URL: https://github.com/apache/datafusion/issues/8844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[PR] feat: Add random timezone to aggregate fuzz test [datafusion]

2024-11-10 Thread via GitHub
jonathanc-n opened a new pull request, #13349: URL: https://github.com/apache/datafusion/pull/13349 ## Which issue does this PR close? Closes #13302 . ## Rationale for this change ## What changes are included in this PR? Added random timezones to fuzz t

Re: [I] Add support for string view to a few functions [datafusion]

2024-11-10 Thread via GitHub
Omega359 commented on issue #13330: URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2467120727 regexp_match looks to not support stringview either. ``` // TODO: Native support Utf8View for regexp_match. TypeSignature::Exact(vec![Utf8, Utf8]),

Re: [PR] feat: support array_append [datafusion-comet]

2024-11-10 Thread via GitHub
andygrove commented on code in PR #1072: URL: https://github.com/apache/datafusion-comet/pull/1072#discussion_r1835886113 ## spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala: ## @@ -2313,4 +2313,22 @@ class CometExpressionSuite extends CometTestBase with Adaptiv

Re: [PR] With Order Support for Memory Tables [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
github-actions[bot] commented on PR #1401: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1401#issuecomment-2467087404 Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or

Re: [I] coalesce function fails in planning if any argument is Utf8View type [datafusion]

2024-11-10 Thread via GitHub
Omega359 commented on issue #13347: URL: https://github.com/apache/datafusion/issues/13347#issuecomment-2467036172 Related to and possibly caused by #12864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Add support for string view to a few functions [datafusion]

2024-11-10 Thread via GitHub
timsaucer commented on issue #13330: URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2466977633 Related: https://github.com/apache/datafusion/issues/13347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] added a BallistaContext to ballista to allow for Remote or standalone [datafusion-ballista]

2024-11-10 Thread via GitHub
tbar4 commented on code in PR #1100: URL: https://github.com/apache/datafusion-ballista/pull/1100#discussion_r1835799586 ## python/src/lib.rs: ## @@ -15,18 +15,66 @@ // specific language governing permissions and limitations // under the License. +use ballista::prelude::*;

[I] coalesce function fails in planning if any argument is Utf8View type [datafusion]

2024-11-10 Thread via GitHub
Omega359 opened a new issue, #13347: URL: https://github.com/apache/datafusion/issues/13347 ### Describe the bug ``` query T select COALESCE(arrow_cast(null, 'Utf8'), arrow_cast('t', 'Utf8View')); t ``` External error: query failed: DataFusion error: Error during

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
buraksenn commented on code in PR #13201: URL: https://github.com/apache/datafusion/pull/13201#discussion_r1835784248 ## datafusion/core/tests/fuzz_cases/window_fuzz.rs: ## @@ -415,36 +414,6 @@ fn get_random_function( ), ); } -window_fn_map.insert(

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
buraksenn commented on code in PR #13201: URL: https://github.com/apache/datafusion/pull/13201#discussion_r1835785304 ## datafusion/physical-plan/src/windows/bounded_window_agg_exec.rs: ## @@ -1500,129 +1494,6 @@ mod tests { Ok(source) } -// Tests NTH_VALUE(n

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
buraksenn commented on code in PR #13201: URL: https://github.com/apache/datafusion/pull/13201#discussion_r1835785085 ## datafusion/proto/tests/cases/roundtrip_physical_plan.rs: ## @@ -272,31 +269,34 @@ fn roundtrip_window() -> Result<()> { let field_b = Field::new("b", Dat

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
buraksenn commented on code in PR #13201: URL: https://github.com/apache/datafusion/pull/13201#discussion_r1835784521 ## datafusion/expr/src/built_in_window_function.rs: ## @@ -17,115 +17,12 @@ //! Built-in functions module contains all the built-in functions definitions. -

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
buraksenn commented on code in PR #13201: URL: https://github.com/apache/datafusion/pull/13201#discussion_r1835784628 ## datafusion/functions-window/src/nth_value.rs: ## @@ -15,143 +15,254 @@ // specific language governing permissions and limitations // under the License. -/

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
buraksenn commented on code in PR #13201: URL: https://github.com/apache/datafusion/pull/13201#discussion_r1835767976 ## datafusion/sqllogictest/test_files/errors.slt: ## @@ -120,7 +120,7 @@ from aggregate_test_100 order by c9 # WindowFunction with BuiltInWindowFunction wron

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
buraksenn commented on code in PR #13201: URL: https://github.com/apache/datafusion/pull/13201#discussion_r1835766292 ## datafusion/proto/proto/datafusion.proto: ## @@ -507,17 +507,6 @@ message ScalarUDFExprNode { enum BuiltInWindowFunction { UNSPECIFIED = 0; // https://p

[PR] build(deps): bump conda-incubator/setup-miniconda from 3.0.4 to 3.1.0 [datafusion-python]

2024-11-10 Thread via GitHub
dependabot[bot] opened a new pull request, #947: URL: https://github.com/apache/datafusion-python/pull/947 Bumps [conda-incubator/setup-miniconda](https://github.com/conda-incubator/setup-miniconda) from 3.0.4 to 3.1.0. Release notes Sourced from https://github.com/conda-incubator

Re: [PR] Convert nth_value builtIn function to UDWF [datafusion]

2024-11-10 Thread via GitHub
buraksenn commented on PR #13201: URL: https://github.com/apache/datafusion/pull/13201#issuecomment-2466856184 > Let us know if you need help Sorry @alamb this dragged on much longer than I anticipated. I could not find time for the recent days and by opening draft PR I kinda blocked

Re: [PR] feat: support array_append [datafusion-comet]

2024-11-10 Thread via GitHub
NoeB commented on code in PR #1072: URL: https://github.com/apache/datafusion-comet/pull/1072#discussion_r1835759807 ## spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala: ## @@ -2313,4 +2313,22 @@ class CometExpressionSuite extends CometTestBase with AdaptiveSpar

Re: [PR] RFC: Add `Precision:AtLeast` and `Precision::AtMost` for more `Statistics`… precision [datafusion]

2024-11-10 Thread via GitHub
findepi commented on PR #13293: URL: https://github.com/apache/datafusion/pull/13293#issuecomment-2466823977 Absolutely, the exact bounds must not be mistaken with estimates/statistics and need to be treated as two different things, even though they look similar. Known data bounds (min/ma

Re: [PR] Support unparsing plans after applying `optimize_projections` rule [datafusion]

2024-11-10 Thread via GitHub
findepi commented on PR #13267: URL: https://github.com/apache/datafusion/pull/13267#issuecomment-2466822596 > With projections optimization we propagate required columns to child nodes so only required columns could be fetched. makes sense! this is important, agreed can yo

Re: [PR] Add source release scripts [datafusion-ray]

2024-11-10 Thread via GitHub
andygrove merged PR #40: URL: https://github.com/apache/datafusion-ray/pull/40 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafu

Re: [PR] Migrate code from invoke to invoke_batch. [datafusion]

2024-11-10 Thread via GitHub
findepi commented on code in PR #13345: URL: https://github.com/apache/datafusion/pull/13345#discussion_r1835746256 ## datafusion/functions/src/datetime/date_bin.rs: ## @@ -491,150 +491,197 @@ mod tests { use chrono::TimeDelta; #[test] -#[allow(deprecated)] // TO

Re: [PR] build(deps): bump datafusion from 42.0.0 to 42.2.0 [datafusion-python]

2024-11-10 Thread via GitHub
dependabot[bot] commented on PR #944: URL: https://github.com/apache/datafusion-python/pull/944#issuecomment-2466810131 Looks like datafusion is up-to-date now, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] build(deps): bump arrow from 53.1.0 to 53.2.0 [datafusion-python]

2024-11-10 Thread via GitHub
dependabot[bot] closed pull request #934: build(deps): bump arrow from 53.1.0 to 53.2.0 URL: https://github.com/apache/datafusion-python/pull/934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] build(deps): bump object_store from 0.11.0 to 0.11.1 [datafusion-python]

2024-11-10 Thread via GitHub
dependabot[bot] commented on PR #932: URL: https://github.com/apache/datafusion-python/pull/932#issuecomment-2466810139 Looks like object_store is up-to-date now, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] build(deps): bump arrow from 53.1.0 to 53.2.0 [datafusion-python]

2024-11-10 Thread via GitHub
dependabot[bot] commented on PR #934: URL: https://github.com/apache/datafusion-python/pull/934#issuecomment-2466810135 Looks like arrow is up-to-date now, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] build(deps): bump object_store from 0.11.0 to 0.11.1 [datafusion-python]

2024-11-10 Thread via GitHub
dependabot[bot] closed pull request #932: build(deps): bump object_store from 0.11.0 to 0.11.1 URL: https://github.com/apache/datafusion-python/pull/932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] build(deps): bump datafusion from 42.0.0 to 42.2.0 [datafusion-python]

2024-11-10 Thread via GitHub
dependabot[bot] closed pull request #944: build(deps): bump datafusion from 42.0.0 to 42.2.0 URL: https://github.com/apache/datafusion-python/pull/944 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] build(deps): bump tokio from 1.40.0 to 1.41.1 [datafusion-python]

2024-11-10 Thread via GitHub
Michael-J-Ward commented on PR #946: URL: https://github.com/apache/datafusion-python/pull/946#issuecomment-2466809982 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Upgrade to Datafusion 43 [datafusion-python]

2024-11-10 Thread via GitHub
Michael-J-Ward merged PR #905: URL: https://github.com/apache/datafusion-python/pull/905 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] feat: support array_append [datafusion-comet]

2024-11-10 Thread via GitHub
codecov-commenter commented on PR #1072: URL: https://github.com/apache/datafusion-comet/pull/1072#issuecomment-2466808384 ## [Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/1072?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_ca

Re: [PR] WIP: Upgrade to Datafusion 43 [datafusion-python]

2024-11-10 Thread via GitHub
Michael-J-Ward commented on PR #905: URL: https://github.com/apache/datafusion-python/pull/905#issuecomment-2466796689 @timsaucer - thanks for finishing this up! I think it's good to go. Especially since this 1st upgrade PR is primarily for the dependencies. -- This is an auto

Re: [PR] feat: support array_append [datafusion-comet]

2024-11-10 Thread via GitHub
andygrove commented on code in PR #1072: URL: https://github.com/apache/datafusion-comet/pull/1072#discussion_r1835729939 ## spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala: ## @@ -2313,4 +2313,22 @@ class CometExpressionSuite extends CometTestBase with Adaptiv

Re: [PR] feat: Add `stringview` for functions to prevent regression [datafusion]

2024-11-10 Thread via GitHub
timsaucer commented on PR #13332: URL: https://github.com/apache/datafusion/pull/13332#issuecomment-2466789423 ConcatFunc: https://github.com/apache/datafusion/pull/13346 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] Bugfix/string view concat [datafusion]

2024-11-10 Thread via GitHub
timsaucer opened a new pull request, #13346: URL: https://github.com/apache/datafusion/pull/13346 ## Which issue does this PR close? This addresses one item of https://github.com/apache/datafusion/issues/13330 ## Rationale for this change Concat simplifier was not correct

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835719464 ## src/parser/mod.rs: ## @@ -11096,6 +11096,19 @@ impl<'a> Parser<'a> { arg, operator: FunctionArgOperator::Assignm

[PR] Migrate code from invoke to invoke_batch. [datafusion]

2024-11-10 Thread via GitHub
irenjj opened a new pull request, #13345: URL: https://github.com/apache/datafusion/pull/13345 ## Which issue does this PR close? Closes #13238 ## Rationale for this change ## What changes are included in this PR? Migrate code in unit test f

Re: [PR] Migrate code from invoke to invoke_batch. [datafusion]

2024-11-10 Thread via GitHub
irenjj commented on PR #13345: URL: https://github.com/apache/datafusion/pull/13345#issuecomment-2466786692 PTAL @findepi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835719464 ## src/parser/mod.rs: ## @@ -11096,6 +11096,19 @@ impl<'a> Parser<'a> { arg, operator: FunctionArgOperator::Assignm

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835721679 ## src/parser/mod.rs: ## @@ -11141,19 +11154,34 @@ impl<'a> Parser<'a> { /// FIRST_VALUE(x IGNORE NULL); /// ``` fn parse_function_arg

Re: [I] Introduce a way to represent constrained statistics / bounds on values in Statistics [datafusion]

2024-11-10 Thread via GitHub
ozankabak commented on issue #8078: URL: https://github.com/apache/datafusion/issues/8078#issuecomment-2466779170 Now that the interval library is in `expr-common`, is it possible we can use ```rust pub enum Estimate { Range { bounds: Interval, value: ScalarValue }, /// No

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835719464 ## src/parser/mod.rs: ## @@ -11096,6 +11096,19 @@ impl<'a> Parser<'a> { arg, operator: FunctionArgOperator::Assignm

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835719464 ## src/parser/mod.rs: ## @@ -11096,6 +11096,19 @@ impl<'a> Parser<'a> { arg, operator: FunctionArgOperator::Assignm

Re: [PR] Fix `scalar_functions.md` for CI [datafusion]

2024-11-10 Thread via GitHub
jonahgao commented on PR #13340: URL: https://github.com/apache/datafusion/pull/13340#issuecomment-2466756630 Merged to get CI green -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Fix `scalar_functions.md` for CI [datafusion]

2024-11-10 Thread via GitHub
jonahgao merged PR #13340: URL: https://github.com/apache/datafusion/pull/13340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [PR] feat: support `DEALLOCATE` to remove prepared statements [datafusion]

2024-11-10 Thread via GitHub
jonahgao merged PR #13327: URL: https://github.com/apache/datafusion/pull/13327 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [PR] feat: support `DEALLOCATE` to remove prepared statements [datafusion]

2024-11-10 Thread via GitHub
jonahgao commented on PR #13327: URL: https://github.com/apache/datafusion/pull/13327#issuecomment-2466749299 Thanks @jatin510 @goldmedal for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] feat: support `DEALLOCATE` to remove prepared statements [datafusion]

2024-11-10 Thread via GitHub
goldmedal commented on code in PR #13327: URL: https://github.com/apache/datafusion/pull/13327#discussion_r1835701840 ## datafusion/sqllogictest/test_files/prepare.slt: ## @@ -77,6 +81,28 @@ EXECUTE my_plan('Foo', 'Bar'); statement error Prepared statement \'my_plan\' already e

[PR] minor: Apply doc update script to scalar_functions [datafusion]

2024-11-10 Thread via GitHub
timsaucer opened a new pull request, #13344: URL: https://github.com/apache/datafusion/pull/13344 ## Which issue does this PR close? None ## Rationale for this change Applying documentation update script to fix CI. ## What changes are included in this PR? Fo

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835701698 ## src/parser/mod.rs: ## @@ -11096,6 +11096,19 @@ impl<'a> Parser<'a> { arg, operator: FunctionArgOperator::Assignm

Re: [PR] feat: support `DEALLOCATE` to remove prepared statements [datafusion]

2024-11-10 Thread via GitHub
goldmedal commented on code in PR #13327: URL: https://github.com/apache/datafusion/pull/13327#discussion_r1835700101 ## datafusion/sqllogictest/test_files/prepare.slt: ## @@ -77,6 +81,28 @@ EXECUTE my_plan('Foo', 'Bar'); statement error Prepared statement \'my_plan\' already e

Re: [PR] feat: support `DEALLOCATE` to remove prepared statements [datafusion]

2024-11-10 Thread via GitHub
goldmedal commented on code in PR #13327: URL: https://github.com/apache/datafusion/pull/13327#discussion_r1835700101 ## datafusion/sqllogictest/test_files/prepare.slt: ## @@ -77,6 +81,28 @@ EXECUTE my_plan('Foo', 'Bar'); statement error Prepared statement \'my_plan\' already e

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
iffyio commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835696061 ## src/parser/mod.rs: ## @@ -11141,19 +11154,34 @@ impl<'a> Parser<'a> { /// FIRST_VALUE(x IGNORE NULL); /// ``` fn parse_function_argume

Re: [PR] feat: Add `stringview` for functions to prevent regression [datafusion]

2024-11-10 Thread via GitHub
timsaucer commented on PR #13332: URL: https://github.com/apache/datafusion/pull/13332#issuecomment-2466735250 Wonderful. Thank you. I'll try to review tomorrow. I did dive into the ConcatFunc and I found (and resolved) two issues: 1) when doing concat on pure literals we aren't all

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835693592 ## src/parser/mod.rs: ## @@ -11096,6 +11096,19 @@ impl<'a> Parser<'a> { arg, operator: FunctionArgOperator::Assignm

Re: [I] [DISCUSSION] 2024 Q4 / 2025 Q1 Roadmap [datafusion]

2024-11-10 Thread via GitHub
matthewmturner commented on issue #13274: URL: https://github.com/apache/datafusion/issues/13274#issuecomment-2466730790 @alamb can you expand on the different runtimes point? Are referring to having a dedicated tokio runtime for CPU bound work? I actually have a [ticket](https://github.com

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
gaoqiangz commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835692334 ## src/parser/mod.rs: ## @@ -11141,19 +11154,34 @@ impl<'a> Parser<'a> { /// FIRST_VALUE(x IGNORE NULL); /// ``` fn parse_function_arg

Re: [PR] Add support for Snowflake SHOW DATABASES/SCHEMAS/TABLES/VIEWS/COLUMNS statements [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
yoavcloud commented on code in PR #1501: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1501#discussion_r1835688630 ## tests/sqlparser_snowflake.rs: ## @@ -2781,3 +2781,68 @@ fn test_parentheses_overflow() { snowflake_with_recursion_limit(max_nesting_leve

[I] [substrait] Replace SessionContext with SessionState [datafusion]

2024-11-10 Thread via GitHub
notfilippo opened a new issue, #13342: URL: https://github.com/apache/datafusion/issues/13342 ### Is your feature request related to a problem or challenge? When producing / consuming a substrait plan it's not always practical to pass the SessionContext. Most of the methods used in th

Re: [PR] feat(substrait): replace SessionContext with SessionState [datafusion]

2024-11-10 Thread via GitHub
notfilippo commented on code in PR #13343: URL: https://github.com/apache/datafusion/pull/13343#discussion_r1835688264 ## datafusion/core/src/execution/session_state.rs: ## @@ -293,7 +293,9 @@ impl SessionState { .resolve(&catalog.default_catalog, &catalog.default_s

[PR] feat(substrait): replace SessionContext with SessionState [datafusion]

2024-11-10 Thread via GitHub
notfilippo opened a new pull request, #13343: URL: https://github.com/apache/datafusion/pull/13343 ## Which issue does this PR close? Closes #13342. ## Rationale for this change See original issue. ## What changes are included in this PR? See original issue.

Re: [PR] Fix `scalar_functions.md` for CI [datafusion]

2024-11-10 Thread via GitHub
goldmedal commented on PR #13340: URL: https://github.com/apache/datafusion/pull/13340#issuecomment-2466722458 Thanks, @Weijun-H and @tlm365. It makes sense to me 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Fix `scalar_functions.md` for CI [datafusion]

2024-11-10 Thread via GitHub
tlm365 commented on PR #13340: URL: https://github.com/apache/datafusion/pull/13340#issuecomment-2466721271 > After some investigation, we can fix this issue through this modification. I also prefer this, it makes the sql example more consistent (no start with tab/spaces) -- Th

Re: [PR] minor(docs): Add docs for `regen.sh` [datafusion]

2024-11-10 Thread via GitHub
Weijun-H merged PR #13321: URL: https://github.com/apache/datafusion/pull/13321 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [PR] minor(docs): Add docs for `regen.sh` [datafusion]

2024-11-10 Thread via GitHub
Weijun-H commented on PR #13321: URL: https://github.com/apache/datafusion/pull/13321#issuecomment-2466719798 Thanks @alamb for reviewing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Fix `scalar_functions.md` for CI [datafusion]

2024-11-10 Thread via GitHub
Weijun-H commented on PR #13340: URL: https://github.com/apache/datafusion/pull/13340#issuecomment-2466718368 After some investigation, we can fix this issue through this modification. https://github.com/apache/datafusion/blob/c48f12d27cc60ae5e9c3d0791d553453f0dd7001/datafusion/functi

Re: [PR] Add support for Snowflake SHOW DATABASES/SCHEMAS/TABLES/VIEWS/COLUMNS statements [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
yoavcloud commented on code in PR #1501: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1501#discussion_r1835682251 ## src/parser/mod.rs: ## @@ -12315,6 +12310,124 @@ impl<'a> Parser<'a> { } false } + +/// Look for all of the expected key

[PR] Introduce `information_schema.parameters` table [datafusion]

2024-11-10 Thread via GitHub
goldmedal opened a new pull request, #13341: URL: https://github.com/apache/datafusion/pull/13341 ## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested?

Re: [PR] Update dev script to publish FFI crate [datafusion]

2024-11-10 Thread via GitHub
timsaucer merged PR #13322: URL: https://github.com/apache/datafusion/pull/13322 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@data

Re: [PR] Add support for Snowflake SHOW DATABASES/SCHEMAS/TABLES/VIEWS/COLUMNS statements [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
yoavcloud commented on code in PR #1501: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1501#discussion_r1835676890 ## src/parser/mod.rs: ## @@ -12315,6 +12310,124 @@ impl<'a> Parser<'a> { } false } + +/// Look for all of the expected key

Re: [PR] Add support for Snowflake SHOW DATABASES/SCHEMAS/TABLES/VIEWS/COLUMNS statements [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
yoavcloud commented on code in PR #1501: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1501#discussion_r1835675842 ## src/parser/mod.rs: ## @@ -12315,6 +12310,124 @@ impl<'a> Parser<'a> { } false } + +/// Look for all of the expected key

Re: [PR] minor: include datafusion-ffi in publishing instructions [datafusion]

2024-11-10 Thread via GitHub
timsaucer merged PR #1: URL: https://github.com/apache/datafusion/pull/1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@data

Re: [PR] WIP: parse true and false as identifiers in mssql [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
lovasoa commented on PR #1510: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1510#issuecomment-2466693830 Ok, I fixed all the tests, this is ready for review @iffyio, @alamb :) -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Improve the coverage of `bit_length` testing [datafusion]

2024-11-10 Thread via GitHub
goldmedal commented on PR #13336: URL: https://github.com/apache/datafusion/pull/13336#issuecomment-2466690671 The CI fail doesn't relate to this PR. It could be fix by https://github.com/apache/datafusion/pull/13340 -- This is an automated message from the Apache Git Service. To respond

[PR] Fix `scalar_functions.md` for CI [datafusion]

2024-11-10 Thread via GitHub
goldmedal opened a new pull request, #13340: URL: https://github.com/apache/datafusion/pull/13340 ## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested?

Re: [PR] Add support for MSSQL's `JSON_ARRAY`/`JSON_OBJECT` expr [datafusion-sqlparser-rs]

2024-11-10 Thread via GitHub
iffyio commented on code in PR #1507: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1507#discussion_r1835643017 ## src/parser/mod.rs: ## @@ -11141,19 +11154,34 @@ impl<'a> Parser<'a> { /// FIRST_VALUE(x IGNORE NULL); /// ``` fn parse_function_argume

[PR] Update dependencies in `requirements.txt` for python3.12 [datafusion]

2024-11-10 Thread via GitHub
AnthonyZhOon opened a new pull request, #13339: URL: https://github.com/apache/datafusion/pull/13339 ## Which issue does this PR close? Closes #13337 ## Rationale for this change [PEP 632 - Deprecate distutils module](https://peps.python.org/pep-0632/) has removed `dist

[PR] Enable `clone_on_ref_ptr` clippy lint on core crate [datafusion]

2024-11-10 Thread via GitHub
getChan opened a new pull request, #13338: URL: https://github.com/apache/datafusion/pull/13338 ## Which issue does this PR close? Part of https://github.com/apache/datafusion/issues/11143. ## Rationale for this change ## What changes are included in this

[I] Docs: Update docs dependencies for python 3.12 [datafusion]

2024-11-10 Thread via GitHub
AnthonyZhOon opened a new issue, #13337: URL: https://github.com/apache/datafusion/issues/13337 ### Describe the bug The `distutils` module has been removed from the standard library in python 3.12 in favour of the `setuptools` package providing and maintaining the module. See [P

Re: [PR] support recursive CTEs logical plans in datafusion-proto [datafusion]

2024-11-10 Thread via GitHub
jonahgao commented on code in PR #13314: URL: https://github.com/apache/datafusion/pull/13314#discussion_r1835638570 ## datafusion/proto/tests/cases/roundtrip_logical_plan.rs: ## @@ -2523,3 +2525,75 @@ fn roundtrip_window() { roundtrip_expr_test(test_expr6, ctx.clone());

Re: [I] Revisit `bit_length` Not Supporting Utf8View Array. [datafusion]

2024-11-10 Thread via GitHub
austin362667 commented on issue #13329: URL: https://github.com/apache/datafusion/issues/13329#issuecomment-2466657720 That's great! Thank you @goldmedal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] Revisit `bit_length` Not Supporting Utf8View Array. [datafusion]

2024-11-10 Thread via GitHub
goldmedal commented on issue #13329: URL: https://github.com/apache/datafusion/issues/13329#issuecomment-2466655055 I added https://github.com/apache/datafusion/pull/13336 to enhance the coverage. When solving this issue, we can move those tests to `string_query.slt.part`. -- This is an

[PR] Improve the coverage of `bit_length` testing [datafusion]

2024-11-10 Thread via GitHub
goldmedal opened a new pull request, #13336: URL: https://github.com/apache/datafusion/pull/13336 ## Which issue does this PR close? The follow-up PR of #13221 ## Rationale for this change ## What changes are included in this PR? - Add tests for `String`,

Re: [I] [DISCUSSION] 2024 Q4 / 2025 Q1 Roadmap [datafusion]

2024-11-10 Thread via GitHub
alamb commented on issue #13274: URL: https://github.com/apache/datafusion/issues/13274#issuecomment-2466651579 I have had a few days to reflect , and I personally think making it easy to integrate DataFusion into the "open data lake" stack might be my top priority over the coming months

Re: [PR] Remove warning logs during the document build process [datafusion]

2024-11-10 Thread via GitHub
alamb commented on code in PR #13324: URL: https://github.com/apache/datafusion/pull/13324#discussion_r1835627885 ## docs/source/conf.py: ## @@ -119,3 +119,9 @@ # enable nice rendering of checkboxes for the task lists myst_enable_extensions = ["colon_fence", "deflist", "task

  1   2   >