phillipleblanc opened a new issue, #10706:
URL: https://github.com/apache/datafusion/issues/10706
### Describe the bug
The schema that is the result of a UNION ALL should not have any table
qualifiers, as the table information has effectively been erased and is no
longer a valid refe
phillipleblanc opened a new pull request, #10707:
URL: https://github.com/apache/datafusion/pull/10707
## Which issue does this PR close?
Closes #10706
## Rationale for this change
The schema that is the result of a UNION ALL should not have any table
qualifiers, as
viirya commented on code in PR #490:
URL: https://github.com/apache/datafusion-comet/pull/490#discussion_r1618312815
##
common/src/main/java/org/apache/comet/vector/CometDictionary.java:
##
@@ -59,121 +46,57 @@ public ValueVector getValueVector() {
}
public boolean decod
lewiszlw opened a new pull request, #10708:
URL: https://github.com/apache/datafusion/pull/10708
## Which issue does this PR close?
Closes #.
## Rationale for this change
Separate proto partitioning so it can be used in user defined shuffle
operator.
viirya commented on code in PR #490:
URL: https://github.com/apache/datafusion-comet/pull/490#discussion_r1618321281
##
common/src/main/java/org/apache/comet/vector/CometDictionary.java:
##
@@ -59,121 +46,57 @@ public ValueVector getValueVector() {
}
public boolean decod
waynexia commented on code in PR #10543:
URL: https://github.com/apache/datafusion/pull/10543#discussion_r1618334298
##
datafusion/physical-plan/src/work_table.rs:
##
@@ -169,7 +169,7 @@ impl ExecutionPlan for WorkTableExec {
&self.cache
}
-fn children(&self)
phillipleblanc commented on code in PR #10707:
URL: https://github.com/apache/datafusion/pull/10707#discussion_r1618338309
##
datafusion/expr/src/logical_plan/builder.rs:
##
@@ -1373,33 +1373,32 @@ pub fn union(left_plan: LogicalPlan, right_plan:
LogicalPlan) -> Resulthttps://g
phillipleblanc commented on code in PR #10707:
URL: https://github.com/apache/datafusion/pull/10707#discussion_r1618341153
##
datafusion/expr/src/logical_plan/builder.rs:
##
@@ -1373,33 +1373,32 @@ pub fn union(left_plan: LogicalPlan, right_plan:
LogicalPlan) -> Result
jonahgao merged PR #10685:
URL: https://github.com/apache/datafusion/pull/10685
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dataf
Blizzara commented on code in PR #10646:
URL: https://github.com/apache/datafusion/pull/10646#discussion_r1618379915
##
datafusion/substrait/src/logical_plan/consumer.rs:
##
@@ -1160,6 +1162,24 @@ pub(crate) fn from_substrait_type(dt:
&substrait::proto::Type) -> Result {
+
waynexia commented on code in PR #10646:
URL: https://github.com/apache/datafusion/pull/10646#discussion_r1618465409
##
datafusion/substrait/src/logical_plan/consumer.rs:
##
@@ -1160,6 +1162,24 @@ pub(crate) fn from_substrait_type(dt:
&substrait::proto::Type) -> Result {
+
jonahgao commented on code in PR #10687:
URL: https://github.com/apache/datafusion/pull/10687#discussion_r1618480297
##
datafusion/expr/src/logical_plan/plan.rs:
##
@@ -56,19 +56,139 @@ use crate::logical_plan::tree_node::unwrap_arc;
pub use datafusion_common::display::{PlanTyp
jonahgao commented on code in PR #10687:
URL: https://github.com/apache/datafusion/pull/10687#discussion_r1618479343
##
datafusion/expr/src/logical_plan/plan.rs:
##
@@ -56,19 +56,139 @@ use crate::logical_plan::tree_node::unwrap_arc;
pub use datafusion_common::display::{PlanTyp
waynexia commented on code in PR #10646:
URL: https://github.com/apache/datafusion/pull/10646#discussion_r1618540788
##
datafusion/substrait/src/logical_plan/consumer.rs:
##
@@ -1160,6 +1162,24 @@ pub(crate) fn from_substrait_type(dt:
&substrait::proto::Type) -> Result {
+
alamb merged PR #10698:
URL: https://github.com/apache/datafusion/pull/10698
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb merged PR #10705:
URL: https://github.com/apache/datafusion/pull/10705
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb commented on code in PR #10702:
URL: https://github.com/apache/datafusion/pull/10702#discussion_r1618607123
##
datafusion/core/src/physical_optimizer/join_selection.rs:
##
@@ -157,7 +157,9 @@ fn swap_join_projection(
}
/// This function swaps the inputs of the given jo
alamb merged PR #10704:
URL: https://github.com/apache/datafusion/pull/10704
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb closed issue #10604: Incorrect statistics read for unsigned integer
columns in parquet
URL: https://github.com/apache/datafusion/issues/10604
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
davidhewitt opened a new issue, #10709:
URL: https://github.com/apache/datafusion/issues/10709
### Describe the bug
I'm experimenting with datafusion 38 (using as a library, not the cli) to
query against parquet files in Google Cloud Storage. For now I'm doing a basic
test, just `sel
samuelcolvin commented on issue #10709:
URL: https://github.com/apache/datafusion/issues/10709#issuecomment-2137048388
Thanks @davidhewitt, one extra note - if it would help anyone looking into
this to see the Logfire live view of the trace, as well as the screenshot,
happy to send some dat
alamb commented on code in PR #10693:
URL: https://github.com/apache/datafusion/pull/10693#discussion_r1618624379
##
datafusion/core/src/datasource/listing/helpers.rs:
##
@@ -675,4 +764,127 @@ mod tests {
// this helper function
assert!(expr_applicable_for_cols
alamb merged PR #10708:
URL: https://github.com/apache/datafusion/pull/10708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb opened a new issue, #10710:
URL: https://github.com/apache/datafusion/issues/10710
### Is your feature request related to a problem or challenge?
In https://github.com/apache/datafusion/pull/10653, @Blizzara added the
beginnings of testing for substrait plans that came from othe
alamb commented on PR #10653:
URL: https://github.com/apache/datafusion/pull/10653#issuecomment-2137098889
> > > though curious if you have any thoughts on how to make it simpler -
given DF doesn't yet produce those compound names, I cannot use a roundtrip
test, and writing the substrait ma
alamb commented on PR #10653:
URL: https://github.com/apache/datafusion/pull/10653#issuecomment-2137099101
Thanks again everyone!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comm
alamb merged PR #10653:
URL: https://github.com/apache/datafusion/pull/10653
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb closed issue #10412: Substrait integration doesn't recognize typed
functions
URL: https://github.com/apache/datafusion/issues/10412
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
alamb commented on PR #10686:
URL: https://github.com/apache/datafusion/pull/10686#issuecomment-2137100092
Thanks for the review @jonahgao 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
alamb commented on code in PR #10687:
URL: https://github.com/apache/datafusion/pull/10687#discussion_r1618661393
##
datafusion/expr/src/logical_plan/plan.rs:
##
@@ -56,19 +56,139 @@ use crate::logical_plan::tree_node::unwrap_arc;
pub use datafusion_common::display::{PlanType,
alamb merged PR #10686:
URL: https://github.com/apache/datafusion/pull/10686
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb commented on issue #10664:
URL: https://github.com/apache/datafusion/issues/10664#issuecomment-2137108974
> In my understanding, window function is a special case of aggregate
function, why would these two have different plan struct?
I think window funtions and aggregate functio
xinlifoobar commented on issue #10453:
URL: https://github.com/apache/datafusion/issues/10453#issuecomment-2137115446
@alamb just hint #10605 is also closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abov
alamb commented on issue #9415:
URL: https://github.com/apache/datafusion/issues/9415#issuecomment-2137119619
Since the the internal metrics in datafusion -- aka
https://docs.rs/datafusion/latest/datafusion/physical_plan/metrics/index.html
have start/stop timestamps on them already, we have
xinlifoobar opened a new pull request, #10711:
URL: https://github.com/apache/datafusion/pull/10711
## Which issue does this PR close?
Closes #10626
## Rationale for this change
## What changes are included in this PR?
Replace the `get_statatistics`
alamb commented on code in PR #10600:
URL: https://github.com/apache/datafusion/pull/10600#discussion_r1618678318
##
datafusion/core/src/datasource/stream.rs:
##
@@ -103,19 +105,46 @@ impl FromStr for StreamEncoding {
}
}
-/// The configuration for a [`StreamTable`]
+///
ozankabak commented on PR #10600:
URL: https://github.com/apache/datafusion/pull/10600#issuecomment-2137168944
@berkaysynnada PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific com
Dandandan commented on code in PR #10702:
URL: https://github.com/apache/datafusion/pull/10702#discussion_r1618726396
##
datafusion/core/src/physical_optimizer/join_selection.rs:
##
@@ -157,7 +157,9 @@ fn swap_join_projection(
}
/// This function swaps the inputs of the give
samuelcolvin opened a new issue, #10712:
URL: https://github.com/apache/datafusion/issues/10712
### Describe the bug
While investigating #10709, I tried using datafusion CLI to require parquet
files to a better size.
But I got a panic:
```
thread 'tokio-runtime-worker
alamb merged PR #10687:
URL: https://github.com/apache/datafusion/pull/10687
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb commented on PR #10687:
URL: https://github.com/apache/datafusion/pull/10687#issuecomment-2137240615
Thanks again for the review @jonahgao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t
yyin-dev opened a new pull request, #10713:
URL: https://github.com/apache/datafusion/pull/10713
## Which issue does this PR close?
Closes #10667 .
## Are these changes tested?
- [x] New proto roundtrip tests.
- [ ] Moved tests into sqllogictests.
--
This is
jayzhan211 commented on code in PR #10700:
URL: https://github.com/apache/datafusion/pull/10700#discussion_r1618774900
##
datafusion/functions/src/macros.rs:
##
@@ -59,6 +59,30 @@ macro_rules! export_functions {
};
}
+macro_rules! make_function {
+// single vector ar
alamb commented on code in PR #10702:
URL: https://github.com/apache/datafusion/pull/10702#discussion_r1618607123
##
datafusion/core/src/physical_optimizer/join_selection.rs:
##
@@ -157,7 +157,9 @@ fn swap_join_projection(
}
/// This function swaps the inputs of the given jo
jayzhan211 commented on code in PR #9906:
URL: https://github.com/apache/datafusion/pull/9906#discussion_r1618798950
##
datafusion-examples/examples/simplify_udwf_expression.rs:
##
@@ -0,0 +1,142 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contr
alamb commented on code in PR #10711:
URL: https://github.com/apache/datafusion/pull/10711#discussion_r1618800964
##
datafusion/core/src/datasource/physical_plan/parquet/statistics.rs:
##
@@ -52,131 +55,311 @@ fn sign_extend_be(b: &[u8]) -> [u8; 16] {
result
}
-/// Extra
waynexia commented on code in PR #10646:
URL: https://github.com/apache/datafusion/pull/10646#discussion_r1618822660
##
datafusion/substrait/src/logical_plan/consumer.rs:
##
@@ -1160,6 +1162,24 @@ pub(crate) fn from_substrait_type(dt:
&substrait::proto::Type) -> Result {
+
Blizzara commented on code in PR #10646:
URL: https://github.com/apache/datafusion/pull/10646#discussion_r1618834391
##
datafusion/substrait/src/logical_plan/consumer.rs:
##
@@ -1160,6 +1162,24 @@ pub(crate) fn from_substrait_type(dt:
&substrait::proto::Type) -> Result {
+
Blizzara commented on issue #8149:
URL: https://github.com/apache/datafusion/issues/8149#issuecomment-2137345554
Values (and EmptyTable) is supported as of #10531 :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
waynexia commented on issue #8149:
URL: https://github.com/apache/datafusion/issues/8149#issuecomment-2137362019
> Values (and EmptyTable) is supported as of #10531 :)
Updated. Thanks for implementing it!
--
This is an automated message from the Apache Git Service.
To respond to the
xinlifoobar commented on code in PR #10711:
URL: https://github.com/apache/datafusion/pull/10711#discussion_r1618850434
##
datafusion/core/src/datasource/physical_plan/parquet/statistics.rs:
##
@@ -211,32 +394,25 @@ pub(crate) fn min_statistics<'a, I: Iterator Result {
-let
xinlifoobar commented on code in PR #10711:
URL: https://github.com/apache/datafusion/pull/10711#discussion_r1618850434
##
datafusion/core/src/datasource/physical_plan/parquet/statistics.rs:
##
@@ -211,32 +394,25 @@ pub(crate) fn min_statistics<'a, I: Iterator Result {
-let
xinlifoobar commented on code in PR #10711:
URL: https://github.com/apache/datafusion/pull/10711#discussion_r1618850434
##
datafusion/core/src/datasource/physical_plan/parquet/statistics.rs:
##
@@ -211,32 +394,25 @@ pub(crate) fn min_statistics<'a, I: Iterator Result {
-let
yyin-dev commented on PR #10713:
URL: https://github.com/apache/datafusion/pull/10713#issuecomment-2137394530
@jayzhan211 I'm working on a change, but can you help me understand the
semantics here:
```
# csv_query_distinct_variance
query R
SELECT var(distinct c2) FROM aggreg
viirya commented on code in PR #490:
URL: https://github.com/apache/datafusion-comet/pull/490#discussion_r1618887167
##
common/src/main/java/org/apache/comet/vector/CometDictionary.java:
##
@@ -59,121 +46,57 @@ public ValueVector getValueVector() {
}
public boolean decod
viirya commented on code in PR #490:
URL: https://github.com/apache/datafusion-comet/pull/490#discussion_r1618887167
##
common/src/main/java/org/apache/comet/vector/CometDictionary.java:
##
@@ -59,121 +46,57 @@ public ValueVector getValueVector() {
}
public boolean decod
xhumanoid commented on code in PR #491:
URL: https://github.com/apache/datafusion-comet/pull/491#discussion_r1618965703
##
spark/src/main/scala/org/apache/comet/parquet/CometParquetFileFormat.scala.old:
##
Review Comment:
do we really need to keep old files?
maybe you c
jayzhan211 opened a new pull request, #10714:
URL: https://github.com/apache/datafusion/pull/10714
## Which issue does this PR close?
Closes #10703.
## Rationale for this change
## What changes are included in this PR?
## Are these changes t
davidhewitt closed issue #10709: Google Cloud Storage requests during query
execution being performed in series
URL: https://github.com/apache/datafusion/issues/10709
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
davidhewitt commented on issue #10709:
URL: https://github.com/apache/datafusion/issues/10709#issuecomment-2137573303
Ok great, so this is happily user error!
After digging around with the query plans and understanding that each file
group is loaded sequentially by design, I found
`L
goldmedal commented on issue #10688:
URL: https://github.com/apache/datafusion/issues/10688#issuecomment-2137613195
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
vaibhawvipul commented on code in PR #471:
URL: https://github.com/apache/datafusion-comet/pull/471#discussion_r1618206450
##
spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala:
##
@@ -1469,5 +1469,36 @@ class CometExpressionSuite extends CometTestBase with
Adapti
berkaysynnada commented on PR #10600:
URL: https://github.com/apache/datafusion/pull/10600#issuecomment-2137637228
I will review it in detail tomorrow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
comphead commented on code in PR #10702:
URL: https://github.com/apache/datafusion/pull/10702#discussion_r1619055381
##
datafusion/core/src/physical_optimizer/join_selection.rs:
##
@@ -157,7 +157,9 @@ fn swap_join_projection(
}
/// This function swaps the inputs of the given
comphead commented on code in PR #10702:
URL: https://github.com/apache/datafusion/pull/10702#discussion_r1619056151
##
datafusion/core/src/physical_optimizer/join_selection.rs:
##
@@ -157,7 +157,9 @@ fn swap_join_projection(
}
/// This function swaps the inputs of the given
jayzhan211 commented on PR #10713:
URL: https://github.com/apache/datafusion/pull/10713#issuecomment-2137673821
> @jayzhan211 I'm working on a change, but can you help me understand the
semantics here:
>
> ```
> # csv_query_distinct_variance
> query R
> SELECT var(distinct c2
huaxingao opened a new pull request, #492:
URL: https://github.com/apache/datafusion-comet/pull/492
## Which issue does this PR close?
Closes [#36](https://github.com/apache/datafusion-comet/issues/36).
## Rationale for this change
## What changes are incl
jayzhan211 commented on code in PR #10714:
URL: https://github.com/apache/datafusion/pull/10714#discussion_r1619082432
##
datafusion/optimizer/src/replace_distinct_aggregate.rs:
##
@@ -163,53 +173,3 @@ impl OptimizerRule for ReplaceDistinctWithAggregate {
Some(BottomUp)
jayzhan211 commented on code in PR #10714:
URL: https://github.com/apache/datafusion/pull/10714#discussion_r1619081626
##
datafusion/optimizer/src/lib.rs:
##
@@ -56,7 +56,7 @@ pub mod single_distinct_to_groupby;
pub mod unwrap_cast_in_comparison;
pub mod utils;
-#[cfg(test)]
viirya commented on issue #10688:
URL: https://github.com/apache/datafusion/issues/10688#issuecomment-2137699009
> Thats weird.
>
> > explain select -128.2::float union all select -128.2;
I think it is because the first `-128.2` is read as float 32, then treated
as float 64. I
viirya commented on code in PR #10702:
URL: https://github.com/apache/datafusion/pull/10702#discussion_r1619096578
##
datafusion/core/src/physical_optimizer/join_selection.rs:
##
@@ -157,7 +157,9 @@ fn swap_join_projection(
}
/// This function swaps the inputs of the given j
goldmedal commented on issue #10688:
URL: https://github.com/apache/datafusion/issues/10688#issuecomment-2137711858
> I think it is because the first `-128.2` is read as float 32, then treated
as float 64. I said "treated" because I don't see there is cast expression. It
may be happened whe
goldmedal commented on issue #10688:
URL: https://github.com/apache/datafusion/issues/10688#issuecomment-2137807135
I wanted to share something I found. I tried to observe the plan
transformation step by step:
sql_to_plan -> analyze -> optimize
```
SQL: select 128.2 union all selec
huaxingao commented on PR #492:
URL: https://github.com/apache/datafusion-comet/pull/492#issuecomment-2137815303
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific co
vaibhawvipul commented on code in PR #471:
URL: https://github.com/apache/datafusion-comet/pull/471#discussion_r1619176000
##
spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala:
##
@@ -1469,5 +1469,36 @@ class CometExpressionSuite extends CometTestBase with
Adapti
vaibhawvipul commented on PR #471:
URL: https://github.com/apache/datafusion-comet/pull/471#issuecomment-2137819580
@kazuyukitanimura requesting a review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
andygrove commented on PR #472:
URL: https://github.com/apache/datafusion-comet/pull/472#issuecomment-2137823739
Thanks for the review @kazuyukitanimura. I have addressed your comments and
also added support for decimal type. I removed the TODO comments and added a
roadmap section to the R
huaxingao closed pull request #492: chore: Use Spark's ParquetFilters
URL: https://github.com/apache/datafusion-comet/pull/492
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
T
sunchao commented on code in PR #490:
URL: https://github.com/apache/datafusion-comet/pull/490#discussion_r1619195722
##
common/src/main/java/org/apache/comet/vector/CometDictionary.java:
##
@@ -59,121 +46,57 @@ public ValueVector getValueVector() {
}
public boolean deco
andygrove commented on code in PR #455:
URL: https://github.com/apache/datafusion-comet/pull/455#discussion_r1619197931
##
docs/spark_expressions_support.md:
##
@@ -0,0 +1,475 @@
+
+
+# Supported Spark Expressions
+
+### agg_funcs
+ - [x] any
+ - [x] any_value
+ - [ ] approx_cou
andygrove commented on code in PR #455:
URL: https://github.com/apache/datafusion-comet/pull/455#discussion_r1619208014
##
docs/spark_expressions_support.md:
##
@@ -0,0 +1,475 @@
+
+
+# Supported Spark Expressions
+
+### agg_funcs
+ - [x] any
+ - [x] any_value
+ - [ ] approx_cou
andygrove commented on code in PR #455:
URL: https://github.com/apache/datafusion-comet/pull/455#discussion_r1619210130
##
docs/spark_expressions_support.md:
##
@@ -0,0 +1,475 @@
+
+
+# Supported Spark Expressions
+
+### agg_funcs
+ - [x] any
+ - [x] any_value
+ - [ ] approx_cou
andygrove commented on code in PR #455:
URL: https://github.com/apache/datafusion-comet/pull/455#discussion_r1619211189
##
docs/spark_expressions_support.md:
##
@@ -0,0 +1,475 @@
+
+
+# Supported Spark Expressions
+
+### agg_funcs
+ - [x] any
+ - [x] any_value
+ - [ ] approx_cou
kazuyukitanimura opened a new pull request, #493:
URL: https://github.com/apache/datafusion-comet/pull/493
## Which issue does this PR close?
Part of https://github.com/apache/datafusion-comet/issues/372
## Rationale for this change
To be ready for Spark
viirya commented on code in PR #455:
URL: https://github.com/apache/datafusion-comet/pull/455#discussion_r1619233974
##
docs/spark_expressions_support.md:
##
@@ -0,0 +1,475 @@
+
+
+# Supported Spark Expressions
+
+### agg_funcs
+ - [x] any
+ - [x] any_value
+ - [ ] approx_count_
yyin-dev commented on PR #10713:
URL: https://github.com/apache/datafusion/pull/10713#issuecomment-2137921026
> > @jayzhan211 I'm working on a change, but can you help me understand the
semantics here:
> > ```
> > # csv_query_distinct_variance
> > query R
> > SELECT var(distinct
parthchandra commented on issue #468:
URL:
https://github.com/apache/datafusion-comet/issues/468#issuecomment-2137936443
IIRC there were differences in output between Spark 3.2 and Spark 3.4 for
the timestamp_ntz type.
Taking a closer look, the definition of timestamp_ntz (in Spark) ess
andygrove commented on code in PR #455:
URL: https://github.com/apache/datafusion-comet/pull/455#discussion_r1619275258
##
docs/spark_expressions_support.md:
##
@@ -0,0 +1,475 @@
+
+
+# Supported Spark Expressions
+
+### agg_funcs
+ - [x] any
+ - [x] any_value
+ - [ ] approx_cou
andygrove commented on code in PR #455:
URL: https://github.com/apache/datafusion-comet/pull/455#discussion_r1619275529
##
docs/spark_expressions_support.md:
##
@@ -0,0 +1,475 @@
+
+
+# Supported Spark Expressions
+
+### agg_funcs
+ - [x] any
+ - [x] any_value
+ - [ ] approx_cou
Abdullahsab3 commented on issue #10602:
URL: https://github.com/apache/datafusion/issues/10602#issuecomment-2137977511
I was looking into this issue again. I am still validating this, but I found
that using `to_char(time AT TIME ZONE 'UTC' AT TIME ZONE 'Europe/Brussels', '%F
%X')` also resu
viirya commented on code in PR #490:
URL: https://github.com/apache/datafusion-comet/pull/490#discussion_r1619295940
##
common/src/main/java/org/apache/comet/vector/CometDictionary.java:
##
@@ -59,121 +46,57 @@ public ValueVector getValueVector() {
}
public boolean decod
viirya commented on code in PR #490:
URL: https://github.com/apache/datafusion-comet/pull/490#discussion_r1619295940
##
common/src/main/java/org/apache/comet/vector/CometDictionary.java:
##
@@ -59,121 +46,57 @@ public ValueVector getValueVector() {
}
public boolean decod
viirya commented on code in PR #10702:
URL: https://github.com/apache/datafusion/pull/10702#discussion_r1619347294
##
datafusion/core/src/physical_optimizer/join_selection.rs:
##
@@ -157,7 +157,9 @@ fn swap_join_projection(
}
/// This function swaps the inputs of the given j
viirya commented on code in PR #10702:
URL: https://github.com/apache/datafusion/pull/10702#discussion_r1619353656
##
datafusion/core/src/physical_optimizer/join_selection.rs:
##
@@ -157,7 +157,9 @@ fn swap_join_projection(
}
/// This function swaps the inputs of the given j
andygrove commented on code in PR #437:
URL: https://github.com/apache/datafusion-comet/pull/437#discussion_r1619362298
##
core/src/execution/datafusion/planner.rs:
##
@@ -978,7 +979,15 @@ impl PhysicalPlanner {
// `EqualNullSafe`, Spark will rewrite it duri
alamb opened a new pull request, #10715:
URL: https://github.com/apache/datafusion/pull/10715
## Which issue does this PR close?
re https://github.com/apache/datafusion/issues/10626
## Rationale for this change
I wanted to prototype another approach that @xinlifo
andygrove commented on code in PR #449:
URL: https://github.com/apache/datafusion-comet/pull/449#discussion_r1619370071
##
core/src/execution/datafusion/expressions/scalar_funcs/hex.rs:
##
@@ -0,0 +1,296 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or mo
viirya commented on code in PR #449:
URL: https://github.com/apache/datafusion-comet/pull/449#discussion_r1619376557
##
core/src/execution/datafusion/expressions/scalar_funcs/hex.rs:
##
@@ -0,0 +1,296 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
alamb commented on code in PR #10715:
URL: https://github.com/apache/datafusion/pull/10715#discussion_r1619383361
##
datafusion/core/src/datasource/physical_plan/parquet/statistics.rs:
##
@@ -211,9 +287,29 @@ pub(crate) fn min_statistics<'a, I: Iterator Result {
-let scalars
alamb commented on code in PR #10711:
URL: https://github.com/apache/datafusion/pull/10711#discussion_r1619384338
##
datafusion/core/src/datasource/physical_plan/parquet/statistics.rs:
##
@@ -211,32 +394,25 @@ pub(crate) fn min_statistics<'a, I: Iterator Result {
-let scalar
1 - 100 of 164 matches
Mail list logo
