Rachelint commented on issue #11463:
URL: https://github.com/apache/datafusion/issues/11463#issuecomment-2227212088
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
Rachelint opened a new issue, #11463:
URL: https://github.com/apache/datafusion/issues/11463
### Is your feature request related to a problem or challenge?
I found hash calculation for Scalar::Map is not impl actually, that leads
error if we enable the `const evaluate` optimization fo
Lordworms commented on PR #11462:
URL: https://github.com/apache/datafusion/pull/11462#issuecomment-2227203523
query 17 got some problem with json file. json files of query 7, 8, 9, 12,
15 are missing (I'll generate them in corresponding repo). I may need two more
PR to finish it.
--
Th
Lordworms opened a new pull request, #11462:
URL: https://github.com/apache/datafusion/pull/11462
## Which issue does this PR close?
part of #10710
Closes #.
## Rationale for this change
## What changes are included in this PR?
## Are these
Rachelint commented on PR #11461:
URL: https://github.com/apache/datafusion/pull/11461#issuecomment-2227203334
cc @alamb @goldmedal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific c
Rachelint opened a new pull request, #11461:
URL: https://github.com/apache/datafusion/pull/11461
## Which issue does this PR close?
Closes #11460 #6485
## Rationale for this change
See #11460
## What changes are included in this PR?
See title.
Rachelint commented on issue #11460:
URL: https://github.com/apache/datafusion/issues/11460#issuecomment-2227201019
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
Rachelint opened a new issue, #11460:
URL: https://github.com/apache/datafusion/issues/11460
### Is your feature request related to a problem or challenge?
Similar to #6485
Now the `map` and `make_map` udfs only return `ColumnarValue::Array` result,
that lead to some sql can't wor
2010YOUY01 commented on code in PR #11455:
URL: https://github.com/apache/datafusion/pull/11455#discussion_r1677007472
##
datafusion/physical-expr/src/expressions/binary.rs:
##
@@ -41,20 +39,23 @@ use datafusion_expr::type_coercion::binary::get_result_type;
use datafusion_expr:
Weijun-H opened a new pull request, #11459:
URL: https://github.com/apache/datafusion/pull/11459
## Which issue does this PR close?
Closes #11410
## Rationale for this change
## What changes are included in this PR?
## Are these changes test
jayzhan211 opened a new pull request, #11458:
URL: https://github.com/apache/datafusion/pull/11458
## Which issue does this PR close?
Closes #.
## Rationale for this change
## What changes are included in this PR?
## Are these changes tested
goldmedal commented on PR #11450:
URL: https://github.com/apache/datafusion/pull/11450#issuecomment-2227174772
Thanks @comphead @alamb !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
github-actions[bot] commented on PR #8143:
URL: https://github.com/apache/datafusion/pull/8143#issuecomment-2227164276
Thank you for your contribution. Unfortunately, this pull request is stale
because it has been open 60 days with no activity. Please remove the stale
label or comment or th
comphead commented on code in PR #11218:
URL: https://github.com/apache/datafusion/pull/11218#discussion_r1676989856
##
datafusion/physical-plan/src/lib.rs:
##
@@ -852,6 +852,30 @@ pub fn spill_record_batches(
Ok(writer.num_rows)
}
+/// Spill the `RecordBatch` to disk as
comphead commented on code in PR #11218:
URL: https://github.com/apache/datafusion/pull/11218#discussion_r1676989380
##
datafusion/physical-plan/src/lib.rs:
##
@@ -852,6 +852,30 @@ pub fn spill_record_batches(
Ok(writer.num_rows)
}
+/// Spill the `RecordBatch` to disk as
comphead commented on code in PR #11218:
URL: https://github.com/apache/datafusion/pull/11218#discussion_r1676989110
##
datafusion/execution/src/memory_pool/mod.rs:
##
@@ -220,6 +220,21 @@ impl MemoryReservation {
self.size = new_size
}
+/// Tries to free `ca
jayzhan211 commented on code in PR #11452:
URL: https://github.com/apache/datafusion/pull/11452#discussion_r1676985308
##
datafusion/sqllogictest/test_files/map.slt:
##
@@ -131,17 +131,23 @@ SELECT MAKE_MAP([1,2], ['a', 'b'], [3,4], ['b']);
{[1, 2]: [a, b], [3, 4]: [b]}
jayzhan211 commented on code in PR #11452:
URL: https://github.com/apache/datafusion/pull/11452#discussion_r1676984126
##
datafusion/functions-array/src/planner.rs:
##
@@ -97,6 +98,27 @@ impl ExprPlanner for ArrayFunctionPlanner {
) -> Result>> {
Ok(PlannerResult::
comphead commented on PR #11218:
URL: https://github.com/apache/datafusion/pull/11218#issuecomment-2227150960
I'll address all the comments shortly, and for multiple batches this comment
makes a lot of sense. I'll double check it
--
This is an automated message from the Apache Git Service
jayzhan211 closed pull request #10363: Standardize the separator in name
URL: https://github.com/apache/datafusion/pull/10363
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
aalexandrov commented on issue #6140:
URL: https://github.com/apache/datafusion/issues/6140#issuecomment-2227118998
@alamb can you please assign me here? I will take a stab at this next week.
If I succeed, you might be able to also close some other issues listed in the
enclosing epic as obs
efredine commented on code in PR #11289:
URL: https://github.com/apache/datafusion/pull/11289#discussion_r1676961201
##
datafusion/core/src/datasource/physical_plan/parquet/statistics.rs:
##
@@ -1234,7 +1250,87 @@ impl<'a> StatisticsConverter<'a> {
arrow_field,
aalexandrov commented on issue #10048:
URL: https://github.com/apache/datafusion/issues/10048#issuecomment-2227117963
I opened a PR (#11456) for the planner extensions required for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on t
aalexandrov opened a new pull request, #11456:
URL: https://github.com/apache/datafusion/pull/11456
## Which issue does this PR close?
Closes the planning part of #10048. The query from the issue will now fail
in the optimization phase similar to #6140.
## Rationale for this ch
alamb commented on code in PR #11425:
URL: https://github.com/apache/datafusion/pull/11425#discussion_r1676959195
##
docs/source/contributor-guide/architecture.md:
##
@@ -25,3 +25,54 @@ possible. You can find the most up to date version in the
[source code].
[crates.io docum
alamb commented on PR #11450:
URL: https://github.com/apache/datafusion/pull/11450#issuecomment-2227111734
🚀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe
alamb merged PR #11450:
URL: https://github.com/apache/datafusion/pull/11450
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb commented on PR #11417:
URL: https://github.com/apache/datafusion/pull/11417#issuecomment-2227111319
Thanks for the reviews @thinkharderdev and @avantgardnerio -- @joroKr21
it appears this branch has conflicts that need to be resolved. Once those are
sorted out I think we can merge
alamb commented on PR #11416:
URL: https://github.com/apache/datafusion/pull/11416#issuecomment-2227111090
Thanks again@
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
alamb merged PR #11416:
URL: https://github.com/apache/datafusion/pull/11416
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb closed issue #11389: Remove the uses of `termtree` dependency
URL: https://github.com/apache/datafusion/issues/11389
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To un
alamb commented on PR #11387:
URL: https://github.com/apache/datafusion/pull/11387#issuecomment-2227111002
Thanks again @lewiszlw
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific com
alamb merged PR #11387:
URL: https://github.com/apache/datafusion/pull/11387
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb closed issue #11350: Support serialization/deserialization for custom
physical exprs in proto
URL: https://github.com/apache/datafusion/issues/11350
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
Rachelint commented on issue #6485:
URL: https://github.com/apache/datafusion/issues/6485#issuecomment-2227106104
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To un
ozankabak commented on code in PR #11425:
URL: https://github.com/apache/datafusion/pull/11425#discussion_r1676942606
##
docs/source/contributor-guide/architecture.md:
##
@@ -25,3 +25,62 @@ possible. You can find the most up to date version in the
[source code].
[crates.io d
zhuliquan opened a new pull request, #11455:
URL: https://github.com/apache/datafusion/pull/11455
it's help for saving compile time for each evaluating of record batch
## Which issue does this PR close?
https://github.com/apache/datafusion/issues/11146
Closes #.
Throne3d commented on issue #2326:
URL: https://github.com/apache/datafusion/issues/2326#issuecomment-2227085035
I think #11445 is related to this epic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
dependabot[bot] opened a new pull request, #760:
URL: https://github.com/apache/datafusion-python/pull/760
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.80 to
0.1.81.
Release notes
Sourced from https://github.com/dtolnay/async-trait/releases";>async-trait's
dependabot[bot] opened a new pull request, #761:
URL: https://github.com/apache/datafusion-python/pull/761
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.68 to 2.0.71.
Release notes
Sourced from https://github.com/dtolnay/syn/releases";>syn's
releases.
2.0.71
Do
dependabot[bot] opened a new pull request, #759:
URL: https://github.com/apache/datafusion-python/pull/759
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.9.1 to 1.10.0.
Release notes
Sourced from https://github.com/uuid-rs/uuid/releases";>uuid's releases.
1.10.0
Depr
dependabot[bot] opened a new pull request, #758:
URL: https://github.com/apache/datafusion-python/pull/758
Bumps [prost-types](https://github.com/tokio-rs/prost) from 0.12.6 to 0.13.1.
Changelog
Sourced from https://github.com/tokio-rs/prost/blob/master/CHANGELOG.md";>prost-types's
dependabot[bot] closed pull request #743: build(deps): bump pyo3-build-config
from 0.21.2 to 0.22.0
URL: https://github.com/apache/datafusion-python/pull/743
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
dependabot[bot] opened a new pull request, #757:
URL: https://github.com/apache/datafusion-python/pull/757
Bumps [pyo3-build-config](https://github.com/pyo3/pyo3) from 0.21.2 to
0.22.1.
Release notes
Sourced from https://github.com/pyo3/pyo3/releases";>pyo3-build-config's
releases
dependabot[bot] commented on PR #743:
URL:
https://github.com/apache/datafusion-python/pull/743#issuecomment-2227067853
Superseded by #757.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
dependabot[bot] opened a new pull request, #756:
URL: https://github.com/apache/datafusion-python/pull/756
Bumps [prost](https://github.com/tokio-rs/prost) from 0.12.6 to 0.13.1.
Changelog
Sourced from https://github.com/tokio-rs/prost/blob/master/CHANGELOG.md";>prost's
changelog.
dependabot[bot] opened a new pull request, #755:
URL: https://github.com/apache/datafusion-python/pull/755
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.47.0
to 0.48.0.
Changelog
Sourced from https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.
nix010 commented on PR #11454:
URL: https://github.com/apache/datafusion/pull/11454#issuecomment-2227006931
My approach to this problem maybe a bit naive! open to suggestions.
Also where should I put the tests for this ?
```rust
use datafusion::prelude::*;
#[tokio::ma
nix010 opened a new pull request, #11454:
URL: https://github.com/apache/datafusion/pull/11454
## Which issue does this PR close?
Closes #11271 .
## Rationale for this change
## What changes are included in this PR?
## Are these changes test
comphead commented on code in PR #11450:
URL: https://github.com/apache/datafusion/pull/11450#discussion_r1676861772
##
docs/source/user-guide/crate-configuration.md:
##
@@ -141,6 +141,48 @@ To obtain a backtrace:
```bash
cargo build --features=backtrace
RUST_BACKTRACE=1 carg
andygrove merged PR #1041:
URL: https://github.com/apache/datafusion-ballista/pull/1041
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
goldmedal commented on code in PR #11452:
URL: https://github.com/apache/datafusion/pull/11452#discussion_r1676854128
##
datafusion/functions/src/core/mod.rs:
##
@@ -81,10 +80,6 @@ pub mod expr_fn {
coalesce,
"Returns `coalesce(args...)`, which evaluates to the
goldmedal commented on code in PR #11452:
URL: https://github.com/apache/datafusion/pull/11452#discussion_r1676853774
##
datafusion/functions-array/src/planner.rs:
##
@@ -97,6 +98,27 @@ impl ExprPlanner for ArrayFunctionPlanner {
) -> Result>> {
Ok(PlannerResult::P
goldmedal commented on code in PR #11452:
URL: https://github.com/apache/datafusion/pull/11452#discussion_r1676853439
##
datafusion/sqllogictest/test_files/map.slt:
##
@@ -131,17 +131,23 @@ SELECT MAKE_MAP([1,2], ['a', 'b'], [3,4], ['b']);
{[1, 2]: [a, b], [3, 4]: [b]}
andygrove merged PR #1022:
URL: https://github.com/apache/datafusion-ballista/pull/1022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
MohamedAbdeen21 opened a new pull request, #11453:
URL: https://github.com/apache/datafusion/pull/11453
## Which issue does this PR close?
Closes #9949.
## Rationale for this change
Recent version of sqlparser addresses some issues with DF
## What chang
andygrove opened a new pull request, #1041:
URL: https://github.com/apache/datafusion-ballista/pull/1041
# Which issue does this PR close?
Closes #.
# Rationale for this change
# What changes are included in this PR?
# Are there any user-facing cha
goldmedal opened a new pull request, #11452:
URL: https://github.com/apache/datafusion/pull/11452
## Which issue does this PR close?
Parietally solve #11434
## Rationale for this change
The benchmark result:
```
Gnuplot not found, using plotters backend
make_
aocsa commented on issue #596:
URL:
https://github.com/apache/datafusion-comet/issues/596#issuecomment-2226968494
https://github.com/user-attachments/assets/61cf5fe3-33be-48bf-abcf-150b2890ddaa";>
 {
}
.run();
}
+// test nested struct
+#[tokio::
andygrove closed issue #1035: 404 in repo Website:
https://datafusion.apache.org/ballista
URL: https://github.com/apache/datafusion-ballista/issues/1035
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
andygrove merged PR #1040:
URL: https://github.com/apache/datafusion-ballista/pull/1040
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
PedroMDuarte commented on PR #506:
URL: https://github.com/apache/datafusion-comet/pull/506#issuecomment-2226941038
> I would rather do this in QueryPlanSerde, E.g.
>
> ```
> case Log2(child) =>
> val childExpr = exprToProtoInternal(nullIfNegative(child),
input
thinkharderdev opened a new issue, #11451:
URL: https://github.com/apache/datafusion/issues/11451
### Is your feature request related to a problem or challenge?
Certain operators (eg `AggregateExec`, `SortExec`) are "greedy" in that they
will continue processing batches as long as the
andygrove opened a new pull request, #1040:
URL: https://github.com/apache/datafusion-ballista/pull/1040
# Which issue does this PR close?
Closes #.
# Rationale for this change
# What changes are included in this PR?
# Are there any user-facing cha
andygrove merged PR #1039:
URL: https://github.com/apache/datafusion-ballista/pull/1039
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
andygrove closed issue #1019: Setting job name is not reflected in the Web UI
URL: https://github.com/apache/datafusion-ballista/issues/1019
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
andygrove closed issue #1019: Setting job name is not reflected in the Web UI
URL: https://github.com/apache/datafusion-ballista/issues/1019
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
2010YOUY01 commented on issue #11408:
URL: https://github.com/apache/datafusion/issues/11408#issuecomment-2226912517
Note to myself: maybe duplicate
```
NoREC oracle violated:
Q1(result size 0):SELECT COUNT(*) FROM t2 table1 FULL JOIN t2 table1 ON
((table1.v1)<(table1.v2)|(-
timsaucer opened a new issue, #754:
URL: https://github.com/apache/datafusion-python/issues/754
**Is your feature request related to a problem or challenge? Please describe
what you are trying to do.**
This is a user enhancement request to make the python interface remove the
default `?t
timsaucer opened a new issue, #753:
URL: https://github.com/apache/datafusion-python/issues/753
**Is your feature request related to a problem or challenge? Please describe
what you are trying to do.**
To make a more user friendly interface, `Expr.cast()` should accept a python
type and
goldmedal commented on issue #11434:
URL: https://github.com/apache/datafusion/issues/11434#issuecomment-2226895700
I found the `MAP` literal isn't supported by the `sqlparser-rs`. I think we
need to implement this syntax on the `sqlparser-rs` side first. However, I
think I can have a PR fo
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676818082
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676816902
##
python/datafusion/functions.py:
##
@@ -15,9 +15,1731 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676814428
##
python/datafusion/record_batch.py:
##
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreeme
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676814381
##
python/datafusion/record_batch.py:
##
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreeme
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676813966
##
python/datafusion/substrait.py:
##
@@ -15,9 +15,156 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
jayzhan211 commented on issue #11419:
URL: https://github.com/apache/datafusion/issues/11419#issuecomment-2226873634
I think it would be a plus If it could be extended from existing `get_field`
function.
--
This is an automated message from the Apache Git Service.
To respond to the messa
jayzhan211 commented on issue #11436:
URL: https://github.com/apache/datafusion/issues/11436#issuecomment-2226871774
MakeMap is possible to be removed after #11434, so you can find out array
support for `MapFunc` only
--
This is an automated message from the Apache Git Service.
To respond
jayzhan211 commented on issue #11359:
URL: https://github.com/apache/datafusion/issues/11359#issuecomment-2226859404
> Cool, maybe I could think about _pulling down_ functions trait from `expr`
instead of _pulling up_ common things to `expr-common` 🤔
`ContextProvider` should be in the
goldmedal opened a new pull request, #11450:
URL: https://github.com/apache/datafusion/pull/11450
## Which issue does this PR close?
No exact issue.
## Rationale for this change
Printing the backtrace directly isn't user-friendly for identifying the root
cause. Here
alamb merged PR #11423:
URL: https://github.com/apache/datafusion/pull/11423
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb closed issue #11414: Crash bug from an inner join query (SQLancer)
URL: https://github.com/apache/datafusion/issues/11414
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
alamb commented on PR #11439:
URL: https://github.com/apache/datafusion/pull/11439#issuecomment-2226837373
Thanks @jonahgao !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
alamb merged PR #11439:
URL: https://github.com/apache/datafusion/pull/11439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
ozankabak commented on issue #11442:
URL: https://github.com/apache/datafusion/issues/11442#issuecomment-2226834443
It would be great to have one or two quarters where we focus on perf. I
think we are at a pretty good place in terms of extensibility/customizability
(evidenced by rapidly inc
jonahgao commented on code in PR #11423:
URL: https://github.com/apache/datafusion/pull/11423#discussion_r1676791583
##
datafusion/optimizer/src/analyzer/type_coercion.rs:
##
@@ -151,9 +151,27 @@ impl<'a> TypeCoercionRewriter<'a> {
})
.collect::>>()?;
berkaysynnada opened a new pull request, #11449:
URL: https://github.com/apache/datafusion/pull/11449
## Which issue does this PR close?
Closes #.
## Rationale for this change
Currently, DF does not utilize `execute_input_stream()` outside of the
`DataSin
jayzhan211 opened a new pull request, #11448:
URL: https://github.com/apache/datafusion/pull/11448
## Which issue does this PR close?
Closes #.
## Rationale for this change
## What changes are included in this PR?
## Are these changes tested
jayzhan211 commented on code in PR #11424:
URL: https://github.com/apache/datafusion/pull/11424#discussion_r1676778820
##
datafusion/core/tests/dataframe/mod.rs:
##
@@ -212,7 +212,6 @@ async fn test_count_wildcard_on_aggregate() -> Result<()> {
let sql_results = ctx
90 matches
Mail list logo
