HyukjinKwon commented on PR #45351:
URL: https://github.com/apache/spark/pull/45351#issuecomment-1975909802
That's fine. My point is that please describe it in the PR description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
HyukjinKwon commented on PR #45366:
URL: https://github.com/apache/spark/pull/45366#issuecomment-1975906508
https://github.com/HyukjinKwon/spark/actions/runs/8136924661/job/22234172189
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SteNicholas commented on PR #45351:
URL: https://github.com/apache/spark/pull/45351#issuecomment-1975895901
> The original PR does not have its test together (#44021). @SteNicholas
Mind describing how you tested this?
@HyukjinKwon, I only tested this bump in production environment of
ericm-db commented on code in PR #45359:
URL: https://github.com/apache/spark/pull/45359#discussion_r1510692025
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateSuite.scala:
##
@@ -195,6 +179,20 @@ class TransformWithStateSuite extends
LuciferYang commented on PR #45367:
URL: https://github.com/apache/spark/pull/45367#issuecomment-1975880459
> this pr is already submitted in master branch
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
TakawaAkirayo commented on PR #45367:
URL: https://github.com/apache/spark/pull/45367#issuecomment-1975874852
@LuciferYang The master branch have this issue too, this pr is already
submitted in master branch. I add [3.5] in the title is because I though I must
have a target version for PR.
cloud-fan commented on code in PR #45282:
URL: https://github.com/apache/spark/pull/45282#discussion_r1510653731
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala:
##
@@ -1009,18 +1009,19 @@ abstract class TreeNode[BaseType <: TreeNode[BaseType]]
cloud-fan commented on code in PR #45282:
URL: https://github.com/apache/spark/pull/45282#discussion_r1510653381
##
sql/core/src/main/scala/org/apache/spark/sql/execution/ExplainUtils.scala:
##
@@ -76,7 +76,7 @@ object ExplainUtils extends AdaptiveSparkPlanHelper {
* 1.
cloud-fan commented on code in PR #45282:
URL: https://github.com/apache/spark/pull/45282#discussion_r1510653381
##
sql/core/src/main/scala/org/apache/spark/sql/execution/ExplainUtils.scala:
##
@@ -76,7 +76,7 @@ object ExplainUtils extends AdaptiveSparkPlanHelper {
* 1.
ulysses-you commented on code in PR #45282:
URL: https://github.com/apache/spark/pull/45282#discussion_r1510631629
##
sql/core/src/main/scala/org/apache/spark/sql/execution/ExplainUtils.scala:
##
@@ -76,7 +76,7 @@ object ExplainUtils extends AdaptiveSparkPlanHelper {
* 1.
anton5798 commented on PR #45133:
URL: https://github.com/apache/spark/pull/45133#issuecomment-1975792360
@cloud-fan updated the code & pr desc!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
ulysses-you commented on code in PR #45282:
URL: https://github.com/apache/spark/pull/45282#discussion_r1510625804
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala:
##
@@ -1009,18 +1009,19 @@ abstract class TreeNode[BaseType <:
cloud-fan commented on code in PR #45282:
URL: https://github.com/apache/spark/pull/45282#discussion_r1510612237
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala:
##
@@ -1009,18 +1009,19 @@ abstract class TreeNode[BaseType <: TreeNode[BaseType]]
cloud-fan commented on PR #45282:
URL: https://github.com/apache/spark/pull/45282#issuecomment-1975770696
I think we can't remove the mutable states (TreeNodeTag) any time soon, we
must live with it and the call sites should be careful when setting it. For
EXPLAIN, my preference is to have
cloud-fan commented on PR #45282:
URL: https://github.com/apache/spark/pull/45282#issuecomment-1975767660
cc @robreeves @liuzqt
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
cloud-fan commented on code in PR #45282:
URL: https://github.com/apache/spark/pull/45282#discussion_r1510609021
##
sql/core/src/main/scala/org/apache/spark/sql/execution/ExplainUtils.scala:
##
@@ -76,7 +76,7 @@ object ExplainUtils extends AdaptiveSparkPlanHelper {
* 1.
doki23 commented on PR #45181:
URL: https://github.com/apache/spark/pull/45181#issuecomment-1975750721
> @doki23 Do you have time to continue working on the pull request? I seems
to me that it's close to completion.
@dtarima Of course, I'm glad to move it forward.
--
This is an
HyukjinKwon commented on PR #45353:
URL: https://github.com/apache/spark/pull/45353#issuecomment-1975750226
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon closed pull request #45353: [SPARK-46094][CORE][FOLLOW-UP] Mark
`writing` as volatile at ExecutorJVMProfiler
URL: https://github.com/apache/spark/pull/45353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
HyukjinKwon commented on code in PR #45363:
URL: https://github.com/apache/spark/pull/45363#discussion_r1510598001
##
python/pyspark/sql/group.py:
##
@@ -450,6 +446,10 @@ def pivot(self, pivot_col: str, values:
Optional[List["LiteralType"]] = None) ->
values : list,
cloud-fan commented on PR #45133:
URL: https://github.com/apache/spark/pull/45133#issuecomment-1975747191
> Semantically, we should wrap in any_value(), but any_value() throws
RuntimeReplaceableAggregate.aggBufferAttributes should not be called and is not
fully supported
It's
cloud-fan commented on code in PR #45357:
URL: https://github.com/apache/spark/pull/45357#discussion_r1510594245
##
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CoalesceShufflePartitions.scala:
##
@@ -126,9 +126,12 @@ case class
LuciferYang commented on PR #45367:
URL: https://github.com/apache/spark/pull/45367#issuecomment-1975736636
@TakawaAkirayo Doesn't master have this issue? The pr should be submitted to
the `master` first, then backported to other branches if need.
--
This is an automated message from the
LuciferYang commented on code in PR #45366:
URL: https://github.com/apache/spark/pull/45366#discussion_r1510589219
##
core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala:
##
@@ -731,6 +731,7 @@ class MasterSuite extends SparkFunSuite
ulysses-you commented on code in PR #45357:
URL: https://github.com/apache/spark/pull/45357#discussion_r1510571235
##
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CoalesceShufflePartitions.scala:
##
@@ -126,9 +126,12 @@ case class
dengziming commented on code in PR #45288:
URL: https://github.com/apache/spark/pull/45288#discussion_r1510560284
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala:
##
@@ -565,7 +565,7 @@ abstract class UnaryExpression extends Expression
dengziming commented on code in PR #45288:
URL: https://github.com/apache/spark/pull/45288#discussion_r1510559579
##
common/utils/src/main/resources/error/error-classes.json:
##
@@ -2884,6 +2890,12 @@
],
"sqlState" : "42809"
},
+ "NOT_A_UNRESOLVED_ENCODER" : {
nchammas commented on code in PR #45363:
URL: https://github.com/apache/spark/pull/45363#discussion_r151059
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala:
##
@@ -259,18 +259,19 @@ class RelationalGroupedDataset
panbingkun closed pull request #45368: [WIP][SPARK-47265][SQL][TESTS] Enable
test of v2 data sources in `FileBasedDataSourceSuite`
URL: https://github.com/apache/spark/pull/45368
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
dtarima commented on PR #45181:
URL: https://github.com/apache/spark/pull/45181#issuecomment-1975636851
@doki23 Do you have time to continue working on the pull request? I seems to
me that it's close to completion.
--
This is an automated message from the Apache Git Service.
To respond
dongjoon-hyun commented on PR #45366:
URL: https://github.com/apache/spark/pull/45366#issuecomment-1975628705
Thank you for trying this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
cloud-fan commented on code in PR #45357:
URL: https://github.com/apache/spark/pull/45357#discussion_r1510551047
##
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CoalesceShufflePartitions.scala:
##
@@ -126,9 +126,12 @@ case class
LuciferYang commented on code in PR #45368:
URL: https://github.com/apache/spark/pull/45368#discussion_r1510546756
##
sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala:
##
@@ -500,8 +500,7 @@ class FileBasedDataSourceSuite extends QueryTest
panbingkun opened a new pull request, #45368:
URL: https://github.com/apache/spark/pull/45368
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
###
LuciferYang commented on PR #45344:
URL: https://github.com/apache/spark/pull/45344#issuecomment-1975570775
late LGTM.
Before this, I always increase `file descriptors` as a workaround ...
--
This is an automated message from the Apache Git Service.
To respond to the message,
cloud-fan closed pull request #45176: [SPARK-46973][SQL] Skip V2 table lookup
when a table is in V1 table cache
URL: https://github.com/apache/spark/pull/45176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
cloud-fan commented on PR #45176:
URL: https://github.com/apache/spark/pull/45176#issuecomment-1975565892
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
wForget commented on PR #45353:
URL: https://github.com/apache/spark/pull/45353#issuecomment-1975565457
> if it is called after finishWriting()
This seems difficult to reproduce, so I have trouble adding a test case to
cover this fix. But I guess the `if (!writing) {` judgment is to
wForget commented on PR #45353:
URL: https://github.com/apache/spark/pull/45353#issuecomment-1975557375
> @wForget mind filling:
>
> > Does this PR introduce any user-facing change?
> > How was this patch tested?
> > Was this patch authored or co-authored using generative AI
cloud-fan closed pull request #45290: [SPARK-46834][SQL][Collations] Support
for aggregates
URL: https://github.com/apache/spark/pull/45290
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
cloud-fan commented on PR #45290:
URL: https://github.com/apache/spark/pull/45290#issuecomment-1975551130
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
waywtdcc commented on PR #40744:
URL: https://github.com/apache/spark/pull/40744#issuecomment-1975541287
Can this PR be merged? I also encountered this scenario
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
ulysses-you commented on code in PR #45357:
URL: https://github.com/apache/spark/pull/45357#discussion_r1510518917
##
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CoalesceShufflePartitions.scala:
##
@@ -126,9 +126,12 @@ case class
LuciferYang commented on PR #45356:
URL: https://github.com/apache/spark/pull/45356#issuecomment-1975533193
Merged into master for Spark 4.0. Thanks @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
LuciferYang closed pull request #45356: [SPARK-47246][SQL] Replace
`InternalRow.fromSeq` with `new GenericInternalRow` to save a collection
conversion
URL: https://github.com/apache/spark/pull/45356
--
This is an automated message from the Apache Git Service.
To respond to the message,
TakawaAkirayo opened a new pull request, #45367:
URL: https://github.com/apache/spark/pull/45367
### What changes were proposed in this pull request?
Add config
spark.scheduler.listenerbus.eventqueue.waitForEventDispatchExitOnStop(default
true).
Before this PR: The event
TakawaAkirayo closed pull request #45364: [SPARK-47253][CORE][3.5] Allow
LiveEventBus to stop without the completely draining of event queue
URL: https://github.com/apache/spark/pull/45364
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
HyukjinKwon commented on PR #45351:
URL: https://github.com/apache/spark/pull/45351#issuecomment-1975495691
The original PR does not have its test together
(https://github.com/apache/spark/pull/44021). @SteNicholas Mind describing how
you tested this?
--
This is an automated message
panbingkun commented on PR #45033:
URL: https://github.com/apache/spark/pull/45033#issuecomment-1975489055
+1, LGTM. Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
panbingkun commented on PR #45235:
URL: https://github.com/apache/spark/pull/45235#issuecomment-1975486898
Thank you for helping to fix this issue. Thank you very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
HyukjinKwon closed pull request #45334: [MINOR] Update outdated comments for
class `o.a.s.s.functions`
URL: https://github.com/apache/spark/pull/45334
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
HyukjinKwon commented on PR #45334:
URL: https://github.com/apache/spark/pull/45334#issuecomment-1975469964
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on PR #45353:
URL: https://github.com/apache/spark/pull/45353#issuecomment-1975466398
@wForget mind filling:
> Does this PR introduce any user-facing change?
> How was this patch tested?
> Was this patch authored or co-authored using generative AI tooling?
HyukjinKwon closed pull request #45358: [SPARK-47249][CONNECT] Fix bug where
all connect executions are considered abandoned regardless of their actual
status
URL: https://github.com/apache/spark/pull/45358
--
This is an automated message from the Apache Git Service.
To respond to the
HyukjinKwon commented on PR #45358:
URL: https://github.com/apache/spark/pull/45358#issuecomment-1975462556
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
SteNicholas commented on PR #45351:
URL: https://github.com/apache/spark/pull/45351#issuecomment-1975460640
@parthchandra, thank you try it out. Have you tried anything wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
HyukjinKwon commented on code in PR #45363:
URL: https://github.com/apache/spark/pull/45363#discussion_r1510453526
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala:
##
@@ -259,18 +259,19 @@ class RelationalGroupedDataset
HyukjinKwon commented on PR #45366:
URL: https://github.com/apache/spark/pull/45366#issuecomment-1975451930
https://github.com/HyukjinKwon/spark/actions/runs/8134128130
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
github-actions[bot] closed pull request #43871: [MINOR][DOCS] Clarify sort
behaviour for structs
URL: https://github.com/apache/spark/pull/43871
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
github-actions[bot] closed pull request #43987: [MINOR][SS] Relax access
constraints on fields of FileStreamSink so the class is extendable, split long
method
URL: https://github.com/apache/spark/pull/43987
--
This is an automated message from the Apache Git Service.
To respond to the
github-actions[bot] closed pull request #43981: [MINOR][DOCS] Add official
documentation on the natural order of structs. Second proposal.
URL: https://github.com/apache/spark/pull/43981
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
github-actions[bot] closed pull request #43938: [SPARK-46037][SQL] When Left
Join build Left and codegen is turned off, ShuffledHashJoinExec may result in
incorrect results
URL: https://github.com/apache/spark/pull/43938
--
This is an automated message from the Apache Git Service.
To
LuciferYang opened a new pull request, #45365:
URL: https://github.com/apache/spark/pull/45365
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
###
63 matches
Mail list logo
