advancedxy commented on PR #46265:
URL: https://github.com/apache/spark/pull/46265#issuecomment-2082705978
> It appears that this PR can enhance performance when there are a large
number of partitions. Could you please share the test results from a real
DatasourceV2 table, such as Iceberg?
yabola commented on PR #46265:
URL: https://github.com/apache/spark/pull/46265#issuecomment-2082672856
@advancedxy It appears that this PR can enhance performance when there are
a large number of partitions. Could you please share the test results from a
real DatasourceV2 table?
advancedxy commented on PR #46265:
URL: https://github.com/apache/spark/pull/46265#issuecomment-2082648145
> In addition, is there a better expireAfterAccess configuration for
NonFateSharingCache?
hmmm, maybe. However, the memory usage of cache should be relatively low.
Let's wait
grundprinzip commented on PR #46182:
URL: https://github.com/apache/spark/pull/46182#issuecomment-2082639091
One thing I'm wondering if it might work out of the box is the ability to
specify an ephemeral port for the spark connect service and pick this up during
startup.
This might
tgravescs commented on code in PR #42352:
URL: https://github.com/apache/spark/pull/42352#discussion_r1583032683
##
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala:
##
@@ -340,6 +385,45 @@ private[spark] class ExecutorAllocationManager(
}
}
+ /**
grundprinzip commented on code in PR #46182:
URL: https://github.com/apache/spark/pull/46182#discussion_r1583029459
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectService.scala:
##
@@ -344,35 +357,79 @@ object SparkConnectService
xuzifu666 commented on PR #46279:
URL: https://github.com/apache/spark/pull/46279#issuecomment-2082514838
> Is there any other outstanding issue similar to SPARK-48010 and this one?
yes，similar Sences@yaooqinn
--
This is an automated message from the Apache Git Service.
To respond
yaooqinn commented on PR #46279:
URL: https://github.com/apache/spark/pull/46279#issuecomment-2082510201
Is there any other outstanding issue similar to SPARK-48010 and this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
yaooqinn closed pull request #46231: [SPARK-47994][SQL] Fix bug with CASE WHEN
column filter push down in SQLServer
URL: https://github.com/apache/spark/pull/46231
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
yaooqinn commented on PR #46231:
URL: https://github.com/apache/spark/pull/46231#issuecomment-208257
Thank you @stefanbuk-db
Merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
zhengruifeng opened a new pull request, #46281:
URL: https://github.com/apache/spark/pull/46281
### What changes were proposed in this pull request?
Cache `DataFrame.isStreaming`
### Why are the changes needed?
In PS, `DataFrame.isStreaming` is used in the construction of
zhengruifeng commented on PR #46277:
URL: https://github.com/apache/spark/pull/46277#issuecomment-2082446455
thanks @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon closed pull request #46277: [SPARK-48039][PYTHON][CONNECT] Update
the error class for `group.apply`
URL: https://github.com/apache/spark/pull/46277
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
HyukjinKwon commented on PR #46277:
URL: https://github.com/apache/spark/pull/46277#issuecomment-2082435935
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on code in PR #46278:
URL: https://github.com/apache/spark/pull/46278#discussion_r1582873705
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/execution/ExecuteThreadRunner.scala:
##
@@ -177,6 +177,10 @@ private[connect] class
grundprinzip commented on code in PR #46182:
URL: https://github.com/apache/spark/pull/46182#discussion_r1582855870
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectService.scala:
##
@@ -344,35 +357,79 @@ object SparkConnectService
grundprinzip commented on code in PR #46182:
URL: https://github.com/apache/spark/pull/46182#discussion_r1582854628
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectService.scala:
##
@@ -344,35 +357,79 @@ object SparkConnectService
grundprinzip commented on code in PR #46182:
URL: https://github.com/apache/spark/pull/46182#discussion_r1582852496
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectServer.scala:
##
@@ -36,21 +32,21 @@ object SparkConnectServer extends
grundprinzip commented on code in PR #46182:
URL: https://github.com/apache/spark/pull/46182#discussion_r1582848826
##
core/src/main/scala/org/apache/spark/SparkContext.scala:
##
@@ -2302,6 +2302,11 @@ class SparkContext(config: SparkConf) extends Logging {
}
grundprinzip commented on code in PR #46182:
URL: https://github.com/apache/spark/pull/46182#discussion_r1582846934
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectServer.scala:
##
@@ -36,21 +32,21 @@ object SparkConnectServer extends
xieshuaihu commented on code in PR #46278:
URL: https://github.com/apache/spark/pull/46278#discussion_r1582847360
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/execution/ExecuteThreadRunner.scala:
##
@@ -177,6 +177,10 @@ private[connect] class
HyukjinKwon commented on code in PR #46278:
URL: https://github.com/apache/spark/pull/46278#discussion_r1582843351
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/execution/ExecuteThreadRunner.scala:
##
@@ -177,6 +177,10 @@ private[connect] class
xieshuaihu commented on code in PR #46278:
URL: https://github.com/apache/spark/pull/46278#discussion_r1582838182
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/execution/ExecuteThreadRunner.scala:
##
@@ -177,6 +177,10 @@ private[connect] class
olaky commented on code in PR #46280:
URL: https://github.com/apache/spark/pull/46280#discussion_r1582837435
##
sql/api/src/main/scala/org/apache/spark/sql/types/DataType.scala:
##
@@ -251,13 +262,25 @@ object DataType {
messageParameters = Map("invalidType" ->
xuzifu666 commented on PR #46279:
URL: https://github.com/apache/spark/pull/46279#issuecomment-2082376106
> Can you fill the PR description please?
Done @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
HyukjinKwon commented on code in PR #46278:
URL: https://github.com/apache/spark/pull/46278#discussion_r1582831117
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/execution/ExecuteThreadRunner.scala:
##
@@ -177,6 +177,10 @@ private[connect] class
cxzl25 commented on code in PR #46273:
URL: https://github.com/apache/spark/pull/46273#discussion_r1582827672
##
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala:
##
@@ -85,8 +86,10 @@ class AdaptiveQueryExecSuite
stefankandic opened a new pull request, #46280:
URL: https://github.com/apache/spark/pull/46280
### What changes were proposed in this pull request?
Changing serialization and deserialization of collated strings so that the
collation information is put in the metadata of each
xieshuaihu commented on code in PR #46278:
URL: https://github.com/apache/spark/pull/46278#discussion_r1582815755
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/execution/ExecuteThreadRunner.scala:
##
@@ -177,6 +177,10 @@ private[connect] class
HyukjinKwon commented on PR #46279:
URL: https://github.com/apache/spark/pull/46279#issuecomment-2082340036
Can you fill the PR description please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
HyukjinKwon commented on code in PR #46279:
URL: https://github.com/apache/spark/pull/46279#discussion_r1582814873
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##
@@ -3816,7 +3816,7 @@ class Analyzer(override val catalogManager:
xuzifu666 commented on PR #46279:
URL: https://github.com/apache/spark/pull/46279#issuecomment-2082334931
@dongjoon-hyun PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on code in PR #46278:
URL: https://github.com/apache/spark/pull/46278#discussion_r1582812044
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/execution/ExecuteThreadRunner.scala:
##
@@ -177,6 +177,10 @@ private[connect] class
xuzifu666 opened a new pull request, #46279:
URL: https://github.com/apache/spark/pull/46279
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How
pan3793 commented on code in PR #46184:
URL: https://github.com/apache/spark/pull/46184#discussion_r1582805042
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala:
##
@@ -73,7 +73,8 @@ private[spark] class
pan3793 commented on code in PR #46184:
URL: https://github.com/apache/spark/pull/46184#discussion_r1582805042
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala:
##
@@ -73,7 +73,8 @@ private[spark] class
pan3793 commented on code in PR #46184:
URL: https://github.com/apache/spark/pull/46184#discussion_r1582805042
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala:
##
@@ -73,7 +73,8 @@ private[spark] class
xieshuaihu opened a new pull request, #46278:
URL: https://github.com/apache/spark/pull/46278
### What changes were proposed in this pull request?
This patch makes spark connect supporting set scheduler pool name like
vanilla spark.
A new field `scheduler_pool` in
pan3793 commented on code in PR #46184:
URL: https://github.com/apache/spark/pull/46184#discussion_r1582767098
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverServiceFeatureStep.scala:
##
@@ -87,16 +77,19 @@ private[spark] class
cloud-fan closed pull request #45791: [SPARK-47567][SQL] Support LOCATE
function to work with collated strings
URL: https://github.com/apache/spark/pull/45791
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
miland-db commented on PR #45791:
URL: https://github.com/apache/spark/pull/45791#issuecomment-2082250288
jenkins merge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
cloud-fan commented on PR #45791:
URL: https://github.com/apache/spark/pull/45791#issuecomment-2082251045
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
zhengruifeng opened a new pull request, #46277:
URL: https://github.com/apache/spark/pull/46277
### What changes were proposed in this pull request?
Update the error class for `group.apply`
### Why are the changes needed?
pan3793 commented on PR #46276:
URL: https://github.com/apache/spark/pull/46276#issuecomment-2082230596
cc @dongjoon-hyun @yaooqinn @LuciferYang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
yaooqinn commented on PR #46231:
URL: https://github.com/apache/spark/pull/46231#issuecomment-2082208303
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
cloud-fan closed pull request #46209: [SPARK-47939][SQL] Implement a new
Analyzer rule to move ParameterizedQuery inside ExplainCommand and
DescribeQueryCommand
URL: https://github.com/apache/spark/pull/46209
--
This is an automated message from the Apache Git Service.
To respond to the
cloud-fan commented on PR #46209:
URL: https://github.com/apache/spark/pull/46209#issuecomment-2082201660
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
stevomitric commented on PR #45791:
URL: https://github.com/apache/spark/pull/45791#issuecomment-2082204032
+1 LGTM, photon checks will be handled in this pr
https://github.com/databricks/runtime/pull/91246
--
This is an automated message from the Apache Git Service.
To respond to the
pan3793 commented on code in PR #46184:
URL: https://github.com/apache/spark/pull/46184#discussion_r1582734681
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala:
##
@@ -749,13 +749,52 @@ private[spark] object Config extends Logging {
yaooqinn commented on code in PR #46231:
URL: https://github.com/apache/spark/pull/46231#discussion_r1582720271
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala:
##
@@ -92,6 +92,7 @@ private case class MsSqlServerDialect() extends JdbcDialect {
yaooqinn commented on code in PR #46231:
URL: https://github.com/apache/spark/pull/46231#discussion_r1582719516
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala:
##
@@ -92,6 +92,7 @@ private case class MsSqlServerDialect() extends JdbcDialect {
stefanbuk-db commented on code in PR #46231:
URL: https://github.com/apache/spark/pull/46231#discussion_r1582716492
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala:
##
@@ -92,6 +92,7 @@ private case class MsSqlServerDialect() extends JdbcDialect {
yaooqinn commented on code in PR #46231:
URL: https://github.com/apache/spark/pull/46231#discussion_r1582712347
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala:
##
@@ -92,6 +92,7 @@ private case class MsSqlServerDialect() extends JdbcDialect {
stefanbuk-db commented on PR #46231:
URL: https://github.com/apache/spark/pull/46231#issuecomment-2082141133
cc @yaooqinn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
pan3793 commented on code in PR #46184:
URL: https://github.com/apache/spark/pull/46184#discussion_r1582682967
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala:
##
@@ -94,6 +96,20 @@ private[spark] class KubernetesDriverConf(
pan3793 commented on code in PR #46184:
URL: https://github.com/apache/spark/pull/46184#discussion_r1582682648
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala:
##
@@ -94,6 +96,20 @@ private[spark] class KubernetesDriverConf(
pan3793 opened a new pull request, #46276:
URL: https://github.com/apache/spark/pull/46276
### What changes were proposed in this pull request?
Promote `driverServiceName` from `DriverServiceFeatureStep` to
`KubernetesDriverConf`.
### Why are the changes needed?
yaooqinn commented on PR #46266:
URL: https://github.com/apache/spark/pull/46266#issuecomment-2082073767
Witnessed an extremely slow benchmark
https://github.com/apache/spark/pull/45453#issuecomment-2082071977
--
This is an automated message from the Apache Git Service.
To respond to the
HyukjinKwon commented on code in PR #45977:
URL: https://github.com/apache/spark/pull/45977#discussion_r1582652464
##
python/pyspark/sql/datasource_internal.py:
##
@@ -0,0 +1,146 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
yaooqinn commented on PR #45453:
URL: https://github.com/apache/spark/pull/45453#issuecomment-2082071977
Hey guys,
I am currently regenerating the complete benchmark result with 20 jobs
running simultaneously. Each job usually takes around 10 to 30 minutes to
complete. However, the
HeartSaVioR commented on code in PR #45977:
URL: https://github.com/apache/spark/pull/45977#discussion_r1582649481
##
python/pyspark/sql/datasource_internal.py:
##
@@ -0,0 +1,146 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license
HyukjinKwon closed pull request #46275: [SPARK-48002][PYTHON][SS][TESTS] Adds
sleep before event testing after query termination
URL: https://github.com/apache/spark/pull/46275
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
HyukjinKwon commented on PR #46275:
URL: https://github.com/apache/spark/pull/46275#issuecomment-2082058695
Merged to master.
I will revert this if that persists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
AngersZh commented on code in PR #46263:
URL: https://github.com/apache/spark/pull/46263#discussion_r1582618679
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/InjectRuntimeFilter.scala:
##
@@ -120,33 +132,61 @@ object InjectRuntimeFilter extends
HyukjinKwon commented on PR #46275:
URL: https://github.com/apache/spark/pull/46275#issuecomment-208168
cc @WweiL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
HyukjinKwon opened a new pull request, #46275:
URL: https://github.com/apache/spark/pull/46275
### What changes were proposed in this pull request?
This PR is a followup of https://github.com/apache/spark/pull/46237 that
makes to wait 5 secs after the query termination to make sure
cloud-fan commented on code in PR #46263:
URL: https://github.com/apache/spark/pull/46263#discussion_r1582597230
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/InjectRuntimeFilter.scala:
##
@@ -120,33 +132,61 @@ object InjectRuntimeFilter extends
pan3793 commented on PR #46184:
URL: https://github.com/apache/spark/pull/46184#issuecomment-2081981982
@dongjoon-hyun sorry for late, I'm a little busy these days, will address
comments soon
--
This is an automated message from the Apache Git Service.
To respond to the message, please
AngersZh commented on code in PR #46263:
URL: https://github.com/apache/spark/pull/46263#discussion_r1582589876
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/InjectRuntimeFilter.scala:
##
@@ -120,34 +132,49 @@ object InjectRuntimeFilter extends
AngersZh commented on code in PR #46263:
URL: https://github.com/apache/spark/pull/46263#discussion_r1582582142
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/InjectRuntimeFilter.scala:
##
@@ -120,34 +132,49 @@ object InjectRuntimeFilter extends
dongjoon-hyun closed pull request #46274: [MINOR][DOCS] Remove space in the
middle of configuration name in Arrow-optimized Python UDF page
URL: https://github.com/apache/spark/pull/46274
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun commented on PR #46274:
URL: https://github.com/apache/spark/pull/46274#issuecomment-2081920450
Merged to master~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dongjoon-hyun closed pull request #46271: [SPARK-48036][DOCS] Update
`sql-ref-ansi-compliance.md` and `sql-ref-identifier.md`
URL: https://github.com/apache/spark/pull/46271
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
dongjoon-hyun commented on PR #46271:
URL: https://github.com/apache/spark/pull/46271#issuecomment-2081891190
I attached the screenshots too.
Merged to master~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
HyukjinKwon opened a new pull request, #46274:
URL: https://github.com/apache/spark/pull/46274
### What changes were proposed in this pull request?
This PR removes a space in the middle of configuration name in
Arrow-optimized Python UDF page.
![Screenshot 2024-04-29 at 1 53
dongjoon-hyun commented on PR #46271:
URL: https://github.com/apache/spark/pull/46271#issuecomment-2081887536
Thank you for helping me revise this doc, @yaooqinn !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
dongjoon-hyun commented on code in PR #46271:
URL: https://github.com/apache/spark/pull/46271#discussion_r1582534717
##
docs/sql-ref-ansi-compliance.md:
##
@@ -67,10 +67,8 @@ The following subsections present behaviour changes in
arithmetic operations, ty
### Arithmetic
dongjoon-hyun closed pull request #46258: [SPARK-48029][INFRA] Update the
packages name removed in building the spark docker image
URL: https://github.com/apache/spark/pull/46258
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
dongjoon-hyun commented on PR #46258:
URL: https://github.com/apache/spark/pull/46258#issuecomment-2081880744
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
cxzl25 opened a new pull request, #46273:
URL: https://github.com/apache/spark/pull/46273
### What changes were proposed in this pull request?
This PR aims to fix SortShuffleWriter lacks shuffle write related metrics
resulting in potentially inaccurate data.
### Why are the
yaooqinn commented on PR #46270:
URL: https://github.com/apache/spark/pull/46270#issuecomment-2081840828
Merged to master(4.0.0), 3.5.2 and 3.4.4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
yaooqinn closed pull request #46270: [SPARK-48034][TESTS] NullPointerException
in MapStatusesSerDeserBenchmark
URL: https://github.com/apache/spark/pull/46270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
dongjoon-hyun commented on PR #46271:
URL: https://github.com/apache/spark/pull/46271#issuecomment-2081838017
Thank you so much, @yaooqinn !
I updated the PR according to your comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
yaooqinn commented on PR #46270:
URL: https://github.com/apache/spark/pull/46270#issuecomment-2081825069
Thank you very much @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
yaooqinn commented on PR #46271:
URL: https://github.com/apache/spark/pull/46271#issuecomment-2081821772
LGTM, only [content
here](https://github.com/apache/spark/pull/46271/files#diff-54eee79bd27cf5ca1288b078a7b0b1b5ae8ae9d8b4ee7fb75f0b0c7cdaef0da8L70-L73)
might need further revison
panbingkun commented on PR #46264:
URL: https://github.com/apache/spark/pull/46264#issuecomment-2081820429
cc @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
panbingkun commented on code in PR #46264:
URL: https://github.com/apache/spark/pull/46264#discussion_r1582501170
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -124,8 +130,8 @@ object LogKeys {
case object DIFF_DELTA extends LogKey
case
dongjoon-hyun commented on PR #46270:
URL: https://github.com/apache/spark/pull/46270#issuecomment-2081817478
Feel free to merge and backport wherever you need this, @yaooqinn .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
dongjoon-hyun commented on PR #46271:
URL: https://github.com/apache/spark/pull/46271#issuecomment-2081817738
Could you review this documentation PR, @yaooqinn ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
panbingkun commented on code in PR #46264:
URL: https://github.com/apache/spark/pull/46264#discussion_r1582501464
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -191,20 +203,23 @@ object LogKeys {
case object HIVE_OPERATION_TYPE extends LogKey
panbingkun commented on code in PR #46264:
URL: https://github.com/apache/spark/pull/46264#discussion_r1582501170
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -124,8 +130,8 @@ object LogKeys {
case object DIFF_DELTA extends LogKey
case
panbingkun commented on code in PR #46264:
URL: https://github.com/apache/spark/pull/46264#discussion_r1582501097
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -111,7 +117,7 @@ object LogKeys {
case object DATA_FILE_NUM extends LogKey
case
panbingkun commented on code in PR #46264:
URL: https://github.com/apache/spark/pull/46264#discussion_r1582500958
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -191,20 +203,23 @@ object LogKeys {
case object HIVE_OPERATION_TYPE extends LogKey
panbingkun commented on code in PR #46264:
URL: https://github.com/apache/spark/pull/46264#discussion_r1582500836
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -276,31 +299,40 @@ object LogKeys {
case object NUM_FILES_REUSED extends LogKey
panbingkun commented on code in PR #46264:
URL: https://github.com/apache/spark/pull/46264#discussion_r1582500556
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -349,11 +387,11 @@ object LogKeys {
case object RETRY_COUNT extends LogKey
case
yaooqinn commented on PR #46266:
URL: https://github.com/apache/spark/pull/46266#issuecomment-2081812844
Pending CI results
https://github.com/yaooqinn/spark/actions/runs/8872679083
https://github.com/yaooqinn/spark/actions/runs/8872179637
--
This is an automated message
dongjoon-hyun opened a new pull request, #46271:
URL: https://github.com/apache/spark/pull/46271
…
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
yaooqinn commented on code in PR #46270:
URL: https://github.com/apache/spark/pull/46270#discussion_r1582497889
##
core/src/test/scala/org/apache/spark/MapStatusesSerDeserBenchmark.scala:
##
@@ -123,7 +123,6 @@ object MapStatusesSerDeserBenchmark extends BenchmarkBase {
}
yaooqinn commented on code in PR #46270:
URL: https://github.com/apache/spark/pull/46270#discussion_r1582497889
##
core/src/test/scala/org/apache/spark/MapStatusesSerDeserBenchmark.scala:
##
@@ -123,7 +123,6 @@ object MapStatusesSerDeserBenchmark extends BenchmarkBase {
}
panbingkun commented on code in PR #46264:
URL: https://github.com/apache/spark/pull/46264#discussion_r1582496007
##
sql/core/src/main/scala/org/apache/spark/sql/execution/r/ArrowRRunner.scala:
##
@@ -161,17 +166,14 @@ class ArrowRRunner(
val input =
1 - 100 of 64153 matches
Mail list logo
