cloud-fan commented on PR #45146:
URL: https://github.com/apache/spark/pull/45146#issuecomment-2048834064
The GA failure is unrelated, I'm merging this to master, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
cloud-fan commented on PR #45589:
URL: https://github.com/apache/spark/pull/45589#issuecomment-2048842691
With hindsight, we shouldn't create the v2 `Predicate` API in the first
place, and should just use the v2 `Expression` API. The `Predicate` trait in
catalyst is not useful as well.
HyukjinKwon commented on PR #45995:
URL: https://github.com/apache/spark/pull/45995#issuecomment-2048857160
cc @WeichenXu123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
gengliangwang commented on code in PR #45923:
URL: https://github.com/apache/spark/pull/45923#discussion_r1560444898
##
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2Listener.scala:
##
@@ -218,7 +232,9 @@ private[thriftserver]
GideonPotok commented on PR #45738:
URL: https://github.com/apache/spark/pull/45738#issuecomment-2048941620
@uros-db No problem at all.
if I understand your refactor correctly, my changes will basically either
stay in the same place or move to the new
cloud-fan commented on code in PR #45978:
URL: https://github.com/apache/spark/pull/45978#discussion_r1560451327
##
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##
@@ -172,19 +183,31 @@ public Collation(
}
/**
- * Auxiliary
cloud-fan commented on code in PR #45978:
URL: https://github.com/apache/spark/pull/45978#discussion_r1560452406
##
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation
itholic commented on code in PR #45923:
URL: https://github.com/apache/spark/pull/45923#discussion_r1560460548
##
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2Listener.scala:
##
@@ -218,7 +232,9 @@ private[thriftserver] class
panbingkun commented on code in PR #45975:
URL: https://github.com/apache/spark/pull/45975#discussion_r1560461520
##
common/utils/src/main/scala/org/apache/spark/internal/Logging.scala:
##
@@ -105,9 +108,10 @@ trait Logging {
val context = new java.util.HashMap[String,
gengliangwang commented on code in PR #45975:
URL: https://github.com/apache/spark/pull/45975#discussion_r1560470002
##
common/utils/src/main/scala/org/apache/spark/internal/Logging.scala:
##
@@ -105,9 +108,10 @@ trait Logging {
val context = new
HyukjinKwon opened a new pull request, #45993:
URL: https://github.com/apache/spark/pull/45993
### What changes were proposed in this pull request?
This PR fixes the documentation of
`spark.sql.execution.arrow.maxRecordsPerBatch` to clarify the relation between
cloud-fan commented on code in PR #45963:
URL: https://github.com/apache/spark/pull/45963#discussion_r1560355740
##
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java:
##
@@ -1636,14 +1699,13 @@ public int levenshteinDistance(UTF8String other, int
beliefer commented on PR #45982:
URL: https://github.com/apache/spark/pull/45982#issuecomment-2048826987
> Do we support scheduling jobs across applications? It's odd to me.
This section is about scheduling across applications.
`Scheduling Within an Application` section is related
cloud-fan closed pull request #45146: [SPARK-47001][SQL] Pushdown verification
in optimizer
URL: https://github.com/apache/spark/pull/45146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
beliefer opened a new pull request, #45994:
URL: https://github.com/apache/spark/pull/45994
### What changes were proposed in this pull request?
This PR propose to make the link of spark properties with YARN more accurate.
### Why are the changes needed?
Currently, the link
wForget commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560376615
##
sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala:
##
@@ -209,7 +209,12 @@ class V2ExpressionBuilder(e: Expression, isPredicate:
cloud-fan commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560376969
##
sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala:
##
@@ -389,6 +394,16 @@ class V2ExpressionBuilder(e: Expression,
wForget commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560377503
##
sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala:
##
@@ -389,6 +394,16 @@ class V2ExpressionBuilder(e: Expression, isPredicate:
wForget commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560376882
##
sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2Suite.scala:
##
@@ -966,6 +966,41 @@ class DataSourceV2Suite extends QueryTest with
HyukjinKwon opened a new pull request, #45995:
URL: https://github.com/apache/spark/pull/45995
### What changes were proposed in this pull request?
This PR proposes to make `pyspark.ml` compatible with `pyspark-connect`.
### Why are the changes needed?
In order for
HyukjinKwon commented on PR #45996:
URL: https://github.com/apache/spark/pull/45996#issuecomment-2048865644
cc @zhengruifeng @WeichenXu123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
cloud-fan commented on code in PR #45997:
URL: https://github.com/apache/spark/pull/45997#discussion_r1560396328
##
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala:
##
@@ -170,18 +170,15 @@ trait ExpressionEvalHelper extends
cloud-fan commented on PR #45997:
URL: https://github.com/apache/spark/pull/45997#issuecomment-2048874386
cc @HyukjinKwon @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
GideonPotok commented on PR #45738:
URL: https://github.com/apache/spark/pull/45738#issuecomment-2048919774
@uros-db this is ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
dongjoon-hyun commented on code in PR #45982:
URL: https://github.com/apache/spark/pull/45982#discussion_r1560434354
##
docs/job-scheduling.md:
##
@@ -92,6 +96,8 @@ In standalone mode, simply start your workers with
`spark.shuffle.service.enable
In YARN mode, follow the
GideonPotok commented on PR #45738:
URL: https://github.com/apache/spark/pull/45738#issuecomment-2048944931
PS: Do you think changes, such as these, which are only to implementations
of `inputTypes` and `replacement`, which do not rely on calling UTFString or
CollationFactory, will need to
panbingkun commented on code in PR #45975:
URL: https://github.com/apache/spark/pull/45975#discussion_r1560455923
##
common/utils/src/test/scala/org/apache/spark/util/MDCSuite.scala:
##
@@ -41,6 +41,21 @@ class MDCSuite
assert(log.context === Map("exit_code" ->
201 - 227 of 227 matches
Mail list logo
