gengliangwang commented on code in PR #45975:
URL: https://github.com/apache/spark/pull/45975#discussion_r1560470002
##
common/utils/src/main/scala/org/apache/spark/internal/Logging.scala:
##
@@ -105,9 +108,10 @@ trait Logging {
val context = new
panbingkun commented on code in PR #45975:
URL: https://github.com/apache/spark/pull/45975#discussion_r1560461520
##
common/utils/src/main/scala/org/apache/spark/internal/Logging.scala:
##
@@ -105,9 +108,10 @@ trait Logging {
val context = new java.util.HashMap[String,
itholic commented on code in PR #45923:
URL: https://github.com/apache/spark/pull/45923#discussion_r1560460548
##
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2Listener.scala:
##
@@ -218,7 +232,9 @@ private[thriftserver] class
panbingkun commented on code in PR #45975:
URL: https://github.com/apache/spark/pull/45975#discussion_r1560460023
##
common/utils/src/main/scala/org/apache/spark/internal/Logging.scala:
##
@@ -37,7 +37,10 @@ import org.apache.spark.util.SparkClassUtils
* The values of the MDC
uros-db commented on PR #45738:
URL: https://github.com/apache/spark/pull/45738#issuecomment-2048962946
@GideonPotok You are correct, this refactor should not greatly affect your
current PR in particular - I expect you'll only need to refactor testing a bit
(shouldn't be too much work)
wForget commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560458632
##
sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala:
##
@@ -187,8 +187,9 @@ class V2ExpressionBuilder(e: Expression, isPredicate:
panbingkun commented on code in PR #45975:
URL: https://github.com/apache/spark/pull/45975#discussion_r1560455923
##
common/utils/src/test/scala/org/apache/spark/util/MDCSuite.scala:
##
@@ -41,6 +41,21 @@ class MDCSuite
assert(log.context === Map("exit_code" ->
gengliangwang closed pull request #45927: [SPARK-47587][SQL] Hive module:
Migrate logWarn with variables to structured logging framework
URL: https://github.com/apache/spark/pull/45927
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
uros-db commented on code in PR #45978:
URL: https://github.com/apache/spark/pull/45978#discussion_r1560453598
##
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##
@@ -172,19 +183,31 @@ public Collation(
}
/**
- * Auxiliary
gengliangwang commented on PR #45927:
URL: https://github.com/apache/spark/pull/45927#issuecomment-2048956649
Thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
uros-db commented on code in PR #45978:
URL: https://github.com/apache/spark/pull/45978#discussion_r1560454235
##
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
cloud-fan commented on code in PR #45978:
URL: https://github.com/apache/spark/pull/45978#discussion_r1560452406
##
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation
cloud-fan commented on code in PR #45978:
URL: https://github.com/apache/spark/pull/45978#discussion_r1560451327
##
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##
@@ -172,19 +183,31 @@ public Collation(
}
/**
- * Auxiliary
GideonPotok commented on PR #45738:
URL: https://github.com/apache/spark/pull/45738#issuecomment-2048944931
PS: Do you think changes, such as these, which are only to implementations
of `inputTypes` and `replacement`, which do not rely on calling UTFString or
CollationFactory, will need to
gengliangwang commented on code in PR #45923:
URL: https://github.com/apache/spark/pull/45923#discussion_r1560444898
##
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2Listener.scala:
##
@@ -218,7 +232,9 @@ private[thriftserver]
GideonPotok commented on PR #45738:
URL: https://github.com/apache/spark/pull/45738#issuecomment-2048941620
@uros-db No problem at all.
if I understand your refactor correctly, my changes will basically either
stay in the same place or move to the new
dongjoon-hyun commented on code in PR #45982:
URL: https://github.com/apache/spark/pull/45982#discussion_r1560434354
##
docs/job-scheduling.md:
##
@@ -92,6 +96,8 @@ In standalone mode, simply start your workers with
`spark.shuffle.service.enable
In YARN mode, follow the
uros-db commented on PR #45738:
URL: https://github.com/apache/spark/pull/45738#issuecomment-2048923852
@GideonPotok nice work, thanks!
Heads up though: we will soon be finishing some code refactoring related to
collation-aware string expression support
gengliangwang closed pull request #45947: [SPARK-47601][GRAPHX] Graphx: Migrate
logs with variables to structured logging framework
URL: https://github.com/apache/spark/pull/45947
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
gengliangwang commented on PR #45947:
URL: https://github.com/apache/spark/pull/45947#issuecomment-2048920978
Thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
GideonPotok commented on PR #45738:
URL: https://github.com/apache/spark/pull/45738#issuecomment-2048919774
@uros-db this is ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
gengliangwang commented on code in PR #45975:
URL: https://github.com/apache/spark/pull/45975#discussion_r1560429054
##
common/utils/src/main/scala/org/apache/spark/internal/Logging.scala:
##
@@ -105,9 +108,10 @@ trait Logging {
val context = new
chaoqin-li1123 commented on code in PR #45977:
URL: https://github.com/apache/spark/pull/45977#discussion_r1560425964
##
python/pyspark/sql/datasource.py:
##
@@ -469,6 +501,188 @@ def stop(self) -> None:
...
+class SimpleInputPartition(InputPartition):
+def
zhengruifeng opened a new pull request, #45998:
URL: https://github.com/apache/spark/pull/45998
### What changes were proposed in this pull request?
Make `handleCreateResourceProfileCommand` private
### Why are the changes needed?
it should not be exposed to users
yaooqinn commented on PR #45981:
URL: https://github.com/apache/spark/pull/45981#issuecomment-2048890101
Merged to master
Thank you @dongjoon-hyun @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
yaooqinn closed pull request #45981: [SPARK-47798][SQL] Enrich the error
message for the reading failures of decimal values
URL: https://github.com/apache/spark/pull/45981
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
anchovYu commented on PR #45935:
URL: https://github.com/apache/spark/pull/45935#issuecomment-2048878434
Hi @ahshahid , thanks for the proposal and the PR. However, the current
Dataframe cache design has a lot of design flaws, I would worry that improving
the cache hit rate in this case
cloud-fan commented on code in PR #45997:
URL: https://github.com/apache/spark/pull/45997#discussion_r1560396328
##
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala:
##
@@ -170,18 +170,15 @@ trait ExpressionEvalHelper extends
cloud-fan commented on PR #45997:
URL: https://github.com/apache/spark/pull/45997#issuecomment-2048874386
cc @HyukjinKwon @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
cloud-fan opened a new pull request, #45997:
URL: https://github.com/apache/spark/pull/45997
### What changes were proposed in this pull request?
There is a bug in the test util `checkExceptionInExpression`. It may fail to
catch bugs when codegen and non-codegen have
dongjoon-hyun closed pull request #45994: [MINOR][DOCS] Make the link of spark
properties with YARN more accurate
URL: https://github.com/apache/spark/pull/45994
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
HyukjinKwon commented on PR #45996:
URL: https://github.com/apache/spark/pull/45996#issuecomment-2048865644
cc @zhengruifeng @WeichenXu123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
HyukjinKwon opened a new pull request, #45995:
URL: https://github.com/apache/spark/pull/45995
### What changes were proposed in this pull request?
This PR proposes to make `pyspark.ml` compatible with `pyspark-connect`.
### Why are the changes needed?
In order for
HyukjinKwon commented on PR #45995:
URL: https://github.com/apache/spark/pull/45995#issuecomment-2048857160
cc @WeichenXu123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
TakawaAkirayo commented on code in PR #45367:
URL: https://github.com/apache/spark/pull/45367#discussion_r1560378130
##
core/src/main/scala/org/apache/spark/internal/config/package.scala:
##
@@ -1014,6 +1014,15 @@ package object config {
.timeConf(TimeUnit.NANOSECONDS)
wForget commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560377503
##
sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala:
##
@@ -389,6 +394,16 @@ class V2ExpressionBuilder(e: Expression, isPredicate:
cloud-fan commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560376969
##
sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala:
##
@@ -389,6 +394,16 @@ class V2ExpressionBuilder(e: Expression,
wForget commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560376882
##
sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2Suite.scala:
##
@@ -966,6 +966,41 @@ class DataSourceV2Suite extends QueryTest with
wForget commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560376615
##
sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala:
##
@@ -209,7 +209,12 @@ class V2ExpressionBuilder(e: Expression, isPredicate:
cloud-fan commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560375676
##
sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala:
##
@@ -209,7 +209,12 @@ class V2ExpressionBuilder(e: Expression,
cloud-fan commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560375469
##
sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2Suite.scala:
##
@@ -966,6 +966,41 @@ class DataSourceV2Suite extends QueryTest with
wForget commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560375353
##
sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2Suite.scala:
##
@@ -966,6 +966,41 @@ class DataSourceV2Suite extends QueryTest with
cloud-fan commented on code in PR #45589:
URL: https://github.com/apache/spark/pull/45589#discussion_r1560375211
##
sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2Suite.scala:
##
@@ -966,6 +966,41 @@ class DataSourceV2Suite extends QueryTest with
cloud-fan commented on PR #45589:
URL: https://github.com/apache/spark/pull/45589#issuecomment-2048842691
With hindsight, we shouldn't create the v2 `Predicate` API in the first
place, and should just use the v2 `Expression` API. The `Predicate` trait in
catalyst is not useful as well.
wForget commented on PR #45589:
URL: https://github.com/apache/spark/pull/45589#issuecomment-2048841083
The `org.apache.spark.sql.catalyst.optimizer.SimplifyBinaryComparison`
optimizer may also fold predicate.
--
This is an automated message from the Apache Git Service.
To respond to the
beliefer opened a new pull request, #45994:
URL: https://github.com/apache/spark/pull/45994
### What changes were proposed in this pull request?
This PR propose to make the link of spark properties with YARN more accurate.
### Why are the changes needed?
Currently, the link
cloud-fan closed pull request #45146: [SPARK-47001][SQL] Pushdown verification
in optimizer
URL: https://github.com/apache/spark/pull/45146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
cloud-fan commented on PR #45146:
URL: https://github.com/apache/spark/pull/45146#issuecomment-2048834064
The GA failure is unrelated, I'm merging this to master, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
beliefer commented on code in PR #45982:
URL: https://github.com/apache/spark/pull/45982#discussion_r1560363139
##
docs/job-scheduling.md:
##
@@ -53,7 +53,11 @@ Resource allocation can be configured as follows, based on
the cluster type:
on the cluster
beliefer commented on PR #45982:
URL: https://github.com/apache/spark/pull/45982#issuecomment-2048826987
> Do we support scheduling jobs across applications? It's odd to me.
This section is about scheduling across applications.
`Scheduling Within an Application` section is related
HyukjinKwon closed pull request #45993: [MINOR][DOCS] Clarify relation between
grouping API and `spark.sql.execution.arrow.maxRecordsPerBatch`
URL: https://github.com/apache/spark/pull/45993
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
HyukjinKwon commented on PR #45993:
URL: https://github.com/apache/spark/pull/45993#issuecomment-2048825061
Merged to master, branch-3.5 and branch-3.4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
cloud-fan commented on code in PR #45963:
URL: https://github.com/apache/spark/pull/45963#discussion_r1560358232
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -89,6 +89,73 @@ class CollationStringExpressionsSuite
cloud-fan commented on code in PR #45963:
URL: https://github.com/apache/spark/pull/45963#discussion_r1560355740
##
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java:
##
@@ -1636,14 +1699,13 @@ public int levenshteinDistance(UTF8String other, int
cloud-fan commented on code in PR #45963:
URL: https://github.com/apache/spark/pull/45963#discussion_r1560352992
##
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java:
##
@@ -1509,12 +1515,62 @@ public boolean semanticEquals(final UTF8String other,
int
panbingkun commented on PR #45927:
URL: https://github.com/apache/spark/pull/45927#issuecomment-2048804834
cc @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon opened a new pull request, #45993:
URL: https://github.com/apache/spark/pull/45993
### What changes were proposed in this pull request?
This PR fixes the documentation of
`spark.sql.execution.arrow.maxRecordsPerBatch` to clarify the relation between
cloud-fan commented on code in PR #45946:
URL: https://github.com/apache/spark/pull/45946#discussion_r1560342794
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala:
##
@@ -1062,4 +1062,11 @@ class CollationSuite extends DatasourceV2SQLBase with
panbingkun commented on code in PR #45927:
URL: https://github.com/apache/spark/pull/45927#discussion_r1560342791
##
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala:
##
@@ -104,21 +105,28 @@ private[hive] class HiveMetastoreCatalog(sparkSession:
panbingkun commented on code in PR #45927:
URL: https://github.com/apache/spark/pull/45927#discussion_r1560341095
##
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##
@@ -229,8 +230,8 @@ private[hive] class HiveClientImpl(
case e:
itholic commented on PR #45377:
URL: https://github.com/apache/spark/pull/45377#issuecomment-2048796376
Thanks @cloud-fan @ueshin @HyukjinKwon @xinrong-meng for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
HyukjinKwon closed pull request #45833: [SPARK-47704][SQL] JSON parsing fails
with "java.lang.ClassCastException" when spark.sql.json.enablePartialResults is
enabled
URL: https://github.com/apache/spark/pull/45833
--
This is an automated message from the Apache Git Service.
To respond to
HyukjinKwon commented on PR #45833:
URL: https://github.com/apache/spark/pull/45833#issuecomment-2048792451
Merged to master and branch-3.5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
anishshri-db commented on PR #45937:
URL: https://github.com/apache/spark/pull/45937#issuecomment-2048792306
> Should we also add some metrics around ttl? (Like keys deleted from state
on ttl expiry?)
Done, added
--
This is an automated message from the Apache Git Service.
To
anishshri-db commented on code in PR #45937:
URL: https://github.com/apache/spark/pull/45937#discussion_r1560336821
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala:
##
@@ -112,6 +116,10 @@ class StatefulProcessorHandleImpl(
cloud-fan closed pull request #45377: [SPARK-47274][PYTHON][SQL] Provide more
useful context for PySpark DataFrame API errors
URL: https://github.com/apache/spark/pull/45377
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
cloud-fan commented on PR #45377:
URL: https://github.com/apache/spark/pull/45377#issuecomment-2048784678
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
cloud-fan commented on code in PR #45985:
URL: https://github.com/apache/spark/pull/45985#discussion_r1560306930
##
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala:
##
@@ -118,12 +117,8 @@ class DataSourceV2Strategy(session:
HyukjinKwon commented on code in PR #45977:
URL: https://github.com/apache/spark/pull/45977#discussion_r1560294470
##
python/pyspark/sql/datasource.py:
##
@@ -469,6 +501,188 @@ def stop(self) -> None:
...
+class SimpleInputPartition(InputPartition):
+def
HyukjinKwon commented on code in PR #45977:
URL: https://github.com/apache/spark/pull/45977#discussion_r1560293229
##
python/pyspark/sql/datasource.py:
##
@@ -469,6 +501,188 @@ def stop(self) -> None:
...
+class SimpleInputPartition(InputPartition):
+def
cloud-fan closed pull request #45987: [SPARK-47802][SQL] Revert (*) from
meaning struct(*) back to meaning *
URL: https://github.com/apache/spark/pull/45987
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
cloud-fan commented on PR #45987:
URL: https://github.com/apache/spark/pull/45987#issuecomment-2048732729
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
chaoqin-li1123 commented on code in PR #45977:
URL: https://github.com/apache/spark/pull/45977#discussion_r1560262338
##
python/pyspark/sql/datasource.py:
##
@@ -469,6 +501,188 @@ def stop(self) -> None:
...
+class SimpleInputPartition(InputPartition):
+def
gene-db commented on PR #45826:
URL: https://github.com/apache/spark/pull/45826#issuecomment-2048671561
> @gene-db seems like the JIRA ID is wrong. Do we have a dedicated JIRA for
this?
hrmmm, I don't think there was a dedicated jira for this. Looks like I will
have to create a jira
HyukjinKwon closed pull request #45992: [SPARK-47725][INFRA][FOLLOW-UP] Do not
run scheduled job in forked repository
URL: https://github.com/apache/spark/pull/45992
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
HyukjinKwon commented on PR #45992:
URL: https://github.com/apache/spark/pull/45992#issuecomment-2048663695
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on code in PR #45977:
URL: https://github.com/apache/spark/pull/45977#discussion_r1560219006
##
python/pyspark/sql/datasource.py:
##
@@ -469,6 +501,188 @@ def stop(self) -> None:
...
+class SimpleInputPartition(InputPartition):
+def
HyukjinKwon commented on code in PR #45977:
URL: https://github.com/apache/spark/pull/45977#discussion_r1560215436
##
python/pyspark/sql/datasource.py:
##
@@ -469,6 +501,188 @@ def stop(self) -> None:
...
+class SimpleInputPartition(InputPartition):
+def
HyukjinKwon commented on PR #45978:
URL: https://github.com/apache/spark/pull/45978#issuecomment-2048651969
Can you please fill the PR description?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
zhengruifeng commented on PR #45614:
URL: https://github.com/apache/spark/pull/45614#issuecomment-2048647268
thanks @HyukjinKwon and @xinrong-meng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
zhengruifeng commented on PR #45980:
URL: https://github.com/apache/spark/pull/45980#issuecomment-2048642692
thanks @xinrong-meng
merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
zhengruifeng closed pull request #45980: [MINOR][PYTHON][TESTS] Enable
`test_udf_cache` parity test
URL: https://github.com/apache/spark/pull/45980
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
holdenk commented on PR #45802:
URL: https://github.com/apache/spark/pull/45802#issuecomment-2048641757
Another possible solution would be to also break up the projection and move
the part of the projection which is used in the filter down with the filter
unless the only thing the
HyukjinKwon commented on PR #45826:
URL: https://github.com/apache/spark/pull/45826#issuecomment-2048636068
@gene-db seems like the JIRA ID is wrong. Do we have a dedicated JIRA for
this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
HyukjinKwon closed pull request #45826: [SPARK-47366][SQL][PYTHON] Add
VariantVal for PySpark
URL: https://github.com/apache/spark/pull/45826
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
HyukjinKwon commented on PR #45826:
URL: https://github.com/apache/spark/pull/45826#issuecomment-2048634802
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
itholic commented on code in PR #45923:
URL: https://github.com/apache/spark/pull/45923#discussion_r1560187499
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -100,6 +101,7 @@ object LogKey extends Enumeration {
val OFFSETS = Value
val
HyukjinKwon closed pull request #45614: [SPARK-41811][PYTHON][CONNECT]
Implement `SQLStringFormatter` with `WithRelations`
URL: https://github.com/apache/spark/pull/45614
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
HyukjinKwon commented on PR #18:
URL: https://github.com/apache/spark-connect-go/pull/18#issuecomment-2048625272
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon closed pull request #18: [MINOR] Make readme easier to follow
URL: https://github.com/apache/spark-connect-go/pull/18
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
HyukjinKwon commented on PR #45614:
URL: https://github.com/apache/spark/pull/45614#issuecomment-2048624804
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
srielau commented on PR #45987:
URL: https://github.com/apache/spark/pull/45987#issuecomment-2048618906
@cloud-fan @gengliangwang This is ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
HyukjinKwon opened a new pull request, #45992:
URL: https://github.com/apache/spark/pull/45992
### What changes were proposed in this pull request?
This is a followup of https://github.com/apache/spark/pull/45870 that skips
the run in forked repository.
### Why are the changes
ericm-db opened a new pull request, #45991:
URL: https://github.com/apache/spark/pull/45991
### What changes were proposed in this pull request?
This PR adds support for expiring state based on TTL for MapState. Using
this functionality, Spark users can specify a TTL Mode for
anchovYu opened a new pull request, #45990:
URL: https://github.com/apache/spark/pull/45990
### What changes were proposed in this pull request?
This PR adds a debug log for Dataframe cache that uses SQL conf to turn on.
It logs necessary information on
* cache hit during cache
chaoqin-li1123 commented on PR #45977:
URL: https://github.com/apache/spark/pull/45977#issuecomment-2048549899
@allisonwang-db @HyukjinKwon @HeartSaVioR PTAL, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
chenhao-db opened a new pull request, #45989:
URL: https://github.com/apache/spark/pull/45989
### What changes were proposed in this pull request?
This PR allows casting another type into the variant type. The changes can
be divided into two major parts:
- The `VariantBuilder`
gengliangwang closed pull request #45910: [SPARK-47595][STREAMING] Streaming:
Migrate logError with variables to structured logging framework
URL: https://github.com/apache/spark/pull/45910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
gengliangwang commented on PR #45910:
URL: https://github.com/apache/spark/pull/45910#issuecomment-2048533652
Thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
gengliangwang commented on code in PR #45923:
URL: https://github.com/apache/spark/pull/45923#discussion_r1560115778
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -100,6 +101,7 @@ object LogKey extends Enumeration {
val OFFSETS = Value
val
1 - 100 of 227 matches
Mail list logo
