cloud-fan commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494763126
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/UpdateFieldsBenchmark.scala
##
@@ -0,0 +1,310 @@
+/*
+ * Licensed to the Apache Software
cloud-fan commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494762975
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/UpdateFieldsBenchmark.scala
##
@@ -0,0 +1,310 @@
+/*
+ * Licensed to the Apache Software
MLnick commented on pull request #29850:
URL: https://github.com/apache/spark/pull/29850#issuecomment-698736348
>
> @MLnick I just do a simple test, and it shows that we can obtain about 11%
speedup.
This seems a worthwhile optimization then, thanks!
dongjoon-hyun commented on a change in pull request #29828:
URL: https://github.com/apache/spark/pull/29828#discussion_r494754892
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeJsonExprs.scala
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to
AmplabJenkins removed a comment on pull request #29869:
URL: https://github.com/apache/spark/pull/29869#issuecomment-698726138
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29869:
URL: https://github.com/apache/spark/pull/29869#issuecomment-698726138
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29869:
URL: https://github.com/apache/spark/pull/29869#issuecomment-698726125
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33723/
SparkQA commented on pull request #29850:
URL: https://github.com/apache/spark/pull/29850#issuecomment-698725725
**[Test build #129103 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129103/testReport)**
for PR 29850 at commit
AmplabJenkins removed a comment on pull request #29800:
URL: https://github.com/apache/spark/pull/29800#issuecomment-698724874
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #29800:
URL: https://github.com/apache/spark/pull/29800#issuecomment-698724864
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33722/
AmplabJenkins commented on pull request #29800:
URL: https://github.com/apache/spark/pull/29800#issuecomment-698724874
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun closed pull request #29863:
URL: https://github.com/apache/spark/pull/29863
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
zhengruifeng commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698722302
@huaxingao Great catch! Yes, I need to modify
https://github.com/apache/spark/pull/29850 to make sure only columns in
`inputCols` can be taken into account. Thanks!
SparkQA commented on pull request #29869:
URL: https://github.com/apache/spark/pull/29869#issuecomment-698721444
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33723/
SparkQA commented on pull request #29800:
URL: https://github.com/apache/spark/pull/29800#issuecomment-698718868
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33722/
cloud-fan commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494745836
##
File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala
##
@@ -901,39 +901,125 @@ class Column(val expr: Expression) extends Logging {
stczwd commented on a change in pull request #29339:
URL: https://github.com/apache/spark/pull/29339#discussion_r494743750
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##
@@ -551,3 +552,31 @@ case class ShowFunctions(
cloud-fan commented on pull request #29339:
URL: https://github.com/apache/spark/pull/29339#issuecomment-698715577
Hi @stczwd , can you follow https://github.com/apache/spark/pull/29866 and
implement the commands using `UnresolvedTableOrView`? cc @imback82
huaxingao commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698714611
The change looks good to me, but do you need to change the implementation in
https://github.com/apache/spark/pull/29850 ? Seems in that PR you are assuming
` categoricalCols`
cloud-fan closed pull request #29866:
URL: https://github.com/apache/spark/pull/29866
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
cloud-fan commented on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698713409
thanks, merging to master!
This is an automated message from the Apache Git Service.
To respond to the
cloud-fan commented on a change in pull request #27604:
URL: https://github.com/apache/spark/pull/27604#discussion_r494738627
##
File path: core/src/main/scala/org/apache/spark/MapOutputTracker.scala
##
@@ -851,8 +852,16 @@ private[spark] class MapOutputTrackerWorker(conf:
SparkQA commented on pull request #29869:
URL: https://github.com/apache/spark/pull/29869#issuecomment-698711104
**[Test build #129102 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129102/testReport)**
for PR 29869 at commit
LantaoJin opened a new pull request #29869:
URL: https://github.com/apache/spark/pull/29869
### What changes were proposed in this pull request?
Add a configuration to
`LISTENER_BUS_ALLOW_EXTERNAL_ACCUMULATORS_ENTER_EVENT` and update the external
accumulators which name is not started
SparkQA commented on pull request #29800:
URL: https://github.com/apache/spark/pull/29800#issuecomment-698709228
**[Test build #129101 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129101/testReport)**
for PR 29800 at commit
AmplabJenkins removed a comment on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698708262
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA removed a comment on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698693481
**[Test build #129100 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129100/testReport)**
for PR 29868 at commit
AmplabJenkins commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698708262
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698707878
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698707878
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698708074
**[Test build #129100 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129100/testReport)**
for PR 29868 at commit
SparkQA commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698707870
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33721/
CodingCat commented on a change in pull request #29831:
URL: https://github.com/apache/spark/pull/29831#discussion_r494734806
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PrunePartitionSuiteBase.scala
##
@@ -46,30 +46,39 @@ abstract class
imback82 commented on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698705568
cc @cloud-fan
This is an automated message from the Apache Git Service.
To respond to the message, please log
imback82 commented on a change in pull request #29866:
URL: https://github.com/apache/spark/pull/29866#discussion_r494632218
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##
@@ -1381,3 +1381,22 @@ case class
AmplabJenkins removed a comment on pull request #29860:
URL: https://github.com/apache/spark/pull/29860#issuecomment-698704837
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #29860:
URL: https://github.com/apache/spark/pull/29860#issuecomment-698704828
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33720/
AmplabJenkins commented on pull request #29860:
URL: https://github.com/apache/spark/pull/29860#issuecomment-698704837
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698703643
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33721/
SparkQA commented on pull request #29860:
URL: https://github.com/apache/spark/pull/29860#issuecomment-698699484
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33720/
holdenk commented on pull request #29854:
URL: https://github.com/apache/spark/pull/29854#issuecomment-698693708
So the launcher tests were broken by a new version of mini kube being
released, it wasn’t broken by any specific PR (except maybe arguably the
original PR adding integration
SparkQA commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698693481
**[Test build #129100 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129100/testReport)**
for PR 29868 at commit
zhengruifeng commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698693361
friendly ping @srowen @huaxingao
This is an automated message from the Apache Git Service.
To respond to
zhengruifeng commented on pull request #29868:
URL: https://github.com/apache/spark/pull/29868#issuecomment-698693191
repl:
```
import org.apache.spark.ml.feature._
import org.apache.spark.ml.linalg.{Vector, Vectors}
val df = Seq((2.0, 1, "foo"),(3.0, 2, "bar")).toDF("real",
zhengruifeng opened a new pull request #29868:
URL: https://github.com/apache/spark/pull/29868
### What changes were proposed in this pull request?
1, remove the comment: `Note, the relevant columns must also be set in
inputCols`;
2, add a check, and if there are `categoricalCols`
HyukjinKwon commented on pull request #29854:
URL: https://github.com/apache/spark/pull/29854#issuecomment-698689697
@holdenk, BTW which PR caused this failure? It's a bit odd that we fix a
test as a followup here whereas you ask to revert at
SparkQA commented on pull request #29860:
URL: https://github.com/apache/spark/pull/29860#issuecomment-698689552
**[Test build #129099 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129099/testReport)**
for PR 29860 at commit
AmplabJenkins removed a comment on pull request #29795:
URL: https://github.com/apache/spark/pull/29795#issuecomment-698688261
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
AmplabJenkins removed a comment on pull request #29795:
URL: https://github.com/apache/spark/pull/29795#issuecomment-698688257
Merged build finished. Test FAILed.
This is an automated message from the Apache Git Service.
To
AmplabJenkins commented on pull request #29795:
URL: https://github.com/apache/spark/pull/29795#issuecomment-698688257
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29795:
URL: https://github.com/apache/spark/pull/29795#issuecomment-698688247
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33719/
HyukjinKwon commented on pull request #29860:
URL: https://github.com/apache/spark/pull/29860#issuecomment-698687917
retest this please
This is an automated message from the Apache Git Service.
To respond to the message,
zhengruifeng commented on a change in pull request #29852:
URL: https://github.com/apache/spark/pull/29852#discussion_r494713601
##
File path: mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala
##
@@ -91,20 +90,13 @@ class HashingTF @Since("3.0.0") private[ml] (
beliefer commented on pull request #29747:
URL: https://github.com/apache/spark/pull/29747#issuecomment-698685083
cc @jiangxb1987
This is an automated message from the Apache Git Service.
To respond to the message, please
AmplabJenkins commented on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698684577
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698684577
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA removed a comment on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698612010
**[Test build #129090 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129090/testReport)**
for PR 29866 at commit
SparkQA commented on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698684084
**[Test build #129090 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129090/testReport)**
for PR 29866 at commit
SparkQA commented on pull request #29795:
URL: https://github.com/apache/spark/pull/29795#issuecomment-698682551
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33719/
AmplabJenkins removed a comment on pull request #29863:
URL: https://github.com/apache/spark/pull/29863#issuecomment-698679229
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29863:
URL: https://github.com/apache/spark/pull/29863#issuecomment-698679229
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
LuciferYang edited a comment on pull request #29857:
URL: https://github.com/apache/spark/pull/29857#issuecomment-698679002
> f you like you can try making the ordering of the Map in this code
deterministic to see if that does it:
> val topNodesForGroup: Map[Int, LearningNode] =
SparkQA removed a comment on pull request #29863:
URL: https://github.com/apache/spark/pull/29863#issuecomment-698650576
**[Test build #129095 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129095/testReport)**
for PR 29863 at commit
LuciferYang commented on pull request #29857:
URL: https://github.com/apache/spark/pull/29857#issuecomment-698679002
> f you like you can try making the ordering of the Map in this code
deterministic to see if that does it:
val topNodesForGroup: Map[Int, LearningNode] =
SparkQA commented on pull request #29863:
URL: https://github.com/apache/spark/pull/29863#issuecomment-698678880
**[Test build #129095 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129095/testReport)**
for PR 29863 at commit
sarutak commented on pull request #29677:
URL: https://github.com/apache/spark/pull/29677#issuecomment-698678194
@c21 @imback82 @maropu @HyukjinKwon
Any other feedback for this change?
This is an automated message from
sarutak commented on pull request #29827:
URL: https://github.com/apache/spark/pull/29827#issuecomment-698677232
@dongjoon-hyun I agree.
This is an automated message from the Apache Git Service.
To respond to the message,
sarutak commented on a change in pull request #29827:
URL: https://github.com/apache/spark/pull/29827#discussion_r494703217
##
File path: .github/workflows/build_and_test.yml
##
@@ -273,6 +273,44 @@ jobs:
cd docs
jekyll build
+ webui-tests-with-chrome:
+
fqaiser94 commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494701237
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/UpdateFieldsPerformanceSuite.scala
##
@@ -0,0 +1,229 @@
+/*
+ * Licensed to the Apache
AmplabJenkins removed a comment on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698675212
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698675212
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA removed a comment on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698592724
**[Test build #129086 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129086/testReport)**
for PR 29866 at commit
fqaiser94 commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494700714
##
File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala
##
@@ -901,39 +901,125 @@ class Column(val expr: Expression) extends Logging {
SparkQA commented on pull request #29795:
URL: https://github.com/apache/spark/pull/29795#issuecomment-698674623
**[Test build #129098 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129098/testReport)**
for PR 29795 at commit
SparkQA commented on pull request #29866:
URL: https://github.com/apache/spark/pull/29866#issuecomment-698674713
**[Test build #129086 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129086/testReport)**
for PR 29866 at commit
fqaiser94 commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494700403
##
File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala
##
@@ -901,39 +901,125 @@ class Column(val expr: Expression) extends Logging {
fqaiser94 commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494698769
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
##
@@ -541,57 +541,105 @@ case class
fqaiser94 commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494698809
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
##
@@ -541,57 +541,105 @@ case class
fqaiser94 commented on a change in pull request #29795:
URL: https://github.com/apache/spark/pull/29795#discussion_r494698625
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
##
@@ -541,57 +541,105 @@ case class
AmplabJenkins removed a comment on pull request #29863:
URL: https://github.com/apache/spark/pull/29863#issuecomment-698667313
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #29863:
URL: https://github.com/apache/spark/pull/29863#issuecomment-698667296
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33716/
AmplabJenkins commented on pull request #29863:
URL: https://github.com/apache/spark/pull/29863#issuecomment-698667313
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698663600
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
AmplabJenkins removed a comment on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698663592
Merged build finished. Test FAILed.
This is an automated message from the Apache Git Service.
To
AmplabJenkins commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698663592
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698663577
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33718/
nssalian commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698662928
@erikerlandson, tests passed. Thanks for your review.
This is an automated message from the Apache Git
SparkQA commented on pull request #29863:
URL: https://github.com/apache/spark/pull/29863#issuecomment-698662720
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33716/
AmplabJenkins removed a comment on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698661308
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698661308
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698661303
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33717/
HyukjinKwon closed pull request #29851:
URL: https://github.com/apache/spark/pull/29851
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
AmplabJenkins removed a comment on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698660054
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA removed a comment on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698652663
**[Test build #129097 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129097/testReport)**
for PR 29844 at commit
github-actions[bot] closed pull request #27604:
URL: https://github.com/apache/spark/pull/27604
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
AmplabJenkins commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698660054
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698659901
**[Test build #129097 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129097/testReport)**
for PR 29844 at commit
SparkQA commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698658661
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33718/
SparkQA commented on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698656746
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33717/
AmplabJenkins removed a comment on pull request #29844:
URL: https://github.com/apache/spark/pull/29844#issuecomment-698656131
This is an automated message from the Apache Git Service.
To respond to the message, please log on
1 - 100 of 521 matches
Mail list logo
