Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21978#discussion_r237585203
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/identifiers.scala ---
@@ -18,48 +18,106 @@
package org.apache.spark.sql.catalyst
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23130
**[Test build #99465 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99465/testReport)**
for PR 23130 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22874
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/23148
Just pushed a tweak to allow closing parens on same line. New pr for that,
or do we want to keep identifying other tweaks first?
I think the args on their own line is triggered once the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23181
**[Test build #99466 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99466/testReport)**
for PR 23181 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22979
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23181
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/23177
Ah, that's the second time I've forgotten this. Yes looks good to me.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23177
**[Test build #4447 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4447/testReport)**
for PR 23177 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23177
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23182
**[Test build #99472 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99472/testReport)**
for PR 23182 at commit
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23168#discussion_r237637501
--- Diff: docs/ml-clustering.md ---
@@ -265,3 +265,38 @@ Refer to the [R API
docs](api/R/spark.gaussianMixture.html) for more details.
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/23175
LGTM - merged in master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23058
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user koertkuipers commented on the issue:
https://github.com/apache/spark/pull/23052
it is pretty common for us to write empty dataframe to parquet and later
read it back in
same for writing to csv with header and reading it back in (with type
inference disabled, we assume
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23178
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99454/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23178
**[Test build #99454 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99454/testReport)**
for PR 23178 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23165
**[Test build #99458 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99458/testReport)**
for PR 23165 at commit
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/23171
@cloud-fan as @aokolnychyi said, `switch` will still be faster than
optimized `Set` without autoboxing when the number of elements are small. As a
result, this PR is still very useful.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23126
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23037
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99471/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23178
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99457/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23178
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23130
**[Test build #99465 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99465/testReport)**
for PR 23130 at commit
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
> Why? And how are mounted files better?
Environment variables leak far more easily than file contents. One can
accidentally `printenv` in a shell attached to the and get the secret
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
(In fact, env variables don't even show up in the UI or event logs, as far
as I can see. Other configs - Spark config, system properties, e.g. - do show
up, and are redacted to mask secrets.)
---
Github user jose-torres closed the pull request at:
https://github.com/apache/spark/pull/20752
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user jose-torres closed the pull request at:
https://github.com/apache/spark/pull/20859
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user jose-torres closed the pull request at:
https://github.com/apache/spark/pull/20906
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23165
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99458/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23165
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22598
**[Test build #99469 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99469/testReport)**
for PR 22598 at commit
Github user AzureQ commented on the issue:
https://github.com/apache/spark/pull/23037
@vanzin @ifilonenko I'm able to capture stdout of `spark-shell` but not
`pyspark` and `sparkR`. Need more investigation on it. Do you happen to know
why?
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22957
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99462/
Test PASSed.
---
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23037
When I tried to write automated tests for pyspark in the past it was kind
of a pain. It doesn't work the way you expect unless you have a
pseudo-terminal, apparently.
Maybe try to write a
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23165
**[Test build #99460 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99460/testReport)**
for PR 23165 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22957
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/23171
@dbtsai @mgaido91 I think we can come back to this question once
[SPARK-26203](https://issues.apache.org/jira/browse/SPARK-26203) is resolved.
That JIRA will give us enough information about
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23130
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22598
**[Test build #99474 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99474/testReport)**
for PR 22598 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23158
Merging to master / 2.4.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23158
Thanks a lot @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/23181
Executor summary should update for history events
## What changes were proposed in this pull request?
The root cause of the problem is, whenever the taskEnd event comes after
stageCompleted
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23173
**[Test build #99467 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99467/testReport)**
for PR 23173 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23086
**[Test build #99461 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99461/testReport)**
for PR 23086 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23037
**[Test build #99471 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99471/testReport)**
for PR 23037 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23031
**[Test build #99463 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99463/testReport)**
for PR 23031 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23178
**[Test build #99457 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99457/testReport)**
for PR 23178 at commit
Github user skonto commented on a diff in the pull request:
https://github.com/apache/spark/pull/23136#discussion_r237642388
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala
---
@@ -30,7 +30,7 @@
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/23171
yes @aokolnychyi , I agree that the work can be done later (not in the
scope of this PR). We can maybe just open a new JIRA about it so we won't
forget.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23130
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99465/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22598
**[Test build #99453 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99453/testReport)**
for PR 22598 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23130
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> via a mounted file
> Also the user should be able to specify their own mounted file
The point is that the user shouldn't need to set this at all. You enable
auth, Spark takes care of
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23130
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> if the secret would be listed under the environment variables in the
Spark UI
Secrets are redacted in the UI and event logs. We already use env variables
in other contexts (e.g.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23181
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23181
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99466/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22979
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99456/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23181
**[Test build #99468 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99468/testReport)**
for PR 23181 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23181
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23181
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AzureQ commented on a diff in the pull request:
https://github.com/apache/spark/pull/23037#discussion_r237618024
--- Diff:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PythonTestsSuite.scala
---
@@ -89,6
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23178
**[Test build #4448 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4448/testReport)**
for PR 23178 at commit
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23136#discussion_r237626777
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
---
@@ -86,11 +88,14 @@
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23072
**[Test build #99470 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99470/testReport)**
for PR 23072 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23161
**[Test build #99473 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99473/testReport)**
for PR 23161 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23182
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23182
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23161
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23173#discussion_r237634432
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -1987,6 +1987,18 @@ class CSVSuite extends
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23173#discussion_r237633638
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -1987,6 +1987,18 @@ class CSVSuite extends
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23037
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5539/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23037
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user squito commented on the issue:
https://github.com/apache/spark/pull/23058
merged to master, thanks @wypoon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21978#discussion_r237578911
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/identifiers.scala ---
@@ -18,48 +18,106 @@
package org.apache.spark.sql.catalyst
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21306
@stczwd, thanks for taking a look at this. What are the differences between
batch and stream DDL that you think will come up?
---
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21978#discussion_r237578805
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/identifiers.scala ---
@@ -18,48 +18,106 @@
package org.apache.spark.sql.catalyst
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23165
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23178
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23126
**[Test build #4445 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4445/testReport)**
for PR 23126 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23158
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23017
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22957
**[Test build #99462 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99462/testReport)**
for PR 22957 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23072
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/23161#discussion_r237628719
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2732,13 +2732,24 @@ setMethod("union",
dataFrame(unioned)
})
-#' Return a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23072
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23037
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5539/
---
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/23171
@dbtsai I see, it would be great, though, to check which is this threshold.
My understanding is that the current solution has better performance even for
several hundreds of items. If this number
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/23171
To sum up, I would set the goal of this PR is to make `In` expressions as
efficient as possible for bytes/shorts/ints. Then we can do benchmarks for `In`
vs `InSet` in
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23175
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23072
**[Test build #99470 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99470/testReport)**
for PR 23072 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23072
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23072
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99470/
Test PASSed.
---
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r237656297
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -688,4 +688,65 @@ package object config {
.stringConf
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r237656366
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala ---
@@ -0,0 +1,200 @@
+/*
+ * Licensed to the Apache
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r237656411
--- Diff:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSecurityHelperSuite.scala
---
@@ -0,0 +1,100 @@
+/*
+ *
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23165
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99451/
Test PASSed.
---
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22959#discussion_r237587099
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
---
@@ -112,125 +72,139 @@ private[spark] case
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/22952
> @zsxwing Btw, how do you think about addressing background move/deletion
(I had thought and
Yeah, this can be done in a separate ticket.
I was playing with
1 - 100 of 672 matches
Mail list logo
