Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22274
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22274
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/21987
It seems that this change caused permission issue:
```
export HADOOP_PROXY_USER=user_a
spark-sql
```
It will create dir `/tmp/hive-$%7Buser.name%7D/user_a/`. then change to
other
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22274
**[Test build #95520 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95520/testReport)**
for PR 22274 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22270
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22270
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95516/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22270
**[Test build #95516 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95516/testReport)**
for PR 22270 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21721
@arunmahadevan, feel free to pick up the commits in my PR in your followup
if they have to be changed. I will close mine.
---
Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21721
@rxin its for streaming sources and sinks as explained in the [doc](
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/22274#discussion_r214246976
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -3633,7 +3633,8 @@ test_that("catalog APIs, currentDatabase,
setCurrentDatabase, listDatabases",
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22183
As discussed in the JIRA, this is a partial fix, and we need to backport
another 2 PRs, which is risky. Can we close it?
---
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/21721
I'm confused by this api. Is this for streaming only? If yes, why are they
not in the stream package? If not, I only found streaming implementation. Maybe
I missed it.
---
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21968#discussion_r214246268
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala
---
@@ -130,6 +134,12 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21968#discussion_r214246211
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala
---
@@ -48,6 +48,12 @@ class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/7#discussion_r214245829
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2546,15 +2546,37 @@ object functions {
def soundex(e: Column):
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/7#discussion_r214245703
--- Diff: python/pyspark/sql/functions.py ---
@@ -1669,20 +1669,36 @@ def repeat(col, n):
return
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/21721
Stuff like this merits api discussions. Not just implementation changes ...
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21721
I actually thought those all of them are part of DataSource V2. Why are we
fine with changing those interfaces but not okay with this one and we consider
reverting it?
Other things
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/7#discussion_r214244981
--- Diff: python/pyspark/sql/functions.py ---
@@ -1669,20 +1669,36 @@ def repeat(col, n):
return
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/7#discussion_r214244918
--- Diff: R/pkg/R/functions.R ---
@@ -3410,13 +3410,14 @@ setMethod("collect_set",
#' \dontrun{
#' head(select(df, split_string(df$Sex, "a")))
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22298
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22298
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22298
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2724/
---
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/22213#discussion_r214244665
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -1144,6 +1144,46 @@ class SparkSubmitSuite
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22298
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95519/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22298
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22298
**[Test build #95519 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95519/testReport)**
for PR 22298 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22274#discussion_r214244580
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -3633,7 +3633,8 @@ test_that("catalog APIs, currentDatabase,
setCurrentDatabase, listDatabases",
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95503/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95503 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95503/testReport)**
for PR 22192 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22281
For clarification, I am okay with targeting this to 3.0.0 since the code
freeze will be very soon if I am not mistaken.
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22291#discussion_r214244359
--- Diff: R/pkg/R/generics.R ---
@@ -799,10 +807,18 @@ setGeneric("array_sort", function(x) {
standardGeneric("array_sort") })
#' @name NULL
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22048
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20637
with the test removed, do we still need this change?
https://github.com/apache/spark/pull/20637/files#diff-41747ec3f56901eb7bfb95d2a217e94dR226
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22281
Yea, but the default fallback should rather be DataSource V2's. Both of you
are super active in DataSource V2. Do you guys have some concerns about
defaulting to DataSource V1's behaviour?
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22298
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2724/
---
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/6#discussion_r214243817
--- Diff: R/pkg/R/functions.R ---
@@ -1697,8 +1697,8 @@ setMethod("to_date",
})
#' @details
-#' \code{to_json}: Converts a
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22281
USING syntax has to be there, but what can USING maybe only data source v1
and file format.
IIUC the agreement is: a data source v2 with catalog can create a table
with USING, and the
Github user ifilonenko commented on a diff in the pull request:
https://github.com/apache/spark/pull/22298#discussion_r214243652
--- Diff:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/SecretsTestsSuite.scala
---
@@
Github user ifilonenko commented on the issue:
https://github.com/apache/spark/pull/22298
@rdblue @holdenk for review. This contains both unit and integration tests
that verify [SPARK-25004] for K8S
---
-
To
GitHub user ifilonenko opened a pull request:
https://github.com/apache/spark/pull/22298
[SPARK-25021][K8S] Add spark.executor.pyspark.memory limit for K8S
## What changes were proposed in this pull request?
Add spark.executor.pyspark.memory limit for K8S
## How
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21721
Note that, data source v2 API is not stable yet and we may even change the
abstraction of the APIs. The design of custom metrics may affect the design of
the streaming source APIs.
I had
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/6#discussion_r214243115
--- Diff: R/pkg/R/functions.R ---
@@ -1697,8 +1697,8 @@ setMethod("to_date",
})
#' @details
-#' \code{to_json}: Converts a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22232
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95508/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22232
**[Test build #95508 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95508/testReport)**
for PR 22232 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21732
> The only tricky thing is, Product is handled specially in the top level,
being flattened into multiple columns.
@cloud-fan Compared with Option of Product which is not supported before,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/7
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/7
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95511/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/7
**[Test build #95511 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95511/testReport)**
for PR 7 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22186
**[Test build #95518 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95518/testReport)**
for PR 22186 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22186
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95518/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22186
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user sadhen commented on the issue:
https://github.com/apache/spark/pull/22264
@srowen A PR for this "bug" is proposed:
https://github.com/scala/scala/pull/7156
Hopefully, Scala 2.12.7 will fix it.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20086
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22264
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r214237818
--- Diff: python/pyspark/sql/session.py ---
@@ -252,6 +252,16 @@ def newSession(self):
"""
return self.__class__(self._sc,
Github user gerashegalov commented on a diff in the pull request:
https://github.com/apache/spark/pull/22213#discussion_r214237801
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
@@ -1144,6 +1144,46 @@ class SparkSubmitSuite
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22273
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95514/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22186
**[Test build #95518 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95518/testReport)**
for PR 22186 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22273
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22197
**[Test build #95517 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95517/testReport)**
for PR 22197 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22273
**[Test build #95514 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95514/testReport)**
for PR 22273 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22186
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22186
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22297
cc @cloud-fan @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/22186
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/22186
I see. Thanks for the explain, I checked the code again, yes you're right.
Let me retrigger the test again, will merge it if everything is fine.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22270
**[Test build #95516 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95516/testReport)**
for PR 22270 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22270
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22270
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21968#discussion_r214235758
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala
---
@@ -141,11 +151,8 @@ class
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21968#discussion_r214235660
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala
---
@@ -141,11 +151,8 @@ class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22297
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22297
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22297
**[Test build #95515 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95515/testReport)**
for PR 22297 at commit
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/22297
[SPARK-25290][Core][Test] Reduce the size of acquired arrays to avoid OOM
error
## What changes were proposed in this pull request?
`BytesToBytesMapOnHeapSuite`.`randomizedStressTest`
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/21860
cc @cloud-fan @maropu @kiszk
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/22279#discussion_r214234325
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -103,6 +103,14 @@ private[spark] class MetricsSystem private (
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/21968
cc @cloud-fan @maropu
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/22279
Hi @LucaCanali do you have an output current AM metrics? I would like to
know what kind of metrics will be output for now.
---
Github user xuejianbest commented on the issue:
https://github.com/apache/spark/pull/22048
I see. A new commit has been done.
Thinks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22197
Seems fine to me too.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22197#discussion_r214233946
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -44,7 +45,14 @@ private[parquet]
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/22273
> I thought the current information is enough to indicate which Arrow or
Pandas we would use and test
Well yeah, it is when they are skipped but my point was that having an
additional
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/22289#discussion_r214233802
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/AbstractCommandBuilder.java ---
@@ -200,6 +200,7 @@ void addOptionString(List cmd, String
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22273
**[Test build #95514 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95514/testReport)**
for PR 22273 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22273
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22273
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22264
Yeah, OK. I think this is acceptable as a potential "known issue" for Scala
2.12 support, which we can accept for a beta release of 2.12 support with Spark
2.4. I think I'd merge this and then see
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22294
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22294
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95498/
Test PASSed.
---
Github user gerashegalov commented on the issue:
https://github.com/apache/spark/pull/22213
@steveloughran Regarding XML format, java.util.Properties has its dedicated
storeTo/loadFromXML methods which Spark does not use, so we don't need to check
this
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22294
**[Test build #95498 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95498/testReport)**
for PR 22294 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22138
**[Test build #95513 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95513/testReport)**
for PR 22138 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22296
**[Test build #95512 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95512/testReport)**
for PR 22296 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22296
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22296
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22173
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22173
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95499/
Test PASSed.
---
1 - 100 of 672 matches
Mail list logo
