[GitHub] spark issue #20480: [Spark-23306] Fix the oom caused by contention

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20480 cc @jiangxb1987 @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-02-01 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/20408 UI looks good to me, and taking a quick look at the code it looks fine, sorry haven't had time to do in depth review,but if @squito is good with it I'm fine with it. ---

[GitHub] spark issue #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/queries with ...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20481 **[Test build #86947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86947/testReport)** for PR 20481 at commit

[GitHub] spark issue #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/queries with ...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20481 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/queries with ...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20481 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/489/

[GitHub] spark issue #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/queries with ...

2018-02-01 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20481 cc @vanzin @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20272: [SPARK-23078] [CORE] [K8s] allow Spark Thrift Server to ...

2018-02-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20272 So how are you expected to contact the thrift server after it's up, and even figure out where it started? Isn't the container unreachable from the outside world unless you do some port mapping on

[GitHub] spark pull request #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/querie...

2018-02-01 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/20481 [SPARK-23307][WEBUI]Sort jobs/stages/tasks/queries with the completed timestamp before cleaning up them ## What changes were proposed in this pull request? Sort jobs/stages/tasks/queries

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-01 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165504748 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,21 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-01 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165504275 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,21 @@ private[v1] class

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20455 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20455 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86941/ Test PASSed. ---

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20455 **[Test build #86941 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86941/testReport)** for PR 20455 at commit

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20476 Since you are being more and more familar with our codes, I believe you can offer us more useful inputs. Let me merge this PR for fixing the bugs. Then, we can have more detailed

[GitHub] spark issue #20470: [SPARK-23296][YARN] Include stacktrace in YARN-app diagn...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20470 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20470: [SPARK-23296][YARN] Include stacktrace in YARN-app diagn...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20470 **[Test build #86945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86945/testReport)** for PR 20470 at commit

[GitHub] spark issue #20470: [SPARK-23296][YARN] Include stacktrace in YARN-app diagn...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20470 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86945/ Test PASSed. ---

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20167 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20167 **[Test build #86946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86946/testReport)** for PR 20167 at commit

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20167 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86946/ Test FAILed. ---

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20167 **[Test build #86946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86946/testReport)** for PR 20167 at commit

[GitHub] spark pull request #20167: [SPARK-16501] [MESOS] Allow providing Mesos princ...

2018-02-01 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20167#discussion_r165497310 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -71,40 +74,64 @@ trait

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-02-01 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r165496876 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -744,7 +744,9 @@ object SparkSubmit extends CommandLineUtils with Logging {

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20167 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20470: [SPARK-23296][YARN] Include stacktrace in YARN-app diagn...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20470 **[Test build #86945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86945/testReport)** for PR 20470 at commit

[GitHub] spark issue #20470: [SPARK-23296][YARN] Include stacktrace in YARN-app diagn...

2018-02-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20470 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20479 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20479 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86940/ Test PASSed. ---

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20479 **[Test build #86940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86940/testReport)** for PR 20479 at commit

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-01 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r165482549 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -420,63 +432,53 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-01 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r165488466 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -246,6 +251,38 @@ class BlockManagerMasterEndpoint(

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-01 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r165490041 --- Diff: core/src/test/scala/org/apache/spark/CacheRecoveryManagerSuite.scala --- @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-01 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r165486679 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -246,6 +251,38 @@ class BlockManagerMasterEndpoint(

[GitHub] spark issue #20480: [Spark-23306] Fix the oom caused by contention

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20480 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20480: [Spark-23306] Fix the oom caused by contention

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20480 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/488/

[GitHub] spark issue #20480: [Spark-23306] Fix the oom caused by contention

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20480 **[Test build #86944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86944/testReport)** for PR 20480 at commit

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20476 Yeah, I did review it, but at the time I wasn't familiar with how the other code paths worked and assumed that it was necessary to introduce this. I wasn't very familiar with how it *should* work,

[GitHub] spark issue #20480: [Spark-23306] Fix the oom caused by contention

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20480 **[Test build #86943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86943/testReport)** for PR 20480 at commit

[GitHub] spark issue #20480: [Spark-23306] Fix the oom caused by contention

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20480 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/487/

[GitHub] spark issue #20480: [Spark-23306] Fix the oom caused by contention

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20480 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-02-01 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20408 cc @tgravescs --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20480: [Spark-23306] Fix the oom caused by contention

2018-02-01 Thread zhzhan
GitHub user zhzhan opened a pull request: https://github.com/apache/spark/pull/20480 [Spark-23306] Fix the oom caused by contention ## What changes were proposed in this pull request? here is race condition in TaskMemoryManger, which may cause OOM. The memory

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-02-01 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/20419 @kiszk For this specific kind of usage, I don't think using a hardcoded stable ID will be a problem. The comment we're talking about is the kind the can only appear once in a single

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20476 https://github.com/apache/spark/pull/19424 is the original PR that introduced the new rule `PushDownOperatorsToDataSource`. Both of us reviewed it. : ) Thank you for your understanding!

[GitHub] spark issue #20462: [SPARK-23020][core] Fix another race in the in-process l...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20462 **[Test build #86942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86942/testReport)** for PR 20462 at commit

[GitHub] spark issue #20462: [SPARK-23020][core] Fix another race in the in-process l...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20462 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20462: [SPARK-23020][core] Fix another race in the in-process l...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/486/

[GitHub] spark pull request #20462: [SPARK-23020][core] Fix another race in the in-pr...

2018-02-01 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20462#discussion_r165462812 --- Diff: launcher/src/main/java/org/apache/spark/launcher/LauncherServer.java --- @@ -363,17 +362,28 @@ public void close() throws IOException {

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20476 @gatorsmile, Do you mean this? > Extensibility is not good and operator push-down capabilities are limited. If so, that's very open to interpretation. I would assume it means that

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-02-01 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20460 Actually I think this may fail some check (though may not throw exceptions) for instance this one:

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20476 @rdblue Operator pushdown is part of the [data source API V2 SPIP](https://docs.google.com/document/d/1n_vUVbF4KD3gxTmkNEon5qdQ-Z8qU5Frf6WMQZ6jJVM/edit#):

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2018-02-01 Thread akopich
Github user akopich commented on the issue: https://github.com/apache/spark/pull/19565 @hhbyyh So, I guess, I should just roll the refactoring back, right? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-02-01 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20460 Do we also want to update the comment of `SPARK_EXECUTOR_CORES` in `spark-env.sh` ? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20476 @gatorsmile, thanks for the context. If we need to redesign push-down, then I think we should do that separately and with a design plan. **I don't think it's a good idea to bundle it into an

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20455 **[Test build #86941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86941/testReport)** for PR 20455 at commit

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20455 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20466: [SPARK-23293][SQL] fix data source v2 self join

2018-02-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20466 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20466: [SPARK-23293][SQL] fix data source v2 self join

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20466 This is another bug fix of the new data source v2 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail:

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20476 To everyone, this is a bug fix we should merge before the next RC of Spark 2.3. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-02-01 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r165449847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -199,7 +200,7 @@ object ExtractFiltersAndInnerJoins

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20476 @rdblue To be honest, the push-down solution in the current code base is not well designed. We got many feedbacks from the community (e.g., SAP and IBM Research). One proposed a bottom-up

[GitHub] spark pull request #17886: [SPARK-13983][SQL] Fix HiveThriftServer2 can not ...

2018-02-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17886 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20466: [SPARK-23293][SQL] fix data source v2 self join

2018-02-01 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20466 +1 Good to get this in before changes to the relation. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20476: [SPARK-23301][SQL] data source column pruning sho...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20476#discussion_r165437683 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownOperatorsToDataSource.scala --- @@ -81,35 +81,34 @@ object

[GitHub] spark issue #17886: [SPARK-13983][SQL] Fix HiveThriftServer2 can not get "--...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17886 LGTM This bug fix is nice to have in 2.3. I will merge it now. Please submit a follow-up PR. Thanks! Merged to master/2.3 ---

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20479 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/485/

[GitHub] spark pull request #17886: [SPARK-13983][SQL] Fix HiveThriftServer2 can not ...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17886#discussion_r165447010 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java --- @@ -221,6 +227,70 @@ private void

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20479 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20476 @cloud-fan, @gatorsmile, this PR demonstrates why we should use PhysicalOperation. I ported the tests from this PR over to our branch and they pass without modifying the push-down code. That's

[GitHub] spark pull request #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyA...

2018-02-01 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/20473#discussion_r165445947 --- Diff: python/run-tests.py --- @@ -151,6 +151,38 @@ def parse_opts(): return opts +def _check_dependencies(python_exec,

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20479 **[Test build #86940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86940/testReport)** for PR 20479 at commit

[GitHub] spark issue #20422: [SPARK-23253][Core][Shuffle]Only write shuffle temporary...

2018-02-01 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20422 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ign...

2018-02-01 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/20479 [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMissingFiles` test case for ORC ## What changes were proposed in this pull request? Like Parquet, Apache Spark ORC already

[GitHub] spark pull request #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyA...

2018-02-01 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/20473#discussion_r165445232 --- Diff: python/run-tests.py --- @@ -151,6 +151,38 @@ def parse_opts(): return opts +def _check_dependencies(python_exec,

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-01 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20474 Unrelated failure: "org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite" --- - To unsubscribe, e-mail:

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86936/ Test FAILed. ---

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #86936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86936/testReport)** for PR 20474 at commit

[GitHub] spark pull request #17886: [SPARK-13983][SQL] Fix HiveThriftServer2 can not ...

2018-02-01 Thread liufengdb
Github user liufengdb commented on a diff in the pull request: https://github.com/apache/spark/pull/17886#discussion_r165441473 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java --- @@ -221,6 +227,70 @@ private void

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #86939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86939/testReport)** for PR 19788 at commit

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86939/ Test FAILed. ---

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #86939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86939/testReport)** for PR 19788 at commit

[GitHub] spark issue #20466: [SPARK-23293][SQL] fix data source v2 self join

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20466 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86934/ Test PASSed. ---

[GitHub] spark issue #20466: [SPARK-23293][SQL] fix data source v2 self join

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20466 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #86938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86938/testReport)** for PR 19788 at commit

[GitHub] spark issue #20466: [SPARK-23293][SQL] fix data source v2 self join

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20466 **[Test build #86934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86934/testReport)** for PR 20466 at commit

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86938/ Test FAILed. ---

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #86938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86938/testReport)** for PR 19788 at commit

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20476 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86933/ Test PASSed. ---

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20476 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20476: [SPARK-23301][SQL] data source column pruning should wor...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20476 **[Test build #86933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86933/testReport)** for PR 20476 at commit

[GitHub] spark issue #20469: [SPARK-23295][Build][Minor]Exclude Waring message when g...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20469 **[Test build #4087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4087/testReport)** for PR 20469 at commit

[GitHub] spark issue #20460: [SPARK-23285][K8S] Allow fractional values for spark.exe...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20460 **[Test build #4088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4088/testReport)** for PR 20460 at commit

[GitHub] spark pull request #20462: [SPARK-23020][core] Fix another race in the in-pr...

2018-02-01 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20462#discussion_r165432306 --- Diff: launcher/src/main/java/org/apache/spark/launcher/LauncherServer.java --- @@ -363,17 +362,28 @@ public void close() throws IOException {

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20455 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #17886: [SPARK-13983][SQL] Fix HiveThriftServer2 can not ...

2018-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17886#discussion_r165425929 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java --- @@ -221,6 +227,70 @@ private void

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20455 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86935/ Test FAILed. ---

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20455 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   6   7   >