[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87664/ Test FAILed. ---

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20675 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20382 **[Test build #87664 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87664/testReport)** for PR 20382 at commit

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20675 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87665/ Test FAILed. ---

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20675 **[Test build #87665 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87665/testReport)** for PR 20675 at commit

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20666 **[Test build #87663 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87663/testReport)** for PR 20666 at commit

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20666 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87663/ Test FAILed. ---

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20666 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-26 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/20675 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20382 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20667: [SPARK-23508][CORE] Use timeStampedHashMap for Bl...

2018-02-26 Thread caneGuy
Github user caneGuy commented on a diff in the pull request: https://github.com/apache/spark/pull/20667#discussion_r170516775 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerId.scala --- @@ -132,10 +133,15 @@ private[spark] object BlockManagerId {

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20675 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20675 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1054/

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1055/

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20675 **[Test build #87666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87666/testReport)** for PR 20675 at commit

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20382 **[Test build #87667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87667/testReport)** for PR 20382 at commit

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170519540 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/UnsafeMemoryAllocator.java --- @@ -19,15 +19,24 @@ import

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170519752 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,135 @@ */ public static final int

[GitHub] spark issue #20667: [SPARK-23508][CORE] Use timeStampedHashMap for Blockmana...

2018-02-26 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/20667 @cloud-fan I just find commit log below `Modified StorageLevel and BlockManagerId to cache common objects and use cached object while deserializing.` I can't figure out why we need cache,

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170520696 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -50,12 +52,11 @@ // These are only updated by

[GitHub] spark pull request #20667: [SPARK-23508][CORE] Use timeStampedHashMap for Bl...

2018-02-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20667#discussion_r170520957 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerId.scala --- @@ -132,10 +133,15 @@ private[spark] object BlockManagerId {

[GitHub] spark issue #20667: [SPARK-23508][CORE] Use softreference for BlockmanagerId...

2018-02-26 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20667 Had a offline chat with @cloud-fan and we feel `com.google.common.cache.Cache` should be used here. You can find a example at `CodeGenerator.cache`. ---

[GitHub] spark issue #20667: [SPARK-23508][CORE] Use softreference for BlockmanagerId...

2018-02-26 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/20667 Nice @jiangxb1987 @cloud-fan I will modify later.Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20611: [SPARK-23425][SQL]When wild card is been used in load co...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20611 **[Test build #87668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87668/testReport)** for PR 20611 at commit

[GitHub] spark pull request #20670: [SPARK-23405] Add constranits

2018-02-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20670#discussion_r170529019 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -29,12 +29,26 @@ trait

[GitHub] spark pull request #20670: [SPARK-23405] Add constranits

2018-02-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20670#discussion_r170528989 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -29,12 +29,26 @@ trait

[GitHub] spark pull request #20662: [SPARK-23475][UI][BACKPORT-2.3] Show also skipped...

2018-02-26 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/20662 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20670: [SPARK-23405] Add constranits

2018-02-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20670#discussion_r170529062 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -29,12 +29,26 @@ trait

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20663: [SPARK-23501][UI] Refactor AllStagesPage in order...

2018-02-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20663#discussion_r170532005 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala --- @@ -143,76 +72,105 @@ private[ui] class AllStagesPage(parent: StagesTab)

[GitHub] spark pull request #20663: [SPARK-23501][UI] Refactor AllStagesPage in order...

2018-02-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20663#discussion_r170532241 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala --- @@ -143,76 +72,105 @@ private[ui] class AllStagesPage(parent: StagesTab)

[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...

2018-02-26 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/20667 Update @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20673: [SPARK-23515] Use input/output streams for large ...

2018-02-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20673#discussion_r170537137 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -100,7 +102,18 @@ private[spark] object JsonProtocol {

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-02-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20678 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20678 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87674/ Test PASSed. ---

[GitHub] spark issue #20664: [SPARK-23496][CORE] Locality of coalesced partitions can...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20664 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20664: [SPARK-23496][CORE] Locality of coalesced partitions can...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20664 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87672/ Test FAILed. ---

[GitHub] spark issue #20628: [SPARK-23449][K8S] Preserve extraJavaOptions ordering

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20628 **[Test build #87680 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87680/testReport)** for PR 20628 at commit

[GitHub] spark pull request #20663: [SPARK-23501][UI] Refactor AllStagesPage in order...

2018-02-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20663#discussion_r170685539 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala --- @@ -67,152 +41,120 @@ private[ui] class AllStagesPage(parent: StagesTab)

[GitHub] spark issue #20628: [SPARK-23449][K8S] Preserve extraJavaOptions ordering

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20628 **[Test build #87680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87680/testReport)** for PR 20628 at commit

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170686427 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,135 @@ */ public static final int

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-26 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20675 Now that I think about it, we may eventually need a way to set the starting partition offset after creation for other reasons, so I'm less confident in those second and third reasons. But on

[GitHub] spark issue #20628: [SPARK-23449][K8S] Preserve extraJavaOptions ordering

2018-02-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20628 Merging to master / 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20669: [SPARK-22839][K8S][WIP] Remove the use of init-container...

2018-02-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20669 If that's the consensus, then great! I thought some people still wanted to keep the init container around for some reason. --- -

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20678 **[Test build #87674 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87674/testReport)** for PR 20678 at commit

[GitHub] spark pull request #20679: [SPARK-23514] Use SessionState.newHadoopConf() to...

2018-02-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20679#discussion_r170666428 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -115,7 +115,9 @@ private[sql] class SessionState(

[GitHub] spark pull request #20628: [SPARK-23449][K8S] Preserve extraJavaOptions orde...

2018-02-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20628 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87673/ Test PASSed. ---

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20678 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-26 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20553 > Do you mean `spark.kubernetes.executor.cores` will only be used with k8s for static allocation? It looks to me that if we wanna k8s work with Spark dynamic allocation better, we have to change

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1066/

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20673 **[Test build #87679 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87679/testReport)** for PR 20673 at commit

[GitHub] spark issue #20669: [SPARK-22839][K8S][WIP] Remove the use of init-container...

2018-02-26 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20669 +1, the benefits outweigh the desire to be k8s-like in the approach. We should have tests that validate the behavior changes though. I'm wondering if we should move the integration tests into the

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-26 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/20673 @HyukjinKwon Added tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20675: [SPARK-23033][SS][Follow Up] Task level retry for...

2018-02-26 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20675#discussion_r170665692 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -219,18 +219,59 @@ class ContinuousSuite

[GitHub] spark pull request #20675: [SPARK-23033][SS][Follow Up] Task level retry for...

2018-02-26 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20675#discussion_r170665920 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala --- @@ -194,6 +194,12 @@ trait StreamTest extends QueryTest with

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-02-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r170682155 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -115,6 +116,19 @@ private[spark] class

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-02-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r170682734 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -115,6 +116,19 @@ private[spark] class

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20679 **[Test build #87675 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87675/testReport)** for PR 20679 at commit

[GitHub] spark issue #20664: [SPARK-23496][CORE] Locality of coalesced partitions can...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20664 **[Test build #87672 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87672/testReport)** for PR 20664 at commit

[GitHub] spark issue #20628: [SPARK-23449][K8S] Preserve extraJavaOptions ordering

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20628 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20628: [SPARK-23449][K8S] Preserve extraJavaOptions ordering

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20628 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87680/ Test PASSed. ---

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20663: [SPARK-23501][UI] Refactor AllStagesPage in order to avo...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20663 **[Test build #87676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87676/testReport)** for PR 20663 at commit

[GitHub] spark issue #20658: [SPARK-23488][python] Add missing catalog methods to pyt...

2018-02-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20658 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20628: [SPARK-23449][K8S] Preserve extraJavaOptions ordering

2018-02-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20628 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/20657 at a high level looks good. I'm glad to see us passing these to the executors over rpc now. I haven't looked in detail at the mesos side. ---

[GitHub] spark issue #20628: [SPARK-23449][K8S] Preserve extraJavaOptions ordering

2018-02-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20628 LGTM, pending tests. I hope we can just use spark-submit here at some point. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20669: [SPARK-22839][K8S][WIP] Remove the use of init-container...

2018-02-26 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20669 My impression was that we did reach a consensus that this was the right idea from the mailing list discussion. Specifically

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20678 **[Test build #87673 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87673/testReport)** for PR 20678 at commit

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87675/ Test PASSed. ---

[GitHub] spark pull request #20622: [SPARK-23491][SS] Remove explicit job cancellatio...

2018-02-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20622 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20663: [SPARK-23501][UI] Refactor AllStagesPage in order to avo...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20663 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20663: [SPARK-23501][UI] Refactor AllStagesPage in order to avo...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20663 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87676/ Test PASSed. ---

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20673 **[Test build #87679 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87679/testReport)** for PR 20673 at commit

[GitHub] spark pull request #20382: [SPARK-23097][SQL][SS] Migrate text socket source...

2018-02-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20382#discussion_r170759995 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/TextSocketStreamSuite.scala --- @@ -0,0 +1,246 @@ +/* + * Licensed

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.3

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20373 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1069/

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.3

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20373 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-26 Thread liufengdb
Github user liufengdb commented on the issue: https://github.com/apache/spark/pull/20681 @felixcheung Can you take a look at the changes in the R tests? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1071/

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #87678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87678/testReport)** for PR 19222 at commit

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.3

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20373 **[Test build #87682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87682/testReport)** for PR 20373 at commit

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20556 **[Test build #87683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87683/testReport)** for PR 20556 at commit

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1070/

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-26 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r170733980 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala --- @@ -541,7 +541,7 @@ object DecisionTreeSuite extends

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87679/ Test PASSed. ---

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20681 **[Test build #87681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87681/testReport)** for PR 20681 at commit

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1068/

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.3

2018-02-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20373 So it looks like the 2.3 release is probably going to go out but Jenkins thinks this can't be merged with master. So lets do a jenkins retest this please and I'll try and take some review cycles

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20556 Thanks for working on this! I like the idea of automated document style checking for Python :) --- - To unsubscribe, e-mail:

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20556 Jenkins retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-02-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13599 So it looks like 2.3 is finishing up. Would it make sense for us to all try and chat around Strata SJ if folks are around? ---

  1   2   3   4   5   >