[GitHub] spark issue #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Optimizer

2018-03-01 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20560 @gatorsmile sorry, do you have time now to take a look at this? Or may I ping you some days later if you are busy? Thanks. ---

[GitHub] spark pull request #20703: [SPARK-19185][SS] Make Kafka consumer cache confi...

2018-03-01 Thread gaborgsomogyi
GitHub user gaborgsomogyi opened a pull request: https://github.com/apache/spark/pull/20703 [SPARK-19185][SS] Make Kafka consumer cache configurable ## What changes were proposed in this pull request? Use property `spark.streaming.kafka.consumer.cache.enabled` in structured

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20043 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171585911 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,135 @@ */ public static final int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171586314 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/UnsafeMemoryAllocator.java --- @@ -36,22 +42,34 @@ public MemoryBlock allocate(long

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20043 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20043 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87843/ Test FAILed. ---

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171588018 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -1380,10 +1428,9 @@ public void write(Kryo kryo, Output out) {

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171588692 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -99,8 +99,15 @@ public static UTF8String fromBytes(byte[]

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20043 **[Test build #87844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87844/testReport)** for PR 20043 at commit

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20701 **[Test build #87830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87830/testReport)** for PR 20701 at commit

[GitHub] spark issue #20696: [SPARK-23525] [SQL] Support ALTER TABLE CHANGE COLUMN CO...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87842/ Test FAILed. ---

[GitHub] spark issue #20696: [SPARK-23525] [SQL] Support ALTER TABLE CHANGE COLUMN CO...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20696 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20449: [SPARK-23040][CORE]: Returns interruptible iterator for ...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20449 **[Test build #87845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87845/testReport)** for PR 20449 at commit

[GitHub] spark issue #20703: [SPARK-19185][SS] Make Kafka consumer cache configurable

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20703 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20043 **[Test build #87843 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87843/testReport)** for PR 20043 at commit

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171587203 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -273,6 +286,7 @@ public long getPrefix() { * @param

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-03-01 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20295 @HyukjinKwon @felixcheung Thank you for the comments. I will take a look and hopefully address them today. --- - To

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171589435 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/ByteArrayMemoryBlock.java --- @@ -0,0 +1,104 @@ +/* + * Licensed to the

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20043 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1197/

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171590355 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SortBenchmark.scala --- @@ -36,7 +36,8 @@ import

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20043 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171590147 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -273,6 +286,7 @@ public long getPrefix() { * @param until

[GitHub] spark issue #20670: [SPARK-23405] Generate additional constraints for Join's...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20670 **[Test build #87836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87836/testReport)** for PR 20670 at commit

[GitHub] spark issue #20682: [SPARK-23522][Python] always use sys.exit over builtin e...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20682 **[Test build #87833 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87833/testReport)** for PR 20682 at commit

[GitHub] spark issue #20670: [SPARK-23405] Generate additional constraints for Join's...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20670 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20703: [SPARK-19185][SS] Make Kafka consumer cache configurable

2018-03-01 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20703 I completely agree with @zsxwing, let understand what the issue is rather than covering it up with a workaround. We should not run into such issue at all. ---

[GitHub] spark pull request #20709: [SPARK-18844][MLLIB] Adding more binary classific...

2018-03-01 Thread sandecho
GitHub user sandecho opened a pull request: https://github.com/apache/spark/pull/20709 [SPARK-18844][MLLIB] Adding more binary classification evaluation metrics ## What changes were proposed in this pull request? The following additional binary classification metrics are

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87849/testReport)** for PR 20705 at commit

[GitHub] spark issue #20639: [SPARK-23288][SS] Fix output metrics with parquet sink

2018-03-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20639 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87851/ Test FAILed. ---

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20706 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87850/ Test FAILed. ---

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20706 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20706 **[Test build #87850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87850/testReport)** for PR 20706 at commit

[GitHub] spark issue #20641: [SPARK-23464][MESOS] Fix mesos cluster scheduler options...

2018-03-01 Thread krcz
Github user krcz commented on the issue: https://github.com/apache/spark/pull/20641 @susanxhuynh Thanks for the example! I have added similar test, covering more cases. After you or someone else reviews it, I'll rebase the pull request. ---

[GitHub] spark pull request #20698: [SPARK-23541][SS] Allow Kafka source to read data...

2018-03-01 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20698#discussion_r171685190 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala --- @@ -0,0 +1,105 @@ +/* + *

[GitHub] spark pull request #20698: [SPARK-23541][SS] Allow Kafka source to read data...

2018-03-01 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20698#discussion_r171685001 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala --- @@ -0,0 +1,105 @@ +/* + *

[GitHub] spark pull request #20698: [SPARK-23541][SS] Allow Kafka source to read data...

2018-03-01 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20698#discussion_r171685641 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala --- @@ -0,0 +1,105 @@ +/* + *

[GitHub] spark pull request #20698: [SPARK-23541][SS] Allow Kafka source to read data...

2018-03-01 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20698#discussion_r171698925 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetRangeCalculator.scala --- @@ -0,0 +1,105 @@ +/* + *

[GitHub] spark issue #20709: [SPARK-18844][MLLIB] Adding more binary classification e...

2018-03-01 Thread sandecho
Github user sandecho commented on the issue: https://github.com/apache/spark/pull/20709 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20704 **[Test build #87848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87848/testReport)** for PR 20704 at commit

[GitHub] spark pull request #20699: [SPARK-23544][SQL]Remove repartition operation fr...

2018-03-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20699#discussion_r171697012 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -613,6 +626,13 @@ object CollapseRepartition

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20706 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1206/

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20706 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20709: [SPARK-18844][MLLIB] Adding more binary classification e...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20709 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20111: [SPARK-22883][ML][TEST] Streaming tests for spark.ml.fea...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20111 **[Test build #87857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87857/testReport)** for PR 20111 at commit

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20618 **[Test build #87860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87860/testReport)** for PR 20618 at commit

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20706 > I do not know whether this change only affects the test cases. I do, because I checked. The tests were using the API incorrectly, actually, and could leave garbage outside the temp dirs,

[GitHub] spark pull request #20703: [SPARK-19185][SS] Make Kafka consumer cache confi...

2018-03-01 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20703#discussion_r171690388 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchReader.scala --- @@ -76,6 +76,10 @@ private[kafka010] class

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16578 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1209/

[GitHub] spark issue #20111: [SPARK-22883][ML][TEST] Streaming tests for spark.ml.fea...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20111 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87857/ Test PASSed. ---

[GitHub] spark issue #20111: [SPARK-22883][ML][TEST] Streaming tests for spark.ml.fea...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20111 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20692: [SPARK-23531][SQL] Show attribute type in explain

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20692 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20703: [SPARK-19185][SS] Make Kafka consumer cache confi...

2018-03-01 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20703#discussion_r171648378 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -376,6 +383,8 @@ The following configurations are optional: +If

[GitHub] spark issue #20692: [SPARK-23531][SQL] Show attribute type in explain

2018-03-01 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20692 > this is not urgent based on our release schedule. Marco is contributing this right now. It is a bad idea to ask contributors to show up in 4 months, if we don't have a better option by

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20706 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1201/

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20706 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171664129 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/UnsafeMemoryAllocator.java --- @@ -36,22 +42,34 @@ public MemoryBlock allocate(long

[GitHub] spark pull request #20705: [SPARK-23553][TESTS] Tests should not assume the ...

2018-03-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20705#discussion_r171668051 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala --- @@ -739,15 +739,15 @@

[GitHub] spark pull request #20702: [SPARK-23547][SQL]Cleanup the .pipeout file when ...

2018-03-01 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20702#discussion_r171668166 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java --- @@ -665,6 +667,25 @@ public void close()

[GitHub] spark issue #20692: [SPARK-23531][SQL] Show attribute type in explain

2018-03-01 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20692 > I think you can add the patch to your fork. My primary concern isn't the feature, although I do think it is useful. My concern is how we work with contributors. My worry here is that the

[GitHub] spark pull request #20707: [SPARK-21209][MLLLIB] Implement Incremental PCA a...

2018-03-01 Thread sandecho
GitHub user sandecho reopened a pull request: https://github.com/apache/spark/pull/20707 [SPARK-21209][MLLLIB] Implement Incremental PCA algorithm ## What changes were proposed in this pull request? A new feature called Incremental Principal Component Analysis

[GitHub] spark pull request #20707: [SPARK-21209][MLLLIB] Implement Incremental PCA a...

2018-03-01 Thread sandecho
Github user sandecho closed the pull request at: https://github.com/apache/spark/pull/20707 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20692: [SPARK-23531][SQL] Show attribute type in explain

2018-03-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20692 Thanks for your suggestion. I think different communities have different cultures. For new contributors, we always try to encourage them and assign them some trivial JIRAs and fixes.

[GitHub] spark issue #20691: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20691 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20691: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20691 **[Test build #87852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87852/testReport)** for PR 20691 at commit

[GitHub] spark issue #20691: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20691 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87852/ Test FAILed. ---

[GitHub] spark pull request #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20706#discussion_r171683202 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -267,44 +264,20 @@ private[spark] object Utils extends Logging { }

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20704 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20704 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87848/ Test FAILed. ---

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20704 The failure is due to flaky test. ``` org.apache.spark.sql.execution.streaming.RateSourceV2Suite.basic microbatch execution ``` ---

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20704 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20704 **[Test build #87854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87854/testReport)** for PR 20704 at commit

[GitHub] spark pull request #20549: SPARK-18844[MLLIB] Add more binary classification...

2018-03-01 Thread sandecho
Github user sandecho closed the pull request at: https://github.com/apache/spark/pull/20549 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20681 **[Test build #87858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87858/testReport)** for PR 20681 at commit

[GitHub] spark issue #20709: [SPARK-18844][MLLIB] Adding more binary classification e...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20709 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20111: [SPARK-22883][ML][TEST] Streaming tests for spark.ml.fea...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20111 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20111: [SPARK-22883][ML][TEST] Streaming tests for spark.ml.fea...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20111 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1207/

[GitHub] spark issue #20639: [SPARK-23288][SS] Fix output metrics with parquet sink

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20639 **[Test build #87856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87856/testReport)** for PR 20639 at commit

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1208/

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16578 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87851/testReport)** for PR 20705 at commit

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16578 **[Test build #87859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87859/testReport)** for PR 16578 at commit

[GitHub] spark issue #20111: [SPARK-22883][ML][TEST] Streaming tests for spark.ml.fea...

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20111 **[Test build #87857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87857/testReport)** for PR 20111 at commit

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20706 Unfortunately, I do not know whether this change only affects the test cases. Any behavior change require a conf. We can get rid of it later if nobody hit the issue. ---

[GitHub] spark issue #20708: [SPARK-21209][MLLLIB] Implement Incremental PCA algorith...

2018-03-01 Thread sandecho
Github user sandecho commented on the issue: https://github.com/apache/spark/pull/20708 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1205/

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20706 Ok, I'm going to revert the `createDirectory` changes because the `Files.createTempDirectory` api forces permissions to `700`, and secure YARN requires that the block manager directories respect the

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20704 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20706 **[Test build #87855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87855/testReport)** for PR 20706 at commit

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87849/ Test FAILed. ---

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20111: [SPARK-22883][ML][TEST] Streaming tests for spark.ml.fea...

2018-03-01 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/20111 Updated! Thanks @WeichenXu123 -- I'll merge this once tests pass. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-03-01 Thread liufengdb
Github user liufengdb commented on the issue: https://github.com/apache/spark/pull/20681 Overall, I think this suite needs a refactoring: split to in-memory catalog one and hive catalog one. The catalog conf should not be manipulated after the spark context is created. The other way

[GitHub] spark issue #20692: [SPARK-23531][SQL] Show attribute type in explain

2018-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20692 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87847/ Test FAILed. ---

[GitHub] spark issue #20692: [SPARK-23531][SQL] Show attribute type in explain

2018-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20692 **[Test build #87847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87847/testReport)** for PR 20692 at commit

[GitHub] spark issue #20692: [SPARK-23531][SQL] Show attribute type in explain

2018-03-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20692 Normally, in our Spark community, if the contributor is not available when we decide to merge, the others (including the committers) will take it over and give the credit back the contributor.

[GitHub] spark pull request #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-01 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/20706 [SPARK-23550][core] Cleanup `Utils`. A few different things going on: - Remove unused methods. - Move JSON methods to the only class that uses them. - Move test-only methods to

<    1   2   3   4   5   6   7   >