[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20727 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88159/ Test PASSed. ---

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20727 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20727 **[Test build #88159 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88159/testReport)** for PR 20727 at commit [`d6e9160`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #20791: [SPARK-23618][K8s][BUILD] Initialize BUILD_ARGS in docke...

2018-03-10 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20791 LGTM! Thanks @jooseong. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20793 Ah, results are different since the number of operations are different. It may be an issue like #20630. I am curious why test are failure when seed is changed. Of course, I understand the seq

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20795 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20795 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20795: [SPARK-23486]cache the function name from the cat...

2018-03-10 Thread kevinyu98
GitHub user kevinyu98 opened a pull request: https://github.com/apache/spark/pull/20795 [SPARK-23486]cache the function name from the catalog for lookupFunctions ## What changes were proposed in this pull request? This PR will cache the function name from spark and external

[GitHub] spark issue #20791: [SPARK-23618][K8s][BUILD] Initialize BUILD_ARGS in docke...

2018-03-10 Thread jooseong
Github user jooseong commented on the issue: https://github.com/apache/spark/pull/20791 Added [K8s] into the PR title. Thanks for the review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For add

[GitHub] spark issue #20791: [SPARK-23618][BUILD] Initialize BUILD_ARGS in docker-ima...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20791 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88157/ Test PASSed. ---

[GitHub] spark issue #20791: [SPARK-23618][BUILD] Initialize BUILD_ARGS in docker-ima...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20791 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20791: [SPARK-23618][BUILD] Initialize BUILD_ARGS in docker-ima...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20791 **[Test build #88157 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88157/testReport)** for PR 20791 at commit [`096e992`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20793 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20793 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88156/ Test FAILed. ---

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20793 **[Test build #88156 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88156/testReport)** for PR 20793 at commit [`bb40ef2`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20793 Does `hashSeed` method produce same hash value after this change? ```scala scala> def hashSeed(seed: Long): Long = { | val bytes = ByteBuffer.allocate(java.lang.Long.SIZE).putL

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20727 **[Test build #88159 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88159/testReport)** for PR 20727 at commit [`d6e9160`](https://github.com/apache/spark/commit/d6

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20727 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20727 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1454/ Tes

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20727 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88158/ Test FAILed. ---

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20727 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20727 **[Test build #88158 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88158/testReport)** for PR 20727 at commit [`97a8422`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20727 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1453/ Tes

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20727 **[Test build #88158 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88158/testReport)** for PR 20727 at commit [`97a8422`](https://github.com/apache/spark/commit/97

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20727 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r173642056 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -42,7 +52,12 @@ class HadoopFileLinesRe

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20785 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88155/ Test PASSed. ---

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20785 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20785 **[Test build #88155 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88155/testReport)** for PR 20785 at commit [`0034a58`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #17774: [SPARK-18371][Streaming] Spark Streaming backpressure ge...

2018-03-10 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/17774 LGTM @tdas @zsxwing absent any objections from you in the next couple of days, I'll merge this --- - To unsubscribe, e-mai

[GitHub] spark issue #19431: [SPARK-18580] [DStreams] [external/kafka-0-10][external/...

2018-03-10 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/19431 @tdas any concerns? If @omuravskiy doesn't express any objections (since these tests are basically taken directly from his linked PR) in the next couple of days, I'm inclined to merge thi

[GitHub] spark pull request #19431: [SPARK-18580] [DStreams] [external/kafka-0-10][ex...

2018-03-10 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/19431#discussion_r173641331 --- Diff: external/kafka-0-8/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala --- @@ -456,6 +455,60 @@ class DirectKafkaStrea

[GitHub] spark issue #20767: [SPARK-23623] [SS] Avoid concurrent use of cached consum...

2018-03-10 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/20767 Can you clarify why you want to allow only 1 cached consumer per topicpartition, closing any others at task end? It seems like opening and closing consumers would be less efficient than a

[GitHub] spark issue #20791: [SPARK-23618][BUILD] Initialize BUILD_ARGS in docker-ima...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20791 **[Test build #88157 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88157/testReport)** for PR 20791 at commit [`096e992`](https://github.com/apache/spark/commit/09

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20793 **[Test build #88156 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88156/testReport)** for PR 20793 at commit [`bb40ef2`](https://github.com/apache/spark/commit/bb

[GitHub] spark issue #20791: [SPARK-23618][BUILD] Initialize BUILD_ARGS in docker-ima...

2018-03-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20791 @foxish --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spar

[GitHub] spark issue #20791: [SPARK-23618][BUILD] Initialize BUILD_ARGS in docker-ima...

2018-03-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20791 could you add [K8s] into PR title --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20791: [SPARK-23618][BUILD] Initialize BUILD_ARGS in docker-ima...

2018-03-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20791 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20793 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r173639932 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextOptions.scala --- @@ -39,9 +39,12 @@ private[text] class TextOpti

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r173639748 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -30,9 +31,19 @@ import org.apache.hado

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20785 **[Test build #88155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88155/testReport)** for PR 20785 at commit [`0034a58`](https://github.com/apache/spark/commit/00

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20785 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20785 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1452/ Tes

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20785 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark pull request #20767: [SPARK-23623] [SS] Avoid concurrent use of cached...

2018-03-10 Thread tedyu
Github user tedyu commented on a diff in the pull request: https://github.com/apache/spark/pull/20767#discussion_r173636109 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala --- @@ -342,80 +415,103 @@ private[kafka010] object Ca

[GitHub] spark pull request #20767: [SPARK-23623] [SS] Avoid concurrent use of cached...

2018-03-10 Thread tedyu
Github user tedyu commented on a diff in the pull request: https://github.com/apache/spark/pull/20767#discussion_r173636002 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala --- @@ -342,80 +415,103 @@ private[kafka010] object Ca

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r173633651 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -30,9 +31,19 @@ import org.apache.hadoop.m

[GitHub] spark issue #20794: [SPARK-23644][CORE][UI] Use absolute path for REST call ...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20794 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88153/ Test PASSed. ---

[GitHub] spark issue #20794: [SPARK-23644][CORE][UI] Use absolute path for REST call ...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20794 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20794: [SPARK-23644][CORE][UI] Use absolute path for REST call ...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20794 **[Test build #88153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88153/testReport)** for PR 20794 at commit [`17ea399`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r173633462 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -42,7 +52,12 @@ class HadoopFileLinesReader

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20779 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88154/ Test PASSed. ---

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20779 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20779 **[Test build #88154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88154/testReport)** for PR 20779 at commit [`603ce0f`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r173632775 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextOptions.scala --- @@ -39,9 +39,12 @@ private[text] class TextOptions(

[GitHub] spark issue #18994: [SPARK-21784][SQL] Adds support for defining information...

2018-03-10 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/18994 @sureshthalamati Hi Suresh, We are planning to proceed with the performance improvements. Will you be able to continue working on this PR? Thanks. --- ---

[GitHub] spark issue #19775: [SPARK-22343][core] Add support for publishing Spark met...

2018-03-10 Thread matyix
Github user matyix commented on the issue: https://github.com/apache/spark/pull/19775 For those who are still interested using Prometheus you can get the standalone package and source code from here: https://github.com/banzaicloud/spark-metrics . Happy monitoring, try to catch the is

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20785 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88152/ Test FAILed. ---

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20785 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20785 **[Test build #88152 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88152/testReport)** for PR 20785 at commit [`0034a58`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20763 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20793 Good catch, LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-03-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20043 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20779 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1451/ Tes

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20779 **[Test build #88154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88154/testReport)** for PR 20779 at commit [`603ce0f`](https://github.com/apache/spark/commit/60

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20779 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20779 Let me reduce the number of loops. Another option is to revert this change to use non-loop version that worked without an exception. --- -

[GitHub] spark issue #20794: [SPARK-23644][CORE][UI] Use absolute path for REST call ...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20794 **[Test build #88153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88153/testReport)** for PR 20794 at commit [`17ea399`](https://github.com/apache/spark/commit/17

[GitHub] spark issue #20794: [SPARK-23644][CORE][UI] Use absolute path for REST call ...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20794 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1450/ Tes

[GitHub] spark issue #20794: [SPARK-23644][CORE][UI] Use absolute path for REST call ...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20794 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20794: [SPARK-23644][CORE][UI] Use absolute path for RES...

2018-03-10 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/20794 [SPARK-23644][CORE][UI] Use absolute path for REST call in SHS ## What changes were proposed in this pull request? SHS is using a relative path for the REST API call to get the list of the

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r173625828 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -72,6 +72,29 @@ class FileBasedDataSourceSuite extends Quer

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20793 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20793 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20793: [SPARK-23643] Shrinking the buffer in hashSeed up...

2018-03-10 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/20793 [SPARK-23643] Shrinking the buffer in hashSeed up to size of the seed parameter ## What changes were proposed in this pull request? The hashSeed method allocates 64 bytes instead of 8. Oth

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88150/ Test PASSed. ---

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20701 **[Test build #88150 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88150/testReport)** for PR 20701 at commit [`b3d0523`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20525 late LGTM too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark pull request #20788: [WIP][SPARK-21030][PYTHON][SQL] Adds more types f...

2018-03-10 Thread DylanGuedes
Github user DylanGuedes commented on a diff in the pull request: https://github.com/apache/spark/pull/20788#discussion_r173623998 --- Diff: python/pyspark/sql/dataframe.py --- @@ -437,10 +437,11 @@ def hint(self, name, *parameters): if not isinstance(name, str):

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20785 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1449/ Tes

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20785 **[Test build #88152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88152/testReport)** for PR 20785 at commit [`0034a58`](https://github.com/apache/spark/commit/00

[GitHub] spark issue #20785: [SPARK-23640][CORE] Fix hadoop config may override spark...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20785 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88149/ Test PASSed. ---

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20717 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20717 **[Test build #88149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88149/testReport)** for PR 20717 at commit [`9e2d993`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20779 I don't think so. There is an option to change the heap size for test execution, but I am not sure we are allowed/it is a good idea to do that. Let's hear others' opinion... ---

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20779 Ah, I increased the heap size (4GB) in my environment with IntelliJ. Should we create a class like https://github.com/apache/spark/pull/20636? ---

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88151/ Test PASSed. ---

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20719 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20719 **[Test build #88151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88151/testReport)** for PR 20719 at commit [`2d64a90`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20790: AccumulatorV2 subclass isZero scaladoc fix

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20790 Wait .. I just found you opened a JIRA - SPARK-23642. Please link it by `[SPARK-23642][DOCS] ...`. see https://spark.apache.org/contributing.html --- --

[GitHub] spark issue #20790: AccumulatorV2 subclass isZero scaladoc fix

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20790 Shall we fix the title to `[MINOR][DOCS] AccumulatorV2 ...` to be consistent with other PRs? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20790: AccumulatorV2 subclass isZero scaladoc fix

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20790#discussion_r173621260 --- Diff: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala --- @@ -290,7 +290,8 @@ class LongAccumulator extends AccumulatorV2[jl.Long, jl.

[GitHub] spark issue #20771: [SPARK-23587][SQL] Add interpreted execution for MapObje...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20771 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88146/ Test PASSed. ---

[GitHub] spark issue #20771: [SPARK-23587][SQL] Add interpreted execution for MapObje...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20771 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20771: [SPARK-23587][SQL] Add interpreted execution for MapObje...

2018-03-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20771 **[Test build #88146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88146/testReport)** for PR 20771 at commit [`e725608`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1448/ Tes

[GitHub] spark issue #20719: [SPARK-23568][ML] Use metadata numAttributes if availabl...

2018-03-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20719 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

  1   2   >