[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-25 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/20675 cc @tdas and @jose-torres #20225 gives a quickly fix for task level retry, this is just an attempt for a maybe better implementation. Please let me know if I do something wrong or have

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20675 **[Test build #87665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87665/testReport)** for PR 20675 at commit

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20675 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1053/

[GitHub] spark issue #20675: [SPARK-23033][SS][Follow Up] Task level retry for contin...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20675 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20675: [SPARK-23033][SS][Follow Up] Task level retry for...

2018-02-25 Thread xuanyuanking
GitHub user xuanyuanking opened a pull request: https://github.com/apache/spark/pull/20675 [SPARK-23033][SS][Follow Up] Task level retry for continuous processing ## What changes were proposed in this pull request? Here we want to reimplement the task level retry for

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2018-02-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18581 Sure, will try to separate this. Will update my PRs soon roughly within this week. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170510883 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,135 @@ */ public static final int

[GitHub] spark pull request #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser be...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20666#discussion_r170510616 --- Diff: python/pyspark/sql/readwriter.py --- @@ -393,13 +395,16 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser be...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20666#discussion_r170510529 --- Diff: python/pyspark/sql/readwriter.py --- @@ -209,13 +209,15 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-25 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170510525 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,135 @@ */ public static final int

[GitHub] spark pull request #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser be...

2018-02-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20666#discussion_r170510027 --- Diff: python/pyspark/sql/readwriter.py --- @@ -209,13 +209,15 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20553 > This is to avoid changing the semantics of spark.executor.cores and spark.task.cpus and their role in task scheduling, task parallelism, dynamic resource allocation, etc. The new configuration

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20618 cc @felixcheung (I saw you and Felix in dev mailing list). So, https://github.com/apache/spark/tree/master/R#generating-documentation does not work? ---

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87662/ Test PASSed. ---

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20673 **[Test build #87662 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87662/testReport)** for PR 20673 at commit

[GitHub] spark issue #20667: [SPARK-23508][CORE] Use timeStampedHashMap for Blockmana...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20667 Why we need this cache? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87661/ Test PASSed. ---

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20673 **[Test build #87661 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87661/testReport)** for PR 20673 at commit

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20657 Will review it soon. 😄 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170505359 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -50,12 +52,11 @@ // These are only updated by

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170504918 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/UnsafeMemoryAllocator.java --- @@ -19,15 +19,24 @@ import

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170504851 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +44,149 @@ */ public static final int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r170504742 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,135 @@ */ public static final int

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-25 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20553 The value of `spark.executor.cores` will be used to set cpu request for the executor pods if `spark.kubernetes.executor.cores` is not set. `spark.driver.cores` already allows fractional values

[GitHub] spark issue #20674: [SPARK-23465][SQL] Introduce new function to rename colu...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20674 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20553 What is the default value if it is not configured, how do K8S control the CPU usage by default? Also it seems that user may configure how to differentiate between k8s executor cores and

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-25 Thread misutoth
Github user misutoth commented on the issue: https://github.com/apache/spark/pull/20618 Sorry, I missed these comments. As I understood we fix all of them here. I am just struggling with the R documentation: it seems the generated doc is incorrect even if I just take the latest

[GitHub] spark issue #20674: [SPARK-23465][SQL] Introduce new function to rename colu...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20674 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20674: [SPARK-23465][SQL] Introduce new function to rena...

2018-02-25 Thread misutoth
GitHub user misutoth opened a pull request: https://github.com/apache/spark/pull/20674 [SPARK-23465][SQL] Introduce new function to rename columns using an algoritm ## What changes were proposed in this pull request? Add an additional convenient method to rename multiple

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1052/

[GitHub] spark pull request #20449: [SPARK-23040][CORE]: Returns interruptible iterat...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20449#discussion_r170501977 --- Diff: core/src/test/scala/org/apache/spark/JobCancellationSuite.scala --- @@ -320,6 +319,55 @@ class JobCancellationSuite extends SparkFunSuite with

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20382 **[Test build #87664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87664/testReport)** for PR 20382 at commit

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20382 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20449: [SPARK-23040][CORE]: Returns interruptible iterat...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20449#discussion_r170501590 --- Diff: core/src/test/scala/org/apache/spark/JobCancellationSuite.scala --- @@ -320,6 +319,55 @@ class JobCancellationSuite extends SparkFunSuite with

[GitHub] spark pull request #20449: [SPARK-23040][CORE]: Returns interruptible iterat...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20449#discussion_r170501479 --- Diff: core/src/test/scala/org/apache/spark/JobCancellationSuite.scala --- @@ -18,15 +18,14 @@ package org.apache.spark import

[GitHub] spark pull request #20449: [SPARK-23040][CORE]: Returns interruptible iterat...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20449#discussion_r170501431 --- Diff: core/src/main/scala/org/apache/spark/shuffle/BlockStoreShuffleReader.scala --- @@ -104,9 +104,16 @@ private[spark] class

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170499705 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -387,6 +390,143 @@ case class CatalogStatistics(

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20666 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1051/

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20666 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser be...

2018-02-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20666#discussion_r170499119 --- Diff: python/pyspark/sql/readwriter.py --- @@ -209,13 +209,15 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark pull request #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser be...

2018-02-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20666#discussion_r170499102 --- Diff: python/pyspark/sql/readwriter.py --- @@ -393,13 +395,16 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20666 **[Test build #87663 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87663/testReport)** for PR 20666 at commit

[GitHub] spark pull request #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser be...

2018-02-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20666#discussion_r170498519 --- Diff: python/pyspark/sql/readwriter.py --- @@ -209,13 +209,15 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2018-02-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18581 It looks like this line separator has to be handled by each data source individually, can we start with, e.g., json, and then csv, text, etc.? Then we can have smaller PRs that would be easier to

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87660/ Test FAILed. ---

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20382 **[Test build #87660 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87660/testReport)** for PR 20382 at commit

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87659/ Test FAILed. ---

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20382 **[Test build #87659 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87659/testReport)** for PR 20382 at commit

[GitHub] spark issue #20557: [SPARK-23364][SQL]'desc table' command in spark-sql add ...

2018-02-25 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/20557 Well, for now, I don't have a better solution. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20543: [SPARK-23357][CORE] 'SHOW TABLE EXTENDED LIKE pattern=ST...

2018-02-25 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/20543 Oh, I just think it adds to make it clearer. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1050/

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20673 **[Test build #87662 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87662/testReport)** for PR 20673 at commit

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1049/

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87658/ Test FAILed. ---

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20673 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20673 **[Test build #87658 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87658/testReport)** for PR 20673 at commit

[GitHub] spark issue #20670: [SPARK-23405] Add constranits

2018-02-25 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the issue: https://github.com/apache/spark/pull/20670 @srowen i redescribe the problem. Now i hive a small table `ls` with one row , and a big table `catalog_sales` with One hundred billion rows. And in the big table, the non null value about

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20673 **[Test build #87661 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87661/testReport)** for PR 20673 at commit

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1048/

[GitHub] spark pull request #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferH...

2018-02-25 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20636#discussion_r170482768 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolderSparkSubmitSuite.scala --- @@ -58,15 +58,20 @@ object

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170474554 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -1059,22 +1054,22 @@ private[spark] class

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170480388 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -154,4 +156,120 @@ case class

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170475159 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/StarSchemaDetection.scala --- @@ -187,11 +187,11 @@ object

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170474720 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -1059,22 +1054,22 @@ private[spark] class

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20382 **[Test build #87660 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87660/testReport)** for PR 20382 at commit

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1047/

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20382 **[Test build #87659 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87659/testReport)** for PR 20382 at commit

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1046/

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20382 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20672: [SPARK-23509][Build] Upgrade commons-net from 2.2 to 3.1

2018-02-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20672 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20673: [SPARK-23515] Use input/output streams for large ...

2018-02-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20673#discussion_r170477999 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -100,7 +102,16 @@ private[spark] object JsonProtocol {

[GitHub] spark pull request #20673: [SPARK-23515] Use input/output streams for large ...

2018-02-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20673#discussion_r170478050 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -17,13 +17,15 @@ package org.apache.spark.util

[GitHub] spark pull request #20673: [SPARK-23515] Use input/output streams for large ...

2018-02-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20673#discussion_r170477844 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -100,7 +102,16 @@ private[spark] object JsonProtocol {

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1045/

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20673 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20673: [SPARK-23515] Use input/output streams for large events ...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20673 **[Test build #87658 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87658/testReport)** for PR 20673 at commit

[GitHub] spark pull request #20673: [SPARK-23515] Use input/output streams for large ...

2018-02-25 Thread brkyvz
GitHub user brkyvz opened a pull request: https://github.com/apache/spark/pull/20673 [SPARK-23515] Use input/output streams for large events in JsonProtocol.sparkEventToJson ## What changes were proposed in this pull request? `def sparkEventToJson(event:

[GitHub] spark issue #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20624 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87657/ Test PASSed. ---

[GitHub] spark issue #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20624 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20624 **[Test build #87657 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87657/testReport)** for PR 20624 at commit

[GitHub] spark issue #20672: [SPARK-23509][Build] Upgrade commons-net from 2.2 to 3.1

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20672 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87656/ Test PASSed. ---

[GitHub] spark issue #20672: [SPARK-23509][Build] Upgrade commons-net from 2.2 to 3.1

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20672 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20672: [SPARK-23509][Build] Upgrade commons-net from 2.2 to 3.1

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20672 **[Test build #87656 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87656/testReport)** for PR 20672 at commit

[GitHub] spark issue #20671: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20671 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20671: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87654/ Test PASSed. ---

[GitHub] spark issue #20671: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20671 **[Test build #87654 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87654/testReport)** for PR 20671 at commit

[GitHub] spark issue #20671: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20671 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20671: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87653/ Test PASSed. ---

[GitHub] spark issue #20671: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20671 **[Test build #87653 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87653/testReport)** for PR 20671 at commit

[GitHub] spark issue #20624: [SPARK-23445] ColumnStat refactoring

2018-02-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20624 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1044/

  1   2   >