[GitHub] spark issue #13807: [SPARK-16097][SQL] Encoders.tuple should handle null obj...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13807 **[Test build #60937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60937/consoleFull)** for PR 13807 at commit

[GitHub] spark pull request #13807: [SPARK-16097][SQL] Encoders.tuple should handle n...

2016-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13807#discussion_r67887470 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -830,6 +830,13 @@ class DatasetSuite extends QueryTest with

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67881287 --- Diff: docs/sparkr.md --- @@ -262,6 +262,83 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13802 @maropu this won't work for other `hive_udf`s since these also maintain internal state, and currently require per group processing. This also has a greater potential of creating out-of-memory

[GitHub] spark issue #13231: [SPARK-15453] [SQL] Sort Merge Join to use bucketing met...

2016-06-21 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/13231 Will re-open when I am ready --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13231: [SPARK-15453] [SQL] Sort Merge Join to use bucket...

2016-06-21 Thread tejasapatil
Github user tejasapatil closed the pull request at: https://github.com/apache/spark/pull/13231 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #13810: [SPARK-16037][SQL] Follow-up: add DataFrameWriter.insert...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13810 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13810: [SPARK-16037][SQL] Follow-up: add DataFrameWriter.insert...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13810 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60934/ Test PASSed. ---

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67880977 --- Diff: docs/sparkr.md --- @@ -262,6 +262,83 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark issue #13810: [SPARK-16037][SQL] Follow-up: add DataFrameWriter.insert...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13810 **[Test build #60934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60934/consoleFull)** for PR 13810 at commit

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67880749 --- Diff: docs/sparkr.md --- @@ -262,6 +262,83 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67880530 --- Diff: docs/sparkr.md --- @@ -262,6 +262,83 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67880209 --- Diff: docs/sparkr.md --- @@ -262,6 +262,83 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark issue #13807: [SPARK-16097][SQL] Encoders.tuple should handle null obj...

2016-06-21 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13807 LGTM except for minor issues. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67880117 --- Diff: docs/sparkr.md --- @@ -262,6 +262,83 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark pull request #13807: [SPARK-16097][SQL] Encoders.tuple should handle n...

2016-06-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13807#discussion_r67880034 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -830,6 +830,13 @@ class DatasetSuite extends QueryTest with

[GitHub] spark issue #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread sun-rui
Github user sun-rui commented on the issue: https://github.com/apache/spark/pull/13660 Can you add documentation for gapply() and gapplyCollect() together here? or @NarineK will do in another PR? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #13807: [SPARK-16097][SQL] Encoders.tuple should handle n...

2016-06-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13807#discussion_r67879774 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala --- @@ -110,16 +110,25 @@ object ExpressionEncoder {

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13802 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60933/ Test PASSed. ---

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13802 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13802 **[Test build #60933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60933/consoleFull)** for PR 13802 at commit

[GitHub] spark issue #13752: [SPARK-16028][SPARKR] spark.lapply can work with active ...

2016-06-21 Thread sun-rui
Github user sun-rui commented on the issue: https://github.com/apache/spark/pull/13752 I think spark.lapply() is a case that demonstrates the need for supporting Dataset in SparkR. Removing the explicit sc parameter is quite helpful to moving to Dataset internally in the future if

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-21 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r67878608 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -0,0 +1,401 @@ +/* + *

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-21 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r67877379 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -0,0 +1,309 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-21 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r67876883 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -0,0 +1,401 @@ +/* + *

[GitHub] spark pull request #13738: [SPARK-11227][CORE] UnknownHostException can be t...

2016-06-21 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/13738#discussion_r67876187 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -421,6 +421,13 @@ object SparkHadoopUtil { val

[GitHub] spark issue #13809: [SPARK-16104][SQL] Do not creaate CSV writer object for ...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13809 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60932/ Test PASSed. ---

[GitHub] spark issue #13809: [SPARK-16104][SQL] Do not creaate CSV writer object for ...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13809 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13809: [SPARK-16104][SQL] Do not creaate CSV writer object for ...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13809 **[Test build #60932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60932/consoleFull)** for PR 13809 at commit

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-21 Thread lfrancke
Github user lfrancke commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r67875791 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -0,0 +1,401 @@ +/* + * Licensed

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-21 Thread lfrancke
Github user lfrancke commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r67875720 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -0,0 +1,401 @@ +/* + * Licensed

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-21 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r67875507 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -0,0 +1,401 @@ +/* + *

[GitHub] spark issue #13738: [SPARK-11227][CORE] UnknownHostException can be thrown w...

2016-06-21 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/13738 the jira says this worked in spark 1.4 but not spark 1.5 or 1.6, do we know why? was spark loading the configuration somewhere else early on in spark 1.4? I must be misunderstanding your

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-21 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r67875307 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -0,0 +1,401 @@ +/* + *

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13804 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60931/ Test PASSed. ---

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13804 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13804 **[Test build #60931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60931/consoleFull)** for PR 13804 at commit

[GitHub] spark issue #13764: [SPARK-16024] [SQL] [TEST] Verify Column Comment for Dat...

2016-06-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13764 Thank you! : ) @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #13022: [SPARK-15248][SQL] Make MetastoreFileCatalog cons...

2016-06-21 Thread yin
Github user yin commented on a diff in the pull request: https://github.com/apache/spark/pull/13022#discussion_r67873879 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -531,18 +531,29 @@ private[hive] class

[GitHub] spark issue #13764: [SPARK-16024] [SQL] [TEST] Verify Column Comment for Dat...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13764 **[Test build #60936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60936/consoleFull)** for PR 13764 at commit

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13806 **[Test build #60935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60935/consoleFull)** for PR 13806 at commit

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13806 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13806 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60927/ Test FAILed. ---

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13806 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13806 **[Test build #60927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60927/consoleFull)** for PR 13806 at commit

[GitHub] spark issue #13808: [SPARK-14480][SQL] Remove meaningless StringIteratorRead...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13808 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13810: [SPARK-16037][SQL] Follow-up: add DataFrameWriter.insert...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13810 **[Test build #60934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60934/consoleFull)** for PR 13810 at commit

[GitHub] spark pull request #13810: [SPARK-16037][SQL] Follow-up: add DataFrameWriter...

2016-06-21 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/13810 [SPARK-16037][SQL] Follow-up: add DataFrameWriter.insertInto() test cases for by position resolution ## What changes were proposed in this pull request? This PR migrates some test cases

[GitHub] spark issue #13808: [SPARK-14480][SQL] Remove meaningless StringIteratorRead...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60930/ Test PASSed. ---

[GitHub] spark issue #13808: [SPARK-14480][SQL] Remove meaningless StringIteratorRead...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13808 **[Test build #60930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60930/consoleFull)** for PR 13808 at commit

[GitHub] spark pull request #13770: [SPARK-16054] [SQL] Verification of Multiple Data...

2016-06-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13770#discussion_r67865519 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -412,6 +417,17 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark pull request #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13780#discussion_r67865315 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetCacheSuite.scala --- @@ -21,11 +21,32 @@ import scala.language.postfixOps import

[GitHub] spark issue #13807: [SPARK-16097][SQL] Encoders.tuple should handle null obj...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13807 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13807: [SPARK-16097][SQL] Encoders.tuple should handle null obj...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13807 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60929/ Test PASSed. ---

[GitHub] spark issue #13807: [SPARK-16097][SQL] Encoders.tuple should handle null obj...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13807 **[Test build #60929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60929/consoleFull)** for PR 13807 at commit

[GitHub] spark pull request #13663: [SPARK-15950][SQL] Eliminate unreachable code at ...

2016-06-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13663#discussion_r67863392 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -26,6 +27,55 @@ import

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13802 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13802 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60928/ Test PASSed. ---

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13802 **[Test build #60928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60928/consoleFull)** for PR 13802 at commit

[GitHub] spark pull request #13802: [SPARK-16094][SQL] Support HashAggregateExec for ...

2016-06-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/13802#discussion_r67861472 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -457,6 +457,36 @@ class DataFrameAggregateSuite extends QueryTest

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13802 **[Test build #60933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60933/consoleFull)** for PR 13802 at commit

[GitHub] spark pull request #13809: [SPARK-16104][SQL] Do not creaate CSV writer obje...

2016-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/13809#discussion_r67860055 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVRelation.scala --- @@ -177,38 +180,61 @@ private[sql] class

[GitHub] spark issue #13809: [SPARK-16104][SQL] Do not creaate CSV writer object for ...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13809 **[Test build #60932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60932/consoleFull)** for PR 13809 at commit

[GitHub] spark pull request #13809: [SPARK-16104][SQL] Do not creaate CSV writer obje...

2016-06-21 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/13809 [SPARK-16104][SQL] Do not creaate CSV writer object for every flush when writing ## What changes were proposed in this pull request? This PR let `CsvWriter` object is not created for

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/13804 Certainly looks more correct --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13780 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60925/ Test PASSed. ---

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13780 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13780 **[Test build #60925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60925/consoleFull)** for PR 13780 at commit

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13804 **[Test build #60931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60931/consoleFull)** for PR 13804 at commit

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/13804 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #12855: [SPARK-10216][SQL] Avoid creating empty files during ove...

2016-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/12855 @DanielMe Yes, actually it seems a different issue when you use `emptyRDD[Row]`. Apparently, this case does not produce any partitions whereas the code provided by @jurriaan produces some empty

[GitHub] spark issue #13764: [SPARK-16024] [SQL] [TEST] Verify Column Comment for Dat...

2016-06-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13764 oh sorry looks like l made a false alarm, and thanks for adding these tests! LGTM, can you resolve the conflict? thanks! --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #13808: [SPARK-14480][SQL] Remove meaningless StringIteratorRead...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13808 **[Test build #60930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60930/consoleFull)** for PR 13808 at commit

[GitHub] spark pull request #13808: [SPARK-14480][SQL] Remove meaningless StringItera...

2016-06-21 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/13808 [SPARK-14480][SQL] Remove meaningless StringIteratorReader for CSV data source. ## What changes were proposed in this pull request? This PR removes meaningless `StringIteratorReader`

[GitHub] spark issue #12855: [SPARK-10216][SQL] Avoid creating empty files during ove...

2016-06-21 Thread DanielMe
Github user DanielMe commented on the issue: https://github.com/apache/spark/pull/12855 I can reproduce the issue that @jurriaan reports on 1.6.0 and on 1.5.2. The issue does not occur on 1.3.1. I have added a comment to the JIRA issue with more detailed instructions how to

[GitHub] spark issue #13765: [SPARK-16052][SQL] Add CollapseRepartitionBy optimizer

2016-06-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13765 Sorry we are very close to 2.0 release (and very busy to get it done) and this is too late to get into 2.0. How about we revisit it after 2.0 release? thanks! --- If your project is set up for

[GitHub] spark pull request #13663: [SPARK-15950][SQL] Eliminate unreachable code at ...

2016-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13663#discussion_r67852633 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -26,6 +27,55 @@ import

[GitHub] spark issue #13807: [SPARK-16097][SQL] Encoders.tuple should handle null obj...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13807 **[Test build #60929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60929/consoleFull)** for PR 13807 at commit

[GitHub] spark issue #13807: [SPARK-16097][SQL] Encoders.tuple should handle null obj...

2016-06-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13807 cc @yhuai @liancheng @clockfly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13807: [SPARK-16097][SQL] Encoders.tuple should handle n...

2016-06-21 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/13807 [SPARK-16097][SQL] Encoders.tuple should handle null object correctly ## What changes were proposed in this pull request? Although the top level input object can not be null, but when we

[GitHub] spark issue #13802: [SPARK-16094][SQL] Support HashAggregateExec for non-par...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13802 **[Test build #60928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60928/consoleFull)** for PR 13802 at commit

[GitHub] spark pull request #13758: [SPARK-16043][SQL] Prepare GenericArrayData imple...

2016-06-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13758#discussion_r67850981 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/GenericArrayData.scala --- @@ -142,3 +164,415 @@ class GenericArrayData(val array:

[GitHub] spark pull request #13758: [SPARK-16043][SQL] Prepare GenericArrayData imple...

2016-06-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13758#discussion_r67849962 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/GenericArrayData.scala --- @@ -142,3 +164,415 @@ class GenericArrayData(val array:

[GitHub] spark pull request #13663: [SPARK-15950][SQL] Eliminate unreachable code at ...

2016-06-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13663#discussion_r67849394 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -26,6 +27,55 @@ import

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13806 **[Test build #60927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60927/consoleFull)** for PR 13806 at commit

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13806 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13806 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60926/ Test FAILed. ---

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13806 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13806 **[Test build #60926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60926/consoleFull)** for PR 13806 at commit

[GitHub] spark issue #13806: [SPARK-16044][SQL] Backport input_file_name() for data s...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13806 **[Test build #60926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60926/consoleFull)** for PR 13806 at commit

[GitHub] spark pull request #13806: [SPARK-16044][SQL] Backport input_file_name() for...

2016-06-21 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/13806 [SPARK-16044][SQL] Backport input_file_name() for data source based on NewHadoopRDD to branch 1.6 ## What changes were proposed in this pull request? This PR backports

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13780 **[Test build #60925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60925/consoleFull)** for PR 13780 at commit

[GitHub] spark pull request #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/13780#discussion_r67846201 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetCacheSuite.scala --- @@ -21,11 +21,32 @@ import scala.language.postfixOps import

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13780 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13780 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60922/ Test PASSed. ---

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13780 **[Test build #60922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60922/consoleFull)** for PR 13780 at commit

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13804 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13804 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60921/ Test FAILed. ---

[GitHub] spark issue #13804: [Minor][Core] Fix display wrong free memory size in the ...

2016-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13804 **[Test build #60921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60921/consoleFull)** for PR 13804 at commit

[GitHub] spark issue #13805: [SPARK-16096][SPARKR] add union and deprecate unionAll

2016-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13805 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60923/ Test PASSed. ---

<    2   3   4   5   6   7   8   9   >