[GitHub] spark issue #16672: [SPARK-19329][SQL]insert data to a not exist location da...

2017-02-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16672 Could you please add the test cases for the scenarios (of non pre existing location) you explained above? Thanks! --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #16832: [WIP][SPARK-19490][SQL] ignore case sensitivity when fil...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16832 **[Test build #72750 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72750/testReport)** for PR 16832 at commit

[GitHub] spark issue #16787: [SPARK-19448][SQL]optimize some duplication functions be...

2017-02-11 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/16787 ok thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16892: [SPARK-19560] Improve DAGScheduler tests.

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16892 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16892: [SPARK-19560] Improve DAGScheduler tests.

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16892 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72744/ Test PASSed. ---

[GitHub] spark issue #16892: [SPARK-19560] Improve DAGScheduler tests.

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16892 **[Test build #72744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72744/testReport)** for PR 16892 at commit

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-11 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r100682019 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,623 @@ +/* +

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-11 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100681997 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala --- @@ -0,0 +1,314 @@ +/* + *

[GitHub] spark issue #16895: [SPARK-15615][SQL] Add an API to load DataFrame from Dat...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16895 **[Test build #72749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72749/testReport)** for PR 16895 at commit

[GitHub] spark issue #16724: [SPARK-19352][SQL] Keep sort order of rows after externa...

2017-02-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16724 > sortWithinPartitions("userId", "timestamp") doesn't it make the userId continuous? Oh, yes. I miss looking... So you recommend that we only optimize this case to preserve the sort

[GitHub] spark issue #16724: [SPARK-19352][SQL] Keep sort order of rows after externa...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16724 `sortWithinPartitions("userId", "timestamp")` doesn't it make the `userId` continuous? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15125 **[Test build #72748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72748/testReport)** for PR 15125 at commit

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r100681839 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCWriteSuite.scala --- @@ -75,7 +75,7 @@ class JDBCWriteSuite extends SharedSQLContext with

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r100681828 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala --- @@ -23,16 +23,30 @@ package

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-02-11 Thread dding3
Github user dding3 commented on the issue: https://github.com/apache/spark/pull/15125 Thank you guys for reviewing the code. I have updated it based on the comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r100681778 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -149,4 +155,29 @@ class

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r100681762 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -149,4 +155,29 @@ class

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r100681754 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -62,6 +62,12 @@ class

[GitHub] spark pull request #16882: [SPARK-19544][SQL] Improve error message when som...

2017-02-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16882#discussion_r100681737 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -116,7 +116,7 @@ object TypeCoercion {

[GitHub] spark pull request #16878: [SPARK-19539][SQL] Block duplicate temp table dur...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16878#discussion_r100681712 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -187,15 +187,18 @@ class CSVSuite extends

[GitHub] spark pull request #16878: [SPARK-19539][SQL] Block duplicate temp table dur...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16878#discussion_r100681701 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -213,27 +216,33 @@ class CSVSuite extends

[GitHub] spark pull request #16878: [SPARK-19539][SQL] Block duplicate temp table dur...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16878#discussion_r100681692 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -187,15 +187,18 @@ class CSVSuite extends

[GitHub] spark issue #16895: [SPARK-15615][SQL] Add an API to load DataFrame from Dat...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16895 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72747/ Test FAILed. ---

[GitHub] spark issue #16895: [SPARK-15615][SQL] Add an API to load DataFrame from Dat...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16895 **[Test build #72747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72747/testReport)** for PR 16895 at commit

[GitHub] spark pull request #16878: [SPARK-19539][SQL] Block duplicate temp table dur...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16878#discussion_r100681642 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -425,7 +425,9 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #16895: [SPARK-15615][SQL] Add an API to load DataFrame from Dat...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16895 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16724: [SPARK-19352][SQL] Keep sort order of rows after externa...

2017-02-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16724 > Just realized df.repartition($"userId").sortWithinPartitions("userId", "timestamp") will produce a result set as we expected, can we optimize this case? I may not understand you

[GitHub] spark issue #16895: [SPARK-15615][SQL] Add an API to load DataFrame from Dat...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16895 **[Test build #72747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72747/testReport)** for PR 16895 at commit

[GitHub] spark pull request #16896: [SPARK-19561][Python] cast TimestampType.toIntern...

2017-02-11 Thread JasonMWhite
Github user JasonMWhite commented on a diff in the pull request: https://github.com/apache/spark/pull/16896#discussion_r100681604 --- Diff: python/pyspark/sql/tests.py --- @@ -1435,6 +1435,12 @@ def test_time_with_timezone(self): self.assertEqual(now, now1)

[GitHub] spark issue #16895: [SPARK-15615][SQL] Add an API to load DataFrame from Dat...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16895 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16895: [SPARK-15615][SQL] Add an API to load DataFrame f...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16895#discussion_r100681579 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/TestJsonData.scala --- @@ -231,4 +231,10 @@ private[json] trait

[GitHub] spark pull request #16895: [SPARK-15615][SQL] Add an API to load DataFrame f...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16895#discussion_r100681578 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1364,9 +1364,9 @@ class JsonSuite extends

[GitHub] spark issue #16672: [SPARK-19329][SQL]insert data to a not exist location da...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16672 **[Test build #72746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72746/testReport)** for PR 16672 at commit

[GitHub] spark pull request #16895: [SPARK-15615][SQL] Add an API to load DataFrame f...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16895#discussion_r100681568 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/TestJsonData.scala --- @@ -231,4 +231,10 @@ private[json] trait

[GitHub] spark pull request #16895: [SPARK-15615][SQL] Add an API to load DataFrame f...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16895#discussion_r100681550 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -328,18 +329,34 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #16672: [SPARK-19329][SQL]insert data to a not exist location da...

2017-02-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16672 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16724: [SPARK-19352][SQL] Keep sort order of rows after externa...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16724 > oh good catch. Then it seems like df.repartition($"userId").sortWithinPartitions("timestamp") won't produce a result set as we expected. Just realized

[GitHub] spark issue #16787: [SPARK-19448][SQL]optimize some duplication functions be...

2017-02-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16787 @windpiger Could you please resolve conflicts? Looks good to me. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16787: [SPARK-19448][SQL]optimize some duplication funct...

2017-02-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16787#discussion_r100681309 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -649,6 +648,16 @@ class VersionsSuite extends QueryTest

[GitHub] spark issue #16724: [SPARK-19352][SQL] Keep sort order of rows after externa...

2017-02-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16724 @cloud-fan OK. I see. If we don't want to add implicit penalty into the existing API, the only way I can think now, is a config to preserve the sort order. This config can be in `SQLConf`, or we can

[GitHub] spark pull request #16787: [SPARK-19448][SQL]optimize some duplication funct...

2017-02-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16787#discussion_r100681187 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -649,6 +648,16 @@ class VersionsSuite extends QueryTest

[GitHub] spark pull request #16787: [SPARK-19448][SQL]optimize some duplication funct...

2017-02-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16787#discussion_r100681160 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -570,7 +570,6 @@ class VersionsSuite extends QueryTest

[GitHub] spark pull request #16894: [SPARK-17897] [SQL] [BACKPORT-2.0] Fixed IsNotNul...

2017-02-11 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/16894 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #16868: [SPARK-19115] [SQL] Supporting Create External Table Lik...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16868 **[Test build #72745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72745/testReport)** for PR 16868 at commit

[GitHub] spark pull request #16870: [SPARK-19496][SQL]to_date udf to return null when...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16870#discussion_r100680992 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -95,9 +95,10 @@ object DateTimeUtils { sdf

[GitHub] spark issue #16868: [SPARK-19115] [SQL] Supporting Create External Table Lik...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16868 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16894: [SPARK-17897] [SQL] [BACKPORT-2.0] Fixed IsNotNull Const...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16894 thanks, merging to 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16896: [SPARK-19561][Python] cast TimestampType.toIntern...

2017-02-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16896#discussion_r100680863 --- Diff: python/pyspark/sql/tests.py --- @@ -1435,6 +1435,12 @@ def test_time_with_timezone(self): self.assertEqual(now, now1)

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-11 Thread dding3
Github user dding3 commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r100680651 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala --- @@ -123,16 +127,25 @@ object Pregel extends Logging { s" but got

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-11 Thread dding3
Github user dding3 commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r100680648 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala --- @@ -123,16 +127,25 @@ object Pregel extends Logging { s" but got

[GitHub] spark pull request #16896: [SPARK-19561][Python] cast TimestampType.toIntern...

2017-02-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16896#discussion_r100680623 --- Diff: python/pyspark/sql/tests.py --- @@ -1435,6 +1435,12 @@ def test_time_with_timezone(self): self.assertEqual(now, now1)

[GitHub] spark issue #16896: [SPARK-19561][Python] cast TimestampType.toInternal outp...

2017-02-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16896 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16892: [SPARK-19560] Improve DAGScheduler tests.

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16892 **[Test build #72744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72744/testReport)** for PR 16892 at commit

[GitHub] spark issue #16892: [SPARK-19560] Improve DAGScheduler tests.

2017-02-11 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16892 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16894: [SPARK-17897] [SQL] [BACKPORT-2.0] Fixed IsNotNull Const...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72743/ Test PASSed. ---

[GitHub] spark issue #16894: [SPARK-17897] [SQL] [BACKPORT-2.0] Fixed IsNotNull Const...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16894 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16894: [SPARK-17897] [SQL] [BACKPORT-2.0] Fixed IsNotNull Const...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16894 **[Test build #72743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72743/consoleFull)** for PR 16894 at commit

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100679282 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala --- @@ -0,0 +1,314 @@ +/*

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r100679242 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,623 @@ +/*

[GitHub] spark issue #16724: [SPARK-19352][SQL] Keep sort order of rows after externa...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16724 If we admit that preserving the sort order is not guaranteed by the API, then the change in this PR is not reasonable, as it has performance penalty. --- If your project is set up for it, you

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r100679183 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1764,4 +1769,117 @@ class JsonSuite extends

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r100679101 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -394,36 +447,32 @@ class JacksonParser( }

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r100678935 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -48,69 +47,110 @@ class JacksonParser(

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16777#discussion_r100678810 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala --- @@ -379,6 +380,75 @@ class TypeCoercionSuite

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16777#discussion_r100678782 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala --- @@ -379,6 +380,75 @@ class TypeCoercionSuite

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16777#discussion_r100678739 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala --- @@ -379,6 +380,75 @@ class TypeCoercionSuite

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100678180 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSQLViewSuite.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100678170 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala --- @@ -452,311 +542,96 @@ class SQLViewSuite extends QueryTest with

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100678107 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -617,13 +617,17 @@ class Analyzer( private

[GitHub] spark issue #16894: [SPARK-17897] [SQL] [BACKPORT-2.0] Fixed IsNotNull Const...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16894 **[Test build #72743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72743/consoleFull)** for PR 16894 at commit

[GitHub] spark issue #16894: [SPARK-17897] [SQL] [BACKPORT-2.0] Fixed IsNotNull Const...

2017-02-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16894 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16896: [SPARK-19561][Python] cast TimestampType.toInternal outp...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16896 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16896: [SPARK-19561][Python] cast TimestampType.toIntern...

2017-02-11 Thread JasonMWhite
GitHub user JasonMWhite opened a pull request: https://github.com/apache/spark/pull/16896 [SPARK-19561][Python] cast TimestampType.toInternal output to long ## What changes were proposed in this pull request? Cast the output of `TimestampType.toInternal` to long to allow

[GitHub] spark issue #16697: [SPARK-19358][CORE] LiveListenerBus shall log the event ...

2017-02-11 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/16697 @mridulm what I thought is that logging the events which are sampled from the timeline may not provide meaningful info to the user the current implementation will clean the events queue

[GitHub] spark issue #16674: [SPARK-19331][SQL][TESTS] Improve the test coverage of S...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16674 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16674: [SPARK-19331][SQL][TESTS] Improve the test coverage of S...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16674 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72742/ Test FAILed. ---

[GitHub] spark issue #16674: [SPARK-19331][SQL][TESTS] Improve the test coverage of S...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16674 **[Test build #72742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72742/testReport)** for PR 16674 at commit

[GitHub] spark issue #16895: [SPARK-15615][SQL] Add an API to load DataFrame from Dat...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16895 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16854: [SPARK-15463][SQL] Add an API to load DataFrame f...

2017-02-11 Thread pjfanning
Github user pjfanning commented on a diff in the pull request: https://github.com/apache/spark/pull/16854#discussion_r100674503 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -361,6 +362,41 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark pull request #16895: [SPARK-15615][SQL] Add an API to load DataFrame f...

2017-02-11 Thread pjfanning
GitHub user pjfanning opened a pull request: https://github.com/apache/spark/pull/16895 [SPARK-15615][SQL] Add an API to load DataFrame from Dataset[String] storing JSON ## What changes were proposed in this pull request? SPARK-15615 proposes replacing the

[GitHub] spark issue #16674: [SPARK-19331][SQL][TESTS] Improve the test coverage of S...

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16674 **[Test build #72742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72742/testReport)** for PR 16674 at commit

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-11 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/16620 Thanks for all the investigation and the write up, @kayousterhout This makes good sense to me, and should take us a long way toward both fixing the immediate bug and improving the code. We

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100672477 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala --- @@ -398,26 +472,6 @@ class SQLViewSuite extends QueryTest with

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100672397 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -624,6 +624,9 @@ class Analyzer( } catch

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100671808 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSQLViewSuite.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the

[GitHub] spark pull request #16871: [SPARK-19550][BUILD][CORE][WIP] Remove Java 7 sup...

2017-02-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16871#discussion_r100671366 --- Diff: project/SparkBuild.scala --- @@ -765,16 +750,6 @@ object CopyDependencies { } -object Java8TestSettings { - import

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16777 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72741/ Test PASSed. ---

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16777 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16777 **[Test build #72741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72741/testReport)** for PR 16777 at commit

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16777 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16777 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72740/ Test PASSed. ---

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16777 **[Test build #72740 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72740/testReport)** for PR 16777 at commit

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100670964 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala --- @@ -452,311 +542,96 @@ class SQLViewSuite extends QueryTest

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100670864 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala --- @@ -452,311 +542,96 @@ class SQLViewSuite extends QueryTest

[GitHub] spark pull request #16854: [SPARK-15463][SQL] Add an API to load DataFrame f...

2017-02-11 Thread pjfanning
Github user pjfanning commented on a diff in the pull request: https://github.com/apache/spark/pull/16854#discussion_r100670326 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -361,6 +362,41 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #16824: [SPARK-18069][PYTHON] Make PySpark doctests for SQL self...

2017-02-11 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16824 Just back from Spark Summit East, I'll try and take a look soon :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15945 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72739/ Test PASSed. ---

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-02-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15945 **[Test build #72739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72739/testReport)** for PR 15945 at commit

[GitHub] spark issue #16882: [SPARK-19544][SQL] Improve error message when some colum...

2017-02-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16882 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72737/ Test PASSed. ---

<    1   2   3   >