[GitHub] spark pull request #17062: [SPARK-17495] [SQL] Support date, timestamp and i...

2017-03-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17062#discussion_r104282564 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HashExpressionsSuite.scala --- @@ -169,6 +171,96 @@ class

[GitHub] spark issue #17125: [SPARK-19211][SQL] Explicitly prevent Insert into View o...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17125 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73894/ Test PASSed. ---

[GitHub] spark issue #17125: [SPARK-19211][SQL] Explicitly prevent Insert into View o...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17125 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17125: [SPARK-19211][SQL] Explicitly prevent Insert into View o...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17125 **[Test build #73894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73894/testReport)** for PR 17125 at commit

[GitHub] spark pull request #16842: [SPARK-19304] [Streaming] [Kinesis] fix kinesis s...

2017-03-03 Thread Gauravshah
Github user Gauravshah commented on a diff in the pull request: https://github.com/apache/spark/pull/16842#discussion_r104282408 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala --- @@ -193,9 +201,10 @@ class

[GitHub] spark pull request #17138: [SPARK-17080] [SQL] join reorder

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17138#discussion_r104281655 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -288,6 +288,9 @@ case class Join(

[GitHub] spark pull request #17160: [SPARK-19701][SQL][PYTHON] Throws a correct excep...

2017-03-03 Thread binhnguyen1512
Github user binhnguyen1512 commented on a diff in the pull request: https://github.com/apache/spark/pull/17160#discussion_r104281624 --- Diff: python/pyspark/sql/column.py --- @@ -180,7 +180,9 @@ def __init__(self, jc): __ror__ = _bin_op("or") # container

[GitHub] spark pull request #17160: [SPARK-19701][SQL][PYTHON] Throws a correct excep...

2017-03-03 Thread binhnguyen1512
Github user binhnguyen1512 commented on a diff in the pull request: https://github.com/apache/spark/pull/17160#discussion_r104281627 --- Diff: python/pyspark/sql/column.py --- @@ -180,7 +180,9 @@ def __init__(self, jc): __ror__ = _bin_op("or") # container

[GitHub] spark pull request #17138: [SPARK-17080] [SQL] join reorder

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17138#discussion_r104281605 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystConf.scala --- @@ -60,6 +60,12 @@ trait CatalystConf { * Enables CBO for

[GitHub] spark pull request #17138: [SPARK-17080] [SQL] join reorder

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17138#discussion_r104281601 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystConf.scala --- @@ -60,6 +60,12 @@ trait CatalystConf { * Enables CBO for

[GitHub] spark pull request #17138: [SPARK-17080] [SQL] join reorder

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17138#discussion_r104281554 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -0,0 +1,274 @@ +/* + * Licensed to

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r104281417 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -773,14 +773,20 @@ case class

[GitHub] spark pull request #17125: [SPARK-19211][SQL] Explicitly prevent Insert into...

2017-03-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17125#discussion_r104281325 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -604,7 +604,7 @@ class Analyzer( def

[GitHub] spark pull request #17125: [SPARK-19211][SQL] Explicitly prevent Insert into...

2017-03-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17125#discussion_r104281300 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1329,6 +1329,15 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73893/ Test PASSed. ---

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-03-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r104281282 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -773,14 +773,20 @@ case class

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73893/testReport)** for PR 16929 at commit

[GitHub] spark pull request #17148: [SPARK-17075][SQL][followup] fix filter estimatio...

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17148#discussion_r104281195 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala --- @@ -157,7 +157,7 @@ class

[GitHub] spark pull request #17148: [SPARK-17075][SQL][followup] fix filter estimatio...

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17148#discussion_r104281194 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -249,17 +251,19 @@ case

[GitHub] spark pull request #17052: [SPARK-19690][SS] Join a streaming DataFrame with...

2017-03-03 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17052#discussion_r104281176 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -31,6 +31,8 @@ abstract class LogicalPlan

[GitHub] spark pull request #17148: [SPARK-17075][SQL][followup] fix filter estimatio...

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17148#discussion_r104280865 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -469,22 +487,28 @@ case

[GitHub] spark pull request #17148: [SPARK-17075][SQL][followup] fix filter estimatio...

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17148#discussion_r104280853 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -414,53 +422,63 @@ case

[GitHub] spark pull request #17148: [SPARK-17075][SQL][followup] fix filter estimatio...

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17148#discussion_r104280815 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -323,7 +331,7 @@ case class

[GitHub] spark pull request #11601: [SPARK-13568] [ML] Create feature transformer to ...

2017-03-03 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/11601#discussion_r104280679 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/ImputerSuite.scala --- @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r104280684 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/StatsConfSuite.scala --- @@ -1,64 +0,0 @@ -/* - * Licensed to

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73892/ Test PASSed. ---

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73892 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73892/testReport)** for PR 16981 at commit

[GitHub] spark issue #17152: [SPARK-18389][SQL] Disallow cyclic view reference

2017-03-03 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/17152 @gatorsmile Currently we don't perform recursive resolution over a temporary view, so perhaps that won't trigger a cyclic view reference. For example: ``` scala> spark.sql("CREATE

[GitHub] spark pull request #16696: [SPARK-19350] [SQL] Cardinality estimation of Lim...

2017-03-03 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/16696#discussion_r104280554 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -773,14 +773,20 @@ case class

[GitHub] spark pull request #17125: [SPARK-19211][SQL] Explicitly prevent Insert into...

2017-03-03 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/17125#discussion_r104280528 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -604,7 +604,7 @@ class Analyzer( def

[GitHub] spark issue #17160: [SPARK-19701][SQL][PYTHON] Throws a correct exception fo...

2017-03-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17160 what if we just remove `__contains__`, `__nonzero__ ` and `__bool__`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17125: [SPARK-19211][SQL] Explicitly prevent Insert into...

2017-03-03 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/17125#discussion_r104280480 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1329,6 +1329,15 @@ class SparkSqlAstBuilder(conf:

[GitHub] spark issue #17125: [SPARK-19211][SQL] Explicitly prevent Insert into View o...

2017-03-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17125 LGTM except 2 questions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17125: [SPARK-19211][SQL] Explicitly prevent Insert into...

2017-03-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17125#discussion_r104280434 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -604,7 +604,7 @@ class Analyzer( def

[GitHub] spark pull request #17125: [SPARK-19211][SQL] Explicitly prevent Insert into...

2017-03-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17125#discussion_r104280410 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1329,6 +1329,15 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #17125: [SPARK-19211][SQL] Explicitly prevent Insert into View o...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17125 **[Test build #73894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73894/testReport)** for PR 17125 at commit

[GitHub] spark issue #17155: [SPARK-19815][SQL] Not orderable should be applied to ri...

2017-03-03 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/17155 @gatorsmile Thanks for reviewing this. I am thinking the logic again. On the surface, the logic may be correct. Since in the join, the left and right key should be the same type. Will close the PR.

[GitHub] spark pull request #17155: [SPARK-19815][SQL] Not orderable should be applie...

2017-03-03 Thread zhzhan
Github user zhzhan closed the pull request at: https://github.com/apache/spark/pull/17155 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17157: [SPARK-16845][SQL][BRANCH-2.0] `GeneratedClass$SpecificO...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17157 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73887/ Test PASSed. ---

[GitHub] spark issue #17157: [SPARK-16845][SQL][BRANCH-2.0] `GeneratedClass$SpecificO...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17157 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17157: [SPARK-16845][SQL][BRANCH-2.0] `GeneratedClass$SpecificO...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17157 **[Test build #73887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73887/consoleFull)** for PR 17157 at commit

[GitHub] spark issue #17158: [SPARK-16845][SQL][BRANCH-1.6] `GeneratedClass$SpecificO...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17158 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73890/ Test PASSed. ---

[GitHub] spark issue #17158: [SPARK-16845][SQL][BRANCH-1.6] `GeneratedClass$SpecificO...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17158 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17158: [SPARK-16845][SQL][BRANCH-1.6] `GeneratedClass$SpecificO...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17158 **[Test build #73890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73890/consoleFull)** for PR 17158 at commit

[GitHub] spark issue #16854: [WIP][SPARK-15463][SQL] Add an API to load DataFrame fro...

2017-03-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16854 Oh, no. It does not need to. I just meant to de-duplicate some logics by https://github.com/apache/spark/pull/16854#discussion_r103894945. Let me just remove that part and leave only

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16929 **[Test build #73893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73893/testReport)** for PR 16929 at commit

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17120 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73885/ Test PASSed. ---

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17120 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17120 **[Test build #73885 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73885/testReport)** for PR 17120 at commit

[GitHub] spark issue #17160: [SPARK-19701][SQL][PYTHON] Throws a correct exception fo...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17160 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17160: [SPARK-19701][SQL][PYTHON] Throws a correct exception fo...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17160 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73891/ Test PASSed. ---

[GitHub] spark issue #17160: [SPARK-19701][SQL][PYTHON] Throws a correct exception fo...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17160 **[Test build #73891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73891/testReport)** for PR 17160 at commit

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17120 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73884/ Test PASSed. ---

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17120 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17120 **[Test build #73884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73884/testReport)** for PR 17120 at commit

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15666 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15666 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73882/ Test PASSed. ---

[GitHub] spark issue #15666: [SPARK-11421] [Core][Python][R] Added ability for addJar...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15666 **[Test build #73882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73882/testReport)** for PR 15666 at commit

[GitHub] spark issue #17155: [SPARK-19815][SQL] Not orderable should be applied to ri...

2017-03-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17155 Add a test case to https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala? --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-03-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16929 Thank you so much. Let me clean up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17160: [SPARK-19701][SQL][PYTHON] Throws a correct excep...

2017-03-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17160#discussion_r104278452 --- Diff: python/pyspark/sql/column.py --- @@ -180,7 +180,9 @@ def __init__(self, jc): __ror__ = _bin_op("or") # container

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistry

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73892/testReport)** for PR 16981 at commit

[GitHub] spark issue #17160: [SPARK-19701][SQL][PYTHON] Throws a correct exception fo...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17160 **[Test build #73891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73891/testReport)** for PR 17160 at commit

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistr...

2017-03-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r104278366 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -566,3 +586,17 @@ case class StructToJson(

[GitHub] spark issue #17160: [SPARK-19701][SQL][PYTHON] Throws a correct exception fo...

2017-03-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17160 cc @cloud-fan, @davies and @holdenk. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17160: [SPARK-19701][SQL][PYTHON] Throws a correct excep...

2017-03-03 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/17160 [SPARK-19701][SQL][PYTHON] Throws a correct exception for 'in' operator against column ## What changes were proposed in this pull request? This PR proposes to remove incorrect

[GitHub] spark issue #17133: [SPARK-19793] Use clock.getTimeMillis when mark task as ...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17133 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73881/ Test PASSed. ---

[GitHub] spark issue #17133: [SPARK-19793] Use clock.getTimeMillis when mark task as ...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17133 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17133: [SPARK-19793] Use clock.getTimeMillis when mark task as ...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17133 **[Test build #73881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73881/testReport)** for PR 17133 at commit

[GitHub] spark issue #16929: [SPARK-19595][SQL] Support json array in from_json

2017-03-03 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16929 @HyukjinKwon Implementation seems fine. Just left a cosmetic comment on your unit tests. Otherwise LGTM --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #16929: [SPARK-19595][SQL] Support json array in from_jso...

2017-03-03 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/16929#discussion_r104278223 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala --- @@ -372,6 +372,58 @@ class

[GitHub] spark pull request #16929: [SPARK-19595][SQL] Support json array in from_jso...

2017-03-03 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/16929#discussion_r104278176 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala --- @@ -372,6 +372,58 @@ class

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-03 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16867 @squito Yes, some of machine learning jobs which do cartesian product in my cluster have over than 100k tasks in the `TaskSetManager`. --- If your project is set up for it, you can reply

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17159 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73888/ Test FAILed. ---

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17159 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17159 **[Test build #73888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73888/testReport)** for PR 17159 at commit

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistr...

2017-03-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r104277851 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -566,3 +586,17 @@ case class StructToJson(

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistr...

2017-03-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r104277780 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -566,3 +586,17 @@ case class StructToJson(

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add to_json in FunctionRegistr...

2017-03-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r104277787 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -566,3 +586,17 @@ case class StructToJson(

[GitHub] spark issue #17158: [SPARK-16845][SQL][BRANCH-1.6] `GeneratedClass$SpecificO...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17158 **[Test build #73890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73890/consoleFull)** for PR 17158 at commit

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-03 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16867 @kayousterhout @squito Thanks a lot for your comments, really helpful :) I really think median heap is a good idea. `slice` is `O(n)` and is not most efficient. I'm doing

[GitHub] spark issue #17158: [SPARK-16845][SQL][BRANCH-1.6] `GeneratedClass$SpecificO...

2017-03-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17158 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17156: [SPARK-19816][SQL][Tests] Fix an issue that DataFrameCal...

2017-03-03 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/17156 merged to branch-2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17156: [SPARK-19816][SQL][Tests] Fix an issue that DataFrameCal...

2017-03-03 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/17156 Let's also merge this to branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17112: [WIP] Measurement for SPARK-16929.

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73889/ Test FAILed. ---

[GitHub] spark issue #17112: [WIP] Measurement for SPARK-16929.

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17112 **[Test build #73889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73889/testReport)** for PR 17112 at commit

[GitHub] spark issue #17112: [WIP] Measurement for SPARK-16929.

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17112 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17156: [SPARK-19816][SQL][Tests] Fix an issue that DataF...

2017-03-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17156 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17159 **[Test build #73888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73888/testReport)** for PR 17159 at commit

[GitHub] spark issue #17112: [WIP] Measurement for SPARK-16929.

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17112 **[Test build #73889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73889/testReport)** for PR 17112 at commit

[GitHub] spark issue #17159: [SPARK-19818][SparkR] union should check for name consis...

2017-03-03 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/17159 The current implementation accepts data frames with different schemas. See issues below: ``` df <- createDataFrame(data.frame(name = c("Michael", "Andy", "Justin"), age = c(1, 30,

[GitHub] spark pull request #17159: [SPARK-19818][SparkR] union should check for name...

2017-03-03 Thread actuaryzhang
GitHub user actuaryzhang opened a pull request: https://github.com/apache/spark/pull/17159 [SPARK-19818][SparkR] union should check for name consistency of input data frames ## What changes were proposed in this pull request? Added checks for name consistency of input data

[GitHub] spark issue #17156: [SPARK-19816][SQL][Tests] Fix an issue that DataFrameCal...

2017-03-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17156 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17156: [SPARK-19816][SQL][Tests] Fix an issue that DataFrameCal...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17156 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73879/ Test PASSed. ---

[GitHub] spark issue #17156: [SPARK-19816][SQL][Tests] Fix an issue that DataFrameCal...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17156 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17156: [SPARK-19816][SQL][Tests] Fix an issue that DataFrameCal...

2017-03-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17156 **[Test build #73879 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73879/testReport)** for PR 17156 at commit

[GitHub] spark issue #17158: [SPARK-16845][SQL][BRANCH-1.6] `GeneratedClass$SpecificO...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17158 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73886/ Test FAILed. ---

[GitHub] spark issue #17158: [SPARK-16845][SQL][BRANCH-1.6] `GeneratedClass$SpecificO...

2017-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17158 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   3   4   5   6   7   >