[GitHub] spark issue #17738: [SPARK-20422][Spark Core] Worker registration retries sh...

2017-05-18 Thread navneetrastogi
Github user navneetrastogi commented on the issue: https://github.com/apache/spark/pull/17738 This is a minor change which will help in configuring registration retries. It can be merged. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #17999: [SPARK-20751][SQL] Add built-in SQL Function - CO...

2017-05-18 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17999#discussion_r117278957 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1991,6 +1991,22 @@ object functions { def tan(columnName: String):

[GitHub] spark issue #13959: [SPARK-14351] [MLlib] [ML] Optimize findBestSplits metho...

2017-05-18 Thread MechCoder
Github user MechCoder commented on the issue: https://github.com/apache/spark/pull/13959 I don't understand. If you don't have time to review that is fine (I've been there too), but there is no need to close a PR due to unavailability of comitters. One of the reasons, that I

[GitHub] spark pull request #17899: [SPARK-20636] Add new optimization rule to flip a...

2017-05-18 Thread ptkool
Github user ptkool commented on a diff in the pull request: https://github.com/apache/spark/pull/17899#discussion_r117238414 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -609,6 +610,19 @@ object CollapseWindow extends

[GitHub] spark pull request #18029: [SPARK-20168][WIP][DStream] Add changes to use ki...

2017-05-18 Thread yssharma
GitHub user yssharma opened a pull request: https://github.com/apache/spark/pull/18029 [SPARK-20168][WIP][DStream] Add changes to use kinesis fetches from specified timestamp ## What changes were proposed in this pull request? Kinesis client can resume from a specified

[GitHub] spark issue #18012: [SPARK-20779][Examples]The ASF header placed in an incor...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18012 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18027: [SPARK-20796] the location of start-master.sh in spark-s...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18027 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17899: [SPARK-20636] Add new optimization rule to flip a...

2017-05-18 Thread ptkool
Github user ptkool commented on a diff in the pull request: https://github.com/apache/spark/pull/17899#discussion_r117240910 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/TransposeWindowSuite.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117250008 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class

[GitHub] spark issue #17996: [SPARK-20506][DOCS] 2.2 migration guide

2017-05-18 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/17996 @MLnick We have separate SparkR guide at http://spark.apache.org/docs/latest/sparkr.html . Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #18022: [SPARK-20790] [MLlib] Correctly handle negative v...

2017-05-18 Thread davideis
Github user davideis commented on a diff in the pull request: https://github.com/apache/spark/pull/18022#discussion_r117260197 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -78,7 +79,7 @@ class ALSSuite val k = 2 val ne0

[GitHub] spark pull request #18022: [SPARK-20790] [MLlib] Correctly handle negative v...

2017-05-18 Thread davideis
Github user davideis commented on a diff in the pull request: https://github.com/apache/spark/pull/18022#discussion_r117264333 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -795,8 +799,8 @@ object ALS extends DefaultParamsReadable[ALS] with

[GitHub] spark pull request #18022: [SPARK-20790] [MLlib] Correctly handle negative v...

2017-05-18 Thread davideis
Github user davideis commented on a diff in the pull request: https://github.com/apache/spark/pull/18022#discussion_r117262532 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -763,11 +763,15 @@ object ALS extends DefaultParamsReadable[ALS] with

[GitHub] spark pull request #18022: [SPARK-20790] [MLlib] Correctly handle negative v...

2017-05-18 Thread davideis
Github user davideis commented on a diff in the pull request: https://github.com/apache/spark/pull/18022#discussion_r117261995 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -348,6 +349,37 @@ class ALSSuite } /** +

[GitHub] spark pull request #18022: [SPARK-20790] [MLlib] Correctly handle negative v...

2017-05-18 Thread davideis
Github user davideis commented on a diff in the pull request: https://github.com/apache/spark/pull/18022#discussion_r117265969 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -1624,15 +1628,15 @@ object ALS extends DefaultParamsReadable[ALS] with

[GitHub] spark issue #18027: [SPARK-20796] the location of start-master.sh in spark-s...

2017-05-18 Thread liu-zhaokun
Github user liu-zhaokun commented on the issue: https://github.com/apache/spark/pull/18027 @srowen I will try my best to slove this type of problem.But I only found one this time.Please help me to merge it.Thanks. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18029 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18029 **[Test build #77043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77043/testReport)** for PR 18029 at commit

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18029 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77043/ Test FAILed. ---

[GitHub] spark pull request #17899: [SPARK-20636] Add new optimization rule to flip a...

2017-05-18 Thread ptkool
Github user ptkool commented on a diff in the pull request: https://github.com/apache/spark/pull/17899#discussion_r117240492 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala --- @@ -423,4 +423,25 @@ class DataFrameWindowFunctionsSuite

[GitHub] spark issue #18030: [SPARK-20798] GenerateUnsafeProjection should check if a...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18030 LGTM - pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18030: [SPARK-20798] GenerateUnsafeProjection should check if a...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18030 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #17996: [SPARK-20506][DOCS] 2.2 migration guide

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17996 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17996: [SPARK-20506][DOCS] 2.2 migration guide

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17996 **[Test build #77052 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77052/testReport)** for PR 17996 at commit

[GitHub] spark issue #17996: [SPARK-20506][DOCS] 2.2 migration guide

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17996 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77052/ Test PASSed. ---

[GitHub] spark pull request #18030: [SPARK-20798] GenerateUnsafeProjection should che...

2017-05-18 Thread ala
GitHub user ala opened a pull request: https://github.com/apache/spark/pull/18030 [SPARK-20798] GenerateUnsafeProjection should check if a value is null before calling the getter ## What changes were proposed in this pull request?

[GitHub] spark issue #18030: [SPARK-20798] GenerateUnsafeProjection should check if a...

2017-05-18 Thread ala
Github user ala commented on the issue: https://github.com/apache/spark/pull/18030 @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17400: [SPARK-19981][SQL] Update output partitioning info. when...

2017-05-18 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17400 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18012: [SPARK-20779][Examples]The ASF header placed in an incor...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18012 **[Test build #77049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77049/testReport)** for PR 18012 at commit

[GitHub] spark issue #18000: [SPARK-20364][SQL] Disable Parquet predicate pushdown fo...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18000 **[Test build #77051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77051/testReport)** for PR 18000 at commit

[GitHub] spark issue #17996: [SPARK-20506][DOCS] 2.2 migration guide

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17996 **[Test build #77052 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77052/testReport)** for PR 17996 at commit

[GitHub] spark issue #18025: [WIP][SparkR] Update doc and examples for sql functions

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18025 **[Test build #77046 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77046/testReport)** for PR 18025 at commit

[GitHub] spark issue #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A]R.

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18019 **[Test build #77048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77048/testReport)** for PR 18019 at commit

[GitHub] spark issue #18028: [DStream][DOC]Add documentation for kinesis retry config...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18028 **[Test build #77044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77044/testReport)** for PR 18028 at commit

[GitHub] spark issue #18026: [SPARK-16202][SQL][DOC] Follow-up to Correct The Descrip...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18026 **[Test build #77045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77045/testReport)** for PR 18026 at commit

[GitHub] spark issue #18025: [WIP][SparkR] Update doc and examples for sql functions

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18025 **[Test build #77046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77046/testReport)** for PR 18025 at commit

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18029 **[Test build #77043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77043/testReport)** for PR 18029 at commit

[GitHub] spark issue #18030: [SPARK-20798] GenerateUnsafeProjection should check if a...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18030 **[Test build #77042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77042/testReport)** for PR 18030 at commit

[GitHub] spark issue #18024: [SPARK-20792][SS] Support same timeout operations in map...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18024 **[Test build #77047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77047/testReport)** for PR 18024 at commit

[GitHub] spark issue #18002: [SPARK-20770][SQL] Improve ColumnStats

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18002 **[Test build #77050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77050/testReport)** for PR 18002 at commit

[GitHub] spark issue #17400: [SPARK-19981][SQL] Update output partitioning info. when...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17400 **[Test build #77053 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77053/testReport)** for PR 17400 at commit

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12646 **[Test build #77055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77055/testReport)** for PR 12646 at commit

[GitHub] spark issue #13959: [SPARK-14351] [MLlib] [ML] Optimize findBestSplits metho...

2017-05-18 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/13959 The lack of bandwidth in MLlib means that sometimes good code that would make an impact just gets ignored. This is kind of the reality of things. However, if we are going to close the PR simply

[GitHub] spark issue #18012: [SPARK-20779][Examples]The ASF header placed in an incor...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77049/ Test PASSed. ---

[GitHub] spark pull request #17899: [SPARK-20636] Add new optimization rule to flip a...

2017-05-18 Thread ptkool
Github user ptkool commented on a diff in the pull request: https://github.com/apache/spark/pull/17899#discussion_r117241321 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/TransposeWindowSuite.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed to

[GitHub] spark issue #17997: [SPARK-20763][SQL]The function of `month` and `day` retu...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17997 The current impl is still missing the special handling of that miracle period. See the outputs of Oracle: ``` Result Set 34 TO_CHAR(TO_DATE('1582-10-04','-MM-DD'),'DDD') 277

[GitHub] spark pull request #18028: [DStream][DOC]Add documentation for kinesis retry...

2017-05-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18028 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2017-05-18 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/16478 The JIRA has 37 watchers (ranked 17 across all Spark JIRAs) and 22 votes (ranked 8 overall across all JIRAs). I'd have thought that's some indication of user interest. --- If your project is set

[GitHub] spark issue #18011: [SPARK-19089][SQL] Add support for nested sequences

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18011 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18011: [SPARK-19089][SQL] Add support for nested sequences

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18011 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77060/ Test FAILed. ---

[GitHub] spark issue #18023: Fix SPARK-12139: REGEX Column Specification for Hive Que...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18023 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18023: Fix SPARK-12139: REGEX Column Specification for Hive Que...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18023 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77062/ Test FAILed. ---

[GitHub] spark issue #18023: Fix SPARK-12139: REGEX Column Specification for Hive Que...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #77062 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77062/testReport)** for PR 18023 at commit

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18023 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77063/ Test FAILed. ---

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18023 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17723: [SPARK-20434][YARN][CORE] Move kerberos delegation token...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17723 **[Test build #77064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77064/testReport)** for PR 17723 at commit

[GitHub] spark issue #18028: [DStream][DOC]Add documentation for kinesis retry config...

2017-05-18 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/18028 merging to master and branch-2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18011: [SPARK-19089][SQL] Add support for nested sequences

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18011 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18016 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18023: Fix SPARK-12139: REGEX Column Specification for Hive Que...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #77062 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77062/testReport)** for PR 18023 at commit

[GitHub] spark issue #18023: Fix SPARK-12139: REGEX Column Specification for Hive Que...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18023 Please update the PR title to `[SPARK-12139] [SQL] REGEX Column Specification` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #77063 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77063/testReport)** for PR 18023 at commit

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

2017-05-18 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r117340556 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopAccessManager.scala --- @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

2017-05-18 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r117341550 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/ConfigurableCredentialManager.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to the

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

2017-05-18 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r117343524 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManager.scala --- @@ -0,0 +1,77 @@

[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17770#discussion_r117343489 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -187,6 +187,9 @@ class Dataset[T] private[sql]( } }

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

2017-05-18 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r117344321 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/YARNHadoopAccessManager.scala --- @@ -21,66 +21,47 @@ import

[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17770#discussion_r117344138 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1545,7 +1551,9 @@ class Analyzer( *

[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17770#discussion_r117344241 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1605,6 +1613,8 @@ class Analyzer(

[GitHub] spark issue #17673: [SPARK-20372] [ML] Word2Vec Continuous Bag of Words mode...

2017-05-18 Thread shubhamchopra
Github user shubhamchopra commented on the issue: https://github.com/apache/spark/pull/17673 Code-review comments/suggestions so far have been incorporated. Thanks for looking into the code. Happy to incorporate more suggestions and feedback. --- If your project is set up for it,

[GitHub] spark issue #18028: [DStream][DOC]Add documentation for kinesis retry config...

2017-05-18 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/18028 LGTM. I have no idea where it got dropped :@ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18023: Fix SPARK-12139: REGEX Column Specification for H...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117326089 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1229,25 +1228,56 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2017-05-18 Thread lossyrob
Github user lossyrob commented on the issue: https://github.com/apache/spark/pull/16478 Using geospatial data types with Spark SQL require UDTs, and solutions are hacking around the fact that they are private to the package. e.g.: -

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

2017-05-18 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r117343724 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/YARNConfigurableCredentialManager.scala --- @@ -0,0 +1,77 @@

[GitHub] spark issue #17723: [SPARK-20434][YARN][CORE] Move kerberos delegation token...

2017-05-18 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/17723 I have: * Made ServiceCredentialProvider private * Made ConfigurableCredentialManager load a hardcoded list of (HDFS, Hive, HBase) providers, rather than service loading * Renamed

[GitHub] spark issue #17698: [SPARK-20403][SQL]Modify the instructions of some functi...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17698 LGTM except one comment: Please add the test cases to the end of file `cast.sql` ``` DESC FUNCTION boolean; DESC FUNCTION EXTENDED boolean; ``` You can generate

[GitHub] spark issue #18023: Fix SPARK-12139: REGEX Column Specification for Hive Que...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18023 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #18023: Fix SPARK-12139: REGEX Column Specification for H...

2017-05-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117326828 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1229,25 +1228,56 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-05-18 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/17680 @gatorsmile, sorry for not responding, I was on vacation for a few days. Should I still review this even though it is merged? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move kerberos delegatio...

2017-05-18 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r117342796 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/ServiceCredentialProvider.scala --- @@ -0,0 +1,59 @@ +/* + * Licensed to the

[GitHub] spark issue #17094: [SPARK-19762][ML] Hierarchy for consolidating ML aggrega...

2017-05-18 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17094 Thanks @MLnick! I am happy to discuss splitting this into smaller bits as well, if it can make things easier. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18016 **[Test build #77061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77061/testReport)** for PR 18016 at commit

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18016 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77061/ Test FAILed. ---

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18016 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18016 **[Test build #77061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77061/testReport)** for PR 18016 at commit

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #77063 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77063/testReport)** for PR 18023 at commit

[GitHub] spark issue #17723: [SPARK-20434][YARN][CORE] Move kerberos delegation token...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17723 **[Test build #77065 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77065/testReport)** for PR 17723 at commit

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117374353 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1229,25 +1228,56 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #77068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77068/testReport)** for PR 18023 at commit

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117367232 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -84,6 +84,33 @@ case class

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r117379878 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1230,24 +1230,49 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18031: Record accurate size of blocks in MapStatus when ...

2017-05-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18031#discussion_r117385321 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -121,48 +126,69 @@ private[spark] class CompressedMapStatus( }

[GitHub] spark issue #18031: Record accurate size of blocks in MapStatus when it's ab...

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18031 **[Test build #77072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77072/testReport)** for PR 18031 at commit

[GitHub] spark issue #17985: Add "full_outer" name to join types

2017-05-18 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/17985 @BartekH Yes, I think we can add that to exception message. Please also add a test case for checking supported join types. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17999 **[Test build #77067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77067/testReport)** for PR 17999 at commit

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17999 **[Test build #77067 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77067/testReport)** for PR 17999 at commit

[GitHub] spark issue #13959: [SPARK-14351] [MLlib] [ML] Optimize findBestSplits metho...

2017-05-18 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/13959 Yes, this is a tough issue. Let's wait and see if @jkbradley has thoughts on this issue. If we don't hear anything, then I'd leave it up to @MechCoder on whether to reopen. Thanks, btw, for taking

[GitHub] spark issue #13959: [SPARK-14351] [MLlib] [ML] Optimize findBestSplits metho...

2017-05-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13959 @MechCoder, I apologize that, probably, it sounds the reason for my suggestion was not clear initially and if it looks without a respect. --- If your project is set up for it, you can reply to

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117378356 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1268,6 +1268,59 @@ case class

[GitHub] spark pull request #18019: [SPARK-20748][SQL] Add built-in SQL function CH[A...

2017-05-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18019#discussion_r117378811 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala --- @@ -117,6 +117,41 @@ class StringFunctionsSuite extends QueryTest

<    1   2   3   4   5   >