[GitHub] spark pull request #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit opti...

2017-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15119#discussion_r95261483 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -283,8 +284,17 @@ object SparkSubmit extends CommandLineUtils { }

[GitHub] spark issue #16464: [SPARK-19066][SparkR]:SparkR LDA doesn't set optimizer c...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71093/ Test PASSed. ---

[GitHub] spark pull request #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit opti...

2017-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15119#discussion_r95261994 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -974,23 +967,102 @@ private[spark] object SparkSubmitUtils { }

[GitHub] spark issue #16464: [SPARK-19066][SparkR]:SparkR LDA doesn't set optimizer c...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16464 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16503: [SPARK-18113] canCommit should return same when called b...

2017-01-09 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16503 Good catch. Looks good to me. @vanzin The RPC layer only guarantees at-most-once. Retry may be still helpful in some case, but the receiver should be idempotent. Either the current change

[GitHub] spark issue #16503: [SPARK-18113] canCommit should return same when called b...

2017-01-09 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16503 > That was the case with akka (I think, not really sure), but the netty RPC layer doesn't drop messages. The new one is "exactly once". It doesn't drop but the connection may be broken.

[GitHub] spark issue #16503: [SPARK-18113] canCommit should return same when called b...

2017-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16503 > It doesn't drop but the connection may be broken In which case the executor will die (see `CoarseGrainedExecutorBackend::onDisconnected`). --- If your project is set up for it, you can

[GitHub] spark issue #16518: [BACKPORT][SPARK-18952] Regex strings not properly escap...

2017-01-09 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/16518 Merging to branch-2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #16522: [SPARK-19137][SQL][SS] Garbage left in source tre...

2017-01-09 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/16522 [SPARK-19137][SQL][SS] Garbage left in source tree after SQL tests ran ## What changes were proposed in this pull request? `DataStreamReaderWriterSuite` makes test files in source

[GitHub] spark issue #16523: [SPARK-19142][SparkR]:spark.kmeans should take seed, ini...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16523 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71101/ Test FAILed. ---

[GitHub] spark issue #16523: [SPARK-19142][SparkR]:spark.kmeans should take seed, ini...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16523 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16522: [SPARK-19137][SQL][SS] Garbage left in source tree after...

2017-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16522 Oh, I see. Then, I'll look inside the `temp folder` generation code and fix that. Thank you for review, @vanzin and @zsxwing . --- If your project is set up for it, you can reply to

[GitHub] spark issue #16523: [SPARK-19142][SparkR]:spark.kmeans should take seed, ini...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16523 **[Test build #71101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71101/testReport)** for PR 16523 at commit

[GitHub] spark pull request #16377: [SPARK-18036][ML][MLLIB] Fixing decision trees ha...

2017-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16377#discussion_r95275614 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -176,6 +203,18 @@ class RandomForestSuite extends SparkFunSuite

[GitHub] spark pull request #16377: [SPARK-18036][ML][MLLIB] Fixing decision trees ha...

2017-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16377#discussion_r95182716 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -828,8 +828,27 @@ private[spark] object RandomForest extends Logging {

[GitHub] spark pull request #16377: [SPARK-18036][ML][MLLIB] Fixing decision trees ha...

2017-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16377#discussion_r95183469 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -161,6 +161,33 @@ class RandomForestSuite extends SparkFunSuite

[GitHub] spark issue #16249: [SPARK-18828][SPARKR] Refactor scripts for R

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71098/ Test FAILed. ---

[GitHub] spark issue #16249: [SPARK-18828][SPARKR] Refactor scripts for R

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16520: [SPARK-19140][SS]Allow update mode for non-aggregation s...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16520 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...

2017-01-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95281046 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +344,102 @@ object

[GitHub] spark pull request #16522: [SPARK-19137][SQL] Fix `withSQLConf` to reset `Op...

2017-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16522#discussion_r95281973 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala --- @@ -94,7 +94,13 @@ private[sql] trait SQLTestUtils */

[GitHub] spark issue #15018: [SPARK-17455][MLlib] Improve PAVA implementation in Isot...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15018 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71087/ Test FAILed. ---

[GitHub] spark issue #16521: [SPARK-19139][core] New auth mechanism for transport lib...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16521 **[Test build #71095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71095/testReport)** for PR 16521 at commit

[GitHub] spark issue #16454: [SPARK-19055][SQL][PySpark] Fix SparkSession initializat...

2017-01-09 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/16454 I just posted a fix to this also. I'll close that one in favor of this and add comments here for what it did differently that we should consider. --- If your project is set up for it, you can reply

[GitHub] spark pull request #16514: [SPARK-19128] [SQL] Refresh Cache after Set Locat...

2017-01-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16514#discussion_r95258103 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -754,6 +754,7 @@ case class AlterTableSetLocationCommand(

[GitHub] spark issue #16518: [BACKPORT][SPARK-18952] Regex strings not properly escap...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16518 **[Test build #71090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71090/testReport)** for PR 16518 at commit

[GitHub] spark issue #16518: [BACKPORT][SPARK-18952] Regex strings not properly escap...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16518 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16518: [BACKPORT][SPARK-18952] Regex strings not properly escap...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71090/ Test PASSed. ---

[GitHub] spark pull request #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit opti...

2017-01-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/15119#discussion_r95272071 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -974,23 +967,102 @@ private[spark] object SparkSubmitUtils { }

[GitHub] spark issue #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit option to o...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15119 **[Test build #71100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71100/testReport)** for PR 15119 at commit

[GitHub] spark issue #16522: [SPARK-19137][SQL][SS] Garbage left in source tree after...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16522 **[Test build #71099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71099/testReport)** for PR 16522 at commit

[GitHub] spark issue #16523: [SPARK-19142][SparkR]:spark.kmeans should take seed, ini...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16523 **[Test build #71101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71101/testReport)** for PR 16523 at commit

[GitHub] spark issue #16522: [SPARK-19137][SQL][SS] Garbage left in source tree after...

2017-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16522 Looks ok since this is what other tests here do; but I wonder why this case isn't handled in `StreamingQueryManager.scala`; it seems to either throw an error or create a new temp directory, but not

[GitHub] spark issue #16522: [SPARK-19137][SQL][SS] Garbage left in source tree after...

2017-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16522 I found the root cause. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16521: [SPARK-19139][core] New auth mechanism for transport lib...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16521 **[Test build #71104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71104/testReport)** for PR 16521 at commit

[GitHub] spark issue #16523: [SPARK-19142][SparkR]:spark.kmeans should take seed, ini...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16523 **[Test build #71103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71103/testReport)** for PR 16523 at commit

[GitHub] spark issue #16514: [SPARK-19128] [SQL] Refresh Cache after Set Location

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16514 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16503: [SPARK-18113] canCommit should return same when called b...

2017-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16503 > The RPC layer only guarantees at-most-once That was the case with akka (I think, not really sure), but the netty RPC layer doesn't drop messages. The new one is "exactly once". --- If

[GitHub] spark issue #16464: [SPARK-19066][SparkR]:SparkR LDA doesn't set optimizer c...

2017-01-09 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16464 @felixcheung I made modifications and don't save the two metrics of DistributedModels. Thanks! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16503: [SPARK-18113] canCommit should return same when called b...

2017-01-09 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16503 > In which case the executor will die (see CoarseGrainedExecutorBackend::onDisconnected). Yeah. Didn't recall that. Then I agree that using `ask` is better. --- If your project is set up

[GitHub] spark pull request #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit opti...

2017-01-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/15119#discussion_r95271119 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -283,8 +284,17 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit opti...

2017-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15119#discussion_r95272422 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -974,23 +967,102 @@ private[spark] object SparkSubmitUtils { }

[GitHub] spark issue #16524: [SPARK-19110][MLLIB][FollowUP]: Add a unit test

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16524 **[Test build #71102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71102/testReport)** for PR 16524 at commit

[GitHub] spark issue #16521: [SPARK-19139][core] New auth mechanism for transport lib...

2017-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16521 Hmm, my final cleanup broke some tests, let me fix those... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16522: [SPARK-19137][SQL] Fix `withSQLConf` to reset `OptionalC...

2017-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16522 Hi, @vanzin and @zsxwing . It was a bug of `withSQLConf`. I think this is correct fix, but we need to see the result of whole result because this is test utility issue. --- If your

[GitHub] spark issue #16522: [SPARK-19137][SQL] Fix `withSQLConf` to reset `OptionalC...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16522 **[Test build #71106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71106/testReport)** for PR 16522 at commit

[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...

2017-01-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95282465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +344,102 @@ object

[GitHub] spark issue #16522: [SPARK-19137][SQL] Fix `withSQLConf` to reset `OptionalC...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16522 **[Test build #71107 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71107/testReport)** for PR 16522 at commit

[GitHub] spark issue #16464: [SPARK-19066][SparkR]:SparkR LDA doesn't set optimizer c...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16464 **[Test build #71093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71093/testReport)** for PR 16464 at commit

[GitHub] spark issue #16514: [SPARK-19128] [SQL] Refresh Cache after Set Location

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16514 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71091/ Test FAILed. ---

[GitHub] spark issue #16361: [SPARK-18952] Regex strings not properly escaped in code...

2017-01-09 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/16361 Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit opti...

2017-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15119#discussion_r95271376 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -283,8 +284,17 @@ object SparkSubmit extends CommandLineUtils { }

[GitHub] spark issue #16522: [SPARK-19137][SQL][SS] Garbage left in source tree after...

2017-01-09 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16522 @dongjoon-hyun The expected behavior is this test should use a temp folder instead. Looks like it gets `` from some place. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #16524: [SPARK-19110][MLLIB][FollowUP]: Add a unit test

2017-01-09 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/16524 [SPARK-19110][MLLIB][FollowUP]: Add a unit test ## What changes were proposed in this pull request? #16491 added the fix to mllib and a unit test to ml. This followup PR, add unit tests

[GitHub] spark issue #16497: [SPARK-19118] [SQL] Percentile support for frequency dis...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16497 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16497: [SPARK-19118] [SQL] Percentile support for frequency dis...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16497 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71094/ Test PASSed. ---

[GitHub] spark issue #16497: [SPARK-19118] [SQL] Percentile support for frequency dis...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16497 **[Test build #71094 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71094/testReport)** for PR 16497 at commit

[GitHub] spark issue #16520: [SPARK-19140][SS]Allow update mode for non-aggregation s...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71096/ Test PASSed. ---

[GitHub] spark issue #16520: [SPARK-19140][SS]Allow update mode for non-aggregation s...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16520 **[Test build #71096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71096/testReport)** for PR 16520 at commit

[GitHub] spark pull request #16522: [SPARK-19137][SQL] Fix `withSQLConf` to reset `Op...

2017-01-09 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16522#discussion_r95281803 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala --- @@ -94,7 +94,13 @@ private[sql] trait SQLTestUtils */

[GitHub] spark pull request #16517: [SPARK-18243][SQL] Implement InsertIntoHiveTable ...

2017-01-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16517#discussion_r95281883 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -99,7 +99,7 @@ class

[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...

2017-01-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95282681 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +344,102 @@ object

[GitHub] spark issue #16517: [SPARK-18243][SQL] Implement InsertIntoHiveTable with Fi...

2017-01-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16517 Maybe a better title is "Port Hive writing to use FileFormat interface"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #16517: [SPARK-18243][SQL] Implement InsertIntoHiveTable ...

2017-01-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/16517#discussion_r95228308 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -128,34 +128,32 @@ object FileFormatWriter extends

[GitHub] spark pull request #16431: [SPARK-19020] [SQL] Cardinality estimation of agg...

2017-01-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16431 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16361: [SPARK-18952] Regex strings not properly escaped in code...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16361 **[Test build #71088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71088/testReport)** for PR 16361 at commit

[GitHub] spark issue #16518: [BACKPORT][SPARK-18952] Regex strings not properly escap...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16518 **[Test build #71090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71090/testReport)** for PR 16518 at commit

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

2017-01-09 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r95237150 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -592,47 +579,6 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

2017-01-09 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r95237792 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -52,7 +55,36 @@ private[spark] class TaskDescription(

[GitHub] spark pull request #16514: [SPARK-19128] [SQL] Refresh Cache after Set Locat...

2017-01-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16514#discussion_r95247659 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -119,7 +119,30 @@ private[hive] class

[GitHub] spark issue #16497: [SPARK-19118] [SQL] Percentile support for frequency dis...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16497 **[Test build #71094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71094/testReport)** for PR 16497 at commit

[GitHub] spark issue #16519: SPARK-19138: Don't return SparkSession for stopped Spark...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16519 **[Test build #71097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71097/testReport)** for PR 16519 at commit

[GitHub] spark issue #16514: [SPARK-19128] [SQL] Refresh Cache after Set Location

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16514 **[Test build #71091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71091/testReport)** for PR 16514 at commit

[GitHub] spark pull request #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit opti...

2017-01-09 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15119#discussion_r95267649 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -974,23 +967,102 @@ private[spark] object SparkSubmitUtils { }

[GitHub] spark pull request #16523: [SPARK-19142][SparkR]:spark.kmeans should take se...

2017-01-09 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/16523 [SPARK-19142][SparkR]:spark.kmeans should take seed, initSteps, and tol as parameters ## What changes were proposed in this pull request? spark.kmeans doesn't have interface to set

[GitHub] spark pull request #16377: [SPARK-18036][ML][MLLIB] Fixing decision trees ha...

2017-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16377#discussion_r95275971 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -161,6 +161,33 @@ class RandomForestSuite extends SparkFunSuite

[GitHub] spark pull request #16377: [SPARK-18036][ML][MLLIB] Fixing decision trees ha...

2017-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16377#discussion_r95275814 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -176,6 +203,18 @@ class RandomForestSuite extends SparkFunSuite

[GitHub] spark issue #16521: [SPARK-19139][core] New auth mechanism for transport lib...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16521 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict ...

2017-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16441#discussion_r95276132 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -248,12 +269,38 @@ class GBTClassificationModel private[ml](

[GitHub] spark issue #16521: [SPARK-19139][core] New auth mechanism for transport lib...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16521 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71095/ Test FAILed. ---

[GitHub] spark issue #16521: [SPARK-19139][core] New auth mechanism for transport lib...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16521 **[Test build #71095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71095/testReport)** for PR 16521 at commit

[GitHub] spark issue #16249: [SPARK-18828][SPARKR] Refactor scripts for R

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16249 **[Test build #71098 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71098/testReport)** for PR 16249 at commit

[GitHub] spark issue #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15211 **[Test build #71105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71105/testReport)** for PR 15211 at commit

[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...

2017-01-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95281248 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +344,102 @@ object

[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...

2017-01-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95281159 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +344,102 @@ object

[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...

2017-01-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95282270 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +344,102 @@ object

[GitHub] spark issue #16522: [SPARK-19137][SQL] Fix `withSQLConf` to reset `OptionalC...

2017-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16522 Thank you. I updated it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16476: [SPARK-19084][SQL][WIP] Implement expression fiel...

2017-01-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r95283107 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1528,6 +1528,18 @@ object functions { def factorial(e: Column):

[GitHub] spark pull request #16518: [BACKPORT][SPARK-18952] Regex strings not properl...

2017-01-09 Thread brkyvz
GitHub user brkyvz opened a pull request: https://github.com/apache/spark/pull/16518 [BACKPORT][SPARK-18952] Regex strings not properly escaped in codegen for aggregations ## What changes were proposed in this pull request? Backport for #16361 to 2.1 branch. ##

[GitHub] spark pull request #16376: [SPARK-18967][SCHEDULER] compute locality levels ...

2017-01-09 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16376#discussion_r95238913 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -54,7 +54,7 @@ import org.apache.spark.util.{AccumulatorV2,

[GitHub] spark issue #16492: [SPARK-19113][SS][Tests]Set UncaughtExceptionHandler in ...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16492 **[Test build #71086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71086/testReport)** for PR 16492 at commit

[GitHub] spark issue #16441: [SPARK-14975][ML] Fixed GBTClassifier to predict probabi...

2017-01-09 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/16441 ping @sethah @jkbradley could you please take another look since I've updated the code review based on your comments? Thank you! --- If your project is set up for it, you can reply to this

[GitHub] spark issue #16519: SPARK-19138: Don't return SparkSession for stopped Spark...

2017-01-09 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/16519 Yeah, it looks like this is basically the same problem. I'll add some review comments to the other issue. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16361: [SPARK-18952] Regex strings not properly escaped in code...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16361 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16361: [SPARK-18952] Regex strings not properly escaped in code...

2017-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16361 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71088/ Test PASSed. ---

[GitHub] spark pull request #15119: [SPARK-17568][CORE][DEPLOY] Add spark-submit opti...

2017-01-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/15119#discussion_r95229575 --- Diff: docs/configuration.md --- @@ -450,8 +452,20 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #16518: [BACKPORT][SPARK-18952] Regex strings not properly escap...

2017-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16518 **[Test build #71089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71089/testReport)** for PR 16518 at commit

[GitHub] spark pull request #16520: [SPARK-19140][SS]Allow update mode for non-aggreg...

2017-01-09 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/16520 [SPARK-19140][SS]Allow update mode for non-aggregation streaming queries ## What changes were proposed in this pull request? This PR allow update mode for non-aggregation streaming

[GitHub] spark pull request #16514: [SPARK-19128] [SQL] Refresh Cache after Set Locat...

2017-01-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16514#discussion_r95258292 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -119,7 +119,30 @@ private[hive] class

[GitHub] spark pull request #16454: [SPARK-19055][SQL][PySpark] Fix SparkSession init...

2017-01-09 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/16454#discussion_r95259073 --- Diff: python/pyspark/sql/session.py --- @@ -214,8 +214,12 @@ def __init__(self, sparkContext, jsparkSession=None): self._wrapped =

  1   2   3   4   5   >