[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16611 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73041/ Test PASSed. ---

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16611 **[Test build #73041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73041/testReport)** for PR 16611 at commit

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698347 --- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala --- @@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698262 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698270 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698275 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698338 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698323 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698331 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698335 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698340 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698315 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698326 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r101698328 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,327 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #16956: [SPARK-19598][SQL]Remove the alias parameter in Unresolv...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16956 **[Test build #73044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73044/testReport)** for PR 16956 at commit

[GitHub] spark pull request #16956: [SPARK-19598][SQL]Remove the alias parameter in U...

2017-02-16 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/16956#discussion_r101698406 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -645,17 +645,21 @@ class AstBuilder extends

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16611 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73039/ Test FAILed. ---

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16611 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16611 **[Test build #73039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73039/testReport)** for PR 16611 at commit

[GitHub] spark issue #16945: [SPARK-19616][SparkR]:weightCol and aggregationDepth sho...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73043/ Test PASSed. ---

[GitHub] spark issue #16945: [SPARK-19616][SparkR]:weightCol and aggregationDepth sho...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16945 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16945: [SPARK-19616][SparkR]:weightCol and aggregationDepth sho...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16945 **[Test build #73043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73043/testReport)** for PR 16945 at commit

[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...

2017-02-16 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/12524 @JoshRosen This is interesting : thanks for the details ! On the face of it, I think @markhamstra's comment about #16620 should apply - but given the additional details, it might possible to

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-16 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r101693864 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,218 @@ +/* + * Licensed

[GitHub] spark issue #16969: [SPARK-19639][SPARKR][Example]:Add spark.svmLinear examp...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16969 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16969: [SPARK-19639][SPARKR][Example]:Add spark.svmLinear examp...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16969 **[Test build #73040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73040/testReport)** for PR 16969 at commit

[GitHub] spark issue #16945: [SPARK-19616][SparkR]:weightCol and aggregationDepth sho...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16945 **[Test build #73043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73043/testReport)** for PR 16945 at commit

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16611 Per 2f78cc7, I ran a build with Scala 2.10 as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16611 **[Test build #73042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73042/testReport)** for PR 16611 at commit

[GitHub] spark issue #16945: [SPARK-19616][SparkR]:weightCol and aggregationDepth sho...

2017-02-16 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16945 I add a test of weightCol for spark.logit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16611 **[Test build #73041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73041/testReport)** for PR 16611 at commit

[GitHub] spark issue #16972: [SPARK-19556][CORE][WIP] Broadcast data is not encrypted...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16972 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73036/ Test FAILed. ---

[GitHub] spark issue #16972: [SPARK-19556][CORE][WIP] Broadcast data is not encrypted...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16972 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16972: [SPARK-19556][CORE][WIP] Broadcast data is not encrypted...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16972 **[Test build #73036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73036/testReport)** for PR 16972 at commit

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

2017-02-16 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r101690256 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +341,91 @@ object

[GitHub] spark issue #16969: [SPARK-19639][SPARKR][Example]:Add spark.svmLinear examp...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16969 **[Test build #73040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73040/testReport)** for PR 16969 at commit

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-02-16 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15770 @thunterdb Thanks for your review! I will address the comments soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16951: [SPARK-18285][SPARKR] SparkR approxQuantile supports inp...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16951 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16951: [SPARK-18285][SPARKR] SparkR approxQuantile supports inp...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16951 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73038/ Test PASSed. ---

[GitHub] spark issue #16951: [SPARK-18285][SPARKR] SparkR approxQuantile supports inp...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16951 **[Test build #73038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73038/testReport)** for PR 16951 at commit

[GitHub] spark issue #16611: [SPARK-17967][SPARK-17878][SQL][PYTHON] Support for arra...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16611 **[Test build #73039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73039/testReport)** for PR 16611 at commit

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

2017-02-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r101688665 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +341,91 @@ object CaseKeyWhen {

[GitHub] spark issue #16785: [SPARK-19443][SQL] The function to generate constraints ...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16785 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16785: [SPARK-19443][SQL] The function to generate constraints ...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16785 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73035/ Test PASSed. ---

[GitHub] spark issue #16785: [SPARK-19443][SQL] The function to generate constraints ...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16785 **[Test build #73035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73035/testReport)** for PR 16785 at commit

[GitHub] spark issue #16785: [SPARK-19443][SQL] The function to generate constraints ...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16785 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73034/ Test PASSed. ---

[GitHub] spark issue #16785: [SPARK-19443][SQL] The function to generate constraints ...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16785 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16785: [SPARK-19443][SQL] The function to generate constraints ...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16785 **[Test build #73034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73034/testReport)** for PR 16785 at commit

[GitHub] spark issue #16971: [SPARK-19573][SQL] Make NaN/null handling consistent in ...

2017-02-16 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16971 @gatorsmile @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16476: [SPARK-19084][SQL] Implement expression field

2017-02-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16476#discussion_r101688198 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -340,3 +343,105 @@ object CaseKeyWhen {

[GitHub] spark issue #16971: [SPARK-19573][SQL] Make NaN/null handling consistent in ...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73033/ Test PASSed. ---

[GitHub] spark issue #16971: [SPARK-19573][SQL] Make NaN/null handling consistent in ...

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16971 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16971: [SPARK-19573][SQL] Make NaN/null handling consistent in ...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16971 **[Test build #73033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73033/testReport)** for PR 16971 at commit

[GitHub] spark issue #16952: [SPARK-19620][SQL]Fix incorrect exchange coordinator id ...

2017-02-16 Thread carsonwang
Github user carsonwang commented on the issue: https://github.com/apache/spark/pull/16952 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16951: [SPARK-18285][SPARKR] SparkR approxQuantile supports inp...

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16951 **[Test build #73038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73038/testReport)** for PR 16951 at commit

[GitHub] spark pull request #16962: [SPARK-18120][SPARK-19557][SQL] Call QueryExecuti...

2017-02-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16962 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101687644 --- Diff: project/MimaExcludes.scala --- @@ -943,7 +948,7 @@ object MimaExcludes {

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16664 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16962: [SPARK-18120][SPARK-19557][SQL] Call QueryExecutionListe...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16962 thanks for the review, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16961: [SPARK-19534][EXAMPLES] Convert Java tests to use lambda...

2017-02-16 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16961 whoa, nice/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16949: [SPARK-16122][CORE] Add rest api for job environment

2017-02-16 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/16949 Thanks @uncleGen and after seeing his code I agree with @vanzin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101687206 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/util/PeriodicGraphCheckpointer.scala --- @@ -87,10 +87,10 @@ private[mllib] class

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101687030 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/GraphOps.scala --- @@ -362,12 +362,14 @@ class GraphOps[VD: ClassTag, ED: ClassTag](graph:

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101687075 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala --- @@ -122,27 +126,37 @@ object Pregel extends Logging {

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101687037 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala --- @@ -113,7 +116,8 @@ object Pregel extends Logging { (graph: Graph[VD,

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101686985 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/GraphOps.scala --- @@ -362,12 +362,14 @@ class GraphOps[VD: ClassTag, ED: ClassTag](graph:

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101687170 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala --- @@ -155,6 +169,8 @@ object Pregel extends Logging { i += 1

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101687092 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala --- @@ -155,6 +169,8 @@ object Pregel extends Logging { i += 1

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101687268 --- Diff: project/MimaExcludes.scala --- @@ -943,7 +948,7 @@ object MimaExcludes {

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16386 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16386: [SPARK-18352][SQL] Support parsing multiline json files

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16386 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101683820 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1802,4 +1806,142 @@ class JsonSuite extends

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101683991 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1802,4 +1806,142 @@ class JsonSuite extends

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101684712 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -0,0 +1,216 @@ +/* + * Licensed to

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101683719 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -394,36 +447,30 @@ class JacksonParser( }

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101685228 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1802,4 +1806,142 @@ class JsonSuite extends

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101683953 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1802,4 +1806,142 @@ class JsonSuite extends

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101683631 --- Diff: core/src/main/scala/org/apache/spark/input/PortableDataStream.scala --- @@ -193,6 +196,10 @@ class PortableDataStream( } }

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101683916 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1802,4 +1806,142 @@ class JsonSuite extends

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101684619 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -37,29 +32,30 @@ import

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101684365 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1802,4 +1806,142 @@ class JsonSuite extends

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101684508 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1764,4 +1769,123 @@ class JsonSuite extends

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101683666 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -48,69 +47,110 @@ class JacksonParser(

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101684686 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -0,0 +1,216 @@ +/* + * Licensed to

[GitHub] spark pull request #16386: [SPARK-18352][SQL] Support parsing multiline json...

2017-02-16 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16386#discussion_r101685374 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1802,4 +1806,142 @@ class JsonSuite extends

[GitHub] spark issue #13467: [SPARK-15642][SQL] Bug report: metadata gets lost when s...

2017-02-16 Thread zommerfelds
Github user zommerfelds commented on the issue: https://github.com/apache/spark/pull/13467 The bug is still present. To be honest I don't think I have time to work on this right now, so we can close this PR if you want. I tried to find the code that was related to this functionality

[GitHub] spark issue #16973: [SPARKR][EXAMPLES] update examples to stop spark session

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16973 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73037/ Test PASSed. ---

[GitHub] spark issue #16973: [SPARKR][EXAMPLES] update examples to stop spark session

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16973 **[Test build #73037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73037/testReport)** for PR 16973 at commit

[GitHub] spark issue #16951: [SPARK-18285][SPARKR] SparkR approxQuantile supports inp...

2017-02-16 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16951 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16951: [SPARK-18285][SPARKR] SparkR approxQuantile suppo...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16951#discussion_r101685201 --- Diff: R/pkg/R/stats.R --- @@ -171,12 +175,17 @@ setMethod("freqItems", signature(x = "SparkDataFrame", cols = "character"), #' } #' @note

[GitHub] spark issue #16973: [SPARKR][EXAMPLES] update examples to stop spark session

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16973 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16951: [SPARK-18285][SPARKR] SparkR approxQuantile suppo...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16951#discussion_r101685081 --- Diff: R/pkg/R/stats.R --- @@ -149,15 +149,19 @@ setMethod("freqItems", signature(x = "SparkDataFrame", cols = "character"), #' This method

[GitHub] spark issue #16330: [SPARK-18817][SPARKR][SQL] change derby log output and m...

2017-02-16 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16330 Hi @gatorsmile @cloud-fan @yhuai could you give us some feedback on this? This is blocking release of Spark to R/CRAN - we don't need a very detailed review - just your thought of whether

[GitHub] spark issue #16386: [SPARK-18352][SQL] Support parsing multiline json files

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73032/ Test PASSed. ---

[GitHub] spark pull request #16966: [SPARK-18409][ML]LSH approxNearestNeighbors shoul...

2017-02-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16966#discussion_r101684524 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -147,6 +148,15 @@ private[ml] abstract class LSHModel[T <: LSHModel[T]]

[GitHub] spark issue #16386: [SPARK-18352][SQL] Support parsing multiline json files

2017-02-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16386 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16386: [SPARK-18352][SQL] Support parsing multiline json files

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16386 **[Test build #73032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73032/testReport)** for PR 16386 at commit

[GitHub] spark issue #16973: [SPARKR][EXAMPLES] update examples to stop spark session

2017-02-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16973 **[Test build #73037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73037/testReport)** for PR 16973 at commit

[GitHub] spark pull request #16973: [SPARKR][EXAMPLES] update examples to stop spark ...

2017-02-16 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/16973 [SPARKR][EXAMPLES] update examples to stop spark session ## What changes were proposed in this pull request? stop session at end of example ## How was this patch tested?

  1   2   3   4   5   6   >