[GitHub] spark issue #17558: [SPARK-20247][CORE] Add jar but this jar is missing late...

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17558 @wangyum what if the task requires that jar? From your fix what I got is that you catch the exception and make it warning log instead, but what if that task requires the jar, will you fix

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110317549 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -328,7 +329,7 @@ object PartitioningUtils

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110298557 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala --- @@ -396,7 +397,7 @@ object

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110317695 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -222,7 +225,7 @@ case class

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110317441 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -128,7 +128,8 @@ object PartitioningUtils

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110314669 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StringKeyHashMap.scala --- @@ -25,7 +27,7 @@ object StringKeyHashMap {

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110315394 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala --- @@ -82,8 +84,8 @@ case class

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110314541 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala --- @@ -26,11 +28,12 @@ package

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110317272 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelation.scala --- @@ -52,7 +54,11 @@ case class

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110318802 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -134,7 +132,7 @@ case class

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110318621 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf {

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/17516 Don't we also need the skip if cran statement ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17516 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17516 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75589/ Test PASSed. ---

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17516 **[Test build #75589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75589/testReport)** for PR 17516 at commit

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17557 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17557 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75588/ Test PASSed. ---

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17557 **[Test build #75588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75588/testReport)** for PR 17557 at commit

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110318101 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17222 I'll try and follow up this weekend. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17494 Thanks @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17494 LGTM as well --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17222 **[Test build #75591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75591/testReport)** for PR 17222 at commit

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/17222 @viirya Thanks for careful review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17494 Thanks @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17222 LGTM, see if @marmbrus or @holdenk have any more comments about this change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJav...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17222#discussion_r110316824 --- Diff: python/pyspark/sql/tests.py --- @@ -436,6 +436,20 @@ def test_udf_with_order_by_and_limit(self): res.explain(True)

[GitHub] spark issue #17558: [SPARK-20247][CORE] Add jar but this jar is missing late...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17558 **[Test build #75590 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75590/testReport)** for PR 17558 at commit

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17546 This looks pretty good over all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110316465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(weight =>

[GitHub] spark pull request #17558: [SPARK-20247][CORE] Add jar but this jar is missi...

2017-04-06 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/17558 [SPARK-20247][CORE] Add jar but this jar is missing later shouldn't affect jobs that doesn't use this jar ## What changes were proposed in this pull request? Catch exception when jar is

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17516 **[Test build #75589 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75589/testReport)** for PR 17516 at commit

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17557 **[Test build #75588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75588/testReport)** for PR 17557 at commit

[GitHub] spark pull request #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth s...

2017-04-06 Thread zero323
GitHub user zero323 opened a pull request: https://github.com/apache/spark/pull/17557 [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support ## What changes were proposed in this pull request? Document fpGrowth in: - vignettes - programming guide - code

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-04-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15770 Any update on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR...

2017-04-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17553#discussion_r110315204 --- Diff: examples/src/main/r/ml/glm.R --- @@ -56,6 +56,15 @@ summary(binomialGLM) # Prediction binomialPredictions <- predict(binomialGLM,

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110314839 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/*

[GitHub] spark issue #17556: [SPARK-16957][MLlib] Use weighted midpoints for split va...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17556 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110314588 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/*

[GitHub] spark pull request #17556: [SPARK-16957][MLlib] Use weighted midpoints for s...

2017-04-06 Thread facaiy
GitHub user facaiy opened a pull request: https://github.com/apache/spark/pull/17556 [SPARK-16957][MLlib] Use weighted midpoints for split values. ## What changes were proposed in this pull request? Use weighted midpoints for split values. ## How was this patch

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110314369 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf {

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110313675 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -54,14 +54,12 @@ case class

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110313661 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -134,7 +132,7 @@ case class

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110313369 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf {

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110313349 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf {

[GitHub] spark pull request #17552: [SPARK-20245][SQL][minor] pass output to LogicalR...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17552#discussion_r110312633 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -18,39 +18,21 @@ package

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17552 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17552 **[Test build #75587 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75587/testReport)** for PR 17552 at commit

[GitHub] spark pull request #17552: [SPARK-20245][SQL][minor] pass output to LogicalR...

2017-04-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17552#discussion_r110311641 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -18,39 +18,21 @@ package

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110309359 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/* + *

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110309073 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/* + *

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110308327 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(weight =>

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110307898 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110307786 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110307666 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110306486 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP

[GitHub] spark pull request #17555: [SPARK-19495][SQL] Make SQLConf slightly more ext...

2017-04-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17555 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17555 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110305903 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75586/ Test PASSed. ---

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17555 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17555 **[Test build #75586 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75586/testReport)** for PR 17555 at commit

[GitHub] spark issue #17495: [SPARK-20172][Core] Add file permission check when listi...

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17495 Ping @vanzin @tgravescs again. Sorry to bother you and really appreciate your time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14617 I see. The current code leverages `SparkListenerBlockUpdated` event to calculate memory usage, let me try to investigate the feasibility of using `taskEnd.taskMetrics.updatedBlocks`, to see if it

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-04-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/14617 yeah, we definitely don't want to start logging more events. But it seems like this info is already available -- taskEnd.taskMetrics.updatedBlocks already has everything, doesn't it? --- If your

[GitHub] spark pull request #17534: [SPARK-20218]'/applications/[app-id]/stages' in R...

2017-04-06 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/17534#discussion_r110303522 --- Diff: docs/monitoring.md --- @@ -299,12 +299,12 @@ can be identified by their `[attempt-id]`. In the API listed below, when running

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110303409 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110302895 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110300420 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14617 Thanks @squito . Regarding showing memory usage in history server. My major concern is that putting so many block update event into event log will significantly increase the file size

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17551 @barnardb only in Spark standalone mode HistoryServer is embedded into Master process for convenience IIRC. You can always start a standalone HistoryServer process. Also

[GitHub] spark issue #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR in pro...

2017-04-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17553 Could you add [SPARKR] to the PR title please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16648 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75585/ Test PASSed. ---

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16648 **[Test build #75585 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75585/testReport)** for PR 16648 at commit

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75584/ Test PASSed. ---

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #75584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75584/testReport)** for PR 15009 at commit

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17555 **[Test build #75586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75586/testReport)** for PR 17555 at commit

[GitHub] spark pull request #17555: [SPARK-19495][SQL] Make SQLConf slightly more ext...

2017-04-06 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/17555 [SPARK-19495][SQL] Make SQLConf slightly more extensible - addendum ## What changes were proposed in this pull request? This is a tiny addendum to SPARK-19495 to remove the private visibility for

[GitHub] spark pull request #17554: [MINOR][DOCS] Fix typo in Hive Examples

2017-04-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17554 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17554: [MINOR][DOCS] Fix typo in Hive Examples

2017-04-06 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17554 Thanks - merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17552 LGTM except only one comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17552: [SPARK-20245][SQL][minor] pass output to LogicalR...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17552#discussion_r110287008 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -18,39 +18,21 @@ package

[GitHub] spark issue #17092: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-04-06 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/17092 Ping. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16966: [SPARK-18409][ML]LSH approxNearestNeighbors should use a...

2017-04-06 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/16966 Ping. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17546 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17546 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75583/ Test PASSed. ---

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17546 **[Test build #75583 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75583/testReport)** for PR 17546 at commit

[GitHub] spark issue #17554: [MINOR][DOCS] Fix typo in Hive Examples

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17554 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17554: [MINOR][DOCS] Fix typo in Hive Examples

2017-04-06 Thread cooper6581
GitHub user cooper6581 opened a pull request: https://github.com/apache/spark/pull/17554 [MINOR][DOCS] Fix typo in Hive Examples ## What changes were proposed in this pull request? Fix typo in hive examples from "DaraFrames" to "DataFrames" ## How was this patch

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16648 **[Test build #75585 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75585/testReport)** for PR 16648 at commit

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #75584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75584/testReport)** for PR 15009 at commit

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread barnardb
Github user barnardb commented on the issue: https://github.com/apache/spark/pull/17551 > It's still running your code, right? Why can't you add a configuration to your own code that tells it to wait some time before shutting down the SparkContext? We're trying to support

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/17546 @wzhfy Yes, star-schema is called from both ```ReorderJoin``` and ```CostBasedJoinReorder```. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17546 **[Test build #75583 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75583/testReport)** for PR 17546 at commit

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17541 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75580/ Test FAILed. ---

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17541 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17541 **[Test build #75580 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75580/testReport)** for PR 17541 at commit

[GitHub] spark issue #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR in pro...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17553 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   3   4   >