[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20620 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20620 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87519/ Test FAILed. ---

[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20620 **[Test build #87519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87519/testReport)** for PR 20620 at commit

[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20620 **[Test build #87522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87522/testReport)** for PR 20620 at commit

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20633 **[Test build #87521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87521/testReport)** for PR 20633 at commit

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20633 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Thank you for retriggering, @gatorsmile . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/946/

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87520/ Test FAILed. ---

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87520/testReport)** for PR 20619 at commit

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20619 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20619 Would it be worth to add this JIRA number in a comment as we did for ORC? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20511: [SPARK-23340][SQL] Upgrade Apache ORC to 1.4.3

2018-02-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20511 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20613: [SPARK-23368][SQL] Avoid unnecessary Exchange or Sort af...

2018-02-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20613 cc @dongjoon-hyun Do you want to make a try to help review this PR? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20511: [SPARK-23340][SQL] Upgrade Apache ORC to 1.4.3

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20511 Thank you, @gatorsmile ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20620 **[Test build #87522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87522/testReport)** for PR 20620 at commit

[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20620 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87522/ Test PASSed. ---

[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20620 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/944/

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87520/testReport)** for PR 20619 at commit

[GitHub] spark issue #20511: [SPARK-23340][SQL] Upgrade Apache ORC to 1.4.3

2018-02-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20511 LGTM Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20620 Seems like unrelated issue. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87523/testReport)** for PR 20619 at commit

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168919128 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -407,6 +407,34 @@ object PartitioningUtils {

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20633 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/945/

[GitHub] spark pull request #20633: [SPARK-23455][ML] Default Params in ML should be ...

2018-02-17 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/20633 [SPARK-23455][ML] Default Params in ML should be saved separately in metadata ## What changes were proposed in this pull request? We save ML's user-supplied params and default params as

[GitHub] spark issue #20620: [SPARK-23438][DSTREAMS] Fix DStreams data loss with WAL ...

2018-02-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20620 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168921851 --- Diff: docs/sql-programming-guide.md --- @@ -1372,6 +1372,13 @@ the following case-insensitive options: This is a JDBC writer related

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168921833 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -102,7 +102,12 @@ object JdbcUtils extends

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20619 Umm, we still see the following exception in [the log](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87523/testReport/) ... ``` Caused by: sbt.ForkMain$ForkError:

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168925264 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite {

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168925267 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite {

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168925279 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite {

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168925247 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } }

[GitHub] spark pull request #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC i...

2018-02-17 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/20634 [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and PPD by default ## What changes were proposed in this pull request? Apache Spark 2.3 introduced `native` ORC supports with

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927631 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala --- @@ -42,4 +42,17 @@ private object MsSqlServerDialect extends

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927633 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MySQLDialect.scala --- @@ -46,4 +46,17 @@ private case object MySQLDialect extends

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927597 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala --- @@ -42,4 +42,17 @@ private object MsSqlServerDialect extends

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927608 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MsSqlServerDialect.scala --- @@ -42,4 +42,17 @@ private object MsSqlServerDialect extends

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927868 --- Diff: docs/sql-programming-guide.md --- @@ -1372,6 +1372,13 @@ the following case-insensitive options: This is a JDBC writer related

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20619 LGTM with one minor comment --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20621 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/948/

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20621 **[Test build #87526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87526/testReport)** for PR 20621 at commit

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20621 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20633 **[Test build #87521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87521/testReport)** for PR 20633 at commit

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87523/testReport)** for PR 20619 at commit

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168924005 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -407,6 +407,34 @@ object PartitioningUtils {

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87524/testReport)** for PR 20057 at commit

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87524/ Test FAILed. ---

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927651 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala --- @@ -94,5 +94,21 @@ private case object OracleDialect extends

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168928044 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala --- @@ -120,11 +121,13 @@ abstract class JdbcDialect extends

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Thank you, @kiszk . I added SPARK-23390 in the PR description. > Would it be worth to add this JIRA number in a comment as we did for ORC? ---

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Oh, @kiszk . The following meat really `comment` in the code. Sorry, I misunderstood. > Would it be worth to add this JIRA number in a comment as we did for ORC? ---

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20633 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20633 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87521/ Test PASSed. ---

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922697 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } } +

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922771 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite {

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922653 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } } +

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922800 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite {

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922677 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } } +

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922746 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } } +

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922736 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } } +

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922556 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -270,8 +269,14 @@ private[tree] class LearningNode( * Convert this

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922725 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -631,6 +654,32 @@ class RandomForestSuite extends SparkFunSuite

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168922781 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -640,4 +689,96 @@ private object RandomForestSuite {

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-02-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20633 cc @jkbradley This is to save default params separately in metadata file. Please help review after 2.3. Thanks! --- - To

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168925339 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -407,6 +407,34 @@ object PartitioningUtils {

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927546 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DB2Dialect.scala --- @@ -49,4 +49,17 @@ private object DB2Dialect extends JdbcDialect {

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20619 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927785 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala --- @@ -64,7 +64,16 @@ private class AggregatedDialect(dialects:

[GitHub] spark pull request #20619: [SPARK-23457][SQL] Register task completion liste...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20619#discussion_r168929352 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -395,16 +395,21 @@ class

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168922006 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -860,14 +860,41 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168922010 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -860,14 +860,41 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927495 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala --- @@ -120,11 +121,13 @@ abstract class JdbcDialect extends

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/950/

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread danielvdende
Github user danielvdende commented on the issue: https://github.com/apache/spark/pull/20057 Thanks for your review @dongjoon-hyun 👍 I've corrected all the indentation problems (didn't show up locally when running scalastyle checks for some reason). Added comments where necessary

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87524/testReport)** for PR 20057 at commit

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168923873 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala --- @@ -1120,4 +1120,16 @@

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168925240 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -631,6 +654,32 @@ class RandomForestSuite extends

[GitHub] spark issue #20632: [SPARK-3159] added subtree pruning in the translation fr...

2018-02-17 Thread asolimando
Github user asolimando commented on the issue: https://github.com/apache/spark/pull/20632 Hello Sean, here is my understanding of the problem and the main intuition of the proposed solution: We want to have a tree such that it does not contain any redundant subtree. A

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168925224 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } }

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-17 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r168925232 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +292,41 @@ private[tree] class LearningNode( } }

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20634 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168927408 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DerbyDialect.scala --- @@ -41,4 +41,16 @@ private object DerbyDialect extends JdbcDialect

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87523/ Test FAILed. ---

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168924965 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -407,6 +407,29 @@ object PartitioningUtils {

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20621#discussion_r168925882 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -407,6 +407,34 @@ object PartitioningUtils {

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20619 **[Test build #87527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87527/testReport)** for PR 20619 at commit

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/949/

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-17 Thread danielvdende
Github user danielvdende commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r168921808 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -119,6 +119,8 @@ class JDBCOptions(

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20634 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/947/

[GitHub] spark issue #20634: [SPARK-23456][SPARK-21783] Turn on `native` ORC impl and...

2018-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20634 **[Test build #87525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87525/testReport)** for PR 20634 at commit

[GitHub] spark issue #20619: [SPARK-23390][SQL] Register task completion listeners fi...

2018-02-17 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Yep. @kiszk . @mgaido91 also reports that, so I'm investigating that more. However, that doesn't mean this approach is not proper. You can see the manual test case example in previous

  1   2   >