[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread zecevicp
Github user zecevicp commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193753965 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/InnerJoinSuite.scala --- @@ -117,101 +131,170 @@ class InnerJoinSuite extends

[GitHub] spark issue #21499: [SPARK-24468][SQL] Handle negative scale when adjusting ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21499 **[Test build #91520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91520/testReport)** for PR 21499 at commit

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193762830 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/InMemoryUnsafeRowQueue.scala --- @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18900 **[Test build #91522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91522/testReport)** for PR 18900 at commit

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #91523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91523/testReport)** for PR 21469 at commit

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #91519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91519/testReport)** for PR 13599 at commit

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193740695 --- Diff: python/pyspark/sql/streaming.py --- @@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None, continuous=None):

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread zecevicp
Github user zecevicp commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193763364 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/InMemoryUnsafeRowQueue.scala --- @@ -0,0 +1,183 @@ +/* + * Licensed to the

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21502 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21502 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3832/

[GitHub] spark pull request #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21502#discussion_r193724774 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala --- @@ -152,6 +152,26 @@ class BroadcastJoinSuite

[GitHub] spark pull request #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21502#discussion_r193742604 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala --- @@ -152,6 +152,26 @@ class BroadcastJoinSuite extends

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193734550 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193735061 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193735681 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193733146 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193736960 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/InnerJoinSuite.scala --- @@ -117,101 +131,170 @@ class InnerJoinSuite extends

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread zecevicp
Github user zecevicp commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193753823 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/InMemoryUnsafeRowQueue.scala --- @@ -0,0 +1,183 @@ +/* + * Licensed to the

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21502 **[Test build #91524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91524/testReport)** for PR 21502 at commit

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21502 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91524/ Test FAILed. ---

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21502 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91519/ Test PASSed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread zecevicp
Github user zecevicp commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193751918 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1205,6 +1205,19 @@ object SQLConf { .booleanConf

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21502 **[Test build #91524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91524/testReport)** for PR 21502 at commit

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread zecevicp
Github user zecevicp commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193754271 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends

[GitHub] spark issue #21499: [SPARK-24468][SQL] Handle negative scale when adjusting ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91520/ Test PASSed. ---

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21467 **[Test build #91521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91521/testReport)** for PR 21467 at commit

[GitHub] spark pull request #18900: [SPARK-21687][SQL] Spark SQL should set createTim...

2018-06-07 Thread debugger87
Github user debugger87 commented on a diff in the pull request: https://github.com/apache/spark/pull/18900#discussion_r193730957 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -1019,6 +1021,8 @@ private[hive] object HiveClientImpl {

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91521/ Test PASSed. ---

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21467 **[Test build #91521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91521/testReport)** for PR 21467 at commit

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21467 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193743605 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1205,6 +1205,19 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193736438 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/InnerJoinSuite.scala --- @@ -70,27 +70,41 @@ class InnerJoinSuite extends

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193737191 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/InMemoryUnsafeRowQueue.scala --- @@ -0,0 +1,183 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #21109: [SPARK-24020][SQL] Sort-merge join inner range op...

2018-06-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21109#discussion_r193735968 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -131,13 +135,100 @@ object ExtractEquiJoinKeys extends

[GitHub] spark issue #21499: [SPARK-24468][SQL] Handle negative scale when adjusting ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21045: [SPARK-23931][SQL] Adds zip function to sparksql

2018-06-07 Thread DylanGuedes
Github user DylanGuedes commented on a diff in the pull request: https://github.com/apache/spark/pull/21045#discussion_r193759057 --- Diff: python/pyspark/sql/functions.py --- @@ -2394,6 +2394,23 @@ def array_repeat(col, count): return

[GitHub] spark pull request #21282: [SPARK-23934][SQL] Adding map_from_entries functi...

2018-06-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21282#discussion_r193958595 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -308,6 +309,234 @@ case class

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #91523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91523/testReport)** for PR 21469 at commit

[GitHub] spark pull request #21501: [SPARK-15064][ML] Locale support in StopWordsRemo...

2018-06-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21501#discussion_r193779131 --- Diff: python/pyspark/ml/feature.py --- @@ -2582,25 +2582,27 @@ class StopWordsRemover(JavaTransformer, HasInputCol, HasOutputCol, JavaMLReadabl

[GitHub] spark pull request #21501: [SPARK-15064][ML] Locale support in StopWordsRemo...

2018-06-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21501#discussion_r193777474 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StopWordsRemover.scala --- @@ -84,7 +86,31 @@ class StopWordsRemover @Since("1.5.0")

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-06-07 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r193801841 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -102,17 +110,30 @@ private[spark]

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21502 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21502 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3833/

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #91529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91529/testReport)** for PR 21488 at commit

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91529/ Test FAILed. ---

[GitHub] spark pull request #21506: [SPARK-24485][SS] Measure and log elapsed time fo...

2018-06-07 Thread HeartSaVioR
GitHub user HeartSaVioR opened a pull request: https://github.com/apache/spark/pull/21506 [SPARK-24485][SS] Measure and log elapsed time for filesystem operations in HDFSBackedStateStoreProvider ## What changes were proposed in this pull request? This patch measures and

[GitHub] spark issue #21483: [SPARK-24477][SPARK-24454][ML][PYTHON] Imports submodule...

2018-06-07 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21483 Ping @jkbradley @MLnick @BryanCutler @holdenk @yanboliang . What do you think about the proposed change? --- - To unsubscribe,

[GitHub] spark issue #21498: [SPARK-24410][SQL][Core] Optimization for Union outputPa...

2018-06-07 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21498 Thanks for you benchmark @viirya. The performance improvement is sensible. And seems no performance regression in the other case. Can we have a similar benchmark also with records with more

[GitHub] spark issue #21506: [SPARK-24485][SS] Measure and log elapsed time for files...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21506 **[Test build #91525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91525/testReport)** for PR 21506 at commit

[GitHub] spark issue #21506: [SPARK-24485][SS] Measure and log elapsed time for files...

2018-06-07 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21506 There're plenty of other debug messages which might hide the log messages added from this patch. Would we want to log them with INFO instead of DEBUG? ---

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21469 retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21501: [SPARK-15064][ML] Locale support in StopWordsRemover

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21501 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21501: [SPARK-15064][ML] Locale support in StopWordsRemover

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21501 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91527/ Test FAILed. ---

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18900 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18900 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91522/ Test FAILed. ---

[GitHub] spark issue #21501: [SPARK-15064][ML] Locale support in StopWordsRemover

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21501 **[Test build #91527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91527/testReport)** for PR 21501 at commit

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18900 **[Test build #91522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91522/testReport)** for PR 18900 at commit

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-06-07 Thread kokes
Github user kokes commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r193805391 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -154,6 +176,24 @@ private[spark] object Config

[GitHub] spark issue #18826: [SPARK-14712][ML] LogisticRegressionModel.toString shoul...

2018-06-07 Thread bravo-zhang
Github user bravo-zhang commented on the issue: https://github.com/apache/spark/pull/18826 @HyukjinKwon It's ready to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91523/ Test FAILed. ---

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21501: [SPARK-15064][ML] Locale support in StopWordsRemo...

2018-06-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21501#discussion_r193781216 --- Diff: python/pyspark/ml/feature.py --- @@ -2582,25 +2582,27 @@ class StopWordsRemover(JavaTransformer, HasInputCol, HasOutputCol, JavaMLReadabl

[GitHub] spark issue #21501: [SPARK-15064][ML] Locale support in StopWordsRemover

2018-06-07 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21501 jenkins add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21501: [SPARK-15064][ML] Locale support in StopWordsRemover

2018-06-07 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21501 jenkins test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #91529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91529/testReport)** for PR 21488 at commit

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #91526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91526/testReport)** for PR 21469 at commit

[GitHub] spark issue #21501: [SPARK-15064][ML] Locale support in StopWordsRemover

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21501 **[Test build #91527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91527/testReport)** for PR 21501 at commit

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-06-07 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r193796798 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -154,6 +176,24 @@ private[spark] object

[GitHub] spark issue #21502: [SPARK-22575][SQL] Add destroy to Dataset

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21502 **[Test build #91528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91528/testReport)** for PR 21502 at commit

[GitHub] spark issue #21506: [SPARK-24485][SS] Measure and log elapsed time for files...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21506 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20677: Event time can't be greater then processing time. 12:21,...

2018-06-07 Thread deil87
Github user deil87 commented on the issue: https://github.com/apache/spark/pull/20677 When we are reading documentation not for example learning on the courses we do not expect such intentional mistakes. --- - To

[GitHub] spark pull request #21483: [SPARK-24477][SPARK-24454][ML][PYTHON] Imports su...

2018-06-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21483#discussion_r193785341 --- Diff: python/pyspark/ml/__init__.py --- @@ -21,5 +21,11 @@ """ from pyspark.ml.base import Estimator, Model, Transformer, UnaryTransformer

[GitHub] spark pull request #21483: [SPARK-24477][SPARK-24454][ML][PYTHON] Imports su...

2018-06-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21483#discussion_r193785229 --- Diff: python/pyspark/ml/__init__.py --- @@ -21,5 +21,11 @@ """ from pyspark.ml.base import Estimator, Model, Transformer, UnaryTransformer

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3834/

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91526/ Test FAILed. ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 **[Test build #91530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91530/testReport)** for PR 21092 at commit

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-06-07 Thread dvogelbacher
Github user dvogelbacher commented on the issue: https://github.com/apache/spark/pull/21366 @foxish did you have a chance to take a look? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18826: [SPARK-14712][ML] LogisticRegressionModel.toString shoul...

2018-06-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18826 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21468: [SPARK-22151] : PYTHONPATH not picked up from the...

2018-06-07 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21468#discussion_r193842887 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -813,8 +813,14 @@ private[spark] class Client(

[GitHub] spark issue #21507: Branch 1.6

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21507 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91530/ Test FAILed. ---

[GitHub] spark issue #21475: [SPARK-24416] Fix configuration specification for killBl...

2018-06-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21475 that wording for the application level sounds good, so we are talking a about changing it to be something like: If set to "true", allow Spark to automatically kill the executors

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-06-07 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21366 That's what should happen, but I think that isn't handled by this code. To handle that I think the abstractions need to be a bit different from what we have here. I'll draft up something. Hopefully

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r192483275 --- Diff: python/pyspark/sql/streaming.py --- @@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None, continuous=None):

[GitHub] spark pull request #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured...

2018-06-07 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/21477#discussion_r193862649 --- Diff: python/pyspark/sql/tests.py --- @@ -1884,7 +1885,164 @@ def test_query_manager_await_termination(self): finally:

[GitHub] spark issue #18826: [SPARK-14712][ML] LogisticRegressionModel.toString shoul...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18826 **[Test build #91531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91531/testReport)** for PR 18826 at commit

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3835/

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3692/ ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3692/ ---

[GitHub] spark issue #21504: [SPARK-24479][SS] Added config for registering streaming...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21504 **[Test build #91532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91532/testReport)** for PR 21504 at commit

[GitHub] spark issue #21366: [SPARK-24248][K8S] Use the Kubernetes API to populate an...

2018-06-07 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21366 Do most controllers use both Watches and polling? I'm wondering if we can just poll very frequently and not have the watch. ---

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21109 **[Test build #91533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91533/testReport)** for PR 21109 at commit

[GitHub] spark pull request #21507: Branch 1.6

2018-06-07 Thread deepaksonu
GitHub user deepaksonu opened a pull request: https://github.com/apache/spark/pull/21507 Branch 1.6 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this

[GitHub] spark issue #18826: [SPARK-14712][ML] LogisticRegressionModel.toString shoul...

2018-06-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18826 **[Test build #91531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91531/testReport)** for PR 18826 at commit

  1   2   3   4   >