[GitHub] spark issue #19580: [SPARK-11334][CORE] Fix bug in Executor allocation manag...

2017-10-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19580 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19617: [SPARK-22347][PySpark][DOC] Add document to notic...

2017-10-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19617#discussion_r147995716 --- Diff: python/pyspark/sql/functions.py --- @@ -2185,6 +2185,12 @@ def udf(f=None, returnType=StringType()): duplicate invocations may be

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19617 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83259/ Test FAILed. ---

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19617 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19617 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19617 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83260/ Test FAILed. ---

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19617 **[Test build #83261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83261/testReport)** for PR 19617 at commit

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19617 **[Test build #83261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83261/testReport)** for PR 19617 at commit

[GitHub] spark issue #19586: [SPARK-22367][WIP][CORE] Separate the serialization of c...

2017-10-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19586 @ConeyLiu what about the below example, does your implementation support this? ```scala trait Base { val name: String } case class A(name: String) extends Base case class

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147991609 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147998308 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147999393 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19617 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #83258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83258/testReport)** for PR 19527 at commit

[GitHub] spark issue #19580: [SPARK-11334][CORE] Fix bug in Executor allocation manag...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19580 **[Test build #83257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83257/testReport)** for PR 19580 at commit

[GitHub] spark pull request #19617: [SPARK-22347][PySpark][DOC] Add document to notic...

2017-10-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19617#discussion_r147995905 --- Diff: python/pyspark/sql/functions.py --- @@ -2185,6 +2185,12 @@ def udf(f=None, returnType=StringType()): duplicate invocations may be

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19272 **[Test build #83262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83262/testReport)** for PR 19272 at commit

[GitHub] spark pull request #19621: [SPARK-11215][ml] Add multiple columns support to...

2017-10-31 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/19621 [SPARK-11215][ml] Add multiple columns support to StringIndexer ## What changes were proposed in this pull request? Add multiple columns support to StringIndexer. ## How was

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147935560 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147989847 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147988875 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147988776 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147989197 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147986087 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147981187 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147980830 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147997611 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83258/ Test PASSed. ---

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19617 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19617 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83261/ Test PASSed. ---

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147935499 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #83258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83258/testReport)** for PR 19527 at commit

[GitHub] spark issue #19617: [SPARK-22347][PySpark][DOC] Add document to notice users...

2017-10-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19617 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-31 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r148013037 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -233,10 +249,15 @@ class

[GitHub] spark issue #19622: [SPARK-22306][SQL][2.2] alter table schema should not er...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19622 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19426: [SPARK-22190][CORE] Add Spark executor task metrics to D...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19426 **[Test build #83268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83268/testReport)** for PR 19426 at commit

[GitHub] spark issue #19622: [SPARK-22306][SQL][2.2] alter table schema should not er...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19622 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83264/ Test FAILed. ---

[GitHub] spark issue #18972: [SPARK-21720][SQL] Fix 64KB JVM bytecode limit problem w...

2017-10-31 Thread bali0019
Github user bali0019 commented on the issue: https://github.com/apache/spark/pull/18972 Hi @kiszk , I have few questions on this PR. Is there any time estimate on when this PR is supposed to merge ? Also, are these 3 commits under this PR planned to be pushed to Spark 2.2 as

[GitHub] spark pull request #19580: [SPARK-11334][CORE] Fix bug in Executor allocatio...

2017-10-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19580 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-10-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 > I'm reluctant to generalize this PR without practical experience applying it to other column-oriented file formats. The only format I'm familiar with and have production experience with is

[GitHub] spark pull request #19622: [SPARK-22306][SQL][2.2] alter table schema should...

2017-10-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19622#discussion_r148060497 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -246,11 +246,11 @@ private[hive] class

[GitHub] spark pull request #19622: [SPARK-22306][SQL][2.2] alter table schema should...

2017-10-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19622#discussion_r148064090 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -246,11 +246,11 @@ private[hive] class

[GitHub] spark pull request #19622: [SPARK-22306][SQL][2.2] alter table schema should...

2017-10-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19622#discussion_r148067066 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -837,20 +849,7 @@ private[hive] object HiveClientImpl

[GitHub] spark issue #19622: [SPARK-22306][SQL][2.2] alter table schema should not er...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19622 **[Test build #83267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83267/testReport)** for PR 19622 at commit

[GitHub] spark issue #19623: [SPARK-22078][SQL] clarify exception behaviors for all d...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19623 **[Test build #83270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83270/testReport)** for PR 19623 at commit

[GitHub] spark issue #19623: [SPARK-22078][SQL] clarify exception behaviors for all d...

2017-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19623 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push ...

2017-10-31 Thread juanrh
Github user juanrh commented on a diff in the pull request: https://github.com/apache/spark/pull/19583#discussion_r148078847 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -51,7 +51,26 @@ private case class ExecutorRegistered(executorId: String)

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19272 **[Test build #83262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83262/testReport)** for PR 19272 at commit

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19272 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19272 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83262/ Test PASSed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83273/testReport)** for PR 19439 at commit

[GitHub] spark issue #19622: [SPARK-22306][SQL][2.2] alter table schema should not er...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19622 **[Test build #83264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83264/testReport)** for PR 19622 at commit

[GitHub] spark issue #19615: [SPARK-19611][SQL][followup] set dataSchema correctly in...

2017-10-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19615 Thank you so much, @cloud-fan ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148053982 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-10-31 Thread MrBago
Github user MrBago commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r148056744 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,456 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19580: [SPARK-11334][CORE] Fix bug in Executor allocation manag...

2017-10-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19580 failed to merge to 2.2... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83269/ Test FAILed. ---

[GitHub] spark issue #19606: [SPARK-22333][SQL][Backport-2.2]timeFunctionCall(CURRENT...

2017-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19606 Thanks! Merged to 2.2 Could you close this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/19439 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83272/ Test FAILed. ---

[GitHub] spark pull request #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push ...

2017-10-31 Thread juanrh
Github user juanrh commented on a diff in the pull request: https://github.com/apache/spark/pull/19583#discussion_r148079448 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -225,6 +270,7 @@ class HeartbeatReceiverSuite

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148054075 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148054159 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148055894 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19580: [SPARK-11334][CORE] Fix bug in Executor allocation manag...

2017-10-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19580 Merging to master / 2.2 / 2.1. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19622: [SPARK-22306][SQL][2.2] alter table schema should not er...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19622 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19605: [SPARK-22394] [SQL] Remove redundant synchronizat...

2017-10-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19605#discussion_r148073401 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -89,10 +89,12 @@ private[spark] class

[GitHub] spark pull request #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push ...

2017-10-31 Thread juanrh
Github user juanrh commented on a diff in the pull request: https://github.com/apache/spark/pull/19583#discussion_r148075569 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -51,7 +51,26 @@ private case class ExecutorRegistered(executorId: String)

[GitHub] spark issue #15178: [SPARK-17556][SQL] Executor side broadcast for broadcast...

2017-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15178 The current solution could OOM executors whose memory sizes are normally much smaller than the driver. We might also see the performance regression when the number of partitions is large and the

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148054322 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-10-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 > @mallman I will try to go through this again. Do you think this can be generalized to data source v2 API? I'm not familiar with that API. I'm reluctant to generalize this PR

[GitHub] spark issue #19426: [SPARK-22190][CORE] Add Spark executor task metrics to D...

2017-10-31 Thread LucaCanali
Github user LucaCanali commented on the issue: https://github.com/apache/spark/pull/19426 Thanks @umehrot2 for the review and comments. Indeed well spotted that I had forgotten a couple of metrics and added one of them twice. This is hopefully fixed with the latest commit. As for

[GitHub] spark issue #19622: [SPARK-22306][SQL][2.2] alter table schema should not er...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19622 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83267/ Test FAILed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83271/ Test FAILed. ---

[GitHub] spark pull request #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE] Push ...

2017-10-31 Thread juanrh
Github user juanrh commented on a diff in the pull request: https://github.com/apache/spark/pull/19583#discussion_r148078337 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -241,6 +243,21 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/19439 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2017-10-31 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/19565 @akopich I'm actually leaning towards "filter after sample". 1. so we don't need to change `miniBatchFraction` in ` docs.sample(withReplacement = sampleWithReplacement,

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83273/testReport)** for PR 19439 at commit

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83273/ Test FAILed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83274/ Test FAILed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/19439 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83275/testReport)** for PR 19439 at commit

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-10-31 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/19272 @vanzin Thanks for the continued review. I addressed the comments you left. PTAL. It seems simplest and more robust to keep the serialized `Credentials` within

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148028651 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148034078 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala ...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19612 **[Test build #3969 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3969/testReport)** for PR 19612 at commit

[GitHub] spark issue #19621: [SPARK-11215][ML] Add multiple columns support to String...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19621 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19621: [SPARK-11215][ML] Add multiple columns support to String...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19621 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83265/ Test FAILed. ---

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148035301 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19622: [SPARK-22306][SQL][2.2] alter table schema should not er...

2017-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19622 **[Test build #83267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83267/testReport)** for PR 19622 at commit

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-10-31 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15770 @WeichenXu123 , for the graph helper, the Mllib has a version takes `Graph[Double, Double]` as a parameter for training. In ML, do we have to provide `DataSet` of `Graph`? Can you specify the

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread MrBago
Github user MrBago commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148049352 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19621: [SPARK-11215][ML] Add multiple columns support to String...

2017-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19621 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83263/ Test FAILed. ---

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r148027184 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,139 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

  1   2   3   >