[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15770 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15770 **[Test build #83230 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83230/testReport)** for PR 15770 at commit [`cfa18af`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19614: [SPARK-22399][ML] update the location of reference paper

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19614 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19614: [SPARK-22399][ML] update the location of reference paper

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19614 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83231/ Test FAILed. ---

[GitHub] spark issue #19614: [SPARK-22399][ML] update the location of reference paper

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19614 **[Test build #83231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83231/testReport)** for PR 19614 at commit [`ddc97ef`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #19614: [SPARK-22399][ML] update the location of reference paper

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19614 **[Test build #83231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83231/testReport)** for PR 19614 at commit [`ddc97ef`](https://github.com/apache/spark/commit/dd

[GitHub] spark pull request #19614: update the location of reference paper

2017-10-30 Thread bomeng
GitHub user bomeng opened a pull request: https://github.com/apache/spark/pull/19614 update the location of reference paper ## What changes were proposed in this pull request? Update the url of reference paper. ## How was this patch tested? It is comments, so nothing

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19611 LGTM pending tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15770 **[Test build #83230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83230/testReport)** for PR 15770 at commit [`cfa18af`](https://github.com/apache/spark/commit/cf

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83229/testReport)** for PR 19611 at commit [`d98ce9e`](https://github.com/apache/spark/commit/d9

[GitHub] spark issue #19613: Fixed a typo

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19613 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19611#discussion_r147824694 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -297,17 +297,41 @@ private[state

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19611#discussion_r147784280 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -297,17 +297,41 @@ private[state

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19611#discussion_r147824364 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -297,17 +297,41 @@ private[state

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19611#discussion_r147826631 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -297,17 +297,41 @@ private[state

[GitHub] spark pull request #19613: Fixed a typo

2017-10-30 Thread ganeshchand
GitHub user ganeshchand opened a pull request: https://github.com/apache/spark/pull/19613 Fixed a typo ready to serialized => ready to be serialized ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was

[GitHub] spark issue #18098: [SPARK-16944][Mesos] Improve data locality when launchin...

2017-10-30 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/18098 @PerilousApricot what do you mean topology? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional co

[GitHub] spark issue #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala ...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19612 **[Test build #83228 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83228/testReport)** for PR 19612 at commit [`e4f5282`](https://github.com/apache/spark/commit/e4

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83220/ Test PASSed. ---

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83220 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83220/testReport)** for PR 19611 at commit [`70df407`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala ...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19612 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala ...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19612 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83227/ Test FAILed. ---

[GitHub] spark issue #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala ...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19612 **[Test build #83227 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83227/testReport)** for PR 19612 at commit [`a2ccbf2`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala ...

2017-10-30 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19612 (Note this is a follow-up to https://github.com/apache/spark/pull/19307) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.a

[GitHub] spark issue #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala ...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19612 **[Test build #83227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83227/testReport)** for PR 19612 at commit [`a2ccbf2`](https://github.com/apache/spark/commit/a2

[GitHub] spark pull request #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for...

2017-10-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19612#discussion_r147812694 --- Diff: repl/scala-2.12/src/main/scala/org/apache/spark/repl/SparkILoop.scala --- @@ -64,10 +61,10 @@ class SparkILoop(in0: Option[BufferedReader], out: J

[GitHub] spark pull request #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for...

2017-10-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19612#discussion_r147812554 --- Diff: pom.xml --- @@ -1921,11 +1921,6 @@ ${antlr4.version} -jline --- End diff -- This cause

[GitHub] spark pull request #19612: [SPARK-14650][REPL][BUILD] Compile Spark REPL for...

2017-10-30 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/19612 [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala 2.12 ## What changes were proposed in this pull request? Spark REPL changes for Scala 2.12.4: use command(), not processLine() in ILo

[GitHub] spark issue #19370: [SPARK-18136] Fix setup of SPARK_HOME variable on Window...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #83226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83226/testReport)** for PR 19370 at commit [`8489a3c`](https://github.com/apache/spark/commit/84

[GitHub] spark issue #19370: [SPARK-18136] Fix setup of SPARK_HOME variable on Window...

2017-10-30 Thread jsnowacki
Github user jsnowacki commented on the issue: https://github.com/apache/spark/pull/19370 I've fixed this `"delims="` part. Let me know if I should change something about the other comments as I explained why this is done like that. --- ---

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83217/ Test PASSed. ---

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19611 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83217/testReport)** for PR 19611 at commit [`33ea2fb`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19610 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83216/ Test PASSed. ---

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19610 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19610 **[Test build #83216 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83216/testReport)** for PR 19610 at commit [`12a3703`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83225/testReport)** for PR 19611 at commit [`e81d142`](https://github.com/apache/spark/commit/e8

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] head() and show() for Columns

2017-10-30 Thread olarayej
Github user olarayej commented on the issue: https://github.com/apache/spark/pull/11336 @shivaram: Have you reviewed this? If the intent is to merge it, I'll gladly update the code. @gatorsmile --- - To unsubscribe

[GitHub] spark pull request #19532: [CORE]Modify the duration real-time calculation a...

2017-10-30 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/19532#discussion_r147795197 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -120,7 +120,7 @@ private[spark] class SparkUI private ( attemptId = Non

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83224 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83224/testReport)** for PR 19439 at commit [`e314327`](https://github.com/apache/spark/commit/e3

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147790710 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,252 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...

2017-10-30 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19388 also @mridulm I think you worked on the committer protocol before, both in spark and MR, so would be great to get your thoughts. ---

[GitHub] spark pull request #19592: [SPARK-22347][SQL][PySpark] Support optionally ru...

2017-10-30 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19592#discussion_r147784119 --- Diff: python/pyspark/worker.py --- @@ -105,8 +105,14 @@ def read_single_udf(pickleSer, infile, eval_type): elif eval_type == PythonEvalType.

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19601 **[Test build #83223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83223/testReport)** for PR 19601 at commit [`b971506`](https://github.com/apache/spark/commit/b9

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19433 **[Test build #83219 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83219/testReport)** for PR 19433 at commit [`7efb1e0`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19433 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19611 I think we can remove the unit test. It's obviously that `loadMap` is not recursive and will not cause StackOverflowError. --- -

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19433 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83219/ Test PASSed. ---

[GitHub] spark pull request #19610: [SPARK-22400] [SQL] rename some APIs and classes ...

2017-10-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19610 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83222/testReport)** for PR 19439 at commit [`6ed56e1`](https://github.com/apache/spark/commit/6e

[GitHub] spark pull request #19608: [SPARK-22396] [SQL] Better Error Message for Inse...

2017-10-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19608 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19610 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19608: [SPARK-22396] [SQL] Better Error Message for InsertIntoD...

2017-10-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19608 Thanks! Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #19370: [SPARK-18136] Fix setup of SPARK_HOME variable on Window...

2017-10-30 Thread jsnowacki
Github user jsnowacki commented on the issue: https://github.com/apache/spark/pull/19370 Sorry I kept you waiting. I'm a bit busy lately and I don't get the notifications unless I get called out. --- - To unsubscrib

[GitHub] spark pull request #19370: [SPARK-18136] Fix setup of SPARK_HOME variable on...

2017-10-30 Thread jsnowacki
Github user jsnowacki commented on a diff in the pull request: https://github.com/apache/spark/pull/19370#discussion_r147771692 --- Diff: bin/find-spark-home.cmd --- @@ -0,0 +1,44 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19610 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark pull request #19370: [SPARK-18136] Fix setup of SPARK_HOME variable on...

2017-10-30 Thread jsnowacki
Github user jsnowacki commented on a diff in the pull request: https://github.com/apache/spark/pull/19370#discussion_r147771026 --- Diff: bin/find-spark-home.cmd --- @@ -0,0 +1,44 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19610 yea, and we can't change it after release. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional co

[GitHub] spark pull request #19370: [SPARK-18136] Fix setup of SPARK_HOME variable on...

2017-10-30 Thread jsnowacki
Github user jsnowacki commented on a diff in the pull request: https://github.com/apache/spark/pull/19370#discussion_r147770207 --- Diff: bin/find-spark-home.cmd --- @@ -0,0 +1,44 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83221/ Test FAILed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83221 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83221/testReport)** for PR 19439 at commit [`9d434ad`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83221/testReport)** for PR 19439 at commit [`9d434ad`](https://github.com/apache/spark/commit/9d

[GitHub] spark issue #19605: [SPARK-22394] [SQL] Remove redundant synchronization for...

2017-10-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19605 Also cc @srinathshankar @JoshRosen @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additio

[GitHub] spark pull request #19605: [SPARK-22394] [SQL] Remove redundant synchronizat...

2017-10-30 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19605#discussion_r147767406 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -89,10 +89,12 @@ private[spark] class HiveExternalCatalog(co

[GitHub] spark pull request #19383: [SPARK-20643][core] Add listener implementation t...

2017-10-30 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19383#discussion_r147766895 --- Diff: project/MimaExcludes.scala --- @@ -36,6 +36,8 @@ object MimaExcludes { // Exclude rules for 2.3.x lazy val v23excludes = v22excl

[GitHub] spark pull request #18251: [SPARK-17788][SPARK-21033][SQL] fix the potential...

2017-10-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18251 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread joseph-torres
Github user joseph-torres commented on the issue: https://github.com/apache/spark/pull/19611 One issue I want to explicitly bring up: this new unit test takes very long, almost 2 minutes on my computer. Creating 10k files isn't going to be super fast no matter what we do, but is there

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-30 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19568 @DonnyZone, I don't know of any cases that use codgen after the fix for `CodegenFallback`, but I think this is still a good idea. If Spark is going to generate code, it should generate correc

[GitHub] spark issue #18251: [SPARK-17788][SPARK-21033][SQL] fix the potential OOM in...

2017-10-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18251 thanks for the review, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional com

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83220/testReport)** for PR 19611 at commit [`70df407`](https://github.com/apache/spark/commit/70

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19611#discussion_r147761708 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -297,17 +297,39 @@ private[state]

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19611#discussion_r147761361 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -297,17 +297,39 @@ private[state]

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19611#discussion_r147761102 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -297,17 +297,39 @@ private[state]

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19611#discussion_r147761742 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -297,17 +297,39 @@ private[state]

[GitHub] spark issue #18139: [SPARK-20787][PYTHON] PySpark can't handle datetimes bef...

2017-10-30 Thread rberenguel
Github user rberenguel commented on the issue: https://github.com/apache/spark/pull/18139 Yes, most solutions are concerned only with strftime (virtualtime and matplotlib above). Since calendar.gmtime can handle this better (or at least for pre-1900) it is just a matter to... not mes

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83218/ Test FAILed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83218/testReport)** for PR 19439 at commit [`030fe38`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83218/testReport)** for PR 19439 at commit [`030fe38`](https://github.com/apache/spark/commit/03

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19433 **[Test build #83219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83219/testReport)** for PR 19433 at commit [`7efb1e0`](https://github.com/apache/spark/commit/7e

[GitHub] spark issue #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19611 **[Test build #83217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83217/testReport)** for PR 19611 at commit [`33ea2fb`](https://github.com/apache/spark/commit/33

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147760700 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,252 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request #19611: [SPARK-22305] Write HDFSBackedStateStoreProvider....

2017-10-30 Thread joseph-torres
GitHub user joseph-torres opened a pull request: https://github.com/apache/spark/pull/19611 [SPARK-22305] Write HDFSBackedStateStoreProvider.loadMap non-recursively ## What changes were proposed in this pull request? Write HDFSBackedStateStoreProvider.loadMap non-recursively. Thi

[GitHub] spark issue #18251: [SPARK-17788][SPARK-21033][SQL] fix the potential OOM in...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18251 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83215/ Test PASSed. ---

[GitHub] spark issue #18251: [SPARK-17788][SPARK-21033][SQL] fix the potential OOM in...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18251 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18251: [SPARK-17788][SPARK-21033][SQL] fix the potential OOM in...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18251 **[Test build #83215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83215/testReport)** for PR 18251 at commit [`13cdaba`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147756618 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,140 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# cont

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19610 **[Test build #83216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83216/testReport)** for PR 19610 at commit [`12a3703`](https://github.com/apache/spark/commit/12

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19610 Is the idea that this is unreleased and can be renamed? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For add

[GitHub] spark issue #19610: [SPARK-22400] [SQL] rename some APIs and classes to make...

2017-10-30 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19610 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spa

[GitHub] spark pull request #19610: [SPARK-22400] [SQL] rename some APIs and classes ...

2017-10-30 Thread wzhfy
GitHub user wzhfy opened a pull request: https://github.com/apache/spark/pull/19610 [SPARK-22400] [SQL] rename some APIs and classes to make their meaning clearer ## What changes were proposed in this pull request? Both `ReadSupport` and `ReadTask` have a method called `cre

[GitHub] spark issue #19608: [SPARK-22396] [SQL] Better Error Message for InsertIntoD...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19608 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83209/ Test PASSed. ---

[GitHub] spark issue #19608: [SPARK-22396] [SQL] Better Error Message for InsertIntoD...

2017-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19608 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19608: [SPARK-22396] [SQL] Better Error Message for InsertIntoD...

2017-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19608 **[Test build #83209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83209/testReport)** for PR 19608 at commit [`863991f`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147744803 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,140 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# cont

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147744313 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,140 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# cont

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-30 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147744010 --- Diff: mllib/src/test/scala/org/apache/spark/ml/image/ImageSchemaSuite.scala --- @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #18139: [SPARK-20787][PYTHON] PySpark can't handle datetimes bef...

2017-10-30 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18139 Thanks for cc'ing me too. This seems a very specific problem. If we can introduce similar and small fix to solve it in PySpark, it should be better for me. I'm not sure if virtualtime solves

<    1   2   3   4   >