[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19733 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19733 **[Test build #83769 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83769/testReport)** for PR 19733 at commit

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19733 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83769/ Test FAILed. ---

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150461348 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,28 @@ case class Concat(children:

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19695 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83765/ Test PASSed. ---

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19695 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150460595 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,28 @@ case class Concat(children:

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19695 **[Test build #83765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83765/testReport)** for PR 19695 at commit

[GitHub] spark issue #19735: [MINOR][CORE] Using bufferedInputStream for dataDeserial...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19735 **[Test build #83773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83773/testReport)** for PR 19735 at commit

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19607 **[Test build #83774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83774/testReport)** for PR 19607 at commit

[GitHub] spark issue #19735: [MINOR][CORE] Using bufferedInputStream for dataDeserial...

2017-11-12 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19735 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11994 **[Test build #83772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83772/testReport)** for PR 11994 at commit

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15049 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15049 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83764/ Test PASSed. ---

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15049 **[Test build #83764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83764/testReport)** for PR 15049 at commit

[GitHub] spark issue #19736: [SPARK-22442][SQL][BRANCH-2.2][FOLLOWUP] ScalaReflection...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19736 **[Test build #83771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83771/testReport)** for PR 19736 at commit

[GitHub] spark pull request #19736: [SPARK-22442][SQL][BRANCH-2.2][FOLLOWUP] ScalaRef...

2017-11-12 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/19736 [SPARK-22442][SQL][BRANCH-2.2][FOLLOWUP] ScalaReflection should produce correct field names for special characters ## What changes were proposed in this pull request? `val TermName:

[GitHub] spark issue #19736: [SPARK-22442][SQL][BRANCH-2.2][FOLLOWUP] ScalaReflection...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19736 cc @felixcheung --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19734 `val TermName: TermNameExtractor` is new in scala 2.11. For 2.10, we should use deprecated `newTermName`. I will submit a follow-up. ---

[GitHub] spark issue #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19728 **[Test build #83770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83770/testReport)** for PR 19728 at commit

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150454183 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -163,13 +190,18 @@ case class

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150454166 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,26 @@ case class Concat(children:

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2017-11-12 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/11994 Sure, let me update the code. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19734 @felixcheung Yes. Looking into it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19733 **[Test build #83769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83769/testReport)** for PR 19733 at commit

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19734 @viirya could you take a look https://amplab.cs.berkeley.edu/jenkins/job/spark-branch-2.2-compile-sbt-scala-2.10/724/ ---

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150451065 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -146,4 +146,172 @@ class QuantileDiscretizerSuite

[GitHub] spark pull request #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19733#discussion_r150451001 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -236,24 +236,30 @@ case class In(value: Expression,

[GitHub] spark issue #19735: [MINOR][CORE] Using bufferedInputStream for dataDeserial...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19735 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150450632 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala --- @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150450502 --- Diff: python/pyspark/ml/image.py --- @@ -48,7 +48,7 @@ def imageSchema(self): """ Returns the image schema. -

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150450412 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala --- @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450334 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -146,4 +146,172 @@ class QuantileDiscretizerSuite

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19734 Thanks @felixcheung --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450305 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -146,4 +146,172 @@ class QuantileDiscretizerSuite

[GitHub] spark issue #19735: [MINOR][CORE] Using bufferedInputStream for dataDeserial...

2017-11-12 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/19735 @srowen Could you take a look? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection sh...

2017-11-12 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/19734 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450319 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -146,4 +146,172 @@ class QuantileDiscretizerSuite

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450280 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -146,4 +146,172 @@ class QuantileDiscretizerSuite

[GitHub] spark pull request #19735: [MINOR][CORE] Using bufferedInputStream for dataD...

2017-11-12 Thread ConeyLiu
GitHub user ConeyLiu opened a pull request: https://github.com/apache/spark/pull/19735 [MINOR][CORE] Using bufferedInputStream for dataDeserializeStream ## What changes were proposed in this pull request? Small fix. Using bufferedInputStream for dataDeserializeStream.

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150449331 --- Diff: python/pyspark/ml/image.py --- @@ -65,7 +65,7 @@ def ocvTypes(self): """ Returns the OpenCV type mapping supported

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-12 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19681 lgtm need to fix the merge conflict though --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150449520 --- Diff: python/pyspark/ml/image.py --- @@ -129,7 +129,7 @@ def toImage(self, array, origin=""): :param array array: The array to

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150449107 --- Diff: python/pyspark/ml/image.py --- @@ -48,7 +48,7 @@ def imageSchema(self): """ Returns the image schema. -

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450222 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -129,34 +152,95 @@ final class QuantileDiscretizer

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150449573 --- Diff: python/pyspark/ml/image.py --- @@ -168,7 +167,7 @@ def readImages(self, path, recursive=False, numPartitions=-1, :param bool

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150449349 --- Diff: python/pyspark/ml/image.py --- @@ -80,7 +80,7 @@ def imageFields(self): """ Returns field names of image columns.

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150449414 --- Diff: python/pyspark/ml/image.py --- @@ -109,7 +109,7 @@ def toNDArray(self, image): Converts an image to a one-dimensional array.

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450151 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -104,7 +126,8 @@ private[feature] trait

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19734 merged to 2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19734 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19734 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83763/ Test PASSed. ---

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19734 ok thanks, I'm above to tag 2.2.1. technically this isn't a regression but I could wait for a few hours (need to wait for jenkins build from the branch) if we could merge this ASAP ---

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19734 **[Test build #83763 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83763/testReport)** for PR 19734 at commit

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83768/testReport)** for PR 19439 at commit

[GitHub] spark issue #19588: [SPARK-12375][ML] VectorIndexerModel support handle unse...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83766/ Test PASSed. ---

[GitHub] spark issue #19588: [SPARK-12375][ML] VectorIndexerModel support handle unse...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19588: [SPARK-12375][ML] VectorIndexerModel support handle unse...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19588 **[Test build #83766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83766/testReport)** for PR 19588 at commit

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/19439 I've updated the code to take care of all comments except this one: "Determinism for sampling (commented above)" I will need to think about this a bit more. @jkbradley

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19733 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19733 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83762/ Test PASSed. ---

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19733 **[Test build #83762 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83762/testReport)** for PR 19733 at commit

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83767/ Test FAILed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83767/testReport)** for PR 19439 at commit

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83767 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83767/testReport)** for PR 19439 at commit

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447624 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,196 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447470 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,196 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447428 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,196 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447369 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447221 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447228 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19695 LGTM pending jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark ...

2017-11-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19459 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19459 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15049 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark ...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19459#discussion_r150445421 --- Diff: python/pyspark/sql/session.py --- @@ -438,28 +438,70 @@ def _get_numpy_record_dtypes(self, rec): curr_type =

[GitHub] spark issue #19588: [SPARK-12375][ML] VectorIndexerModel support handle unse...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19588 **[Test build #83766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83766/testReport)** for PR 19588 at commit

[GitHub] spark pull request #19588: [SPARK-12375][ML] VectorIndexerModel support hand...

2017-11-12 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19588#discussion_r150445283 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala --- @@ -37,7 +38,25 @@ import org.apache.spark.sql.types.{StructField,

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150444966 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -126,18 +137,34 @@ case class

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150444902 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,26 @@ case class Concat(children:

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83760/ Test PASSed. ---

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19695 **[Test build #83765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83765/testReport)** for PR 19695 at commit

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150444269 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,26 @@ case class Concat(children:

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19651 **[Test build #83760 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83760/testReport)** for PR 19651 at commit

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19695 Just rebased and fixed the comments a little bit more. Let me merge this one once it passes the tests. --- - To

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83761/ Test PASSed. ---

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19459 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19459 **[Test build #83761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83761/testReport)** for PR 19459 at commit

[GitHub] spark pull request #19389: [SPARK-22165][SQL] Resolve type conflicts between...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19389#discussion_r150443594 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -468,14 +460,16 @@ object

[GitHub] spark issue #19722: [SPARK-21693][R][ML] Reduce max iterations in Linear SVM...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19722 Thank you @srowen and @felixcheung for review! and thanks @mgaido91 and @dongjoon-hyun for your thumbs up :D. --- - To

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15049 **[Test build #83764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83764/testReport)** for PR 15049 at commit

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83759/ Test PASSed. ---

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19651 **[Test build #83759 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83759/testReport)** for PR 19651 at commit

[GitHub] spark pull request #19077: [SPARK-21860][core]Improve memory reuse for heap ...

2017-11-12 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/19077#discussion_r150442437 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -48,6 +49,15 @@ public long size() { }

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150441354 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -163,13 +190,18 @@ case class

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150441287 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,26 @@ case class Concat(children:

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150441323 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -126,18 +137,34 @@ case class

  1   2   3   4   >