[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-12 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150430605 --- Diff: core/src/main/resources/org/apache/spark/ui/static/historypage.js --- @@ -38,8 +38,17 @@ function makeIdNumeric(id) { } function

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-12 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150430610 --- Diff: core/src/main/resources/org/apache/spark/ui/static/historypage.js --- @@ -38,8 +38,17 @@ function makeIdNumeric(id) { } function

[GitHub] spark issue #19726: [SPARK-22490][DOC] Add PySpark doc for SparkSession.buil...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19726 I see. Thank you, @HyukjinKwon . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-12 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r150432221 --- Diff: mllib/src/main/scala/org/apache/spark/ml/linalg/JsonMatrixConverter.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-11-12 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/19272 Hello @vanzin thanks for the continued help with this, anything else needed? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150444902 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,26 @@ case class Concat(children:

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447428 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,196 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447470 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,196 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83767/ Test FAILed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83767/testReport)** for PR 19439 at commit

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150450412 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala --- @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450334 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -146,4 +146,172 @@ class QuantileDiscretizerSuite

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19734 Thanks @felixcheung --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150451065 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -146,4 +146,172 @@ class QuantileDiscretizerSuite

[GitHub] spark pull request #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19733#discussion_r150451001 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -236,24 +236,30 @@ case class In(value: Expression,

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15049 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15049 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83764/ Test PASSed. ---

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19651 **[Test build #83759 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83759/testReport)** for PR 19651 at commit

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150431325 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150432473 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark pull request #19729: [SPARK-22499][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19729#discussion_r150437176 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -604,6 +604,8 @@ case class Least(children:

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19651 **[Test build #83759 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83759/testReport)** for PR 19651 at commit

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83759/ Test PASSed. ---

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450151 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -104,7 +126,8 @@ private[feature] trait

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19734 merged to 2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19734 @felixcheung Yes. Looking into it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19695 **[Test build #83765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83765/testReport)** for PR 19695 at commit

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150430468 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150430423 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark pull request #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection sh...

2017-11-12 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/19734 [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should produce correct field names for special characters ## What changes were proposed in this pull request? For a class with field name of

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150444966 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -126,18 +137,34 @@ case class

[GitHub] spark issue #19588: [SPARK-12375][ML] VectorIndexerModel support handle unse...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19588 **[Test build #83766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83766/testReport)** for PR 19588 at commit

[GitHub] spark issue #19588: [SPARK-12375][ML] VectorIndexerModel support handle unse...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83766/ Test PASSed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83768/testReport)** for PR 19439 at commit

[GitHub] spark issue #19588: [SPARK-12375][ML] VectorIndexerModel support handle unse...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19720 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83754/ Test PASSed. ---

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19720 **[Test build #83754 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83754/testReport)** for PR 19720 at commit

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19720 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150430180 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala --- @@ -39,3 +58,120 @@ private[sql] object

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150431217 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark issue #19726: [SPARK-22490][DOC] Add PySpark doc for SparkSession.buil...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19726 **[Test build #83758 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83758/testReport)** for PR 19726 at commit

[GitHub] spark issue #19726: [SPARK-22490][DOC] Add PySpark doc for SparkSession.buil...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19726 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83758/ Test PASSed. ---

[GitHub] spark issue #19726: [SPARK-22490][DOC] Add PySpark doc for SparkSession.buil...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19726 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19459 **[Test build #83761 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83761/testReport)** for PR 19459 at commit

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-12 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r150432943 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -2769,6 +2769,20 @@ class LogisticRegressionSuite

[GitHub] spark pull request #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/19733 [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem with in ## What changes were proposed in this pull request? This PR changes `In` code generation to place generated code for expression

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2017-11-12 Thread Nickersoft
Github user Nickersoft commented on the issue: https://github.com/apache/spark/pull/13617 @JeremyNixon @avulanov Any update on this? I noticed neither this PR or the one on the deeplearning package was ever merged, and it is the only resource I can find regarding neural net-based

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19733 **[Test build #83762 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83762/testReport)** for PR 19733 at commit

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150441354 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -163,13 +190,18 @@ case class

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150441323 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -126,18 +137,34 @@ case class

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150441287 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,26 @@ case class Concat(children:

[GitHub] spark issue #19722: [SPARK-21693][R][ML] Reduce max iterations in Linear SVM...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19722 Thank you @srowen and @felixcheung for review! and thanks @mgaido91 and @dongjoon-hyun for your thumbs up :D. --- - To

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447624 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,196 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19734 @viirya could you take a look https://amplab.cs.berkeley.edu/jenkins/job/spark-branch-2.2-compile-sbt-scala-2.10/724/ ---

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15049 **[Test build #83764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83764/testReport)** for PR 15049 at commit

[GitHub] spark issue #19736: [SPARK-22442][SQL][BRANCH-2.2][FOLLOWUP] ScalaReflection...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19736 **[Test build #83771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83771/testReport)** for PR 19736 at commit

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150460595 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,28 @@ case class Concat(children:

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19695 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19695 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83765/ Test PASSed. ---

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-12 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r150430257 --- Diff: mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala --- @@ -476,6 +476,10 @@ class DenseMatrix @Since("2.0.0") (

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150430842 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala --- @@ -0,0 +1,138 @@ +/* + * Licensed to

[GitHub] spark pull request #19633: [SPARK-22411][SQL] Disable the heuristic to calcu...

2017-11-12 Thread vgankidi
Github user vgankidi commented on a diff in the pull request: https://github.com/apache/spark/pull/19633#discussion_r150431620 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -424,11 +424,19 @@ case class FileSourceScanExec(

[GitHub] spark issue #19726: [SPARK-22490][DOC] Add PySpark doc for SparkSession.buil...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19726 Oh, I had to be clear, I think Sphinx 1.6.6 is in development and upgrading it is actually a bigger issue I think. I tried other ways I could think ahead actually, but failed to figure

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19734 **[Test build #83763 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83763/testReport)** for PR 19734 at commit

[GitHub] spark issue #19727: [WIP][SPARK-22497][SQL] Project reuse

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19727 Simply reusing `ProjectExec` doesn't really reduce the scan. The duplication execution of CTE is a well known issue. I've addressed it before. But seems no solution to deal all possible cases yet.

[GitHub] spark pull request #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19733#discussion_r150440568 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -236,24 +236,30 @@ case class In(value: Expression,

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19459 **[Test build #83761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83761/testReport)** for PR 19459 at commit

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19459 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447221 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447369 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19728 **[Test build #83770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83770/testReport)** for PR 19728 at commit

[GitHub] spark pull request #19736: [SPARK-22442][SQL][BRANCH-2.2][FOLLOWUP] ScalaRef...

2017-11-12 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/19736 [SPARK-22442][SQL][BRANCH-2.2][FOLLOWUP] ScalaReflection should produce correct field names for special characters ## What changes were proposed in this pull request? `val TermName:

[GitHub] spark issue #19736: [SPARK-22442][SQL][BRANCH-2.2][FOLLOWUP] ScalaReflection...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19736 cc @felixcheung --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19735: [MINOR][CORE] Using bufferedInputStream for dataDeserial...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19735 **[Test build #83773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83773/testReport)** for PR 19735 at commit

[GitHub] spark pull request #19728: [SPARK-22498][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-12 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19728#discussion_r150461348 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -63,15 +63,28 @@ case class Concat(children:

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19651 **[Test build #83760 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83760/testReport)** for PR 19651 at commit

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19459 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15049 **[Test build #83764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83764/testReport)** for PR 15049 at commit

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19733 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83762/ Test PASSed. ---

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19733 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19733 **[Test build #83762 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83762/testReport)** for PR 19733 at commit

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/19439 I've updated the code to take care of all comments except this one: "Determinism for sampling (commented above)" I will need to think about this a bit more. @jkbradley

[GitHub] spark issue #19735: [MINOR][CORE] Using bufferedInputStream for dataDeserial...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19735 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150450502 --- Diff: python/pyspark/ml/image.py --- @@ -48,7 +48,7 @@ def imageSchema(self): """ Returns the image schema. -

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150450632 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala --- @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19733: [SPARK-22501][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19733 **[Test build #83769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83769/testReport)** for PR 19733 at commit

[GitHub] spark issue #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should pr...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19734 `val TermName: TermNameExtractor` is new in scala 2.11. For 2.10, we should use deprecated `newTermName`. I will submit a follow-up. ---

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150430065 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcDeserializer.scala --- @@ -0,0 +1,217 @@ +/* + *

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150430312 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcSerializer.scala --- @@ -0,0 +1,205 @@ +/* + * Licensed

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83760/ Test PASSed. ---

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19695 **[Test build #83765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83765/testReport)** for PR 19695 at commit

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15049 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-12 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150447228 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-12 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19695 LGTM pending jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83767 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83767/testReport)** for PR 19439 at commit

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450280 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -146,4 +146,172 @@ class QuantileDiscretizerSuite

[GitHub] spark pull request #19734: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection sh...

2017-11-12 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/19734 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19715: [SPARK-22397][ML]add multiple columns support to ...

2017-11-12 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/19715#discussion_r150450222 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -129,34 +152,95 @@ final class QuantileDiscretizer

  1   2   3   4   >