[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-12-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15780 LGTM except a very minor style comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15843: [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWr...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15843 **[Test build #69476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69476/consoleFull)** for PR 15843 at commit

[GitHub] spark issue #15843: [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWr...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15843 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16093: [SPARK-18663][SQL] Simplify CountMinSketch aggregate imp...

2016-12-01 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/16093 LGTM except one minor point about test case. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15843: [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWr...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15843 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69476/ Test PASSed. ---

[GitHub] spark issue #15843: [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWr...

2016-12-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15843 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16100: [SPARK-18674][SQL] improve the error message of n...

2016-12-01 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/16100 [SPARK-18674][SQL] improve the error message of natural join ## What changes were proposed in this pull request? The current error message of USING join is quite confusing, for example:

[GitHub] spark issue #15831: [SPARK-18385][ML] Make the transformer's natively in ml ...

2016-12-01 Thread techaddict
Github user techaddict commented on the issue: https://github.com/apache/spark/pull/15831 @sethah @yanboliang I've started with migrating `IDF`, can you review the WIP and if i'm going in the right direction https://github.com/techaddict/spark/pull/2/files there is some code

[GitHub] spark pull request #16101: [WIP] Migrate IDF to not used mllib

2016-12-01 Thread techaddict
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/16101 [WIP] Migrate IDF to not used mllib You can merge this pull request into a Git repository by running: $ git pull https://github.com/techaddict/spark migrate-idf Alternatively you can

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15780 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-12-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90447908 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -498,7 +498,8 @@ object ScalaReflection extends

[GitHub] spark issue #16098: [WIP][SPARK-18672][CORE] Close recordwriter in SparkHado...

2016-12-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16098 Build started: [CORE] `org.apache.spark.JavaAPISuite`

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #69477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69477/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12004 **[Test build #69480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69480/consoleFull)** for PR 12004 at commit

[GitHub] spark pull request #16100: [SPARK-18674][SQL] improve the error message of n...

2016-12-01 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16100#discussion_r90460053 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/joinTypes.scala --- @@ -100,7 +100,7 @@ case class NaturalJoin(tpe: JoinType)

[GitHub] spark issue #16100: [SPARK-18674][SQL] improve the error message of natural ...

2016-12-01 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16100 cc @rxin @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16089: [SPARK-18658][SQL] Write text records directly to...

2016-12-01 Thread NathanHowell
Github user NathanHowell commented on a diff in the pull request: https://github.com/apache/spark/pull/16089#discussion_r90468563 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CodecStreams.scala --- @@ -0,0 +1,73 @@ +/* + * Licensed to the

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-12-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90447815 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala --- @@ -396,12 +396,14 @@ object JavaTypeInference {

[GitHub] spark issue #16079: [SPARK-18645][Deploy] Fix spark-daemon.sh arguments erro...

2016-12-01 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16079 Merged to master/2.1. This doesn't actually affect earlier versions because it seems to be introduced by https://issues.apache.org/jira/browse/SPARK-11653 I didn't realize the 'local' would have

[GitHub] spark issue #16098: [WIP][SPARK-18672][CORE] Close recordwriter in SparkHado...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16098 **[Test build #69475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69475/consoleFull)** for PR 16098 at commit

[GitHub] spark issue #15843: [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWr...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15843 **[Test build #69476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69476/consoleFull)** for PR 15843 at commit

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #69478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69478/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-01 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/14079 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16100: [SPARK-18674][SQL] improve the error message of natural ...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16100 **[Test build #69481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69481/consoleFull)** for PR 16100 at commit

[GitHub] spark pull request #16089: [SPARK-18658][SQL] Write text records directly to...

2016-12-01 Thread NathanHowell
Github user NathanHowell commented on a diff in the pull request: https://github.com/apache/spark/pull/16089#discussion_r90468858 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextFileFormat.scala --- @@ -132,39 +128,17 @@ class

[GitHub] spark pull request #16101: [WIP] Migrate IDF to not used mllib

2016-12-01 Thread techaddict
Github user techaddict closed the pull request at: https://github.com/apache/spark/pull/16101 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #16086: [SPARK-18653][SQL] Fix incorrect space padding fo...

2016-12-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16086#discussion_r90452307 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -275,36 +308,43 @@ class Dataset[T] private[sql]( val numCols =

[GitHub] spark pull request #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrderi...

2016-12-01 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/15480#discussion_r90454770 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala --- @@ -118,7 +118,42 @@ object

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15994 **[Test build #69471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69471/consoleFull)** for PR 15994 at commit

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12004 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12004 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69480/ Test FAILed. ---

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12004 **[Test build #69480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69480/consoleFull)** for PR 12004 at commit

[GitHub] spark issue #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #69482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69482/consoleFull)** for PR 14079 at commit

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-12-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90446944 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -171,15 +171,18 @@ case class StaticInvoke(

[GitHub] spark pull request #16086: [SPARK-18653][SQL] Fix incorrect space padding fo...

2016-12-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16086#discussion_r90449169 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -236,6 +238,37 @@ class Dataset[T] private[sql]( } } +

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #69478 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69478/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69478/ Test FAILed. ---

[GitHub] spark pull request #16086: [SPARK-18653][SQL] Fix incorrect space padding fo...

2016-12-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16086#discussion_r90448773 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -236,6 +238,37 @@ class Dataset[T] private[sql]( } } +

[GitHub] spark issue #16056: [SPARK-18623][SQL] Add `returnNullable` to `StaticInvoke...

2016-12-01 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16056 I will update the comment again after #15780 is merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69477/ Test FAILed. ---

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #69477 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69477/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #16086: [SPARK-18653][SQL] Fix incorrect space padding for unico...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16086 **[Test build #69483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69483/consoleFull)** for PR 16086 at commit

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16098 **[Test build #69475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69475/consoleFull)** for PR 16098 at commit

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-01 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/16098 close is idempotent - simply close it before 'committer.commitTask(taskContext)' and you should be done. The second close in finally will become no-op in case of successful commit, and will handle

[GitHub] spark pull request #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrderi...

2016-12-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15480#discussion_r90474669 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala --- @@ -118,7 +118,42 @@ object

[GitHub] spark pull request #16086: [SPARK-18653][SQL] Fix incorrect space padding fo...

2016-12-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16086#discussion_r90451294 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -275,36 +308,43 @@ class Dataset[T] private[sql]( val numCols =

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15994 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69471/ Test PASSed. ---

[GitHub] spark issue #15843: [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWr...

2016-12-01 Thread techaddict
Github user techaddict commented on the issue: https://github.com/apache/spark/pull/15843 @jkbradley @holdenk @viirya PR updated --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16098 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69475/ Test FAILed. ---

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16098 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15780 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69473/ Test PASSed. ---

[GitHub] spark issue #16098: [WIP][SPARK-18672][CORE] Close recordwriter in SparkHado...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16098 **[Test build #69474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69474/consoleFull)** for PR 16098 at commit

[GitHub] spark pull request #16086: [SPARK-18653][SQL] Fix incorrect space padding fo...

2016-12-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/16086#discussion_r90451485 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -275,36 +308,43 @@ class Dataset[T] private[sql]( val numCols =

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15780 **[Test build #69479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69479/consoleFull)** for PR 15780 at commit

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-12-01 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/12004 The latest patch 1. keeps the cloud package separate from hadoop-2.7. This is important avoid outstanding problems related to org.json licensed artifacts in the aws SDK JARs. The

[GitHub] spark pull request #16086: [SPARK-18653][SQL] Fix incorrect space padding fo...

2016-12-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/16086#discussion_r90465622 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -275,36 +308,43 @@ class Dataset[T] private[sql]( val numCols =

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15780 **[Test build #69473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69473/consoleFull)** for PR 15780 at commit

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-12-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90482686 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -498,7 +498,8 @@ object ScalaReflection extends

[GitHub] spark pull request #16086: [SPARK-18653][SQL] Fix incorrect space padding fo...

2016-12-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/16086#discussion_r90482897 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -236,6 +238,37 @@ class Dataset[T] private[sql]( } } +

[GitHub] spark issue #15910: [SPARK-18476][SPARKR][ML]:SparkR Logistic Regression sho...

2016-12-01 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15910 I found the ```summary``` of ```spark.logit``` return incorrect result when reviewing this PR. Actually it should return ```coefficients``` rather than binary logistic regression summary that R

[GitHub] spark pull request #16104: [SPARK-18675][SQL] CTAS for hive serde table shou...

2016-12-01 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/16104 [SPARK-18675][SQL] CTAS for hive serde table should work for all hive versions ## What changes were proposed in this pull request? Before hive 1.1, when inserting into a table, hive

[GitHub] spark issue #16089: [SPARK-18658][SQL] Write text records directly to a File...

2016-12-01 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/16089 AFAIK, the big thing the FileOutputFormat really adds is not the compression, but the output committer and the stuff to go with that (working directories, paths, etc etc). If you aren't going

[GitHub] spark pull request #15998: [SPARK-18572][SQL] Add a method `listPartitionNam...

2016-12-01 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/15998#discussion_r90503688 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClient.scala --- @@ -155,6 +155,25 @@ private[hive] trait HiveClient { }

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16098 **[Test build #69474 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69474/consoleFull)** for PR 16098 at commit

[GitHub] spark issue #16086: [SPARK-18653][SQL] Fix incorrect space padding for unico...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16086 **[Test build #69486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69486/consoleFull)** for PR 16086 at commit

[GitHub] spark pull request #16086: [SPARK-18653][SQL] Fix incorrect space padding fo...

2016-12-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/16086#discussion_r90482959 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -236,6 +238,37 @@ class Dataset[T] private[sql]( } } +

[GitHub] spark issue #16086: [SPARK-18653][SQL] Fix incorrect space padding for unico...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16086 **[Test build #69487 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69487/consoleFull)** for PR 16086 at commit

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-01 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16104 cc @yhuai @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16089: [SPARK-18658][SQL] Write text records directly to...

2016-12-01 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/16089#discussion_r90497882 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala --- @@ -194,4 +194,8 @@ private[sql] class

[GitHub] spark pull request #16089: [SPARK-18658][SQL] Write text records directly to...

2016-12-01 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/16089#discussion_r90499765 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -147,6 +147,17 @@ public void writeTo(ByteBuffer buffer) {

[GitHub] spark pull request #16089: [SPARK-18658][SQL] Write text records directly to...

2016-12-01 Thread NathanHowell
Github user NathanHowell commented on a diff in the pull request: https://github.com/apache/spark/pull/16089#discussion_r90501927 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala --- @@ -194,4 +194,8 @@ private[sql] class

[GitHub] spark issue #16086: [SPARK-18653][SQL] Fix incorrect space padding for unico...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16086 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16086: [SPARK-18653][SQL] Fix incorrect space padding for unico...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16086 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69483/ Test PASSed. ---

[GitHub] spark issue #15910: [SPARK-18476][SPARKR][ML]:SparkR Logistic Regression sho...

2016-12-01 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15910 The summary returns the same as scala side summary, including roc, areaUnderROC, pr, fMeasureByThreshold etc. I think we can add `coefficients` as additional item. --- If your project is set

[GitHub] spark issue #16037: [SPARK-18471][MLLIB] In LBFGS, avoid sending huge vector...

2016-12-01 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/16037 By the way this same issue may also impact the `ml` optimizers that use L-BFGS. We should check the various gradient aggregators for `LogisticRegression`, `LinearRegression`, `MLP` etc. cc @sethah

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 Thank you for approving, @srowen ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15780 **[Test build #69479 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69479/consoleFull)** for PR 15780 at commit

[GitHub] spark issue #15713: [SPARK-18196] [CORE] Optimise CompactBuffer implementati...

2016-12-01 Thread a-roberts
Github user a-roberts commented on the issue: https://github.com/apache/spark/pull/15713 Performance results against the Spark master branch on a 48 core machine running PageRank with 500k pages follow **Vanilla CompactBuffer, no changes, run time and throughput (bytes per

[GitHub] spark issue #16100: [SPARK-18674][SQL] improve the error message of using jo...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16100 **[Test build #69481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69481/consoleFull)** for PR 16100 at commit

[GitHub] spark pull request #16103: [SPARK-18374][ML]Incorrect words in StopWords/eng...

2016-12-01 Thread hhbyyh
GitHub user hhbyyh opened a pull request: https://github.com/apache/spark/pull/16103 [SPARK-18374][ML]Incorrect words in StopWords/english.txt ## What changes were proposed in this pull request? Currently English stop words list in MLlib contains only the argumented words

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16089: [SPARK-18658][SQL] Write text records directly to...

2016-12-01 Thread NathanHowell
Github user NathanHowell commented on a diff in the pull request: https://github.com/apache/spark/pull/16089#discussion_r90503024 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CodecStreams.scala --- @@ -0,0 +1,74 @@ +/* + * Licensed to the

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #69485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69485/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #16102: [SPARK-18586][BUILD] netty-3.8.0.Final.jar has vulnerabi...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16102 **[Test build #69484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69484/consoleFull)** for PR 16102 at commit

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-12-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90482546 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -969,6 +972,53 @@ class DatasetSuite extends QueryTest with SharedSQLContext

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-12-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90482655 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala --- @@ -396,12 +396,14 @@ object JavaTypeInference {

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-12-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90482617 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -171,15 +171,18 @@ case class StaticInvoke(

[GitHub] spark issue #16069: [WIP][SPARK-18638][BUILD] Upgrade sbt, Zinc, and Maven p...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16069 +1 for 2.1! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15780 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69479/ Test PASSed. ---

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15780 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16104 **[Test build #69490 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69490/consoleFull)** for PR 16104 at commit

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69485/ Test FAILed. ---

[GitHub] spark issue #15219: [SPARK-14098][SQL] Generate Java code to build CachedCol...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #69485 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69485/consoleFull)** for PR 15219 at commit

[GitHub] spark pull request #16089: [SPARK-18658][SQL] Write text records directly to...

2016-12-01 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/16089#discussion_r90502669 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala --- @@ -194,4 +194,8 @@ private[sql] class

[GitHub] spark issue #16103: [SPARK-18374][ML]Incorrect words in StopWords/english.tx...

2016-12-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16103 **[Test build #69489 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69489/consoleFull)** for PR 16103 at commit

[GitHub] spark issue #16103: [SPARK-18374][ML]Incorrect words in StopWords/english.tx...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16103 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16103: [SPARK-18374][ML]Incorrect words in StopWords/english.tx...

2016-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16103 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69489/ Test PASSed. ---

  1   2   3   4   5   6   7   >