[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21511900 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -54,47 +54,95 @@ private[hive] abstract class HiveFunctionRegistry

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512172 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -54,47 +54,95 @@ private[hive] abstract class HiveFunctionRegistry

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512252 --- Diff: sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala --- @@ -49,6 +49,16 @@ import

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512352 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -54,47 +54,95 @@ private[hive] abstract class HiveFunctionRegistry

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-66247996 [Test build #24242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24242/consoleFull) for PR 3247 at commit

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512453 --- Diff: sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala --- @@ -48,6 +48,17 @@ import scala.language.implicitConversions

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-66248753 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512634 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -54,47 +54,95 @@ private[hive] abstract class HiveFunctionRegistry

[GitHub] spark pull request: [SPARK-4233] [SQL] WIP:Simplify the UDAF API (...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3247#issuecomment-66248750 [Test build #24242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24242/consoleFull) for PR 3247 at commit

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512701 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -54,47 +54,95 @@ private[hive] abstract class HiveFunctionRegistry

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512696 --- Diff: sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala --- @@ -48,6 +48,17 @@ import scala.language.implicitConversions

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512868 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -54,47 +54,95 @@ private[hive] abstract class HiveFunctionRegistry

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3640#discussion_r21512962 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -54,47 +54,95 @@ private[hive] abstract class HiveFunctionRegistry

[GitHub] spark pull request: [SPARK-4793] [Deploy] ensure .jar at end of li...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3641#issuecomment-66250028 [Test build #24241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24241/consoleFull) for PR 3641 at commit

[GitHub] spark pull request: [SPARK-4793] [Deploy] ensure .jar at end of li...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3641#issuecomment-66250034 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66251380 Appreciate a lot for fixing this case! The serialization wrapper class makes sense. However, would like to make some refactoring. A summary of my comments above:

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/3642 [SPARK-4795][Core] Redesign the primitive type = Writable implicit APIs to make them be activated automatically Try to redesign the primitive type = Writable implicit APIs to make them be

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66252423 [Test build #24243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24243/consoleFull) for PR 3642 at commit

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66253299 [Test build #24244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24244/consoleFull) for PR 3642 at commit

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66259222 [Test build #24243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24243/consoleFull) for PR 3642 at commit

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66259234 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66260184 [Test build #24244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24244/consoleFull) for PR 3642 at commit

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66260189 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3891][SQL] Add array support to percent...

2014-12-09 Thread gvramana
Github user gvramana commented on a diff in the pull request: https://github.com/apache/spark/pull/2802#discussion_r21518099 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -172,6 +177,8 @@ private[hive] case class

[GitHub] spark pull request: Replace breezeSquaredDistance

2014-12-09 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/3643 Replace breezeSquaredDistance This PR replaces slow breezeSquaredDistance. A simple calculation involving 4 vectors of 2 dims shows: * breezeSquaredDistance: ~12 secs * This

[GitHub] spark pull request: Replace breezeSquaredDistance

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3643#issuecomment-66262214 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4483][SQL]Optimization about reduce mem...

2014-12-09 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/3375#discussion_r21519366 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashOuterJoin.scala --- @@ -68,62 +68,59 @@ case class HashOuterJoin(

[GitHub] spark pull request: [SPARK-4483][SQL]Optimization about reduce mem...

2014-12-09 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/3375#discussion_r21519410 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashOuterJoin.scala --- @@ -68,62 +68,59 @@ case class HashOuterJoin(

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/3642#discussion_r21520954 --- Diff: graphx/src/test/scala/org/apache/spark/graphx/lib/ShortestPathsSuite.scala --- @@ -40,7 +40,7 @@ class ShortestPathsSuite extends FunSuite with

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66268347 [Test build #24245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24245/consoleFull) for PR 3642 at commit

[GitHub] spark pull request: [SPARK-3891][SQL] Add array support to percent...

2014-12-09 Thread gvramana
Github user gvramana commented on a diff in the pull request: https://github.com/apache/spark/pull/2802#discussion_r21524226 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -165,6 +165,11 @@ private[hive] case class

[GitHub] spark pull request: [SPARK-3891][SQL] Add array support to percent...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2802#issuecomment-66274239 [Test build #24246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24246/consoleFull) for PR 2802 at commit

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66277314 [Test build #24245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24245/consoleFull) for PR 3642 at commit

[GitHub] spark pull request: [SPARK-4795][Core] Redesign the primitive typ...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3642#issuecomment-66277323 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4793] [Deploy] ensure .jar at end of li...

2014-12-09 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3641#issuecomment-66278031 LGTM. Are there other instances like this in other scripts? couldn't hurt to attach `$` elsewhere where the pattern is clearly matching at the end of a filename. Worth a

[GitHub] spark pull request: [SPARK-4792] Add error message when making loc...

2014-12-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3635#discussion_r21526216 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala --- @@ -67,11 +67,29 @@ private[spark] class DiskBlockManager(blockManager:

[GitHub] spark pull request: [SPARK-4792] Add error message when making loc...

2014-12-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3635#discussion_r21526232 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala --- @@ -67,11 +67,29 @@ private[spark] class DiskBlockManager(blockManager:

[GitHub] spark pull request: [SPARK-2096][SQL] support dot notation on arbi...

2014-12-09 Thread sziep
Github user sziep commented on the pull request: https://github.com/apache/spark/pull/2405#issuecomment-66280104 are there any plans on merging this soon? This is a pretty useful feature. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-2096][SQL] support dot notation on arbi...

2014-12-09 Thread ayoub-benali
Github user ayoub-benali commented on the pull request: https://github.com/apache/spark/pull/2405#issuecomment-66280190 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3891][SQL] Add array support to percent...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2802#issuecomment-66281289 [Test build #24246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24246/consoleFull) for PR 2802 at commit

[GitHub] spark pull request: [SPARK-3891][SQL] Add array support to percent...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2802#issuecomment-66281294 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2096][SQL] support dot notation on arbi...

2014-12-09 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/2405#issuecomment-66282781 This PR is blocked by https://github.com/apache/spark/pull/2543. I'll update the code tomorrow and make it work :) --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-4792] Add error message when making loc...

2014-12-09 Thread XuTingjun
Github user XuTingjun commented on the pull request: https://github.com/apache/spark/pull/3635#issuecomment-66283335 @srowen , thank you for your suggestion, I have modified the method. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-4789] [mllib] Standardize ML Prediction...

2014-12-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3637#discussion_r21527840 --- Diff: mllib/src/main/scala/org/apache/spark/ml/LabeledPoint.scala --- @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-4792] Add error message when making loc...

2014-12-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3635#discussion_r21527997 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala --- @@ -67,11 +67,20 @@ private[spark] class DiskBlockManager(blockManager:

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66286027 [Test build #24247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24247/consoleFull) for PR 3640 at commit

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66287710 [Test build #24248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24248/consoleFull) for PR 3640 at commit

[GitHub] spark pull request: [SPARK-4483][SQL]Optimization about reduce mem...

2014-12-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3375#discussion_r21529898 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashOuterJoin.scala --- @@ -68,62 +68,59 @@ case class HashOuterJoin(

[GitHub] spark pull request: [SPARK-3891][SQL] Add array support to percent...

2014-12-09 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/2802#discussion_r21530532 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -172,6 +177,8 @@ private[hive] case class

[GitHub] spark pull request: [SPARK-4699][SQL] make caseSensitive configura...

2014-12-09 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/3558#issuecomment-66291005 That's a good point, Can we make another interface that SqlConf inherit from? Or can we move the SqlConf into the module `Catalyst`? Any other idea? --- If your

[GitHub] spark pull request: [SPARK-4793] [Deploy] ensure .jar at end of li...

2014-12-09 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/3641#issuecomment-66291537 I have checked those files and found another on https://github.com/adrian-wang/spark/blob/jar/bin/compute-classpath.sh#L111 Do you think I should also change

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2014-12-09 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/3607#discussion_r21531881 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -39,23 +39,34 @@ private[spark] class ClientArguments(args:

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread akopich
Github user akopich commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66292305 @jkbradley Tests fail again... Stab in the dark: looks like something is changed in the testing environment. (2) Regular and Robust in the same

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66295835 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66295827 [Test build #24247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24247/consoleFull) for PR 3640 at commit

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread chazchandler
Github user chazchandler commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66296113 re: (5) you can add a remote: `git remote add upstream https://github.com/apache/spark.git` fetch the latest state: `git fetch upstream`

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66297896 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66297888 [Test build #24248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24248/consoleFull) for PR 3640 at commit

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread akopich
Github user akopich commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66299321 @chazchandler, thank you very much for your quick reply! It did the trick. Now I'm a bit confused about ml/ folder. What's it for? --- If your project is set

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread akopich
Github user akopich commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66306112 It seems like something went wrong. I've got multiple compilation errors like ``` [error]

[GitHub] spark pull request: [SPARK-4483][SQL]Optimization about reduce mem...

2014-12-09 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/3375#discussion_r21538764 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashOuterJoin.scala --- @@ -68,62 +68,59 @@ case class HashOuterJoin(

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/3644 [SPARK-4798][SQL] A new set of Parquet testing API and test suites This PR provides a set Parquet testing API (see trait `ParquetTest`) that enables developers to write more concise test cases. A

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3644#discussion_r21538934 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala --- @@ -181,6 +181,10 @@ private[sql] trait SQLConf { */ def

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66309267 [Test build #24249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24249/consoleFull) for PR 3644 at commit

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread karlhigley
Github user karlhigley commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66309244 Re: (2) Regular and Robust in the same class It's possible to implement, but I don't want to turn class hierarchy inside out. It just violates OOP principles.

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread chazchandler
Github user chazchandler commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66311502 @akopich , rebasing can be tricky, especially if you've been off on a branch for a while. `git reflog` can be helpful in getting back to previous states if you end

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66312060 [Test build #24250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24250/consoleFull) for PR 3644 at commit

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66314310 [Test build #24250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24250/consoleFull) for PR 3644 at commit

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66314319 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2309][MLlib] Generalize the binary logi...

2014-12-09 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1379#issuecomment-66318526 @dbtsai 1) Could you elaborate on what kind of optimizations did you do? Probably, they could be applied to the broader MLlib, which is beneficial. 2) Do you know the

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66319557 The current design of this PR is derived from some background knowledges. I'd like to provide a brief summary here for future reference. As mentioned in the PR

[GitHub] spark pull request: [SPARK-2309][MLlib] Generalize the binary logi...

2014-12-09 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1379#issuecomment-66320878 @jkbradley Thank you! They took some time. - I totally agree with you, I need to perform tests on the original test set. It contains less attributes, i.e. 778 vs

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread akopich
Github user akopich commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66321327 @karlhigley, yes I've heard something about abstract classes. Though, I see no way to employ this concept here. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66321392 [Test build #24249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24249/consoleFull) for PR 3644 at commit

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66321407 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66324190 Although it passed Jenkins, the first failure is rather weird. It seems that partitions collected via `SchemaRDD.collect()` can sometimes be out of order. --- If

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66324222 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-09 Thread smola
GitHub user smola opened a pull request: https://github.com/apache/spark/pull/3645 [SPARK-4799] Use IP address instead of local hostname in ConnectionManager See https://issues.apache.org/jira/browse/SPARK-4799 Spark fails when a node hostname is not resolvable by

[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3645#issuecomment-66324983 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66325520 [Test build #24251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24251/consoleFull) for PR 3644 at commit

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread akopich
Github user akopich commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66325635 @chazchandler, thank you very much for your help. I shouldn't have rebase on master. Rebase on 1.2 was successful. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [MLLIB][SPARK-4675] Find similar products and ...

2014-12-09 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/3536#issuecomment-66331985 I'd agree that cosine similarity is preferred. Can't really think of a case where I've *not* used cosine sim for a similar items or similar users computation. Of course,

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3640 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66332833 Thanks a lot guys for digging into this! Merged to master and branch 1.2 --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3567#issuecomment-66333686 Jenkins, test this please. LGTM pending tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3567#issuecomment-66334417 [Test build #24252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24252/consoleFull) for PR 3567 at commit

[GitHub] spark pull request: [SPARK-4791] [sql] Infer schema from case clas...

2014-12-09 Thread jkbradley
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/3646 [SPARK-4791] [sql] Infer schema from case class with multiple constructors Modified ScalaReflection.schemaFor to take primary constructor of Product when there are multiple constructors. Added

[GitHub] spark pull request: [SPARK-2309][MLlib] Generalize the binary logi...

2014-12-09 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/1379#issuecomment-66336110 @avulanov 1. I did the same optimization for MLlib in [my recently PRs](https://github.com/apache/spark/commits/master?author=dbtsai). * Accessing the

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66336150 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66336134 [Test build #24251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24251/consoleFull) for PR 3644 at commit

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3215#issuecomment-66336371 We actually don't need to wait on the RC to merge this since this is only going into master. I'll take a quick look and will likely merge it after that since this is

[GitHub] spark pull request: [SPARK-4791] [sql] Infer schema from case clas...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3646#issuecomment-66336585 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4791] [sql] Infer schema from case clas...

2014-12-09 Thread KirthiRaman
Github user KirthiRaman commented on the pull request: https://github.com/apache/spark/pull/3646#issuecomment-66336768 unsubscribe On Tue, Dec 9, 2014 at 1:57 PM, UCB AMPLab notificati...@github.com wrote: Test FAILed. Refer to this link for build results

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3215#discussion_r21552599 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -178,21 +178,25 @@ private[spark] class ClientArguments(args:

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3215#discussion_r21552652 --- Diff: yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClientImpl.scala --- @@ -1,118 +0,0 @@ -/* - * Licensed to the Apache

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3215#issuecomment-66337338 Hey this LGTM. I'm just gonna merge this. My comments are minor enough that we can slip them in later. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3215 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3607#issuecomment-66338560 By the way you will need to rebase to master since we just removed the support for yarn alpha. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3607#discussion_r21553460 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -39,23 +39,34 @@ private[spark] class

[GitHub] spark pull request: [SPARK-4461][YARN] pass extra java options to ...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3409#issuecomment-66339357 Hey regarding the naming, I would actually prefer `spark.yarn.am.*` over `spark.yarn.clientmode.am.*`. Although I agree that it's clearer to have clientmode in there,

  1   2   3   4   5   >