[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-01-18 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/4087#issuecomment-70446766 [~leahmcguire], Thanks for the patch! A few comments: 1. PySpark calls the Scala API for MLlib, so for API compatibility, we can't use enumerations

[GitHub] spark pull request: [SQL] fix typo in class description

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4100#issuecomment-70453298 [Test build #25746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25746/consoleFull) for PR 4100 at commit

[GitHub] spark pull request: SPARK-5217 Spark UI should report pending stag...

2015-01-18 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/4043#issuecomment-70453703 @pwendell - patch updated to latest master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4098#issuecomment-70453833 [Test build #25740 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25740/consoleFull) for PR 4098 at commit

[GitHub] spark pull request: Bug fix for SPARK-5242: ec2/spark_ec2.py lauc...

2015-01-18 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/4038#issuecomment-70442384 cc @shivaram I haven't had a chance to look at this more closely yet, and likely won't until next weekend. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/3946#discussion_r23142124 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suite.scala --- @@ -384,4 +388,32 @@ class

[GitHub] spark pull request: [SPARK-3650] Fix TriangleCount handling of rev...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2495#issuecomment-70447973 [Test build #25737 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25737/consoleFull) for PR 2495 at commit

[GitHub] spark pull request: [SPARK-3650] Fix TriangleCount handling of rev...

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2495#issuecomment-70447975 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70448381 [Test build #25738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25738/consoleFull) for PR 3897 at commit

[GitHub] spark pull request: [SPARK-4001][MLlib] adding apriori and fp-grow...

2015-01-18 Thread jackylk
Github user jackylk commented on the pull request: https://github.com/apache/spark/pull/2847#issuecomment-70450975 Yes, I have tested the parallel FP-Growth algorithm using a open data set from http://fimi.ua.ac.be/data/, performance test result can be found at

[GitHub] spark pull request: [SPARK-4699][SQL] make caseSensitive configura...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3558#issuecomment-70456370 [Test build #25751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25751/consoleFull) for PR 3558 at commit

[GitHub] spark pull request: [SPARK-4699][SQL] make caseSensitive configura...

2015-01-18 Thread jackylk
Github user jackylk commented on the pull request: https://github.com/apache/spark/pull/3558#issuecomment-70456295 I have updated the code based on SPARK-3965 (SPARK-5168) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70446535 [Test build #25735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25735/consoleFull) for PR 3946 at commit

[GitHub] spark pull request: [SPARK-5278][SQL] complete the check of ambigu...

2015-01-18 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/4068#issuecomment-70447398 The problem is that: Currently the `GetField` class is an operation which picks the first field whose name equal to the required `fieldName` with case sensitive. As I

[GitHub] spark pull request: [SQL] fix typo in class description

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4100#issuecomment-70452986 [Test build #25744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25744/consoleFull) for PR 4100 at commit

[GitHub] spark pull request: [SPARK-4961] [CORE] Put HadoopRDD.getPartition...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3794#issuecomment-70452982 [Test build #25745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25745/consoleFull) for PR 3794 at commit

[GitHub] spark pull request: [SQL] fix typo in class description

2015-01-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4100#issuecomment-70457188 Thanks. Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5282][mllib]: RowMatrix easily gets int...

2015-01-18 Thread hhbyyh
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/4069#issuecomment-70441745 @srowen Would you mind to take another look? Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-4908][SQL]narrow the scope of synchroni...

2015-01-18 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/4001#issuecomment-70441804 `HiveShim.getCommandProcess` delegates to methods defined in `CommandProcessorFactory`, which tries to find a cached `Driver` object and initialize it. The underlying

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70445416 [Test build #25734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25734/consoleFull) for PR 3897 at commit

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread tianyi
Github user tianyi commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70446387 rebased from latest master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2848] Shade Guava in uber-jars.

2015-01-18 Thread mfawzymkh
Github user mfawzymkh commented on the pull request: https://github.com/apache/spark/pull/1813#issuecomment-70446580 do we have an ETA to get this pull request merged to master? The guava shading issue is causing a problem for client libs that has a dependency on swift-service when

[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-01-18 Thread rnowling
Github user rnowling commented on a diff in the pull request: https://github.com/apache/spark/pull/4087#discussion_r23142812 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala --- @@ -75,9 +106,12 @@ class NaiveBayesModel private[mllib] ( *

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70449375 [Test build #25733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25733/consoleFull) for PR 3946 at commit

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70449381 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5186] [MLLIB] Vector.equals and Vector....

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3997#issuecomment-70449484 [Test build #25739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25739/consoleFull) for PR 3997 at commit

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/4098 [SPARK-5307] SerializationDebugger - take 2 This patch adds a SerializationDebugger that is used to add serialization path to a NotSerializableException. When a NotSerializableException is

[GitHub] spark pull request: [SPARK-4987] [SQL] parquet timestamp type supp...

2015-01-18 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/3820#issuecomment-70450284 I've tested this PR but the result seems to be off. Parquet generated from Hive with timestamp values set by 'from_utc_timestamp('1970-01-01 08:00:00','PST')'

[GitHub] spark pull request: [SPARK-4001][MLlib] adding apriori and fp-grow...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2847#issuecomment-70450297 [Test build #25742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25742/consoleFull) for PR 2847 at commit

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4098#issuecomment-70450293 [Test build #25741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25741/consoleFull) for PR 4098 at commit

[GitHub] spark pull request: [SQL] fix typo in class description

2015-01-18 Thread jackylk
GitHub user jackylk opened a pull request: https://github.com/apache/spark/pull/4100 [SQL] fix typo in class description You can merge this pull request into a Git repository by running: $ git pull https://github.com/jackylk/spark patch-9 Alternatively you can review and

[GitHub] spark pull request: [SPARK-5278][SQL] complete the check of ambigu...

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4068#issuecomment-70454202 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5297][Streaming] Fix Java file stream t...

2015-01-18 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/4101 [SPARK-5297][Streaming] Fix Java file stream type erasure problem Current Java file stream doesn't support custom key/value type because of loss of type information, details can be seen in

[GitHub] spark pull request: [SPARK-5278][SQL] complete the check of ambigu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4068#issuecomment-70454200 [Test build #25747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25747/consoleFull) for PR 4068 at commit

[GitHub] spark pull request: [SPARK-3586][streaming]Support nested director...

2015-01-18 Thread wangxiaojing
Github user wangxiaojing commented on the pull request: https://github.com/apache/spark/pull/2765#issuecomment-70437483 @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-01-18 Thread rnowling
Github user rnowling commented on a diff in the pull request: https://github.com/apache/spark/pull/4087#discussion_r23142620 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala --- @@ -75,9 +106,12 @@ class NaiveBayesModel private[mllib] ( *

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4098#issuecomment-70449497 Link to the earlier attempt: https://github.com/apache/spark/pull/4093 by me and https://github.com/apache/spark/issues/3518 by @ilganeli --- If your project is set

[GitHub] spark pull request: [SPARK-5022] [Sql] Change VectorUDT to object

2015-01-18 Thread MechCoder
GitHub user MechCoder opened a pull request: https://github.com/apache/spark/pull/4099 [SPARK-5022] [Sql] Change VectorUDT to object You can merge this pull request into a Git repository by running: $ git pull https://github.com/MechCoder/spark spark-5022 Alternatively you

[GitHub] spark pull request: [SPARK-5022] [Sql] Change VectorUDT to object

2015-01-18 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/4099#issuecomment-70450637 cc @rxin I am unable to understand how to change this line `@SQLUserDefinedType(udt = classOf[VectorUDT])` . I tried doing `@SQLUserDefinedType(udt =

[GitHub] spark pull request: SPARK-2630 Input data size of CoalescedRDD cou...

2015-01-18 Thread ash211
Github user ash211 commented on the pull request: https://github.com/apache/spark/pull/2310#issuecomment-70440912 Sounds good, I concur. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5257] [MLlib] SparseVector indices must...

2015-01-18 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/4096#issuecomment-70441989 Alright, but maybe the documentation can be updated that the indices should be non-negative? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: use defaultParallelism for defaultMinPartition...

2015-01-18 Thread idanz
Github user idanz commented on the pull request: https://github.com/apache/spark/pull/4094#issuecomment-70443024 I see, I don't want to repeat old discussions so to be more pragmatic, the real problem for me is to set the partition size when using sparksql. My cluster uses

[GitHub] spark pull request: [WIP][SPARK-4131][SQL] Writing data into the f...

2015-01-18 Thread nieldomingo
Github user nieldomingo commented on the pull request: https://github.com/apache/spark/pull/2997#issuecomment-70443145 this would really help me --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3218, SPARK-3219, SPARK-3261, SPARK-342...

2015-01-18 Thread derrickburns
Github user derrickburns commented on the pull request: https://github.com/apache/spark/pull/2634#issuecomment-70443890 @mengxr I have implemented several variants of Kullback-Leibler divergence in my separate GitHub repository

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70444727 [Test build #25733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25733/consoleFull) for PR 3946 at commit

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread jongyoul
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70444764 Rebase is not finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-01-18 Thread rnowling
Github user rnowling commented on a diff in the pull request: https://github.com/apache/spark/pull/4087#discussion_r23142579 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala --- @@ -32,28 +42,42 @@ import org.apache.spark.rdd.RDD * @param

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread jongyoul
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70445944 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread jongyoul
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70455615 @mateiz I've rebased this PR and finished tests successfully. Merge this, please. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-5278][SQL] complete the check of ambigu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4068#issuecomment-70455616 [Test build #25750 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25750/consoleFull) for PR 4068 at commit

[GitHub] spark pull request: [SPARK-5278][SQL] complete the check of ambigu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4068#issuecomment-70456466 [Test build #25750 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25750/consoleFull) for PR 4068 at commit

[GitHub] spark pull request: [SPARK-5278][SQL] complete the check of ambigu...

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4068#issuecomment-70456470 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-01-18 Thread rnowling
Github user rnowling commented on a diff in the pull request: https://github.com/apache/spark/pull/4087#discussion_r23142512 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala --- @@ -32,28 +42,42 @@ import org.apache.spark.rdd.RDD * @param

[GitHub] spark pull request: [SPARK-4894][mllib] Added Bernoulli option to ...

2015-01-18 Thread rnowling
Github user rnowling commented on a diff in the pull request: https://github.com/apache/spark/pull/4087#discussion_r23142533 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala --- @@ -32,28 +42,42 @@ import org.apache.spark.rdd.RDD * @param

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70445835 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70445832 [Test build #25732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25732/consoleFull) for PR 3897 at commit

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70446696 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70446695 [Test build #25735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25735/consoleFull) for PR 3946 at commit

[GitHub] spark pull request: [SPARK-3650] Fix TriangleCount handling of rev...

2015-01-18 Thread ankurdave
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/2495#issuecomment-70447655 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3650] Fix TriangleCount handling of rev...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2495#issuecomment-70447688 [Test build #25737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25737/consoleFull) for PR 2495 at commit

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70447740 [Test build #25734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25734/consoleFull) for PR 3897 at commit

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70447746 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70452235 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70452231 [Test build #25738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25738/consoleFull) for PR 3897 at commit

[GitHub] spark pull request: [SPARK-5186] [MLLIB] Vector.equals and Vector....

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3997#issuecomment-70453518 [Test build #25739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25739/consoleFull) for PR 3997 at commit

[GitHub] spark pull request: [SPARK-5186] [MLLIB] Vector.equals and Vector....

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3997#issuecomment-70453523 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5278][SQL] complete the check of ambigu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4068#issuecomment-70453627 [Test build #25747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25747/consoleFull) for PR 4068 at commit

[GitHub] spark pull request: SPARK-5217 Spark UI should report pending stag...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4043#issuecomment-70453629 [Test build #25748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25748/consoleFull) for PR 4043 at commit

[GitHub] spark pull request: [SPARK-4984][CORE][WEBUI] Adding a pop-up cont...

2015-01-18 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/3819#issuecomment-70437399 Hmm, agree with you, but have not found a easy way to spot truncated description. if we add `...` for truncated decs, we will consider the case of window scaling and

[GitHub] spark pull request: [SPARK-5088] Use spark-class for running execu...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3897#issuecomment-70441182 [Test build #25732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25732/consoleFull) for PR 3897 at commit

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4098#issuecomment-70449764 [Test build #25740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25740/consoleFull) for PR 4098 at commit

[GitHub] spark pull request: [SPARK-5022] [Sql] Change VectorUDT to object

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4099#issuecomment-70451197 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5022] [Sql] Change VectorUDT to object

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4099#issuecomment-70451196 [Test build #25743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25743/consoleFull) for PR 4099 at commit

[GitHub] spark pull request: [SPARK-4001][MLlib] adding apriori and fp-grow...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2847#issuecomment-70452828 [Test build #25742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25742/consoleFull) for PR 2847 at commit

[GitHub] spark pull request: [SPARK-4001][MLlib] adding apriori and fp-grow...

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2847#issuecomment-70452833 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4098#issuecomment-70453837 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/4098#discussion_r23146186 --- Diff: core/src/test/scala/org/apache/spark/serializer/SerializationDebuggerSuite.scala --- @@ -0,0 +1,140 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70447201 [Test build #25736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25736/consoleFull) for PR 3946 at commit

[GitHub] spark pull request: use defaultParallelism for defaultMinPartition...

2015-01-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4094#issuecomment-70448190 Hey @idanz first of all, we should add some comments to the code referencing SPARK-822, so that we don't go through this all over again for the core Spark API.

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70450761 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5100][SQL] add thriftserver-ui support

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3946#issuecomment-70450760 [Test build #25736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25736/consoleFull) for PR 3946 at commit

[GitHub] spark pull request: [SPARK-5022] [Sql] Change VectorUDT to object

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4099#issuecomment-70450843 [Test build #25743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25743/consoleFull) for PR 4099 at commit

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4098#issuecomment-70454498 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger - take 2

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4098#issuecomment-70454492 [Test build #25741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25741/consoleFull) for PR 4098 at commit

[GitHub] spark pull request: [SPARK-5297][Streaming] Fix Java file stream t...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4101#issuecomment-70454568 [Test build #25749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25749/consoleFull) for PR 4101 at commit

[GitHub] spark pull request: [SQL][Minor] Refactors deeply nested FP style ...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4091#issuecomment-70400460 [Test build #25718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25718/consoleFull) for PR 4091 at commit

[GitHub] spark pull request: [SPARK-5186] [MLLIB] Vector.equals and Vector....

2015-01-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3997#issuecomment-70400053 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5186] [MLLIB] Vector.equals and Vector....

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3997#issuecomment-70400050 [Test build #25714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25714/consoleFull) for PR 3997 at commit

[GitHub] spark pull request: [SPARK-5307] SerializationDebugger to help deb...

2015-01-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4093#issuecomment-70400534 [Test build #25716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25716/consoleFull) for PR 4093 at commit

[GitHub] spark pull request: [SPARK-1405] [mllib] Latent Dirichlet Allocati...

2015-01-18 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4047#issuecomment-70434374 @hhbyyh Yes, please review the design doc linked from the JIRA. There is quite a bit of functionality which will not be in this initial PR. --- If your project is

[GitHub] spark pull request: [SQL][minor] Put DataTypes.java in java dir.

2015-01-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4097 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...

2015-01-18 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3571#issuecomment-70435538 Hi @jacek-lewandowski, Thanks for bringing this up to date. I took a quick pass through and left some minor comments. Just to clarify: this only adds

[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...

2015-01-18 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/4074#discussion_r23139343 --- Diff: core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala --- @@ -436,6 +436,12 @@ trait JavaRDDLike[T, This : JavaRDDLike[T, This]]

[GitHub] spark pull request: SPARK-5270 [CORE] Elegantly check if RDD is em...

2015-01-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4074#issuecomment-70435986 LGTM @srowen - are you still working on it or is it good from your end? Will leave a bit of time for others to comment as well. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-5249] Added type specific set functions...

2015-01-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4042#issuecomment-70436107 Okay - @AdamGS thanks for sending this patch but I think we'll pass on adding this API. Overall we're pretty conservative with adding API's like this if there isn't a

[GitHub] spark pull request: [SPARK-5249] Added type specific set functions...

2015-01-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4042#issuecomment-70436115 Let's close this issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4920][UI]: back port the PR-3763 to bra...

2015-01-18 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/3768 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: use defaultParallelism for defaultMinPartition...

2015-01-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4094#issuecomment-70436485 Yeah, this has always been broken. What's even more confusing is what Hadoop actually does with this minSplits if you trace the code through Hadoop - I remember looking

[GitHub] spark pull request: use defaultParallelism for defaultMinPartition...

2015-01-18 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4094#issuecomment-70436546 Here's some links to prior discussions of this: - https://issues.apache.org/jira/browse/SPARK-822 - https://github.com/mesos/spark/pull/718 --- If your

[GitHub] spark pull request: [SPARK-5257] [MLlib] SparseVector indices must...

2015-01-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4096#issuecomment-70436617 @MechCoder Similar to #3791, this will significantly hurt performance. Having indices being nonnegative and ordered is a contract. If you want to ensure these, please use

  1   2   >