[GitHub] spark pull request: Spark 1442 step 1

2014-10-27 Thread guowei2
GitHub user guowei2 opened a pull request: https://github.com/apache/spark/pull/2953 Spark 1442 step 1 You can merge this pull request into a Git repository by running: $ git pull https://github.com/guowei2/spark SPARK-1442-STEP-1 Alternatively you can review and apply these

[GitHub] spark pull request: [SPARK-4030] Make destroy public for broadcast...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2922#issuecomment-60553565 [Test build #22277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22277/consoleFull) for PR 2922 at commit

[GitHub] spark pull request: [SPARK-1442] [SQL] window function implement

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2953#issuecomment-60553550 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-1442] [SQL] window function implement

2014-10-27 Thread guowei2
Github user guowei2 commented on the pull request: https://github.com/apache/spark/pull/2953#issuecomment-60553573 aims of step1 ( this pr): 1、support parse sql with complex window define 2、support most of aggregate-functions with window-spec 3、support window range

[GitHub] spark pull request: [SPARK-4087] use broadcast for task only when ...

2014-10-27 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2933#issuecomment-60553581 @JoshRosen I think we still have it (in tests at tonight): ``` [info] org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 11.0

[GitHub] spark pull request: [SPARK-4030] Make destroy public for broadcast...

2014-10-27 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/2922#issuecomment-60553667 @pwendell - I made destroy blocking by default and only made that version public (its not clear we need the non-blocking version to also be public -- we can add it

[GitHub] spark pull request: [SPARK-4087] use broadcast for task only when ...

2014-10-27 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2933#issuecomment-60553702 This is really strange; I thought that the unexpected exception type would have been addressed by https://github.com/apache/spark/pull/2932 --- If your project is set

[GitHub] spark pull request: [SPARK-3795] Heuristics for dynamically scalin...

2014-10-27 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/2746#discussion_r19389557 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3795] Heuristics for dynamically scalin...

2014-10-27 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/2746#discussion_r19389583 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4087] use broadcast for task only when ...

2014-10-27 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2933#issuecomment-60553953 Can you point me to the commit that produced that stacktrace? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4095][YARN][Minor]extract val isLaunchi...

2014-10-27 Thread WangTaoTheTonic
GitHub user WangTaoTheTonic opened a pull request: https://github.com/apache/spark/pull/2954 [SPARK-4095][YARN][Minor]extract val isLaunchingDriver in ClientBase Instead of checking if `args.userClass` is null repeatedly, we extract it to an global val as in `ApplicationMaster`.

[GitHub] spark pull request: [SPARK-4087] use broadcast for task only when ...

2014-10-27 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2933#issuecomment-60554225 @JoshRosen @pwendell The test branch (internal) did not have that commit. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-4095][YARN][Minor]extract val isLaunchi...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2954#issuecomment-60554332 [Test build #22278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22278/consoleFull) for PR 2954 at commit

[GitHub] spark pull request: [SPARK-3904] [SQL] add constant objectinspecto...

2014-10-27 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2762#issuecomment-60554357 test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4030] Make destroy public for broadcast...

2014-10-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2922#issuecomment-60554586 Oh right yeah. Great, LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-60554724 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-60554714 [Test build #22273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22273/consoleFull) for PR 2716 at commit

[GitHub] spark pull request: Streaming KMeans [MLLIB][SPARK-3254]

2014-10-27 Thread anantasty
Github user anantasty commented on the pull request: https://github.com/apache/spark/pull/2942#issuecomment-60554980 Should we create another PR for the python bindings/example? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-27 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2771#issuecomment-6072 test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread WangTaoTheTonic
GitHub user WangTaoTheTonic opened a pull request: https://github.com/apache/spark/pull/2955 [SPARK-4096][YARN]Update executor memory description in the help message Here `ApplicationMaster` accept executor memory argument only in number format, we should update the description in

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60555883 [Test build #22279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22279/consoleFull) for PR 2955 at commit

[GitHub] spark pull request: [SPARK-4094][CORE] checkpoint should still be ...

2014-10-27 Thread liyezhang556520
GitHub user liyezhang556520 opened a pull request: https://github.com/apache/spark/pull/2956 [SPARK-4094][CORE] checkpoint should still be available after any rdd actions JIRA URL: [SPARK-4094](https://issues.apache.org/jira/browse/SPARK-4094) You can merge this pull request into

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2771#issuecomment-60555869 [Test build #22280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22280/consoleFull) for PR 2771 at commit

[GitHub] spark pull request: [SPARK-3904] [SQL] add constant objectinspecto...

2014-10-27 Thread gvramana
Github user gvramana commented on the pull request: https://github.com/apache/spark/pull/2762#issuecomment-60556024 Yes, one more advantage of passing object inspector as parameter is, constants need not be allocated every time, same value in constantObjectInpector can be reused. I

[GitHub] spark pull request: [SPARK-4094][CORE] checkpoint should still be ...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2956#issuecomment-60556120 [Test build #22281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22281/consoleFull) for PR 2956 at commit

[GitHub] spark pull request: [Spark 3922] Refactor spark-core to use Utils....

2014-10-27 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/2781#issuecomment-60556812 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60556909 Yeah, I like that idea. On Oct 26, 2014 8:43 PM, wangfei notificati...@github.com wrote: @marmbrus https://github.com/marmbrus, how about make a new sub

[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-60556908 [Test build #22275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22275/consoleFull) for PR 2753 at commit

[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2753#issuecomment-60556917 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3988][SQL] add public API for date type

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2901#issuecomment-60556986 [Test build #22274 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22274/consoleFull) for PR 2901 at commit

[GitHub] spark pull request: [SPARK-3988][SQL] add public API for date type

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2901#issuecomment-60556992 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4030] Make destroy public for broadcast...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2922#issuecomment-60557108 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4030] Make destroy public for broadcast...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2922#issuecomment-60557102 [Test build #22277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22277/consoleFull) for PR 2922 at commit

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-27 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60557917 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4095][YARN][Minor]extract val isLaunchi...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2954#issuecomment-60558127 [Test build #22278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22278/consoleFull) for PR 2954 at commit

[GitHub] spark pull request: [SPARK-4095][YARN][Minor]extract val isLaunchi...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2954#issuecomment-60558131 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2771#issuecomment-60558681 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2771#issuecomment-60558676 [Test build #22280 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22280/consoleFull) for PR 2771 at commit

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60558850 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60558845 [Test build #22279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22279/consoleFull) for PR 2955 at commit

[GitHub] spark pull request: [SPARK-4094][CORE] checkpoint should still be ...

2014-10-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2956#discussion_r19391269 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1204,6 +1204,8 @@ abstract class RDD[T: ClassTag]( } else if

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60558978 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60559230 It seems much more desirable to just support 3g or 200m in this argument, as was intended. `Utils.memoryStringToMb` can do the conversion. --- If your project is set up

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60559246 [Test build #22282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22282/consoleFull) for PR 2955 at commit

[GitHub] spark pull request: [SPARK-3997][Build]scalastyle should output th...

2014-10-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2846#issuecomment-60559452 The `awk` command here is still a little wrong, as it matches any string `error`. `grep \[error\]` definitely works on `scalastyle.txt` to match `[error]`. There's a

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-27 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60559682 Sorry, are we going to support both Hive 0.12 0.13.1 in long term? I am working on SerDe stuff #2570, seems lots of method signatures changed after upgrading to

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60559959 @srowen I thought this way at beginning, but after more dig I found in `ClientBase.scala` there is an initilization of `ApplicationMaster` using arguments from

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-60560204 [Test build #22283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22283/consoleFull) for PR 2570 at commit

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-27 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-60560261 @marmbrus, I've rebase dthe code with latest master (with Hive 0.13.1 supported, but not compatible with Hive 0.12). Please let me know if you have concerns on

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-60560430 [Test build #22283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22283/consoleFull) for PR 2570 at commit

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-60560431 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4094][CORE] checkpoint should still be ...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2956#issuecomment-60560699 [Test build #22281 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22281/consoleFull) for PR 2956 at commit

[GitHub] spark pull request: [SPARK-4094][CORE] checkpoint should still be ...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2956#issuecomment-60560702 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4097] Fix the race condition of 'thread...

2014-10-27 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/2957 [SPARK-4097] Fix the race condition of 'thread' There is a chance that `thread` is null when calling `thread.interrupt()`. ```Scala override def cancel(): Unit = this.synchronized {

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392124 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -556,6 +556,8 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392140 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/RangeJoins.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392150 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/RangeJoins.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392165 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/RangeJoinImpl.scala --- @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392161 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/RangeJoinImpl.scala --- @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392217 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala --- @@ -188,8 +191,18 @@ class SqlParser extends AbstractSparkSQLParser {

[GitHub] spark pull request: [SPARK-4097] Fix the race condition of 'thread...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2957#issuecomment-60561668 [Test build #22284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22284/consoleFull) for PR 2957 at commit

[GitHub] spark pull request: [MLLIB] SPARK-1547: Adding Gradient Boosting t...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2607#issuecomment-60561667 [Test build #22285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22285/consoleFull) for PR 2607 at commit

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392262 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/RangeJoinImpl.scala --- @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392296 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/RangeJoins.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392285 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/RangeJoins.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392318 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/RangeJoins.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392340 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -102,6 +102,13 @@ private[sql] abstract class SparkStrategies

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392360 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLRangeJoinSuite.scala --- @@ -0,0 +1,81 @@ +/** + * Licensed to Big Data Genomics

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392347 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLRangeJoinSuite.scala --- @@ -0,0 +1,81 @@ +/** + * Licensed to Big Data Genomics

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2939#discussion_r19392368 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLRangeJoinSuite.scala --- @@ -0,0 +1,81 @@ +/** + * Licensed to Big Data Genomics

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60562034 [Test build #473 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/473/consoleFull) for PR 2884 at commit

[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...

2014-10-27 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-60562052 @pwendell I tried your reproducer after changing a few things, so I am not sure whether I fixed it accidentally or I could not reproduce at all. --- If your project

[GitHub] spark pull request: [SPARK-4098][YARN]use appUIAddress instead of ...

2014-10-27 Thread WangTaoTheTonic
GitHub user WangTaoTheTonic opened a pull request: https://github.com/apache/spark/pull/2958 [SPARK-4098][YARN]use appUIAddress instead of appUIHostPort in yarn-client mode https://issues.apache.org/jira/browse/SPARK-4098 You can merge this pull request into a Git repository by

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-27 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-60562071 Build failed because this PR is only compatible with Hive 0,13.1 (not 0.12 any more). --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: Add range join support to spark-sql

2014-10-27 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2939#issuecomment-60562141 Hi @kozanitis , did you try to run sbt/sbt -Phive scalastyle ? That command helps you, telling which code is invalid style. Maybe, the length of some lines is over

[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-60562506 [Test build #22287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22287/consoleFull) for PR 2615 at commit

[GitHub] spark pull request: [SPARK-4098][YARN]use appUIAddress instead of ...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2958#issuecomment-60562505 [Test build #22286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22286/consoleFull) for PR 2958 at commit

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60562729 [Test build #473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/473/consoleFull) for PR 2884 at commit

[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-60562791 [Test build #22287 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22287/consoleFull) for PR 2615 at commit

[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-60562793 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-3223 runAsSparkUser cannot change HDFS w...

2014-10-27 Thread jongyoul
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/2126#issuecomment-60563727 @andrewor14 You mean that a clean way to merge this patch into main stream is that I code this patch from current master branch and make a pull requst again, isn't it?

[GitHub] spark pull request: [SPARK-4097] Fix the race condition of 'thread...

2014-10-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2957#issuecomment-60563934 You can also just take a local reference to the thread, and operate on it. The local reference will of course be null in both cases or not-null in both cases. --- If

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60564242 Hm, how do you mean? the rest of the code already expects this to be an `Int` and a number of megabytes. Nothing else can be further parsing this (or else that's a bug).

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60564958 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60564950 [Test build #22282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22282/consoleFull) for PR 2955 at commit

[GitHub] spark pull request: [SPARK-4097] Fix the race condition of 'thread...

2014-10-27 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/2957#issuecomment-60565342 You can also just take a local reference to the thread, and operate on it. The local reference will of course be null in both cases or not-null in both cases.

[GitHub] spark pull request: [SPARK-4097] Fix the race condition of 'thread...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2957#issuecomment-60565582 [Test build #22289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22289/consoleFull) for PR 2957 at commit

[GitHub] spark pull request: [SPARK-4096][YARN]Update executor memory descr...

2014-10-27 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/2955#issuecomment-60566095 Guess I didn't make it clear. So I paste some code segment to help. Excuse me for this. In ClientBase.scala we could see: ` val amClass =

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2906#issuecomment-60566259 [Test build #22290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22290/consoleFull) for PR 2906 at commit

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2906#issuecomment-60566367 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2906#issuecomment-60566364 [Test build #22290 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22290/consoleFull) for PR 2906 at commit

[GitHub] spark pull request: [SPARK-4097] Fix the race condition of 'thread...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2957#issuecomment-60566474 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-27 Thread yu-iskw
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/2906#issuecomment-60566674 @mengxr thank you for your feedback. Is there a paper that you used as reference? If so, please cite it in the doc. Yes. I added the comment into the doc.

[GitHub] spark pull request: [SPARK-4098][YARN]use appUIAddress instead of ...

2014-10-27 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/2958#issuecomment-60567168 Same replacement happened in https://github.com/apache/spark/pull/2276, same change in `runExecutorLauncher` is mentioned in that PR but done nothing. --- If

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2906#issuecomment-60567740 [Test build #22291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22291/consoleFull) for PR 2906 at commit

[GitHub] spark pull request: [MLLIB] SPARK-1547: Adding Gradient Boosting t...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2607#issuecomment-60568456 [Test build #22285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22285/consoleFull) for PR 2607 at commit

[GitHub] spark pull request: [MLLIB] SPARK-1547: Adding Gradient Boosting t...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2607#issuecomment-60568462 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4097] Fix the race condition of 'thread...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2957#issuecomment-60568910 [Test build #22284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22284/consoleFull) for PR 2957 at commit

[GitHub] spark pull request: [SPARK-4097] Fix the race condition of 'thread...

2014-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2957#issuecomment-60568924 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4032] Deprecate YARN alpha support in S...

2014-10-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2878#issuecomment-60569271 [Test build #22292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22292/consoleFull) for PR 2878 at commit

  1   2   3   4   5   6   >