[GitHub] spark issue #15723: [SPARK-18214][SQL] Simplify RuntimeReplaceable type coer...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15723 **[Test build #67935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67935/consoleFull)** for PR 15723 at commit

[GitHub] spark pull request #15723: [SPARK-18214][SQL] Simplify RuntimeReplaceable ty...

2016-11-01 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/15723 [SPARK-18214][SQL] Simplify RuntimeReplaceable type coercion ## What changes were proposed in this pull request? RuntimeReplaceable is used to create aliases for expressions, but the way it deals

[GitHub] spark issue #9183: [SPARK-11215] [ML] Add multiple columns support to String...

2016-11-01 Thread pramitchoudhary
Github user pramitchoudhary commented on the issue: https://github.com/apache/spark/pull/9183 @yanboliang This is a very helpful initiative by you. Thanks for taking it up. Let me know, if you need any help for this PR. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15618: [WIP][SPARK-14914][CORE] Fix Resource not closed after u...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15618 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15702: [SPARK-18124] Observed delay based Event Time Wat...

2016-11-01 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15702#discussion_r86053925 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/EventTimeWatermark.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed

[GitHub] spark issue #15618: [WIP][SPARK-14914][CORE] Fix Resource not closed after u...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15618 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67928/ Test FAILed. ---

[GitHub] spark issue #15618: [WIP][SPARK-14914][CORE] Fix Resource not closed after u...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15618 **[Test build #67928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67928/consoleFull)** for PR 15618 at commit

[GitHub] spark pull request #15700: [SPARK-17964][SparkR] Enable SparkR with Mesos cl...

2016-11-01 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/15700#discussion_r86052116 --- Diff: core/src/main/scala/org/apache/spark/api/r/RUtils.scala --- @@ -84,7 +84,7 @@ private[spark] object RUtils { } } else {

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-11-01 Thread jiexiong
Github user jiexiong commented on the issue: https://github.com/apache/spark/pull/15722 Here is my understanding: after spilling, it would call reset() to release the memory. In the reset() function, it deletes all the memory pages, but it did not release any memory from longArray().

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13617 **[Test build #67934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67934/consoleFull)** for PR 13617 at commit

[GitHub] spark pull request #15647: [SPARK-18088][ML] Various ChiSqSelector cleanups

2016-11-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15647 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15703 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67930/ Test FAILed. ---

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15703 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15703 **[Test build #67930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67930/consoleFull)** for PR 15703 at commit

[GitHub] spark issue #15647: [SPARK-18088][ML] Various ChiSqSelector cleanups

2016-11-01 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15647 I'll go ahead and merge this, but please comment if it needs any follow-ups. Merging with master Thanks for the review! --- If your project is set up for it, you can reply to

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-11-01 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15722 @jiexiong Do you have a theory why this will cause OOM? To me, the current code will use more memory than needed but less allocation, why it will cause OOM? --- If your project is set up for it,

[GitHub] spark pull request #15698: [SPARK-18182] Expose ReplayListenerBus.read() ove...

2016-11-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15698 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15541 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67924/ Test PASSed. ---

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15541 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-11-01 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15698 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15541 **[Test build #67924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67924/consoleFull)** for PR 15541 at commit

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a memory lea...

2016-11-01 Thread jiexiong
Github user jiexiong commented on the issue: https://github.com/apache/spark/pull/15722 This is production query. Sorry, I could not share it. It is doing a join between two big tables. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13617 **[Test build #67933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67933/consoleFull)** for PR 13617 at commit

[GitHub] spark issue #15715: [SPARK-18198][Doc][Streaming] Highlight code snippets

2016-11-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15715 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15703 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67927/ Test FAILed. ---

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15703 **[Test build #67927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67927/consoleFull)** for PR 15703 at commit

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15703 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15722: [SPARK-18208] [Shuffle] Executor OOM due to a mem...

2016-11-01 Thread jiexiong
Github user jiexiong commented on a diff in the pull request: https://github.com/apache/spark/pull/15722#discussion_r86048313 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -903,11 +906,12 @@ public void reset() { numKeys = 0;

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13617 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13617 **[Test build #67932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67932/consoleFull)** for PR 13617 at commit

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13617 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67932/ Test FAILed. ---

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13617 **[Test build #67932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67932/consoleFull)** for PR 13617 at commit

[GitHub] spark pull request #14906: [SPARK-17350][SQL] Disable default use of KryoSer...

2016-11-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14906 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a memory lea...

2016-11-01 Thread jiexiong
Github user jiexiong commented on the issue: https://github.com/apache/spark/pull/15722 Here is the query: Here is the query: INSERT OVERWRITE TABLE lookalike_trainer_campaign_conv_users_with_country_shadow PARTITION(ds='2016-10-19') SELECT c.source_id,

[GitHub] spark pull request #15722: [SPARK-18208] [Shuffle] Executor OOM due to a mem...

2016-11-01 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15722#discussion_r86046017 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -903,11 +906,12 @@ public void reset() { numKeys = 0;

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a memory lea...

2016-11-01 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15722 OK to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a memory lea...

2016-11-01 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15722 @jiexiong I don't think this is a memory leak, BytesToBytesMap does not release all memory for each spilling based on the assumption that the memory will be acquired back soon. What's the query that

[GitHub] spark issue #14906: [SPARK-17350][SQL] Disable default use of KryoSerializer...

2016-11-01 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14906 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15027: [SPARK-17475] [STREAMING] Delete CRC files if the filesy...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15027 **[Test build #67931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67931/consoleFull)** for PR 15027 at commit

[GitHub] spark issue #15027: [SPARK-17475] [STREAMING] Delete CRC files if the filesy...

2016-11-01 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15027 It turns out there is a bug in Hadoop's FileContext that doesn't rename the checksum file. LGTM pending tests. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a memory lea...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15722 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67921/ Test PASSed. ---

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15704 **[Test build #67921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67921/consoleFull)** for PR 15704 at commit

[GitHub] spark pull request #15722: [SPARK-18208] [Shuffle] Executor OOM due to a mem...

2016-11-01 Thread jiexiong
GitHub user jiexiong opened a pull request: https://github.com/apache/spark/pull/15722 [SPARK-18208] [Shuffle] Executor OOM due to a memory leak in BytesToBytesMap ## What changes were proposed in this pull request? Fixed the OOM problem in BytesToBytesMap. ## How

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15703 **[Test build #67930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67930/consoleFull)** for PR 15703 at commit

[GitHub] spark issue #15718: [SPARK-16839][SQL] Simplify Struct creation code path

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15718 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15718: [SPARK-16839][SQL] Simplify Struct creation code path

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15718 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67918/ Test PASSed. ---

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13617 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13617 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67929/ Test FAILed. ---

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13617 **[Test build #67929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67929/consoleFull)** for PR 13617 at commit

[GitHub] spark issue #15718: [SPARK-16839][SQL] Simplify Struct creation code path

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15718 **[Test build #67918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67918/consoleFull)** for PR 15718 at commit

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67919/ Test PASSed. ---

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15704 **[Test build #67919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67919/consoleFull)** for PR 15704 at commit

[GitHub] spark issue #15697: [SparkR][Test]:remove unnecessary suppressWarnings

2016-11-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15697 Ah, yes actually that's a problem now unfortunately. It seems even comitters can't retrigger. Actually, that's why I ran this via another account and left the comment above. I am trying to find

[GitHub] spark issue #15697: [SparkR][Test]:remove unnecessary suppressWarnings

2016-11-01 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15697 No, I just launched Jenkins test. I don't know how to manually trigger the AppVeyor test. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-11-01 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15673 @ericl I can do that, yes. I'm current tied down. I will push a new commit later today or tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15703 **[Test build #67927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67927/consoleFull)** for PR 15703 at commit

[GitHub] spark issue #15618: [WIP][SPARK-14914][CORE] Fix Resource not closed after u...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15618 **[Test build #67928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67928/consoleFull)** for PR 15618 at commit

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

2016-11-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15618 Oh wait, it seems the failed test is related. Will take a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-11-01 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/15703 It turned out that I didn't initialize Hive UDAF evaluators properly. Quoted from commit message of my previous commit: > Hive UDAFs are sensitive to aggregation mode, and must be

[GitHub] spark pull request #15618: [SPARK-14914][CORE] Fix Resource not closed after...

2016-11-01 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/15618 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #15618: [SPARK-14914][CORE] Fix Resource not closed after...

2016-11-01 Thread HyukjinKwon
GitHub user HyukjinKwon reopened a pull request: https://github.com/apache/spark/pull/15618 [SPARK-14914][CORE] Fix Resource not closed after using, mostly for unit tests ## What changes were proposed in this pull request? Close `FileStreams`, `ZipFiles` etc to release the

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

2016-11-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15618 Ll --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15618: [SPARK-14914][CORE] Fix Resource not closed after using,...

2016-11-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15618 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15697: [SparkR][Test]:remove unnecessary suppressWarnings

2016-11-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15697 Oh, AppVeyor test is not triggered again with the Jebkins one. Or, do you mean you launched another build via another account and manually tested it? --- If your project is set up for it,

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-11-01 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15553 Closing PR given pushback to commit'ing it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15553: [SPARK-18008] [build] Add support for -Dmaven.tes...

2016-11-01 Thread mridulm
Github user mridulm closed the pull request at: https://github.com/apache/spark/pull/15553 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15698 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67917/ Test PASSed. ---

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15698 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-11-01 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15659 I think we should try and get so that it can be a part of the 2.1 release so we can have publishing to PyPI added in 2.1.1 or 2.2. We've been looking at making PySpark installable with pip since

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-11-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15354 Thank you for merging this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15659 **[Test build #67926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67926/consoleFull)** for PR 15659 at commit

[GitHub] spark issue #15647: [SPARK-18088][ML] Various ChiSqSelector cleanups

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15647 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15647: [SPARK-18088][ML] Various ChiSqSelector cleanups

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67923/ Test PASSed. ---

[GitHub] spark issue #15647: [SPARK-18088][ML] Various ChiSqSelector cleanups

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15647 **[Test build #67923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67923/consoleFull)** for PR 15647 at commit

[GitHub] spark issue #15718: [SPARK-16839][SQL] Simplify Struct creation code path

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15718 **[Test build #67925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67925/consoleFull)** for PR 15718 at commit

[GitHub] spark issue #15721: [SPARK-17772][ML][TEST] Add test functions for ML sample...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15721 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15721: [SPARK-17772][ML][TEST] Add test functions for ML sample...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15721 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67922/ Test PASSed. ---

[GitHub] spark issue #15721: [SPARK-17772][ML][TEST] Add test functions for ML sample...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15721 **[Test build #67922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67922/consoleFull)** for PR 15721 at commit

[GitHub] spark issue #15719: [SPARK-18114][HOTFIX] Fix line-too-long style error from...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15719 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67914/ Test PASSed. ---

[GitHub] spark issue #15719: [SPARK-18114][HOTFIX] Fix line-too-long style error from...

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15719 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15719: [SPARK-18114][HOTFIX] Fix line-too-long style error from...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15719 **[Test build #67914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67914/consoleFull)** for PR 15719 at commit

[GitHub] spark issue #14435: [SPARK-16756][SQL][WIP] Add `sql` function to LogicalPla...

2016-11-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14435 I close this PR since the issue is closed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14435: [SPARK-16756][SQL][WIP] Add `sql` function to Log...

2016-11-01 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/14435 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15637: [SPARK-18000] [SQL] Aggregation function for computing e...

2016-11-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15637 BTW, please follow the PR https://github.com/apache/spark/issues/15677 to rewrite your function description. Add an example for the new function if possible. --- If your project is set up for

[GitHub] spark pull request #15637: [SPARK-18000] [SQL] Aggregation function for comp...

2016-11-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15637#discussion_r86031611 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/MapAggregate.scala --- @@ -0,0 +1,332 @@ +/* + *

[GitHub] spark pull request #15675: [SPARK-18144][SQL] logging StreamingQueryListener...

2016-11-01 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15675#discussion_r86030559 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala --- @@ -290,7 +290,10 @@ class StreamingQuerySuite extends

[GitHub] spark pull request #15675: [SPARK-18144][SQL] logging StreamingQueryListener...

2016-11-01 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15675#discussion_r86030545 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingQueryListenerBus.scala --- @@ -50,7 +52,14 @@ class

[GitHub] spark issue #15671: [SPARK-14567][ML]Add instrumentation logs to ML training...

2016-11-01 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15671 So, if we consistently use `instr.logParams(params: _*)` (even in cases where it's acceptable) then we run the risk of adding some param in the future that could "overload" the logs (like

[GitHub] spark pull request #15637: [SPARK-18000] [SQL] Aggregation function for comp...

2016-11-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15637#discussion_r86030388 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/MapAggregate.scala --- @@ -0,0 +1,332 @@ +/* + *

[GitHub] spark issue #15700: [SPARK-17964][SparkR] Enable SparkR with Mesos client mo...

2016-11-01 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/15700 @srowen LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #13880: SPARK-16178: Remove unnecessary Hive partition ch...

2016-11-01 Thread rdblue
Github user rdblue closed the pull request at: https://github.com/apache/spark/pull/13880 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14906: [SPARK-17350][SQL] Disable default use of KryoSerializer...

2016-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14906 **[Test build #3390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3390/consoleFull)** for PR 14906 at commit

[GitHub] spark issue #15088: SPARK-17532: Add lock debugging info to thread dumps.

2016-11-01 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/15088 @srowen, if you have a chance, could you look at this again? I think it will be helpful for tracking down live-lock issues. Thanks! --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #15700: [SPARK-17964][SparkR] Enable SparkR with Mesos client mo...

2016-11-01 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/15700 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #15700: [SPARK-17964][SparkR] Enable SparkR with Mesos cl...

2016-11-01 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/15700#discussion_r86029381 --- Diff: core/src/main/scala/org/apache/spark/api/r/RUtils.scala --- @@ -84,7 +84,7 @@ private[spark] object RUtils { } } else {

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost

2016-11-01 Thread vlad17
Github user vlad17 commented on the issue: https://github.com/apache/spark/pull/14547 @jkbradley There seems to be more issues with deprecating impurity: [error] [warn]

[GitHub] spark issue #15720: [SPARK-18167] Disable flaky SQLQuerySuite test

2016-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15720 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

<    1   2   3   4   5   6   7   8   >