Github user carsonwang closed the pull request at:
https://github.com/apache/spark/pull/9991
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user carsonwang commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160876864
Close this and resubmit #10061
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does no
Github user carsonwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r46245206
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extend
Github user carsonwang commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160858339
> It's a bit different but not in the way @carsonwang explained; whether
you use the hook or handle SparkListenerApplicationEnd, the listener will be
cleared when the
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160770491
Head's up: since discussion here is still ongoing and I think there's still
more work to do, I'm going to revert #9297 in order to un-break the master
Maven tests. Cou
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r46207010
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extends
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r46206482
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1619,6 +1620,14 @@ class SparkContext(config: SparkConf) extends
Logging with Exec
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160717545
> It's a bit different because the location of postApplicationEnd() is at
the beginning of stop?
It's a bit different but not in the way @carsonwang explained; wh
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160714495
> Sorry, I don't follow. SparkListenerApplicationEnd is posted by
SparkContext.stop, which is the same place where you're adding the hook to
clean up the listener. So it
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160691520
> I didn't clean up the SQLListener after a application end event because
another SQLContext created later still wants to use the same SQLListener.
Sorry, I don'
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160512250
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160512249
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160512207
**[Test build #46862 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46862/consoleFull)**
for PR 9991 at commit
[`00df329`](https://git
Github user carsonwang commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160495888
@zsxwing , do you have any further comments regarding how the `SQLListener`
is cleaned up?
---
If your project is set up for it, you can reply to this email and have
Github user carsonwang commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160494777
@vanzin , I wrapped the calls to the hooks with
`Utils.tryLogNonFatalError`. I didn't clean up the `SQLListener` after a
application end event because another `SQLCon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160493581
**[Test build #46862 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46862/consoleFull)**
for PR 9991 at commit
[`00df329`](https://gith
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160371284
Haven't followed discussion in detail yet, but just wanted to flag this
PR/discussion as a high priority item to get resolved soon, since the failing
memory leak test
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160352795
Would things work if `SQLListener` cleaned up after itself when
`SparkListenerApplicationEnd` is received? That would avoid adding hooks in
`SparkContext`.
Also,
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r46086313
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1764,6 +1773,7 @@ class SparkContext(config: SparkConf) extends Logging
with Executor
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160250393
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160250394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160250373
**[Test build #46834 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46834/consoleFull)**
for PR 9991 at commit
[`b694e27`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160242817
**[Test build #46834 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46834/consoleFull)**
for PR 9991 at commit
[`b694e27`](https://gith
Github user carsonwang commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160242651
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160204205
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160204204
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160204175
**[Test build #46828 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46828/consoleFull)**
for PR 9991 at commit
[`b694e27`](https://git
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160196391
**[Test build #46828 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46828/consoleFull)**
for PR 9991 at commit
[`b694e27`](https://gith
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160195573
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160195567
> Test build #46807 has finished for PR 9991 at commit b694e27.
>
> This patch fails from timeout after a configured wait of `250m`.
> This patch merges clean
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160118552
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160118520
**[Test build #46807 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46807/consoleFull)**
for PR 9991 at commit
[`b694e27`](https://git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160118554
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160112718
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160112569
**[Test build #46805 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46805/consoleFull)**
for PR 9991 at commit
[`4549f62`](https://git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160112715
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160057079
**[Test build #46807 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46807/consoleFull)**
for PR 9991 at commit
[`b694e27`](https://gith
Github user carsonwang commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160055041
The original purpose of this PR is to fix the `SQLListenerMemoryLeakSuite`
test failure. This can be resolved by clearing `SQLContext.sqlListener` before
the test.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-160053650
**[Test build #46805 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46805/consoleFull)**
for PR 9991 at commit
[`4549f62`](https://gith
Github user carsonwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r46014967
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extend
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r46008320
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite ext
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-159848848
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-159848849
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-159848673
**[Test build #46751 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46751/consoleFull)**
for PR 9991 at commit
[`8ca3031`](https://git
Github user carsonwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r45952482
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extend
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r45949524
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extends S
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r45949221
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extends S
Github user carsonwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r45948964
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extend
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r45948582
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extends S
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9991#issuecomment-159825648
**[Test build #46751 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46751/consoleFull)**
for PR 9991 at commit
[`8ca3031`](https://gith
GitHub user carsonwang opened a pull request:
https://github.com/apache/spark/pull/9991
[SPARK-11206] (Followup) Fix SQLListenerMemoryLeakSuite test error
A followup to #9297, fix the SQLListenerMemoryLeakSuite test error. The
[failure](https://amplab.cs.berkeley.edu/jenkins/job/Spa
51 matches
Mail list logo
