Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/20584
I created a PR, https://github.com/apache/spark/pull/20590 .
---
-
To unsubscribe, e-mail:
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/20584
For the following case, I'll make a PR for Spark ORC columnar reader very
soon.
> 2) the orc columnar reader's close method doesn't close the file stream.
---
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/20584
This patch helps `sbt/hadoop2.7`. So, I'm seriously monitoring the latest
consecutive failures at `sbt` and `hadoop-2.6` branch, too.
- 4210 (Running)
- 4209 Failed with
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/20584
My bad. Thank you, guys. For the following, I'll investigate it.
> According to the log, the leaked file stream was created when building
the ORC columnar reader.
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20584
Great!
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-2.3-test-sbt-hadoop-2.7/
becomes green again!!!
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20584
I think I rushed to take a look at the first time. Thanks for fixing this.
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20584
> I am suspicious about relationship between afterEach() and
addTaskCompletionListener (call close()). But, not sure. Let us try this
approach first.
This is one of my speculations.
Github user sameeragarwal commented on the issue:
https://github.com/apache/spark/pull/20584
merging this to master/2.3. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20584
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87321/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20584
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20584
**[Test build #87321 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87321/testReport)**
for PR 20584 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/20584
I am also thinking about this. I agree with this.
> According to the log, the leaked file stream was created when building
the ORC columnar reader.
I am suspicious about relationship
Github user sameeragarwal commented on the issue:
https://github.com/apache/spark/pull/20584
LGTM, seems plausible!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20584
You are right. I have run out of ideas. LGTM too for a try if it happens
more frequently in spark-branch-2.3-test-sbt-hadoop-2.7.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20584
> BTW, my rough wild guess was that case 2. (reading it but not closing it)
happens in schema inference path.
According to the log, the leaked file stream was created when building the
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20584
I won't get in the way but I am less sure on this. I thought this is also
flaky in PR builder too anyway.
---
-
To
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20584
LGTM, I would merge this first and see whether this can help fix the flaky
tests.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20584
BTW, my rough wild guess was that case 2. (reading it but not closing it)
happens in schema inference path.
---
-
To
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20584
BTW, I would bet for case 2. in the PR description (just a rough wild
guess).
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20584
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20584
**[Test build #87321 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87321/testReport)**
for PR 20584 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20584
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/802/
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20584
cc @sameeragarwal @dongjoon-hyun @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
23 matches
Mail list logo
