[
https://issues.apache.org/jira/browse/BEAM-2831?focusedWorklogId=81841&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81841
]
ASF GitHub Bot logged work on BEAM-2831:
----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Mar/18 13:44
Start Date: 19/Mar/18 13:44
Worklog Time Spent: 10m
Work Description: dawidwys opened a new pull request #4892: [BEAM-2831]
Do not wrap IOException in SerializableCoder
URL: https://github.com/apache/beam/pull/4892
DESCRIPTION HERE
------------------------
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ ] Make sure there is a [JIRA
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the
change (usually before you start working on it). Trivial changes like typos do
not require a JIRA issue. Your pull request should address just this issue,
without pulling in other changes.
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue.
- [ ] Write a pull request description that is detailed enough to
understand:
- [ ] What the pull request does
- [ ] Why it does it
- [ ] How it does it
- [ ] Why this approach
- [ ] Each commit in the pull request should have a meaningful subject line
and body.
- [ ] Run `mvn clean verify` to make sure basic checks pass. A more
thorough check will be performed on your pull request automatically.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 81841)
Time Spent: 10m
Remaining Estimate: 0h
> Pipeline crashes due to Beam encoder breaking Flink memory management
> ---------------------------------------------------------------------
>
> Key: BEAM-2831
> URL: https://issues.apache.org/jira/browse/BEAM-2831
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.0.0, 2.1.0
> Environment: Flink 1.2.1 and 1.3.0, Java HotSpot and OpenJDK 8, macOS
> 10.12.6 and unknown Linux
> Reporter: Reinier Kip
> Assignee: Aljoscha Krettek
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I’ve been running a Beam pipeline on Flink. Depending on the dataset size and
> the heap memory configuration of the jobmanager and taskmanager, I may run
> into an EOFException, which causes the job to fail.
> As [discussed on Flink's
> mailinglist|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/EOFException-related-to-memory-segments-during-run-of-Beam-pipeline-on-Flink-td15255.html]
> (stacktrace enclosed), Flink catches these EOFExceptions and activates disk
> spillover. Because Beam wraps these exceptions, this mechanism fails, the
> exception travels up the stack, and the job aborts.
> Hopefully this is enough information and this is something that can be
> adjusted for in Beam. I'd be glad to provide more information where needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
