GitHub user ericvandenbergfb opened a pull request:
https://github.com/apache/spark/pull/18673
[SPARK-21447][WEB UI] Spark history server fails to render compressed
inprogress history file in some cases.
Add failure handling for EOFException that can be thrown during
decompression of an inprogress spark history file, treat same as case
where can't parse the last line.
## What changes were proposed in this pull request?
Failure handling for case of EOFException thrown within the
ReplayListenerBus.replay method to handle the case analogous to json parse fail
case. This path can arise in compressed inprogress history files since an
incomplete compression block could be read (not flushed by writer on a block
boundary). See the stack trace of this occurrence in the jira ticket
(https://issues.apache.org/jira/browse/SPARK-21447)
## How was this patch tested?
Added a unit test that specifically targets validating the failure handling
path appropriately when maybeTruncated is true and false.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ericvandenbergfb/spark
fix_inprogress_compr_history_file
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18673.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18673
----
commit caeac288e2285d16c327a0df6abd6c9fccecdd50
Author: Eric Vandenberg <[email protected]>
Date: 2017-07-18T16:58:12Z
[SPARK-21447][WEB UI] Spark history server fails to render compressed
inprogress history file in some cases.
Add failure handling for EOFException that can be thrown during
decompression of an inprogress spark history file, treat same as case
where can't parse the last line.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
