GitHub user vanzin opened a pull request:
https://github.com/apache/spark/pull/7097
[SPARK-8372] Do not show applications that haven't recorded their app ID
yet.
Showing these applications may lead to weird behavior in the History
Server. For old logs, if
the app ID is recorded later, you may end up with a duplicate entry. For
new logs, the app might
be listed with a ".inprogress" suffix.
So ignore those, but still allow old applications that don't record app IDs
at all (1.0 and 1.1) to be shown.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vanzin/spark SPARK-8372
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7097.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7097
----
commit 1eca3fe8d2b6c56c22c01ec8bde1d86563258e58
Author: Carson Wang <[email protected]>
Date: 2015-06-17T20:41:36Z
[SPARK-8372] History server shows incorrect information for application not
started
The history server may show an incorrect App ID for an incomplete
application like <App ID>.inprogress. This app info will never disappear even
after the app is completed.
The cause of the issue is that a log path name is used as the app id when
app id cannot be got during replay.
Author: Carson Wang <[email protected]>
Closes #6827 from carsonwang/SPARK-8372 and squashes the following commits:
cdbb089 [Carson Wang] Fix code style
3e46b35 [Carson Wang] Update code style
90f5dde [Carson Wang] Add a unit test
d8c9cd0 [Carson Wang] Replaying events only return information when app is
started
commit 7b91b74f366bf44b913112040e6419e06b042a07
Author: Marcelo Vanzin <[email protected]>
Date: 2015-06-29T20:25:08Z
Handle logs generated by 1.0 and 1.1.
These logs don't have app IDs, so they should not be filtered.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
