Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/17551
@barnardb only in Spark standalone mode HistoryServer is embedded into
Master process for convenience IIRC. You can always start a standalone
HistoryServer process.
Also `FsHistoryProvider` is not bound to HDFS, other Hadoop compatible FS
could be supported, like wasb, s3 and other object stores that has Hadoop FS
compatible layer. I would think even in your cluster environment (k8s), you
probably have a object store. And at least you could implement a customized
`ApplicationHistoryProvider`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
