FatalLin commented on PR #966: URL: https://github.com/apache/submarine/pull/966#issuecomment-1157194033
> ```python > class HDFSPreHandler(FsPreHandler): > def __init__(self): > self.hdfs_host=os.environ["HDFS_HOST"] > self.hdfs_port=int(os.environ["HDFS_PORT"]) > self.hdfs_source=os.environ["HDFS_SOURCE"] > self.dest_path=os.environ["DEST_PATH"] > self.enable_kerberos=os.environ["ENABLE_KERBEROS"] > ``` > > Do we need to consider Hadoop HA? In HA case, the host may be mapped to `dfs.nameservices`, so that we may need to upload `core-sit.xml` and `hdfs-site.xml` files (of course, this is a Java way, will Python simplify it?). And sometimes some Hadoop host names may be set independently by the internal server in `/etc/hosts` rather than obtained in DNS. Should we add `hostaliases` to the pod/deployment resource of submarine? well, as a client, I don't think Submarine need to worry about the server side behavior here. If the cluster has any HA mechanism and it also knows there's any client would connect the cluster service, the cluster administrator should find out how to provide stable serving when the swtich-over happened. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
