[
https://issues.apache.org/jira/browse/HDDS-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17417568#comment-17417568
]
István Fajth commented on HDDS-5447:
------------------------------------
Hi [~elek], let me re-iterate, I am 100% on your side and agree to have the
HTTPFS GW common code in hadoop-common at the end of the day, and I also agree
that it has a cost to maintain a codebase in two places.
I feel important to note that even Hadoop has a duplicate implementation, one
in the NameNode, and one in the separate HTTPFS GateWay. So from Hadoop's
perspective, this is a big refactor not just a small cleanup, which might be
risky as well (affects the NN internally also), so it has to be well
established as we do not want to see possible regressions there. From Ozone's
perspective this is a new feature, so we have some more wiggle room to run into
some regressions and problems along the way while the new architecture matures,
and when it does, it is way more safer to contribute back to Hadoop a somewhat
matured code we do not see any issues with.
So I firmly believe doing it in the proposed way - even though it seems to be
unnatural, and admittedly it has a maintenance overhead - is safer, and more
stable from Hadoop's point of view, while from Ozone's point of view it is more
faster to have the feature in a half backed but evolving way. I think these
properties are overweighing the cost of duplicated maintenance for some time.
Speed is an interesting property here, as we can ease migrations from HDFS in
some cases if we provide this API, one of those software migrations (with HUE
as we already mentioned it) we also would like to finish in a reasonable
timeframe.
I really hope that considering the above you can support the idea of going
forward on this route, not just because we already started it, but because we
are committed to get to the envisioned end state, and because it sounds
reasonable.
> HttpFS support in Ozone
> -----------------------
>
> Key: HDDS-5447
> URL: https://issues.apache.org/jira/browse/HDDS-5447
> Project: Apache Ozone
> Issue Type: New Feature
> Components: Ozone Client, Ozone Manager
> Reporter: Aravindan Vijayan
> Assignee: István Fajth
> Priority: Major
> Attachments: HTTPFS interface for Ozone.pdf
>
>
> There are several tools out there mainly written in Python, that uses the
> webhdfs interface to connect to HDFS. Even there are quite a few other
> filesystem implementations that provide access via the same rest interface
> that HDFS provides.
> HUE also implements the HDFS file browser by accessing HDFS via the REST API
> either on the NameNodes or on HTTPFS Server instances added to the HDFS
> service.
> This gave the inspiration to check and experiment what is required to support
> a similar REST endpoint over Ozone.
> The advantage is that we can ease the migration of tools developed in-house
> that are using this interface of HDFS, while we can add the possibility to
> browse Ozone from HUE.
> There is literally no disadvantage of having such an interface, as we can
> implement it as a separate module which does not have any interference with
> the rest of the code, as the REST endpoint as with HTTPFS will use a regular
> Java based Ozone client to serve any requests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
