[
https://issues.apache.org/jira/browse/HDDS-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614609#comment-16614609
]
Elek, Marton commented on HDDS-446:
-----------------------------------
Thank you very much the questions. I am very happy that we can discuss the
distribution specific questions.
1. It doesn't require an more jars. You can start namenode + datanode from
hadoop 3.1 distribution with hadoop-ozone-filesystem.jar. And you should start
scm + om from the ozone distribution. So the single jar file is enough to
extend an existing hadoop datanode.
2. Exactly. This is topology is different from the simplified ozone-only
topology (scm + om + ozone datanode). I would like to run the acceptance test
with both of the docker clusters (ozone-only and ozone+hdfs)
3. Right. But updating maven shade plugin is more risky. I propose to do it in
a separated jira (just to make it easier to revert in case of problems). Until
that I would use the newer version.
4. It's more ore less just laziness. these two jars are copied to different
location and it was harder to handle version in the names. If you need it 1) we
can add the current version to the jar (eg. META-INF/...). It's more safe as it
can survive renames. 2) we can improve the build process to replace versions in
the docker files
5/a. One of the old hadoop scripts checks it for safety (I have to check it
again which was it). We don't need the content of the folder but wouldn't like
to change the scripts.
5/b. sure, you are right. I will fix it. I am also working on a jira where I
fix a few minor release relate type (wrong name of the assembly descriptor,
ozone related + readme), it could be a better place to fix it.
As an additional note: I think sooner or later we need to switch to a
customized release process. Currently we can't release ozone together with
hadoop-3.1 as many project file comes from the hadoop-3.2-SNAPSHOT source tree
(dist-layout-stitching copies them directly)
> Provide shaded artifact to start ozone service as a datanode plugin
> -------------------------------------------------------------------
>
> Key: HDDS-446
> URL: https://issues.apache.org/jira/browse/HDDS-446
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
> Fix For: 0.2.1
>
> Attachments: HDDS-446-ozone-0.2.001.patch
>
>
> Ozone datanode service could be started in two different way:
> # as a standalone process
> # as a datanode plugin
> We tested the second scenarion by compiling both ozone + hadoop trunk
> together and created a combined artifact. But we have no more full hadoop
> distribution in the ozone release package any more. And we had no answer how
> it could be started with an existing hadoop release.
> We need a well defined way to add the datanode-service to the classpath of
> the hadoop datanode.
> I propose here to create a lightweight shaded artifact (and here the shading
> only combines all the classes together without package name refactoring) to
> make it easier to extend an existing hadoop installation.
> In this patch I add the shade plugin execution to the ozone
> objectstore-service and the shaded file will be copied to the distribution.
> A new docker-compose based execution environment (ozone-hdfs)
> demonstrates/tests how it could work.
> Tested with hadoop 3.1.0 and datanode service could be started without any
> problem
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
