[
https://issues.apache.org/jira/browse/HDDS-447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622761#comment-16622761
]
Bharat Viswanadham edited comment on HDDS-447 at 9/20/18 9:56 PM:
------------------------------------------------------------------
Thank You, [~elek] for the patch.
I am able to compile now.
Few changes, I have not understood why it is done.
{quote}The classpath of 'ozone fs' is solved in a more easy way. I just added
the ozonefs project as a dependency to the tools. Now the classpath of the
tools project could be used for all the tools (ozone scmcli, ozone fs, ...).
But it introduced a circular dependency. I fixed it with moving 4 test classes
to the tools. (which also helped to get the classes and the test classes in the
same projects).
{quote}
1. Why the above mentioned change is needed and why moving the test files is
done?
2. Now, we don't have a separate folder for each project like hdds,ozone in
ozone tar ball. We have all tozone-related jars in share/ozone, but for each
component, as we have generated classpath, during running that component, we
use only jars needed for that component, by adding only the jars required for
that component from the generated classpath file?
3. And do we need empty folder share/hadoop/ozone and share/hadoop/hdds?
4. In bin/ozone, Line 86: ";;a", this should be changed to ";;". Because of
this when I try to start docker clusters, they are not coming up. (Thanks to
smoke tests, which help to figure out immediately)
was (Author: bharatviswa):
Thank You, [~elek] for the patch.
I am able to compile now.
Few changes, I have not understood why it is done.
{quote}The classpath of 'ozone fs' is solved in a more easy way. I just added
the ozonefs project as a dependency to the tools. Now the classpath of the
tools project could be used for all the tools (ozone scmcli, ozone fs, ...).
But it introduced a circular dependency. I fixed it with moving 4 test classes
to the tools. (which also helped to get the classes and the test classes in the
same projects).
{quote}
1. Why the above mentioned change is needed and why moving the test files is
done?
2. Now, we don't have a separate folder for each project like hdds,ozone in
ozone tar ball. We have all tozone-related jars in share/ozone, but for each
component, as we have generated classpath, during running that component, we
use only jars needed for that component, by adding only the jars required for
that component from the generated classpath file?
3. And do we need empty folder share/hadoop/ozone and share/hadoop/hdds?
> separate ozone-dist and hadoop-dist projects with real classpath separation
> ---------------------------------------------------------------------------
>
> Key: HDDS-447
> URL: https://issues.apache.org/jira/browse/HDDS-447
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
> Attachments: HDDS-447-ozone-0.2.001.patch, HDDS-447.003.patch,
> HDDS-447.004.patch, HDDS-447.005.patch
>
>
> Currently we have the same hadoop-dist project to create both the ozone and
> and the hadoop distribution.
> To decouple ozone and hadoop build it would be great to create two different
> dist project.
> The hadoop-dist should be cloned to hadoop-ozone/dist and from
> hadoop-dist/pom.xml we can remove the hdds/ozone related items and from
> hadoop-ozone/dist/pom.xml we can remove the core hadoop related part.
> An other issue with the current distribution schema is the lack of real
> classpath separation.
> The current hadoop distribution model is defined in the hadoop-project-dist
> which is parent of all the component projects and the output of the
> distribution generation will be copied by the dist-layout-stitching. There is
> no easy way to use command specific classpath as the classpath is defined in
> component level (hdfs/yarn/mapreduce).
> With this approach we will have a lot of unnecessary dependencies on the
> classpath (which were not on the classpath at the time of the unit tests) and
> it's not possible (as an example) use different type of jaxrs stack for
> different services (s3gateway vs scm).
> As a simplified but more effective approach I propose to use the following
> method:
> 1. don't use hadoop-project-dist for ozone projects any more
> 2. During the build generate a classpath descriptor (with the
> dependency:build-classpath maven plugin/goal) for all the projects
> 3. During the distribution copy all the required dependencies (with
> dependency:copy maven plugin/goal) to a lib folder (share/ozone/lib)
> 4. During the distribution copy all the classpath descriptors to the
> classpath folder (share/ozone/classpath)
> 5. Put only the required jar files to the classpath with reading the
> classpath descriptor
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
