[
https://issues.apache.org/jira/browse/HDDS-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elek, Marton updated HDDS-1521:
-------------------------------
Description:
The conversation of the current ozone directory structure is started here:
HDDS-1458
By [~eyang]
{quote}The proposal is to generate test artifacts in
ozone-[version]/share/ozone/tests for fault injection test framework in release
binary tarball.
{quote}
By [~anu]:
{quote}Let us drop the share and make it tests/blockade, and tests/smoketests
etc. That way, all tests that we ship can be found easily. Otherwise I am +1 on
this change.
{quote}
By [~elek]
{quote}I agree with Anu. I think It's time to revisit the current structure of
ozone distribution tar files. The original structure is inherited from
HADOOP-6255 which is defined for a project which includes multiple subproject
(hadoop, yarn,...) and should support the creation of different RPM package
from the same tar. I think for ozone we can improve the usability with
reorganizing the dirs to a better structure.
But I am fine to do it in a separated jira and keep it in the share/test until
that to make progress.
{quote}
By [~eyang]
{quote}The reason for HADOOP-6255 was more than just RPM packaging. The
motivation behind the reorganization was to make a directory structure that can
work as standalone tarball as well as follow the general guideline for
[Filesystem Hierarchy
Standard|https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard]. This was
proposed because a several people saw the need to make the structure more unix
like for sharing dependencies for larger eco-system to work. This is the reason
that Hadoop HDFS, Mapreduce have good integration between projects to reduce
shell script bootstrap cost. Earlier version of YARN did not follow the
conventional wisdom and it was hard to integrate with rest of Hadoop, YARN
community struggled on classpath issue for at least 2+ years and the time to
hype YARN framework had already passed. Given there is a high probability that
we want to make ozone as universal as possible for applications to integrate
with us. It given us more incentive to make the structure as flexible as
possible. This is only a advice from my own past scaring. There are no perfect
solution, but the conventional wisdom usually have endure test of time and save
energy.
{quote}
I propose the following method:
# Start an independent discussion, let's don't mix this question with
HDDS-1458 and don't block it.
# Let's propose a more userfriendly directory structure first
# Let's ask [~eyang] to defined _technical_ problems regarding to the proposal.
There are two different view here:
# Technical limitations which were mentioned earlier
# Theoretical questions (do we need to have the same structure for core Hadoop
and Ozone packages)
I propose to start the discussion with the first one.
was:
The conversation of the current ozone directory structure is started here:
HDDS-1458
By [~eyang]
{quote}The proposal is to generate test artifacts in
ozone-[version]/share/ozone/tests for fault injection test framework in release
binary tarball.
{quote}
By [~anu]:
{quote}Let us drop the share and make it tests/blockade, and tests/smoketests
etc. That way, all tests that we ship can be found easily. Otherwise I am +1 on
this change.
{quote}
By [~elek]
{quote}I agree with Anu. I think It's time to revisit the current structure of
ozone distribution tar files. The original structure is inherited from
HADOOP-6255 which is defined for a project which includes multiple subproject
(hadoop, yarn,...) and should support the creation of different RPM package
from the same tar. I think for ozone we can improve the usability with
reorganizing the dirs to a better structure.
But I am fine to do it in a separated jira and keep it in the share/test until
that to make progress.
{quote}
By [~eyang]
{quote}The reason for HADOOP-6255 was more than just RPM packaging. The
motivation behind the reorganization was to make a directory structure that can
work as standalone tarball as well as follow the general guideline for
[Filesystem Hierarchy
Standard|https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard]. This was
proposed because a several people saw the need to make the structure more unix
like for sharing dependencies for larger eco-system to work. This is the reason
that Hadoop HDFS, Mapreduce have good integration between projects to reduce
shell script bootstrap cost. Earlier version of YARN did not follow the
conventional wisdom and it was hard to integrate with rest of Hadoop, YARN
community struggled on classpath issue for at least 2+ years and the time to
hype YARN framework had already passed. Given there is a high probability that
we want to make ozone as universal as possible for applications to integrate
with us. It given us more incentive to make the structure as flexible as
possible. This is only a advice from my own past scaring. There are no perfect
solution, but the conventional wisdom usually have endure test of time and save
energy.
{quote}
I propose the following method:
# Start an independent discussion, let's don't mix this question with
HDDS-1458 and don't block it.
# Let's propose a more use friendly directory structure first
# Let's ask [~eyang] to defined _technical_ problems regarding to the proposal.
There are two different view here:
# Technical limitations which were mentioned earlier
# Theoretical questions (do we need to have the same structure for core Hadoop
and Ozone packages)
I propose to start the discussion with the first one.
> Use more natural directory structure for ozone
> ----------------------------------------------
>
> Key: HDDS-1521
> URL: https://issues.apache.org/jira/browse/HDDS-1521
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Elek, Marton
> Priority: Major
>
> The conversation of the current ozone directory structure is started here:
> HDDS-1458
> By [~eyang]
> {quote}The proposal is to generate test artifacts in
> ozone-[version]/share/ozone/tests for fault injection test framework in
> release binary tarball.
> {quote}
> By [~anu]:
> {quote}Let us drop the share and make it tests/blockade, and tests/smoketests
> etc. That way, all tests that we ship can be found easily. Otherwise I am +1
> on this change.
> {quote}
> By [~elek]
> {quote}I agree with Anu. I think It's time to revisit the current structure
> of ozone distribution tar files. The original structure is inherited from
> HADOOP-6255 which is defined for a project which includes multiple subproject
> (hadoop, yarn,...) and should support the creation of different RPM package
> from the same tar. I think for ozone we can improve the usability with
> reorganizing the dirs to a better structure.
> But I am fine to do it in a separated jira and keep it in the share/test
> until that to make progress.
> {quote}
> By [~eyang]
> {quote}The reason for HADOOP-6255 was more than just RPM packaging. The
> motivation behind the reorganization was to make a directory structure that
> can work as standalone tarball as well as follow the general guideline for
> [Filesystem Hierarchy
> Standard|https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard]. This
> was proposed because a several people saw the need to make the structure more
> unix like for sharing dependencies for larger eco-system to work. This is the
> reason that Hadoop HDFS, Mapreduce have good integration between projects to
> reduce shell script bootstrap cost. Earlier version of YARN did not follow
> the conventional wisdom and it was hard to integrate with rest of Hadoop,
> YARN community struggled on classpath issue for at least 2+ years and the
> time to hype YARN framework had already passed. Given there is a high
> probability that we want to make ozone as universal as possible for
> applications to integrate with us. It given us more incentive to make the
> structure as flexible as possible. This is only a advice from my own past
> scaring. There are no perfect solution, but the conventional wisdom usually
> have endure test of time and save energy.
> {quote}
> I propose the following method:
> # Start an independent discussion, let's don't mix this question with
> HDDS-1458 and don't block it.
> # Let's propose a more userfriendly directory structure first
> # Let's ask [~eyang] to defined _technical_ problems regarding to the
> proposal.
> There are two different view here:
> # Technical limitations which were mentioned earlier
> # Theoretical questions (do we need to have the same structure for core
> Hadoop and Ozone packages)
> I propose to start the discussion with the first one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
