[
https://issues.apache.org/jira/browse/HDDS-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838929#comment-16838929
]
Eric Yang edited comment on HDDS-1521 at 5/13/19 10:54 PM:
-----------------------------------------------------------
The current directories look like this:
{code}
.
├── bin
├── compose
│ ├── common
│ │ ├── grafana
│ │ │ ├── conf
│ │ │ ├── dashboards
│ │ │ └── provisioning
│ │ │ ├── dashboards
│ │ │ └── datasources
│ │ └── prometheus
│ ├── ozone
│ ├── ozoneblockade
│ ├── ozonefs
│ ├── ozone-hdfs
│ ├── ozone-om-ha
│ ├── ozoneperf
│ ├── ozone-recon
│ ├── ozones3
│ ├── ozonescripts
│ ├── ozonesecure
│ │ └── docker-image
│ │ └── docker-krb5
│ ├── ozonesecure-mr
│ │ └── docker-image
│ │ └── docker-krb5
│ └── ozonetrace
├── etc
│ └── hadoop
│ └── shellprofile.d
├── kubernetes
│ ├── definitions
│ │ ├── jaeger
│ │ ├── ozone
│ │ │ ├── definitions
│ │ │ ├── freon
│ │ │ └── transformations
│ │ └── prometheus
│ │ └── definitions
│ └── examples
│ ├── minikube
│ │ ├── flekszible
│ │ └── freon
│ ├── ozone
│ │ ├── flekszible
│ │ └── freon
│ └── ozone-dev
│ ├── flekszible
│ └── freon
├── libexec
│ └── shellprofile.d
├── logs
├── sbin
├── share
│ ├── hadoop
│ │ ├── common
│ │ ├── hdds
│ │ ├── hdfs
│ │ ├── mapreduce
│ │ ├── ozone
│ │ ├── ozoneplugin
│ │ └── yarn
│ └── ozone
│ ├── classpath
│ ├── lib
│ └── web
├── smoketest
│ ├── auditparser
│ ├── basic
│ ├── ozonefs
│ ├── s3
│ └── security
└── tests
├── blockade
│ ├── blockadeUtils
│ ├── clusterUtils
│ └── __pycache__
└── compose
{code}
The recommended directory looks like this:
{code}
.
├── bin
├── etc
│ └── ozone
│ └── shellprofile.d
├── libexec
│ └── shellprofile.d
├── logs
├── sbin
└── share
├── hadoop
│ ├── common
│ ├── hdds
│ ├── hdfs
│ ├── mapreduce
│ ├── ozone
│ ├── ozoneplugin
│ └── yarn
└── ozone
├── classpath
├── compose
│ ├── common
│ │ ├── grafana
│ │ │ ├── conf
│ │ │ ├── dashboards
│ │ │ └── provisioning
│ │ │ ├── dashboards
│ │ │ └── datasources
│ │ └── prometheus
│ ├── ozone
│ ├── ozoneblockade
│ ├── ozonefs
│ ├── ozone-hdfs
│ ├── ozone-om-ha
│ ├── ozoneperf
│ ├── ozone-recon
│ ├── ozones3
│ ├── ozonescripts
│ ├── ozonesecure
│ │ └── docker-image
│ │ └── docker-krb5
│ ├── ozonesecure-mr
│ │ └── docker-image
│ │ └── docker-krb5
│ └── ozonetrace
├── kubernetes
│ ├── definitions
│ │ ├── jaeger
│ │ ├── ozone
│ │ │ ├── definitions
│ │ │ ├── freon
│ │ │ └── transformations
│ │ └── prometheus
│ │ └── definitions
│ └── examples
│ ├── minikube
│ │ ├── flekszible
│ │ └── freon
│ ├── ozone
│ │ ├── flekszible
│ │ └── freon
│ └── ozone-dev
│ ├── flekszible
│ └── freon
├── lib
├── smoketest
│ ├── auditparser
│ ├── basic
│ ├── ozonefs
│ ├── s3
│ └── security
├── tests
│ ├── blockade
│ │ ├── blockadeUtils
│ │ ├── clusterUtils
│ │ └── __pycache__
│ └── compose
└── web
{code}
The main difference are the location of config directory, and other files are
moved into share/ozone subdirectories. This allows the tarball to expand in a
subdirectory or overlay on top of FHS location using the same structure.
was (Author: eyang):
The current directories look like this:
{code}
.
├── bin
├── compose
│ ├── common
│ │ ├── grafana
│ │ │ ├── conf
│ │ │ ├── dashboards
│ │ │ └── provisioning
│ │ │ ├── dashboards
│ │ │ └── datasources
│ │ └── prometheus
│ ├── ozone
│ ├── ozoneblockade
│ ├── ozonefs
│ ├── ozone-hdfs
│ ├── ozone-om-ha
│ ├── ozoneperf
│ ├── ozone-recon
│ ├── ozones3
│ ├── ozonescripts
│ ├── ozonesecure
│ │ └── docker-image
│ │ └── docker-krb5
│ ├── ozonesecure-mr
│ │ └── docker-image
│ │ └── docker-krb5
│ └── ozonetrace
├── etc
│ └── hadoop
│ └── shellprofile.d
├── kubernetes
│ ├── definitions
│ │ ├── jaeger
│ │ ├── ozone
│ │ │ ├── definitions
│ │ │ ├── freon
│ │ │ └── transformations
│ │ └── prometheus
│ │ └── definitions
│ └── examples
│ ├── minikube
│ │ ├── flekszible
│ │ └── freon
│ ├── ozone
│ │ ├── flekszible
│ │ └── freon
│ └── ozone-dev
│ ├── flekszible
│ └── freon
├── libexec
│ └── shellprofile.d
├── logs
├── sbin
├── share
│ ├── hadoop
│ │ ├── common
│ │ ├── hdds
│ │ ├── hdfs
│ │ ├── mapreduce
│ │ ├── ozone
│ │ ├── ozoneplugin
│ │ └── yarn
│ └── ozone
│ ├── classpath
│ ├── lib
│ └── web
├── smoketest
│ ├── auditparser
│ ├── basic
│ ├── ozonefs
│ ├── s3
│ └── security
└── tests
├── blockade
│ ├── blockadeUtils
│ ├── clusterUtils
│ └── __pycache__
└── compose
{code}
The recommended directory looks like this:
{code}
{code}
.
├── bin
├── etc
│ └── ozone
│ └── shellprofile.d
├── libexec
│ └── shellprofile.d
├── logs
├── sbin
└── share
├── hadoop
│ ├── common
│ ├── hdds
│ ├── hdfs
│ ├── mapreduce
│ ├── ozone
│ ├── ozoneplugin
│ └── yarn
└── ozone
├── classpath
├── compose
│ ├── common
│ │ ├── grafana
│ │ │ ├── conf
│ │ │ ├── dashboards
│ │ │ └── provisioning
│ │ │ ├── dashboards
│ │ │ └── datasources
│ │ └── prometheus
│ ├── ozone
│ ├── ozoneblockade
│ ├── ozonefs
│ ├── ozone-hdfs
│ ├── ozone-om-ha
│ ├── ozoneperf
│ ├── ozone-recon
│ ├── ozones3
│ ├── ozonescripts
│ ├── ozonesecure
│ │ └── docker-image
│ │ └── docker-krb5
│ ├── ozonesecure-mr
│ │ └── docker-image
│ │ └── docker-krb5
│ └── ozonetrace
├── kubernetes
│ ├── definitions
│ │ ├── jaeger
│ │ ├── ozone
│ │ │ ├── definitions
│ │ │ ├── freon
│ │ │ └── transformations
│ │ └── prometheus
│ │ └── definitions
│ └── examples
│ ├── minikube
│ │ ├── flekszible
│ │ └── freon
│ ├── ozone
│ │ ├── flekszible
│ │ └── freon
│ └── ozone-dev
│ ├── flekszible
│ └── freon
├── lib
├── smoketest
│ ├── auditparser
│ ├── basic
│ ├── ozonefs
│ ├── s3
│ └── security
├── tests
│ ├── blockade
│ │ ├── blockadeUtils
│ │ ├── clusterUtils
│ │ └── __pycache__
│ └── compose
└── web
{code}
The main difference are the location of config directory, and other files are
moved into share/ozone subdirectories. This allows the tarball to expand in a
subdirectory or overlay on top of FHS location using the same structure.
> Use more natural directory structure for ozone
> ----------------------------------------------
>
> Key: HDDS-1521
> URL: https://issues.apache.org/jira/browse/HDDS-1521
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Elek, Marton
> Priority: Major
>
> The conversation of the current ozone directory structure is started here:
> HDDS-1458
> By [~eyang]
> {quote}The proposal is to generate test artifacts in
> ozone-[version]/share/ozone/tests for fault injection test framework in
> release binary tarball.
> {quote}
> By [~anu]:
> {quote}Let us drop the share and make it tests/blockade, and tests/smoketests
> etc. That way, all tests that we ship can be found easily. Otherwise I am +1
> on this change.
> {quote}
> By [~elek]
> {quote}I agree with Anu. I think It's time to revisit the current structure
> of ozone distribution tar files. The original structure is inherited from
> HADOOP-6255 which is defined for a project which includes multiple subproject
> (hadoop, yarn,...) and should support the creation of different RPM package
> from the same tar. I think for ozone we can improve the usability with
> reorganizing the dirs to a better structure.
> But I am fine to do it in a separated jira and keep it in the share/test
> until that to make progress.
> {quote}
> By [~eyang]
> {quote}The reason for HADOOP-6255 was more than just RPM packaging. The
> motivation behind the reorganization was to make a directory structure that
> can work as standalone tarball as well as follow the general guideline for
> [Filesystem Hierarchy
> Standard|https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard]. This
> was proposed because a several people saw the need to make the structure more
> unix like for sharing dependencies for larger eco-system to work. This is the
> reason that Hadoop HDFS, Mapreduce have good integration between projects to
> reduce shell script bootstrap cost. Earlier version of YARN did not follow
> the conventional wisdom and it was hard to integrate with rest of Hadoop,
> YARN community struggled on classpath issue for at least 2+ years and the
> time to hype YARN framework had already passed. Given there is a high
> probability that we want to make ozone as universal as possible for
> applications to integrate with us. It given us more incentive to make the
> structure as flexible as possible. This is only a advice from my own past
> scaring. There are no perfect solution, but the conventional wisdom usually
> have endure test of time and save energy.
> {quote}
> I propose the following method:
> # Start an independent discussion, let's don't mix this question with
> HDDS-1458 and don't block it.
> # Let's propose a more use friendly directory structure first
> # Let's ask [~eyang] to defined _technical_ problems regarding to the
> proposal.
> There are two different view here:
> # Technical limitations which were mentioned earlier
> # Theoretical questions (do we need to have the same structure for core
> Hadoop and Ozone packages)
> I propose to start the discussion with the first one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
