Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree [discussion -> lazy vote]

2019-10-13 Thread Elek, Marton



Sure. Will do it that way.

Thanks the feedback.

Marton

On 10/12/19 7:27 PM, Vinod Kumar Vavilapalli wrote:

On Mon, Sep 23, 2019 at 4:02 PM Elek, Marton  wrote:


  As the overall feedback was positive (in fact many of the answers were
simple +1 votes) I don't think the thread should be repeated under
[VOTE] subject. Therefore I call it for a lazy consensus.



Let's please not do this in the future.

These are large enough changes that we'd like to get formal votes both for
immediate visibility (VOTE threads are more in your face) as well as for
record-keeping in posterity.

Thanks
+Vinod



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree [discussion -> lazy vote]

2019-10-12 Thread Vinod Kumar Vavilapalli
On Mon, Sep 23, 2019 at 4:02 PM Elek, Marton  wrote:

>  As the overall feedback was positive (in fact many of the answers were
> simple +1 votes) I don't think the thread should be repeated under
> [VOTE] subject. Therefore I call it for a lazy consensus.
>

Let's please not do this in the future.

These are large enough changes that we'd like to get formal votes both for
immediate visibility (VOTE threads are more in your face) as well as for
record-keeping in posterity.

Thanks
+Vinod


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree [discussion -> lazy vote]

2019-09-23 Thread Elek, Marton
> Do you see a Submarine like split-also-into-a-TLP for Ozone? If not 
now, sometime further down the line?


Good question, and I don't know what is the best answer right now. It's 
definitely an option, But Submarine move hasn't been finished, so it's 
not yet possible to learn form the experiences (which can be a usefull 
input for the decision).


I think it's a bigger/more important question and I would prefer to 
start a new thread about it.


>  If so, why not do both at the same time?

That's an easier question: I think the repo separation is an easier 
step, with immediate benefits, therefore I would prefer to do it as soon 
as possible.


Moving to a separated TLP may take months (discussion, vote, proposal, 
board approval, etc.). While this code organization step can be done 
easily after the 0.4.1 Ozone release (which is very close, I hope).


As it should be done anyway (with or without separated TLP) I propose to 
do it after the next Ozone release (in the next 1-2 weeks).




As the overall feedback was positive (in fact many of the answers were 
simple +1 votes) I don't think the thread should be repeated under 
[VOTE] subject. Therefore I call it for a lazy consensus. If you have 
any objections (against doing the repo separation now or doing it at 
all) please express in the next 3 days...


Thanks a lot,
Marton

On 9/22/19 4:02 PM, Vinod Kumar Vavilapalli wrote:

Looks to me that the advantages of this additional step are only incremental 
given that you've already decoupled releases and dependencies.

Do you see a Submarine like split-also-into-a-TLP for Ozone? If not now, 
sometime further down the line? If so, why not do both at the same time? I felt 
the same way with Submarine, but couldn't follow up in time.

Thanks
+Vinod


On Sep 18, 2019, at 4:04 AM, Wangda Tan  wrote:

+1 (binding).

 From my experiences of Submarine project, I think moving to a separate repo
helps.

- Wangda

On Tue, Sep 17, 2019 at 11:41 AM Subru Krishnan  wrote:


+1 (binding).

IIUC, there will not be an Ozone module in trunk anymore as that was my
only concern from the original discussion thread? IMHO, this should be the
default approach for new modules.

On Tue, Sep 17, 2019 at 9:58 AM Salvatore LaMendola (BLOOMBERG/ 731 LEX) <
slamendo...@bloomberg.net> wrote:


+1

From: e...@apache.org At: 09/17/19 05:48:32To:

hdfs-...@hadoop.apache.org,

mapreduce-...@hadoop.apache.org,  common-dev@hadoop.apache.org,
yarn-...@hadoop.apache.org
Subject: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree


TLDR; I propose to move Ozone related code out from Hadoop trunk and
store it in a separated *Hadoop* git repository apache/hadoop-ozone.git


When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
be part of the source tree but with separated release cadence, mainly
because it had the hadoop-trunk/SNAPSHOT as compile time dependency.

During the last Ozone releases this dependency is removed to provide
more stable releases. Instead of using the latest trunk/SNAPSHOT build
from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).

As we have no more strict dependency between Hadoop trunk SNAPSHOT and
Ozone trunk I propose to separate the two code base from each other with
creating a new Hadoop git repository (apache/hadoop-ozone.git):

With moving Ozone to a separated git repository:

  * It would be easier to contribute and understand the build (as of now
we always need `-f pom.ozone.xml` as a Maven parameter)
  * It would be possible to adjust build process without breaking
Hadoop/Ozone builds.
  * It would be possible to use different Readme/.asf.yaml/github
template for the Hadoop Ozone and core Hadoop. (For example the current
github template [2] has a link to the contribution guideline [3]. Ozone
has an extended version [4] from this guideline with additional
information.)
  * Testing would be more safe as it won't be possible to change core
Hadoop and Hadoop Ozone in the same patch.
  * It would be easier to cut branches for Hadoop releases (based on the
original consensus, Ozone should be removed from all the release
branches after creating relase branches from trunk)


What do you think?

Thanks,
Marton

[1]:



https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8

c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
[2]:



https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md

[3]:

https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

[4]:



https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org








-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: 

Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-22 Thread Vinod Kumar Vavilapalli
Looks to me that the advantages of this additional step are only incremental 
given that you've already decoupled releases and dependencies.

Do you see a Submarine like split-also-into-a-TLP for Ozone? If not now, 
sometime further down the line? If so, why not do both at the same time? I felt 
the same way with Submarine, but couldn't follow up in time.

Thanks
+Vinod

> On Sep 18, 2019, at 4:04 AM, Wangda Tan  wrote:
> 
> +1 (binding).
> 
> From my experiences of Submarine project, I think moving to a separate repo
> helps.
> 
> - Wangda
> 
> On Tue, Sep 17, 2019 at 11:41 AM Subru Krishnan  wrote:
> 
>> +1 (binding).
>> 
>> IIUC, there will not be an Ozone module in trunk anymore as that was my
>> only concern from the original discussion thread? IMHO, this should be the
>> default approach for new modules.
>> 
>> On Tue, Sep 17, 2019 at 9:58 AM Salvatore LaMendola (BLOOMBERG/ 731 LEX) <
>> slamendo...@bloomberg.net> wrote:
>> 
>>> +1
>>> 
>>> From: e...@apache.org At: 09/17/19 05:48:32To:
>> hdfs-...@hadoop.apache.org,
>>> mapreduce-...@hadoop.apache.org,  common-dev@hadoop.apache.org,
>>> yarn-...@hadoop.apache.org
>>> Subject: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
>>> source tree
>>> 
>>> 
>>> TLDR; I propose to move Ozone related code out from Hadoop trunk and
>>> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>>> 
>>> 
>>> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
>>> be part of the source tree but with separated release cadence, mainly
>>> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>>> 
>>> During the last Ozone releases this dependency is removed to provide
>>> more stable releases. Instead of using the latest trunk/SNAPSHOT build
>>> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>>> 
>>> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
>>> Ozone trunk I propose to separate the two code base from each other with
>>> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>>> 
>>> With moving Ozone to a separated git repository:
>>> 
>>>  * It would be easier to contribute and understand the build (as of now
>>> we always need `-f pom.ozone.xml` as a Maven parameter)
>>>  * It would be possible to adjust build process without breaking
>>> Hadoop/Ozone builds.
>>>  * It would be possible to use different Readme/.asf.yaml/github
>>> template for the Hadoop Ozone and core Hadoop. (For example the current
>>> github template [2] has a link to the contribution guideline [3]. Ozone
>>> has an extended version [4] from this guideline with additional
>>> information.)
>>>  * Testing would be more safe as it won't be possible to change core
>>> Hadoop and Hadoop Ozone in the same patch.
>>>  * It would be easier to cut branches for Hadoop releases (based on the
>>> original consensus, Ozone should be removed from all the release
>>> branches after creating relase branches from trunk)
>>> 
>>> 
>>> What do you think?
>>> 
>>> Thanks,
>>> Marton
>>> 
>>> [1]:
>>> 
>>> 
>> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8
>>> c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
>>> [2]:
>>> 
>>> 
>> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
>>> [3]:
>> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
>>> [4]:
>>> 
>>> 
>> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>>> 
>>> -
>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>> 
>>> 
>>> 
>> 


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-21 Thread Wanqiang Ji
+1 (non-binding)

BR,
Wanqiang Ji

On Tue, Sep 17, 2019 at 5:48 PM Elek, Marton  wrote:

>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
>   * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
>   * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
>   * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
>   * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
>   * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-20 Thread Peter Bacsko
+1 (non-binding)

On Fri, Sep 20, 2019 at 8:01 AM Rakesh Radhakrishnan 
wrote:

> +1
>
> Rakesh
>
> On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri  wrote:
>
> > +1 (binding)
> >
> > Thanks to the Ozone folks for their efforts at maintaining good
> separation
> > with HDFS and common. I took a lot of heat for the unpopular opinion that
> > they should  be separate, so I am glad the process has worked out well
> for
> > both codebases. It looks like my concerns were addressed and I appreciate
> > it.  It is cool to see the evolution here.
> >
> > Aaron
> >
> >
> > On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran
>  > >
> > wrote:
> >
> > > in that case,
> > >
> > > +1 from me (binding)
> > >
> > > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton  wrote:
> > >
> > > >  > one thing to consider here as you are giving up your ability to
> make
> > > >  > changes in hadoop-* modules, including hadoop-common, and their
> > > >  > dependencies, in sync with your own code. That goes for filesystem
> > > > contract
> > > >  > tests.
> > > >  >
> > > >  > are you happy with that?
> > > >
> > > >
> > > > Yes. I think we can live with it.
> > > >
> > > > Fortunatelly the Hadoop parts which are used by Ozone (security +
> rpc)
> > > > are stable enough, we didn't need bigger changes until now (small
> > > > patches are already included in 3.1/3.2).
> > > >
> > > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > > worst (best?) case we can try to do more frequent patch releases from
> > > > Hadoop (if required).
> > > >
> > > >
> > > > m.
> > > >
> > > >
> > > >
> > >
> >
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-20 Thread Rakesh Radhakrishnan
+1

Rakesh

On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri  wrote:

> +1 (binding)
>
> Thanks to the Ozone folks for their efforts at maintaining good separation
> with HDFS and common. I took a lot of heat for the unpopular opinion that
> they should  be separate, so I am glad the process has worked out well for
> both codebases. It looks like my concerns were addressed and I appreciate
> it.  It is cool to see the evolution here.
>
> Aaron
>
>
> On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran  >
> wrote:
>
> > in that case,
> >
> > +1 from me (binding)
> >
> > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton  wrote:
> >
> > >  > one thing to consider here as you are giving up your ability to make
> > >  > changes in hadoop-* modules, including hadoop-common, and their
> > >  > dependencies, in sync with your own code. That goes for filesystem
> > > contract
> > >  > tests.
> > >  >
> > >  > are you happy with that?
> > >
> > >
> > > Yes. I think we can live with it.
> > >
> > > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > > are stable enough, we didn't need bigger changes until now (small
> > > patches are already included in 3.1/3.2).
> > >
> > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > worst (best?) case we can try to do more frequent patch releases from
> > > Hadoop (if required).
> > >
> > >
> > > m.
> > >
> > >
> > >
> >
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-19 Thread Aaron Fabbri
+1 (binding)

Thanks to the Ozone folks for their efforts at maintaining good separation
with HDFS and common. I took a lot of heat for the unpopular opinion that
they should  be separate, so I am glad the process has worked out well for
both codebases. It looks like my concerns were addressed and I appreciate
it.  It is cool to see the evolution here.

Aaron


On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran 
wrote:

> in that case,
>
> +1 from me (binding)
>
> On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton  wrote:
>
> >  > one thing to consider here as you are giving up your ability to make
> >  > changes in hadoop-* modules, including hadoop-common, and their
> >  > dependencies, in sync with your own code. That goes for filesystem
> > contract
> >  > tests.
> >  >
> >  > are you happy with that?
> >
> >
> > Yes. I think we can live with it.
> >
> > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > are stable enough, we didn't need bigger changes until now (small
> > patches are already included in 3.1/3.2).
> >
> > I think it's better to use released Hadoop bits in Ozone anyway, and
> > worst (best?) case we can try to do more frequent patch releases from
> > Hadoop (if required).
> >
> >
> > m.
> >
> >
> >
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-19 Thread Steve Loughran
in that case,

+1 from me (binding)

On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton  wrote:

>  > one thing to consider here as you are giving up your ability to make
>  > changes in hadoop-* modules, including hadoop-common, and their
>  > dependencies, in sync with your own code. That goes for filesystem
> contract
>  > tests.
>  >
>  > are you happy with that?
>
>
> Yes. I think we can live with it.
>
> Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> are stable enough, we didn't need bigger changes until now (small
> patches are already included in 3.1/3.2).
>
> I think it's better to use released Hadoop bits in Ozone anyway, and
> worst (best?) case we can try to do more frequent patch releases from
> Hadoop (if required).
>
>
> m.
>
>
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread 俊平堵
+1.

Thanks,

Junping

Elek, Marton  于2019年9月17日周二 下午5:48写道:

>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
>   * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
>   * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
>   * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
>   * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
>   * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread Arpit Agarwal
+1


> On Sep 17, 2019, at 2:49 AM, Elek, Marton  wrote:
> 
> 
> 
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it 
> in a separated *Hadoop* git repository apache/hadoop-ozone.git
> 
> 
> 
> 
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be 
> part of the source tree but with separated release cadence, mainly because it 
> had the hadoop-trunk/SNAPSHOT as compile time dependency.
> 
> During the last Ozone releases this dependency is removed to provide more 
> stable releases. Instead of using the latest trunk/SNAPSHOT build from 
> Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
> 
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone 
> trunk I propose to separate the two code base from each other with creating a 
> new Hadoop git repository (apache/hadoop-ozone.git):
> 
> With moving Ozone to a separated git repository:
> 
> * It would be easier to contribute and understand the build (as of now we 
> always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone 
> builds.
> * It would be possible to use different Readme/.asf.yaml/github template for 
> the Hadoop Ozone and core Hadoop. (For example the current github template 
> [2] has a link to the contribution guideline [3]. Ozone has an extended 
> version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop 
> and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the 
> original consensus, Ozone should be removed from all the release branches 
> after creating relase branches from trunk)
> 
> 
> What do you think?
> 
> Thanks,
> Marton
> 
> [1]: 
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: 
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: 
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
> 
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> 


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread Elek, Marton

> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem 
contract

> tests.
>
> are you happy with that?


Yes. I think we can live with it.

Fortunatelly the Hadoop parts which are used by Ozone (security + rpc) 
are stable enough, we didn't need bigger changes until now (small 
patches are already included in 3.1/3.2).


I think it's better to use released Hadoop bits in Ozone anyway, and 
worst (best?) case we can try to do more frequent patch releases from 
Hadoop (if required).



m.


On 9/18/19 12:06 PM, Steve Loughran wrote:

one thing to consider here as you are giving up your ability to make
changes in hadoop-* modules, including hadoop-common, and their
dependencies, in sync with your own code. That goes for filesystem contract
tests.

are you happy with that?

On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton  wrote:




TLDR; I propose to move Ozone related code out from Hadoop trunk and
store it in a separated *Hadoop* git repository apache/hadoop-ozone.git




When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
be part of the source tree but with separated release cadence, mainly
because it had the hadoop-trunk/SNAPSHOT as compile time dependency.

During the last Ozone releases this dependency is removed to provide
more stable releases. Instead of using the latest trunk/SNAPSHOT build
from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).

As we have no more strict dependency between Hadoop trunk SNAPSHOT and
Ozone trunk I propose to separate the two code base from each other with
creating a new Hadoop git repository (apache/hadoop-ozone.git):

With moving Ozone to a separated git repository:

   * It would be easier to contribute and understand the build (as of now
we always need `-f pom.ozone.xml` as a Maven parameter)
   * It would be possible to adjust build process without breaking
Hadoop/Ozone builds.
   * It would be possible to use different Readme/.asf.yaml/github
template for the Hadoop Ozone and core Hadoop. (For example the current
github template [2] has a link to the contribution guideline [3]. Ozone
has an extended version [4] from this guideline with additional
information.)
   * Testing would be more safe as it won't be possible to change core
Hadoop and Hadoop Ozone in the same patch.
   * It would be easier to cut branches for Hadoop releases (based on the
original consensus, Ozone should be removed from all the release
branches after creating relase branches from trunk)


What do you think?

Thanks,
Marton

[1]:

https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
[2]:

https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
[3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
[4]:

https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org






-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread Steve Loughran
one thing to consider here as you are giving up your ability to make
changes in hadoop-* modules, including hadoop-common, and their
dependencies, in sync with your own code. That goes for filesystem contract
tests.

are you happy with that?

On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton  wrote:

>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
>   * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
>   * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
>   * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
>   * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
>   * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread Sandeep Nemuri
+1 (non-binding)



On Wed, Sep 18, 2019 at 5:33 AM Weiwei Yang  wrote:

> +1 (binding)
>
> Thanks
> Weiwei
>
> On Wed, Sep 18, 2019 at 6:35 AM Wangda Tan  wrote:
>
> > +1 (binding).
> >
> > From my experiences of Submarine project, I think moving to a separate
> repo
> > helps.
> >
> > - Wangda
> >
> > On Tue, Sep 17, 2019 at 11:41 AM Subru Krishnan 
> wrote:
> >
> > > +1 (binding).
> > >
> > > IIUC, there will not be an Ozone module in trunk anymore as that was my
> > > only concern from the original discussion thread? IMHO, this should be
> > the
> > > default approach for new modules.
> > >
> > > On Tue, Sep 17, 2019 at 9:58 AM Salvatore LaMendola (BLOOMBERG/ 731
> LEX)
> > <
> > > slamendo...@bloomberg.net> wrote:
> > >
> > > > +1
> > > >
> > > > From: e...@apache.org At: 09/17/19 05:48:32To:
> > > hdfs-...@hadoop.apache.org,
> > > > mapreduce-...@hadoop.apache.org,  common-dev@hadoop.apache.org,
> > > > yarn-...@hadoop.apache.org
> > > > Subject: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
> > > > source tree
> > > >
> > > >
> > > > TLDR; I propose to move Ozone related code out from Hadoop trunk and
> > > > store it in a separated *Hadoop* git repository
> apache/hadoop-ozone.git
> > > >
> > > >
> > > > When Ozone was adopted as a new Hadoop subproject it was proposed[1]
> to
> > > > be part of the source tree but with separated release cadence, mainly
> > > > because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
> > > >
> > > > During the last Ozone releases this dependency is removed to provide
> > > > more stable releases. Instead of using the latest trunk/SNAPSHOT
> build
> > > > from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
> > > >
> > > > As we have no more strict dependency between Hadoop trunk SNAPSHOT
> and
> > > > Ozone trunk I propose to separate the two code base from each other
> > with
> > > > creating a new Hadoop git repository (apache/hadoop-ozone.git):
> > > >
> > > > With moving Ozone to a separated git repository:
> > > >
> > > >   * It would be easier to contribute and understand the build (as of
> > now
> > > > we always need `-f pom.ozone.xml` as a Maven parameter)
> > > >   * It would be possible to adjust build process without breaking
> > > > Hadoop/Ozone builds.
> > > >   * It would be possible to use different Readme/.asf.yaml/github
> > > > template for the Hadoop Ozone and core Hadoop. (For example the
> current
> > > > github template [2] has a link to the contribution guideline [3].
> Ozone
> > > > has an extended version [4] from this guideline with additional
> > > > information.)
> > > >   * Testing would be more safe as it won't be possible to change core
> > > > Hadoop and Hadoop Ozone in the same patch.
> > > >   * It would be easier to cut branches for Hadoop releases (based on
> > the
> > > > original consensus, Ozone should be removed from all the release
> > > > branches after creating relase branches from trunk)
> > > >
> > > >
> > > > What do you think?
> > > >
> > > > Thanks,
> > > > Marton
> > > >
> > > > [1]:
> > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8
> > > > c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> > > > [2]:
> > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> > > > [3]:
> > > https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> > > > [4]:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
> > > >
> > > > -
> > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > > >
> > > >
> > > >
> > >
> >
>


-- 
*  Regards*
*  Sandeep Nemuri*


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-17 Thread Weiwei Yang
+1 (binding)

Thanks
Weiwei

On Wed, Sep 18, 2019 at 6:35 AM Wangda Tan  wrote:

> +1 (binding).
>
> From my experiences of Submarine project, I think moving to a separate repo
> helps.
>
> - Wangda
>
> On Tue, Sep 17, 2019 at 11:41 AM Subru Krishnan  wrote:
>
> > +1 (binding).
> >
> > IIUC, there will not be an Ozone module in trunk anymore as that was my
> > only concern from the original discussion thread? IMHO, this should be
> the
> > default approach for new modules.
> >
> > On Tue, Sep 17, 2019 at 9:58 AM Salvatore LaMendola (BLOOMBERG/ 731 LEX)
> <
> > slamendo...@bloomberg.net> wrote:
> >
> > > +1
> > >
> > > From: e...@apache.org At: 09/17/19 05:48:32To:
> > hdfs-...@hadoop.apache.org,
> > > mapreduce-...@hadoop.apache.org,  common-dev@hadoop.apache.org,
> > > yarn-...@hadoop.apache.org
> > > Subject: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
> > > source tree
> > >
> > >
> > > TLDR; I propose to move Ozone related code out from Hadoop trunk and
> > > store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
> > >
> > >
> > > When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> > > be part of the source tree but with separated release cadence, mainly
> > > because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
> > >
> > > During the last Ozone releases this dependency is removed to provide
> > > more stable releases. Instead of using the latest trunk/SNAPSHOT build
> > > from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
> > >
> > > As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> > > Ozone trunk I propose to separate the two code base from each other
> with
> > > creating a new Hadoop git repository (apache/hadoop-ozone.git):
> > >
> > > With moving Ozone to a separated git repository:
> > >
> > >   * It would be easier to contribute and understand the build (as of
> now
> > > we always need `-f pom.ozone.xml` as a Maven parameter)
> > >   * It would be possible to adjust build process without breaking
> > > Hadoop/Ozone builds.
> > >   * It would be possible to use different Readme/.asf.yaml/github
> > > template for the Hadoop Ozone and core Hadoop. (For example the current
> > > github template [2] has a link to the contribution guideline [3]. Ozone
> > > has an extended version [4] from this guideline with additional
> > > information.)
> > >   * Testing would be more safe as it won't be possible to change core
> > > Hadoop and Hadoop Ozone in the same patch.
> > >   * It would be easier to cut branches for Hadoop releases (based on
> the
> > > original consensus, Ozone should be removed from all the release
> > > branches after creating relase branches from trunk)
> > >
> > >
> > > What do you think?
> > >
> > > Thanks,
> > > Marton
> > >
> > > [1]:
> > >
> > >
> >
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8
> > > c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> > > [2]:
> > >
> > >
> >
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> > > [3]:
> > https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> > > [4]:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
> > >
> > > -
> > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > >
> > >
> > >
> >
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-17 Thread Wangda Tan
+1 (binding).

>From my experiences of Submarine project, I think moving to a separate repo
helps.

- Wangda

On Tue, Sep 17, 2019 at 11:41 AM Subru Krishnan  wrote:

> +1 (binding).
>
> IIUC, there will not be an Ozone module in trunk anymore as that was my
> only concern from the original discussion thread? IMHO, this should be the
> default approach for new modules.
>
> On Tue, Sep 17, 2019 at 9:58 AM Salvatore LaMendola (BLOOMBERG/ 731 LEX) <
> slamendo...@bloomberg.net> wrote:
>
> > +1
> >
> > From: e...@apache.org At: 09/17/19 05:48:32To:
> hdfs-...@hadoop.apache.org,
> > mapreduce-...@hadoop.apache.org,  common-dev@hadoop.apache.org,
> > yarn-...@hadoop.apache.org
> > Subject: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
> > source tree
> >
> >
> > TLDR; I propose to move Ozone related code out from Hadoop trunk and
> > store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
> >
> >
> > When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> > be part of the source tree but with separated release cadence, mainly
> > because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
> >
> > During the last Ozone releases this dependency is removed to provide
> > more stable releases. Instead of using the latest trunk/SNAPSHOT build
> > from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
> >
> > As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> > Ozone trunk I propose to separate the two code base from each other with
> > creating a new Hadoop git repository (apache/hadoop-ozone.git):
> >
> > With moving Ozone to a separated git repository:
> >
> >   * It would be easier to contribute and understand the build (as of now
> > we always need `-f pom.ozone.xml` as a Maven parameter)
> >   * It would be possible to adjust build process without breaking
> > Hadoop/Ozone builds.
> >   * It would be possible to use different Readme/.asf.yaml/github
> > template for the Hadoop Ozone and core Hadoop. (For example the current
> > github template [2] has a link to the contribution guideline [3]. Ozone
> > has an extended version [4] from this guideline with additional
> > information.)
> >   * Testing would be more safe as it won't be possible to change core
> > Hadoop and Hadoop Ozone in the same patch.
> >   * It would be easier to cut branches for Hadoop releases (based on the
> > original consensus, Ozone should be removed from all the release
> > branches after creating relase branches from trunk)
> >
> >
> > What do you think?
> >
> > Thanks,
> > Marton
> >
> > [1]:
> >
> >
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8
> > c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> > [2]:
> >
> >
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> > [3]:
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> > [4]:
> >
> >
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> >
> >
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-17 Thread Subru Krishnan
+1 (binding).

IIUC, there will not be an Ozone module in trunk anymore as that was my
only concern from the original discussion thread? IMHO, this should be the
default approach for new modules.

On Tue, Sep 17, 2019 at 9:58 AM Salvatore LaMendola (BLOOMBERG/ 731 LEX) <
slamendo...@bloomberg.net> wrote:

> +1
>
> From: e...@apache.org At: 09/17/19 05:48:32To:  hdfs-...@hadoop.apache.org,
> mapreduce-...@hadoop.apache.org,  common-dev@hadoop.apache.org,
> yarn-...@hadoop.apache.org
> Subject: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
> source tree
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
>   * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
>   * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
>   * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
>   * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
>   * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8
> c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>
>


Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-17 Thread Dinesh Chitlangia
+1

-Dinesh




On Tue, Sep 17, 2019 at 12:58 PM Salvatore LaMendola (BLOOMBERG/ 731 LEX) <
slamendo...@bloomberg.net> wrote:

> +1
>
> From: e...@apache.org At: 09/17/19 05:48:32To:  hdfs-...@hadoop.apache.org,
> mapreduce-...@hadoop.apache.org,  common-dev@hadoop.apache.org,
> yarn-...@hadoop.apache.org
> Subject: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
> source tree
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
>   * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
>   * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
>   * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
>   * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
>   * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8
> c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>
>


Re:[DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-17 Thread Salvatore LaMendola (BLOOMBERG/ 731 LEX)
+1

From: e...@apache.org At: 09/17/19 05:48:32To:  hdfs-...@hadoop.apache.org,  
mapreduce-...@hadoop.apache.org,  common-dev@hadoop.apache.org,  
yarn-...@hadoop.apache.org
Subject: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree


TLDR; I propose to move Ozone related code out from Hadoop trunk and 
store it in a separated *Hadoop* git repository apache/hadoop-ozone.git


When Ozone was adopted as a new Hadoop subproject it was proposed[1] to 
be part of the source tree but with separated release cadence, mainly 
because it had the hadoop-trunk/SNAPSHOT as compile time dependency.

During the last Ozone releases this dependency is removed to provide 
more stable releases. Instead of using the latest trunk/SNAPSHOT build 
from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).

As we have no more strict dependency between Hadoop trunk SNAPSHOT and 
Ozone trunk I propose to separate the two code base from each other with 
creating a new Hadoop git repository (apache/hadoop-ozone.git):

With moving Ozone to a separated git repository:

  * It would be easier to contribute and understand the build (as of now 
we always need `-f pom.ozone.xml` as a Maven parameter)
  * It would be possible to adjust build process without breaking 
Hadoop/Ozone builds.
  * It would be possible to use different Readme/.asf.yaml/github 
template for the Hadoop Ozone and core Hadoop. (For example the current 
github template [2] has a link to the contribution guideline [3]. Ozone 
has an extended version [4] from this guideline with additional 
information.)
  * Testing would be more safe as it won't be possible to change core 
Hadoop and Hadoop Ozone in the same patch.
  * It would be easier to cut branches for Hadoop releases (based on the 
original consensus, Ozone should be removed from all the release 
branches after creating relase branches from trunk)


What do you think?

Thanks,
Marton

[1]: 
https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8
c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
[2]: 
https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
[3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
[4]: 
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org




Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-17 Thread Anu Engineer
+1
—Anu

> On Sep 17, 2019, at 2:49 AM, Elek, Marton  wrote:
> 
> 
> 
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it 
> in a separated *Hadoop* git repository apache/hadoop-ozone.git
> 
> 
> 
> 
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be 
> part of the source tree but with separated release cadence, mainly because it 
> had the hadoop-trunk/SNAPSHOT as compile time dependency.
> 
> During the last Ozone releases this dependency is removed to provide more 
> stable releases. Instead of using the latest trunk/SNAPSHOT build from 
> Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
> 
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone 
> trunk I propose to separate the two code base from each other with creating a 
> new Hadoop git repository (apache/hadoop-ozone.git):
> 
> With moving Ozone to a separated git repository:
> 
> * It would be easier to contribute and understand the build (as of now we 
> always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone 
> builds.
> * It would be possible to use different Readme/.asf.yaml/github template for 
> the Hadoop Ozone and core Hadoop. (For example the current github template 
> [2] has a link to the contribution guideline [3]. Ozone has an extended 
> version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop 
> and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the 
> original consensus, Ozone should be removed from all the release branches 
> after creating relase branches from trunk)
> 
> 
> What do you think?
> 
> Thanks,
> Marton
> 
> [1]: 
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: 
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: 
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
> 
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> 

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org