Re: segment-tar depending on oak-core

[Julian Sedding]

Hi all

My preference is also with a higher degree of modularity. Compared to
a monolithic application it is a trade-off that leads to both, higher
complexity and higher flexibility. Provided we are willing to change
and learn, I am sure we can easily manage the complexity. Numerous
benefits of the extra flexibility have been mentioned in this thread
before, so I won't repeat them.

As I understand it the Oak package structure was designed to
facilitate modularity very early on. As Jukka wrote back in 2012:

"[...] Ultimately such extra plugin components may well end up as
separate Maven components, but until the related service interfaces
and plugin boundaries are well defined it's better to keep all such
code together and simply use Java package boundaries to separate them.
That's the rationale behind the .oak.plugins package [...]"[0].

IMHO, now that the API boundaries are well defined (I hope), it would
be great to finally move the structure of the code-base and release
artifacts towards a more modular approach.

Regards
Julian

[0] http://markmail.org/thread/cs34a637dr26xscj


On Fri, Oct 28, 2016 at 8:29 AM, Francesco Mari
 wrote:
> Hi
>
> 2016-10-27 19:08 GMT+02:00 Alexander Klimetschek :
>> Maybe looking at this step by step would help.
>
> The oak-segment-tar bundle was supposed to be the first step.
>
>>For example, start with the nodestore implementations and extract everything 
>>into separate modules that is necessary for this - i.e. an oak-store-api 
>>along with the impls. But keep other apis in oak-core in that first step, to 
>>limit the effort. (And try not renaming the API packages, as well as keeping 
>>them backwards compatible, i.e. no major version bump, if possible).
>
> This didn't happen because of lack of consensus. See my previous
> answer to Michael Marth.
>
>>See how that works out and if positive, continue with more.
>
> The reaction to the modularization effort was not positive, so
> oak-segment-tar backed up.
>
>>
>> Cheers,
>> Alex
>>
>> Am 27. Okt. 2016, 03:48 -0700 schrieb Francesco Mari 
>> :
>> Something did happen: the first NodeStore implementation living in its
>> own module was oak-segment-tar. We just decided to go back to the old
>> model exactly because we didn't reach consensus about modularizing its
>> upstream and downstream dependencies.
>>
>> 2016-10-27 12:22 GMT+02:00 Michael Marth :
>> fwiw: last year a concrete proposal was made that seemed to have consensus
>>
>> “Move NodeStore implementations into their own modules"
>> http://markmail.org/message/6ylxk4twdi2lzfdz
>>
>> Agree that nothing happened - but I believe that this move might again find 
>> consenus
>>
>>
>>
>> On 27/10/16 10:49, "Francesco Mari"  wrote:
>>
>> We keep having this conversation regularly but nothing ever changes.
>> As much as I would like to push the modularization effort forward, I
>> recognize that the majority of the team is either not in favour or
>> openly against it. I don't want to disrupt the way most of us are used
>> to work. Michael Dürig already provided an extensive list of what we
>> will be missing if we keep writing software the way we do, so I'm not
>> going to repeat it. The most sensible thing to do is, in my humble
>> opinion, accept the decision of the majority.
>>
>> 2016-10-27 11:05 GMT+02:00 Davide Giannella :
>> On 27/10/2016 08:53, Michael Dürig wrote:
>>
>> +1.
>>
>> It would also help re. backporting, continuous integration, releasing,
>> testing, longevity, code reuse, maintainability, reducing technical
>> debt, deploying, stability, etc, etc...
>>
>> While I can agree on the above, and the fact that now we have
>> https://issues.apache.org/jira/browse/OAK-5007 in place, just for the
>> sake or argument I would say that if we want to have any part of Oak
>> with an independent release cycle we need to
>>
>> Have proper API packages that abstract things. Specially from oak-core
>>
>> As soon as we introduce a separate release cycle for a single module we
>> have to look at a wider picture. What other modules are affected?
>>
>> Taking the example of segment-tar we saw that we need
>>
>> - oak-core-api (name can be changed)
>> - independent releases of the oak tools: oak-run, oak-upgrade, ...
>> - independent release cycle for parent/pom.xml
>> - anything I'm missing?
>>
>> So if we want to go down that route than we have to do it properly and
>> for good. Not half-way.
>>
>> Davide
>>
>>

Re: segment-tar depending on oak-core

[Francesco Mari]

Hi

2016-10-27 19:08 GMT+02:00 Alexander Klimetschek :
> Maybe looking at this step by step would help.

The oak-segment-tar bundle was supposed to be the first step.

>For example, start with the nodestore implementations and extract everything 
>into separate modules that is necessary for this - i.e. an oak-store-api along 
>with the impls. But keep other apis in oak-core in that first step, to limit 
>the effort. (And try not renaming the API packages, as well as keeping them 
>backwards compatible, i.e. no major version bump, if possible).

This didn't happen because of lack of consensus. See my previous
answer to Michael Marth.

>See how that works out and if positive, continue with more.

The reaction to the modularization effort was not positive, so
oak-segment-tar backed up.

>
> Cheers,
> Alex
>
> Am 27. Okt. 2016, 03:48 -0700 schrieb Francesco Mari 
> :
> Something did happen: the first NodeStore implementation living in its
> own module was oak-segment-tar. We just decided to go back to the old
> model exactly because we didn't reach consensus about modularizing its
> upstream and downstream dependencies.
>
> 2016-10-27 12:22 GMT+02:00 Michael Marth :
> fwiw: last year a concrete proposal was made that seemed to have consensus
>
> “Move NodeStore implementations into their own modules"
> http://markmail.org/message/6ylxk4twdi2lzfdz
>
> Agree that nothing happened - but I believe that this move might again find 
> consenus
>
>
>
> On 27/10/16 10:49, "Francesco Mari"  wrote:
>
> We keep having this conversation regularly but nothing ever changes.
> As much as I would like to push the modularization effort forward, I
> recognize that the majority of the team is either not in favour or
> openly against it. I don't want to disrupt the way most of us are used
> to work. Michael Dürig already provided an extensive list of what we
> will be missing if we keep writing software the way we do, so I'm not
> going to repeat it. The most sensible thing to do is, in my humble
> opinion, accept the decision of the majority.
>
> 2016-10-27 11:05 GMT+02:00 Davide Giannella :
> On 27/10/2016 08:53, Michael Dürig wrote:
>
> +1.
>
> It would also help re. backporting, continuous integration, releasing,
> testing, longevity, code reuse, maintainability, reducing technical
> debt, deploying, stability, etc, etc...
>
> While I can agree on the above, and the fact that now we have
> https://issues.apache.org/jira/browse/OAK-5007 in place, just for the
> sake or argument I would say that if we want to have any part of Oak
> with an independent release cycle we need to
>
> Have proper API packages that abstract things. Specially from oak-core
>
> As soon as we introduce a separate release cycle for a single module we
> have to look at a wider picture. What other modules are affected?
>
> Taking the example of segment-tar we saw that we need
>
> - oak-core-api (name can be changed)
> - independent releases of the oak tools: oak-run, oak-upgrade, ...
> - independent release cycle for parent/pom.xml
> - anything I'm missing?
>
> So if we want to go down that route than we have to do it properly and
> for good. Not half-way.
>
> Davide
>
>

Re: segment-tar depending on oak-core

[Alexander Klimetschek]

Maybe looking at this step by step would help. For example, start with the 
nodestore implementations and extract everything into separate modules that is 
necessary for this - i.e. an oak-store-api along with the impls. But keep other 
apis in oak-core in that first step, to limit the effort. (And try not renaming 
the API packages, as well as keeping them backwards compatible, i.e. no major 
version bump, if possible). See how that works out and if positive, continue 
with more.

Cheers,
Alex

Am 27. Okt. 2016, 03:48 -0700 schrieb Francesco Mari :
Something did happen: the first NodeStore implementation living in its
own module was oak-segment-tar. We just decided to go back to the old
model exactly because we didn't reach consensus about modularizing its
upstream and downstream dependencies.

2016-10-27 12:22 GMT+02:00 Michael Marth :
fwiw: last year a concrete proposal was made that seemed to have consensus

“Move NodeStore implementations into their own modules"
http://markmail.org/message/6ylxk4twdi2lzfdz

Agree that nothing happened - but I believe that this move might again find 
consenus



On 27/10/16 10:49, "Francesco Mari"  wrote:

We keep having this conversation regularly but nothing ever changes.
As much as I would like to push the modularization effort forward, I
recognize that the majority of the team is either not in favour or
openly against it. I don't want to disrupt the way most of us are used
to work. Michael Dürig already provided an extensive list of what we
will be missing if we keep writing software the way we do, so I'm not
going to repeat it. The most sensible thing to do is, in my humble
opinion, accept the decision of the majority.

2016-10-27 11:05 GMT+02:00 Davide Giannella :
On 27/10/2016 08:53, Michael Dürig wrote:

+1.

It would also help re. backporting, continuous integration, releasing,
testing, longevity, code reuse, maintainability, reducing technical
debt, deploying, stability, etc, etc...

While I can agree on the above, and the fact that now we have
https://issues.apache.org/jira/browse/OAK-5007 in place, just for the
sake or argument I would say that if we want to have any part of Oak
with an independent release cycle we need to

Have proper API packages that abstract things. Specially from oak-core

As soon as we introduce a separate release cycle for a single module we
have to look at a wider picture. What other modules are affected?

Taking the example of segment-tar we saw that we need

- oak-core-api (name can be changed)
- independent releases of the oak tools: oak-run, oak-upgrade, ...
- independent release cycle for parent/pom.xml
- anything I'm missing?

So if we want to go down that route than we have to do it properly and
for good. Not half-way.

Davide

Re: segment-tar depending on oak-core

[Francesco Mari]

Something did happen: the first NodeStore implementation living in its
own module was oak-segment-tar. We just decided to go back to the old
model exactly because we didn't reach consensus about modularizing its
upstream and downstream dependencies.

2016-10-27 12:22 GMT+02:00 Michael Marth :
> fwiw: last year a concrete proposal was made that seemed to have consensus
>
> “Move NodeStore implementations into their own modules"
> http://markmail.org/message/6ylxk4twdi2lzfdz
>
> Agree that nothing happened - but I believe that this move might again find 
> consenus
>
>
>
> On 27/10/16 10:49, "Francesco Mari"  wrote:
>
>>We keep having this conversation regularly but nothing ever changes.
>>As much as I would like to push the modularization effort forward, I
>>recognize that the majority of the team is either not in favour or
>>openly against it. I don't want to disrupt the way most of us are used
>>to work. Michael Dürig already provided an extensive list of what we
>>will be missing if we keep writing software the way we do, so I'm not
>>going to repeat it. The most sensible thing to do is, in my humble
>>opinion, accept the decision of the majority.
>>
>>2016-10-27 11:05 GMT+02:00 Davide Giannella :
>>> On 27/10/2016 08:53, Michael Dürig wrote:

 +1.

 It would also help re. backporting, continuous integration, releasing,
 testing, longevity, code reuse, maintainability, reducing technical
 debt, deploying, stability, etc, etc...
>>>
>>> While I can agree on the above, and the fact that now we have
>>> https://issues.apache.org/jira/browse/OAK-5007 in place, just for the
>>> sake or argument I would say that if we want to have any part of Oak
>>> with an independent release cycle we need to
>>>
>>> Have proper API packages that abstract things. Specially from oak-core
>>>
>>> As soon as we introduce a separate release cycle for a single module we
>>> have to look at a wider picture. What other modules are affected?
>>>
>>> Taking the example of segment-tar we saw that we need
>>>
>>> - oak-core-api (name can be changed)
>>> - independent releases of the oak tools: oak-run, oak-upgrade, ...
>>> - independent release cycle for parent/pom.xml
>>> - anything I'm missing?
>>>
>>> So if we want to go down that route than we have to do it properly and
>>> for good. Not half-way.
>>>
>>> Davide
>>>
>>>

Re: segment-tar depending on oak-core

[Michael Marth]

fwiw: last year a concrete proposal was made that seemed to have consensus

“Move NodeStore implementations into their own modules"
http://markmail.org/message/6ylxk4twdi2lzfdz

Agree that nothing happened - but I believe that this move might again find 
consenus



On 27/10/16 10:49, "Francesco Mari"  wrote:

>We keep having this conversation regularly but nothing ever changes.
>As much as I would like to push the modularization effort forward, I
>recognize that the majority of the team is either not in favour or
>openly against it. I don't want to disrupt the way most of us are used
>to work. Michael Dürig already provided an extensive list of what we
>will be missing if we keep writing software the way we do, so I'm not
>going to repeat it. The most sensible thing to do is, in my humble
>opinion, accept the decision of the majority.
>
>2016-10-27 11:05 GMT+02:00 Davide Giannella :
>> On 27/10/2016 08:53, Michael Dürig wrote:
>>>
>>> +1.
>>>
>>> It would also help re. backporting, continuous integration, releasing,
>>> testing, longevity, code reuse, maintainability, reducing technical
>>> debt, deploying, stability, etc, etc...
>>
>> While I can agree on the above, and the fact that now we have
>> https://issues.apache.org/jira/browse/OAK-5007 in place, just for the
>> sake or argument I would say that if we want to have any part of Oak
>> with an independent release cycle we need to
>>
>> Have proper API packages that abstract things. Specially from oak-core
>>
>> As soon as we introduce a separate release cycle for a single module we
>> have to look at a wider picture. What other modules are affected?
>>
>> Taking the example of segment-tar we saw that we need
>>
>> - oak-core-api (name can be changed)
>> - independent releases of the oak tools: oak-run, oak-upgrade, ...
>> - independent release cycle for parent/pom.xml
>> - anything I'm missing?
>>
>> So if we want to go down that route than we have to do it properly and
>> for good. Not half-way.
>>
>> Davide
>>
>>

Re: segment-tar depending on oak-core

[Francesco Mari]

We keep having this conversation regularly but nothing ever changes.
As much as I would like to push the modularization effort forward, I
recognize that the majority of the team is either not in favour or
openly against it. I don't want to disrupt the way most of us are used
to work. Michael Dürig already provided an extensive list of what we
will be missing if we keep writing software the way we do, so I'm not
going to repeat it. The most sensible thing to do is, in my humble
opinion, accept the decision of the majority.

2016-10-27 11:05 GMT+02:00 Davide Giannella :
> On 27/10/2016 08:53, Michael Dürig wrote:
>>
>> +1.
>>
>> It would also help re. backporting, continuous integration, releasing,
>> testing, longevity, code reuse, maintainability, reducing technical
>> debt, deploying, stability, etc, etc...
>
> While I can agree on the above, and the fact that now we have
> https://issues.apache.org/jira/browse/OAK-5007 in place, just for the
> sake or argument I would say that if we want to have any part of Oak
> with an independent release cycle we need to
>
> Have proper API packages that abstract things. Specially from oak-core
>
> As soon as we introduce a separate release cycle for a single module we
> have to look at a wider picture. What other modules are affected?
>
> Taking the example of segment-tar we saw that we need
>
> - oak-core-api (name can be changed)
> - independent releases of the oak tools: oak-run, oak-upgrade, ...
> - independent release cycle for parent/pom.xml
> - anything I'm missing?
>
> So if we want to go down that route than we have to do it properly and
> for good. Not half-way.
>
> Davide
>
>

Re: segment-tar depending on oak-core

[Davide Giannella]

On 27/10/2016 08:53, Michael Dürig wrote:
>
> +1.
>
> It would also help re. backporting, continuous integration, releasing,
> testing, longevity, code reuse, maintainability, reducing technical
> debt, deploying, stability, etc, etc...

While I can agree on the above, and the fact that now we have
https://issues.apache.org/jira/browse/OAK-5007 in place, just for the
sake or argument I would say that if we want to have any part of Oak
with an independent release cycle we need to

Have proper API packages that abstract things. Specially from oak-core

As soon as we introduce a separate release cycle for a single module we
have to look at a wider picture. What other modules are affected?

Taking the example of segment-tar we saw that we need

- oak-core-api (name can be changed)
- independent releases of the oak tools: oak-run, oak-upgrade, ...
- independent release cycle for parent/pom.xml
- anything I'm missing?

So if we want to go down that route than we have to do it properly and
for good. Not half-way.

Davide

Re: segment-tar depending on oak-core

[Michael Dürig]

On 26.10.16 9:44 , Alexander Klimetschek wrote:

FWIW, I would prefer oak to have better modularization.

It also overlaps with Michael Marth's other recent thread "On adding new APIs" 
- the exported APIs in Oak are all ad hoc because of it's monolithic design (i.e. all in 
oak-core) and no clear API packages designed upfront. Someone had to figure out what 
packages need to exported after the fact, and a lot of things are exported that should 
really not be, and because of that, you have to update a lot more oak code when something 
changes in these, because it inadvertently depends on these exported classes, although it 
doesn't use them.

Building a patch for Oak to try something out as an outside is very difficult, 
as things aren't well modularized. I was recently involved with 
oak-auth-external, and you can only really work on it if you always rebuild and 
redeploy the whole of oak. It's pretty painful… builds are also slow, as 
because of the snapshot dependencies you have to build everything which takes a 
lot of time with all the tests. With released dependencies between the modules, 
one could work on a higher level module without worrying about the lower layer, 
as it stays in the same released version. I would argue you might get more 
contributions if things were easier on this front.

Releasing modularized stuff can be easier: if there is only a change in one 
module, you only have to release one, instead of all of them. Faster 
turnaround. A


+1.

It would also help re. backporting, continuous integration, releasing, 
testing, longevity, code reuse, maintainability, reducing technical 
debt, deploying, stability, etc, etc...


Michael



Just my 2 cents,
Alex

Re: segment-tar depending on oak-core

[Alexander Klimetschek]

FWIW, I would prefer oak to have better modularization.

It also overlaps with Michael Marth's other recent thread "On adding new APIs" 
- the exported APIs in Oak are all ad hoc because of it's monolithic design 
(i.e. all in oak-core) and no clear API packages designed upfront. Someone had 
to figure out what packages need to exported after the fact, and a lot of 
things are exported that should really not be, and because of that, you have to 
update a lot more oak code when something changes in these, because it 
inadvertently depends on these exported classes, although it doesn't use them.

Building a patch for Oak to try something out as an outside is very difficult, 
as things aren't well modularized. I was recently involved with 
oak-auth-external, and you can only really work on it if you always rebuild and 
redeploy the whole of oak. It's pretty painful… builds are also slow, as 
because of the snapshot dependencies you have to build everything which takes a 
lot of time with all the tests. With released dependencies between the modules, 
one could work on a higher level module without worrying about the lower layer, 
as it stays in the same released version. I would argue you might get more 
contributions if things were easier on this front.

Releasing modularized stuff can be easier: if there is only a change in one 
module, you only have to release one, instead of all of them. Faster 
turnaround. A

Just my 2 cents,
Alex

Re: segment-tar depending on oak-core

[Michael Dürig]

On 25.10.16 11:05 , Davide Giannella wrote:

On 24/10/2016 14:30, Julian Reschke wrote:

On 2016-10-24 15:10, Davide Giannella wrote:

Wow, quite some replies :)
...
However we still have the original problem to address: how are we
planning to solve segment depending on oak depending on segment on third
party applications? I have two proposals here that we could vote on if
we want

1) Going back to monolithic? Read it as, segment-tar won't be released
any more on its own cycle.

2) We correctly address dependencies. For example adding oak-core-api.
...


If I had to do Oak releases, I'd clearly vote for 1) - this model has
drawbacks, but it's simple and easy to understand.


yes from a release pov it's way easier. However I'd like to ask to the
people who decided to go for the independent segment-tar release. What's
the rationale behind the decision? Why did we go in that direction that
was not possible to do with a monolithic?

Again, to be clear: I'm not championing neither one or the other
solution. I simply would like to a a solution done properly, and
hopefully quickly so that we ease our lives later on rather than
complicate it.


See my earlier reply to Julian Sedding where I stated the requirement to 
that respect.


Michael

Re: segment-tar depending on oak-core

[Davide Giannella]

On 24/10/2016 14:30, Julian Reschke wrote:
> On 2016-10-24 15:10, Davide Giannella wrote:
>> Wow, quite some replies :)
>> ...
>> However we still have the original problem to address: how are we
>> planning to solve segment depending on oak depending on segment on third
>> party applications? I have two proposals here that we could vote on if
>> we want
>>
>> 1) Going back to monolithic? Read it as, segment-tar won't be released
>> any more on its own cycle.
>>
>> 2) We correctly address dependencies. For example adding oak-core-api.
>> ...
>
> If I had to do Oak releases, I'd clearly vote for 1) - this model has
> drawbacks, but it's simple and easy to understand.

yes from a release pov it's way easier. However I'd like to ask to the
people who decided to go for the independent segment-tar release. What's
the rationale behind the decision? Why did we go in that direction that
was not possible to do with a monolithic?

Again, to be clear: I'm not championing neither one or the other
solution. I simply would like to a a solution done properly, and
hopefully quickly so that we ease our lives later on rather than
complicate it.

Davide

Re: segment-tar depending on oak-core

[Thomas Mueller]

Hi,

There are two "extreme" cases, and both are used and work fine (please
nobody says "it's a joke", and "monolithic" is worse):

* "Monolithic": Linux, Apache Lucene, and so on: one version for everything

* "Fine grained": Apache Sling: separate, independent versions for
everything

(actually I don't know more examples of "Fine grained")

Apache Sling doesn't really maintain multiple branches in the same way we
do in Oak. I argue that having to maintain multiple branches is easier
with the "monolithic" approach.

Re: segment-tar depending on oak-core

[Davide Giannella]

Wow, quite some replies :)

See in-line, trying to replying to everyone in one email

On 21/10/2016 13:08, Thomas Mueller wrote:
> I would prefer if we get oak-segment-tar in line with the rest of oak
> (release it at the same time and so on). I understand, there are some
> disadvantages. But I think all alternatives also have disadvantages.

As one of the people who deal with the releases mainly this will help
greatly. However I have to ask here: is this what we want? I'm not
expressing any opinion. I'm simply asking what are we expecting to have
from oak. If we're expecting to reuse segment-tar somewhere else, in
other projects, then modularity is the way to go. However anyone willing
to use segment-tar outside of Oak will have deal with quite a bunch of
dependencies: oak-core in primis.

If we're speaking of having a separate release cycle just for being free
from the Oak rc, then we better reconsider the move, as we're releasing
trunk every two weeks, which is quite frequent and allows for testing in
third party products.

On 21/10/2016 14:04, Julian Sedding wrote:
> While this may be possible, an arguably simpler solution would be to
> give oak-run and oak-upgrade a separate lifecycle. They are consumers
> of both segment-tar and oak-core (+ other bundles with same release
> cycle). Hence they require interoperable releases of both *before*
> they themselves can be released.

+1 In general. However, while it will solve the oak dependencies from
itself, which is already a big win, it won't solve third party
dependencies problems. Read it other applications leveraging
Oak+segment. For those we still need either oak-core-api (for example)
or a single monolithic release. Again I don't mind one or the other, but
we have to decide and act for good, not half solutions.

On 21/10/2016 14:09, Francesco Mari wrote:
> ...I don't even know how many releases of our parent POM we
> have, every one of them equal to the other, and this is nonsense.

Yes it's true. However I don't see any issues here. It's simplifying a
lot our process and no one else is depending on oak-parent afaik. If
segment-tar depends on oak-parent then it makes sense to have a
different release cycle for oak-parent as well. We're back at the
previous point though, we have to do it well or not. We can't stay in
the middle-ground.

On 21/10/2016 16:41, Alex Parvulescu wrote:
> ...everyone asks what the *oak* version is that
> goes with each segment release,...

This has been another topic I've been willing to bring up for a while.
I've eased my life a bit with a script (if that could help)
https://github.com/davidegiannella/misc/blob/master/oaky-stuff/oak-dependecies.sh.
However as of the interdependencies we're speaking of it doesn't really
reflect the truth. See oak 1.5.12 and segment 0.0.14.

On 24/10/2016 12:48, Michael Marth wrote:
(and Tommaso)
> I agree with your assessment that this discussion is actually about the 
> delivery granularity and user’s consumption of Oak. Taking the freedom to 
> re-phrase what you said above:
>
>   *   either a complete library that is consumed as a whole (and where the 
> various internal modules are implementation details)
>   *   Or a set of modules where users are expected and allowed to access the 
> modules directly and deploy arbitrary subsets of modules

and here's the point. As said above. Where are willing to go? What are
the advantages? Disadvantages? Let's make a decision and then move for
good. Not half-way.

However we still have the original problem to address: how are we
planning to solve segment depending on oak depending on segment on third
party applications? I have two proposals here that we could vote on if
we want

1) Going back to monolithic? Read it as, segment-tar won't be released
any more on its own cycle.

2) We correctly address dependencies. For example adding oak-core-api.

Davide

Re: segment-tar depending on oak-core

[Michael Marth]

Sorry, my email client made a mess.
Resending my text only




On 24/10/16 13:48, "Michael Marth"  wrote:

>Hi Tommaso,
>
>I agree with your assessment that this discussion is actually about the 
>delivery granularity and user’s consumption of Oak. Taking the freedom to 
>re-phrase what you said above:
>
>  *   either a complete library that is consumed as a whole (and where the 
> various internal modules are implementation details)
>  *   Or a set of modules where users are expected and allowed to access the 
> modules directly and deploy arbitrary subsets of modules
>
>At least so far, in my view we saw Oak as one library. If we were to change 
>that then we would need to be much more careful about the interactions between 
>the various (internal) modules. So far, these are an implementation detail 
>which we can change at will if needed  - which obviously allows for quite some 
>flexibility on internal changes.
>
>my2c
>Michael

Re: segment-tar depending on oak-core

[Michael Marth]

Hi Tommaso,

In my opinion what we're discussing is our view on how Oak should be
architectured, either as a big (layered) blackbox or as a set of reusable
(and interoperable) software components.
The "release all at once with version x.y" approach sounds to me more
inline with the former while the "release every module separately and
abstract APIs as much as possible" sounds more inline with the latter.

I agree with your assessment that this discussion is actually about the 
delivery granularity and user’s consumption of Oak. Taking the freedom to 
re-phrase what you said above:

  *   either a complete library that is consumed as a whole (and where the 
various internal modules are implementation details)
  *   Or a set of modules where users are expected and allowed to access the 
modules directly and deploy arbitrary subsets of modules

At least so far, in my view we saw Oak as one library. If we were to change 
that then we would need to be much more careful about the interactions between 
the various (internal) modules. So far, these are an implementation detail 
which we can change at will if needed  - which obviously allows for quite some 
flexibility on internal changes.

my2c
Michael

Re: segment-tar depending on oak-core

[Tommaso Teofili]

Il giorno ven 21 ott 2016 alle ore 16:08 Thomas Mueller 
ha scritto:

> Hi,
>
> >The release process in Oak is a joke.
>
> I don't think it's a joke.
>
> > Releasing every two weeks by
> >using version numbers as counters just for the sake of it is
> >embarrassing.
>
> Why? It's simple.
>
> > I don't even know how many releases of our parent POM we
> >have, every one of them equal to the other, and this is nonsense.
>
> "Nonsense"... again a word without explanation.
>

my understanding of it is that probably it doesn't make much sense to
release every time a new version of oak-parent that doesn't contain
anything new.


>
> >We shouldn't go backward, but forward.
>
> It depends on what "backward is". I would prefer if we make things
> "simpler".
>
> > We need to extract APIs into
> >their own independently released bundles.
>
> I don't think we need to do that. The "release everything at once" sounds
> good to me.
>
> > We should split oak-run in
> >different CLI utility modules
>
> Split, split, and again split. Why? What is the advantage?
>
> >, so that every implementation can take
> >better care of their own utilities.
>
> It's the Oak utilities. I think the current organization is just fine.
>
> >Oak is not a pet project
>
> Again, you are using strong words ("pet"), but without real explanation...
> How is it that your definition of "pet" is the only valid one?
>
> >and we
> >have to admit that its current level of complexity doesn't allow us to
> >use oak-core and oak-run as dumping grounds anymore.
>
> Again a strong word... "dump".
>
> I just don't see how making tiny "ravioli" modules makes things any
> better.


sorry, as an Italian I need a clarification on this (ravioli architecture,
nice sounding).


> It surely makes things more complex, as we see with
> oak-segment-tar: it forces to add even more and more modules, to be able
> to deal with the consequences of adding modules.
>

In my opinion what we're discussing is our view on how Oak should be
architectured, either as a big (layered) blackbox or as a set of reusable
(and interoperable) software components.
The "release all at once with version x.y" approach sounds to me more
inline with the former while the "release every module separately and
abstract APIs as much as possible" sounds more inline with the latter.

My 2 cents,
Tommaso


>
> Regards,
> Thomas
>
>

Re: segment-tar depending on oak-core

[Michael Dürig]

On 21.10.16 3:04 , Julian Sedding wrote:

an arguably simpler solution would be to
give oak-run and oak-upgrade a separate lifecycle. They are consumers
of both segment-tar and oak-core (+ other bundles with same release
cycle). Hence they require interoperable releases of both *before*
they themselves can be released.


+1. This would match the requirement for Segment Tar as it allows for 
running upgrades and tools against specific versions. Something that is 
indispensable for effective and continuous monitoring of performance, 
disk foot print and memory usage and for efficiently identifying 
regressions in any of these areas.



Michael

Re: segment-tar depending on oak-core

[Alex Parvulescu]

I fully agree with what Thomas is saying.
Being on the inside of the segment-tar release cycles I don't really see
the need or the advantages of a decoupled release, it seems it only managed
to separate us even more (we even have a different release email template,
if that was ever needed) without really providing any benefits: there is no
segment-tar without oak-*, no one would expect to be able to just drop in
the latest segment-tar bundle, everyone asks what the *oak* version is that
goes with each segment release, maybe this will change, maybe not, but
that's how it is now. So for me personally, the overhead of the separate
release greatly outweighs the benefits.

> The release process in Oak is a joke. Releasing every two weeks by
using version numbers as counters just for the sake of it is
embarrassing. I don't even know how many releases of our parent POM we
have, every one of them equal to the other, and this is nonsense.

There's a lot of misrepresentation packed into this. First it would be good
to maintain a respectful & objective attitude towards the list. Second, we
release every two weeks from a development branch to reduce the feedback
loop, if something breaks we can catch it early. Releases are unstable and
frequent, I saw no one else complain about this before.
Third, releases are generally cheap, why does it matter how often we
release the parent POM? Are you worried about disk space? Confused
customers? What part of this seems _nonsense_?

I'm all for going forward on the things that matter to the overall project,
but I'm not convinced separate release cycles for all modules is what was
keeping Oak from graduating from 'pet project' level to the big league. I
would rather keep this as simple as it can be for everyone, not just
segment-tar.

best,
alex


On Fri, Oct 21, 2016 at 4:22 PM, Julian Reschke 
wrote:

> On 2016-10-21 16:15, Thomas Mueller wrote:
>
>> Hi,
>>
>> You are sure using many emotional, judgmental words and sentences like
>> "joke", "embarrassing", "nonsense", "We shouldn't go backward, but
>> forward", "pet project", "admit", "level of complexity", "doesn't allow",
>> "dumping grounds". Your whole mail is very judgmental.
>>
>> OK, I see you would like to split everything into tiny, tiny modules.
>>
>> Right now we already have many modules, and using a different release
>> cycle for oak-segment-tar is not a problem.
>>
>> So your solution is to split things into even more modules. I see that, as
>> you seem to be very emotional about that.
>>
>> But I don't agree that's the best solution. I prefer simple solutions,
>> that don't require a lot of bureaucracy and overhead.
>>
>> I see no big value in "being able" to release things independently. In
>> fact I think it's added overhead, with no value.
>>
>> Regards,
>> Thomas
>>
>
> Big +1.
>
> Best regards, Julian
>
>

Re: segment-tar depending on oak-core

[Thomas Mueller]

>
>and using a different release
>cycle for oak-segment-tar is not a problem.

Sorry, I wanted to write "and using a different release cycle for
oak-segment-tar *created new problems*"

Re: segment-tar depending on oak-core

[Julian Reschke]

On 2016-10-21 16:15, Thomas Mueller wrote:

Hi,

You are sure using many emotional, judgmental words and sentences like
"joke", "embarrassing", "nonsense", "We shouldn't go backward, but
forward", "pet project", "admit", "level of complexity", "doesn't allow",
"dumping grounds". Your whole mail is very judgmental.

OK, I see you would like to split everything into tiny, tiny modules.

Right now we already have many modules, and using a different release
cycle for oak-segment-tar is not a problem.

So your solution is to split things into even more modules. I see that, as
you seem to be very emotional about that.

But I don't agree that's the best solution. I prefer simple solutions,
that don't require a lot of bureaucracy and overhead.

I see no big value in "being able" to release things independently. In
fact I think it's added overhead, with no value.

Regards,
Thomas


Big +1.

Best regards, Julian

Re: segment-tar depending on oak-core

[Thomas Mueller]

Hi,

You are sure using many emotional, judgmental words and sentences like
"joke", "embarrassing", "nonsense", "We shouldn't go backward, but
forward", "pet project", "admit", "level of complexity", "doesn't allow",
"dumping grounds". Your whole mail is very judgmental.

OK, I see you would like to split everything into tiny, tiny modules.

Right now we already have many modules, and using a different release
cycle for oak-segment-tar is not a problem.

So your solution is to split things into even more modules. I see that, as
you seem to be very emotional about that.

But I don't agree that's the best solution. I prefer simple solutions,
that don't require a lot of bureaucracy and overhead.

I see no big value in "being able" to release things independently. In
fact I think it's added overhead, with no value.

Regards,
Thomas


On 21/10/16 15:09, "Francesco Mari"  wrote:

>Luckily for us this is not a computer science problem but an easier
>software engineering concern.
>
>The release process in Oak is a joke. Releasing every two weeks by
>using version numbers as counters just for the sake of it is
>embarrassing. I don't even know how many releases of our parent POM we
>have, every one of them equal to the other, and this is nonsense.
>
>We shouldn't go backward, but forward. We need to extract APIs into
>their own independently released bundles. We should split oak-run in
>different CLI utility modules, so that every implementation can take
>better care of their own utilities. Oak is not a pet project and we
>have to admit that its current level of complexity doesn't allow us to
>use oak-core and oak-run as dumping grounds anymore.
>
>2016-10-21 14:08 GMT+02:00 Thomas Mueller :
>> Hi,
>>
>>> could adding an oak-core-api with independent lifecycle solve the
>>>situation?
>>
>> "All problems in computer science can be solved by another level of
>> indirection"
>>
>> I would prefer if we get oak-segment-tar in line with the rest of oak
>> (release it at the same time and so on). I understand, there are some
>> disadvantages. But I think all alternatives also have disadvantages.
>>
>> Regards,
>> Thomas
>>
>>
>>
>>
>> On 21/10/16 12:46, "Davide Giannella"  wrote:
>>
>>>Hello team,
>>>
>>>while integrating Oak with segment-tar in other products, I'm facing
>>>quite a struggle with a sort-of circular dependencies. We have
>>>segment-tar that depends on oak-core and then we have tools like oak-run
>>>or oak-upgrade which depends on both oak-core and segment-tar.
>>>
>>>this may not be an issue but in case of changes in the API, like for
>>>1.5.12 we have the following situation. 1.5.12 has been released with
>>>segment-tar 0.0.14 but this mix doesn't actually work on OSGi
>>>environment as of API changes. On the other hand, in order to release
>>>0.0.16 we need oak-core 1.5.12 with the changes.
>>>
>>>Now oak-run and other tools may fail, or at least be in an unknown
>>>situation.
>>>
>>>All of this is my understanding and I may be wrong, so please correct me
>>>if I'm wrong. I'm right, could adding an oak-core-api with independent
>>>lifecycle solve the situation?
>>>
>>>Davide
>>>
>>>
>>

Re: segment-tar depending on oak-core

[Thomas Mueller]

Hi,

>The release process in Oak is a joke.

I don't think it's a joke.

> Releasing every two weeks by
>using version numbers as counters just for the sake of it is
>embarrassing.

Why? It's simple.

> I don't even know how many releases of our parent POM we
>have, every one of them equal to the other, and this is nonsense.

"Nonsense"... again a word without explanation.

>We shouldn't go backward, but forward.

It depends on what "backward is". I would prefer if we make things
"simpler".

> We need to extract APIs into
>their own independently released bundles.

I don't think we need to do that. The "release everything at once" sounds
good to me.

> We should split oak-run in
>different CLI utility modules

Split, split, and again split. Why? What is the advantage?

>, so that every implementation can take
>better care of their own utilities.

It's the Oak utilities. I think the current organization is just fine.

>Oak is not a pet project

Again, you are using strong words ("pet"), but without real explanation...
How is it that your definition of "pet" is the only valid one?

>and we
>have to admit that its current level of complexity doesn't allow us to
>use oak-core and oak-run as dumping grounds anymore.

Again a strong word... "dump".

I just don't see how making tiny "ravioli" modules makes things any
better. It surely makes things more complex, as we see with
oak-segment-tar: it forces to add even more and more modules, to be able
to deal with the consequences of adding modules.

Regards,
Thomas

Re: segment-tar depending on oak-core

[Francesco Mari]

Luckily for us this is not a computer science problem but an easier
software engineering concern.

The release process in Oak is a joke. Releasing every two weeks by
using version numbers as counters just for the sake of it is
embarrassing. I don't even know how many releases of our parent POM we
have, every one of them equal to the other, and this is nonsense.

We shouldn't go backward, but forward. We need to extract APIs into
their own independently released bundles. We should split oak-run in
different CLI utility modules, so that every implementation can take
better care of their own utilities. Oak is not a pet project and we
have to admit that its current level of complexity doesn't allow us to
use oak-core and oak-run as dumping grounds anymore.

2016-10-21 14:08 GMT+02:00 Thomas Mueller :
> Hi,
>
>> could adding an oak-core-api with independent lifecycle solve the
>>situation?
>
> "All problems in computer science can be solved by another level of
> indirection"
>
> I would prefer if we get oak-segment-tar in line with the rest of oak
> (release it at the same time and so on). I understand, there are some
> disadvantages. But I think all alternatives also have disadvantages.
>
> Regards,
> Thomas
>
>
>
>
> On 21/10/16 12:46, "Davide Giannella"  wrote:
>
>>Hello team,
>>
>>while integrating Oak with segment-tar in other products, I'm facing
>>quite a struggle with a sort-of circular dependencies. We have
>>segment-tar that depends on oak-core and then we have tools like oak-run
>>or oak-upgrade which depends on both oak-core and segment-tar.
>>
>>this may not be an issue but in case of changes in the API, like for
>>1.5.12 we have the following situation. 1.5.12 has been released with
>>segment-tar 0.0.14 but this mix doesn't actually work on OSGi
>>environment as of API changes. On the other hand, in order to release
>>0.0.16 we need oak-core 1.5.12 with the changes.
>>
>>Now oak-run and other tools may fail, or at least be in an unknown
>>situation.
>>
>>All of this is my understanding and I may be wrong, so please correct me
>>if I'm wrong. I'm right, could adding an oak-core-api with independent
>>lifecycle solve the situation?
>>
>>Davide
>>
>>
>

Re: segment-tar depending on oak-core

[Julian Sedding]

> All of this is my understanding and I may be wrong, so please correct me
> if I'm wrong. I'm right, could adding an oak-core-api with independent
> lifecycle solve the situation?

While this may be possible, an arguably simpler solution would be to
give oak-run and oak-upgrade a separate lifecycle. They are consumers
of both segment-tar and oak-core (+ other bundles with same release
cycle). Hence they require interoperable releases of both *before*
they themselves can be released.

The other alternative, as Thomas mentioned, is to release everything
at once, including segment-tar.

Regards
Julian


On Fri, Oct 21, 2016 at 12:46 PM, Davide Giannella  wrote:
> Hello team,
>
> while integrating Oak with segment-tar in other products, I'm facing
> quite a struggle with a sort-of circular dependencies. We have
> segment-tar that depends on oak-core and then we have tools like oak-run
> or oak-upgrade which depends on both oak-core and segment-tar.
>
> this may not be an issue but in case of changes in the API, like for
> 1.5.12 we have the following situation. 1.5.12 has been released with
> segment-tar 0.0.14 but this mix doesn't actually work on OSGi
> environment as of API changes. On the other hand, in order to release
> 0.0.16 we need oak-core 1.5.12 with the changes.
>
> Now oak-run and other tools may fail, or at least be in an unknown
> situation.
>
> All of this is my understanding and I may be wrong, so please correct me
> if I'm wrong. I'm right, could adding an oak-core-api with independent
> lifecycle solve the situation?
>
> Davide
>
>

Re: segment-tar depending on oak-core

[Thomas Mueller]

Hi,

> could adding an oak-core-api with independent lifecycle solve the
>situation?

"All problems in computer science can be solved by another level of
indirection"

I would prefer if we get oak-segment-tar in line with the rest of oak
(release it at the same time and so on). I understand, there are some
disadvantages. But I think all alternatives also have disadvantages.

Regards,
Thomas




On 21/10/16 12:46, "Davide Giannella"  wrote:

>Hello team,
>
>while integrating Oak with segment-tar in other products, I'm facing
>quite a struggle with a sort-of circular dependencies. We have
>segment-tar that depends on oak-core and then we have tools like oak-run
>or oak-upgrade which depends on both oak-core and segment-tar.
>
>this may not be an issue but in case of changes in the API, like for
>1.5.12 we have the following situation. 1.5.12 has been released with
>segment-tar 0.0.14 but this mix doesn't actually work on OSGi
>environment as of API changes. On the other hand, in order to release
>0.0.16 we need oak-core 1.5.12 with the changes.
>
>Now oak-run and other tools may fail, or at least be in an unknown
>situation.
>
>All of this is my understanding and I may be wrong, so please correct me
>if I'm wrong. I'm right, could adding an oak-core-api with independent
>lifecycle solve the situation?
>
>Davide
>
>

segment-tar depending on oak-core

[Davide Giannella]

Hello team,

while integrating Oak with segment-tar in other products, I'm facing
quite a struggle with a sort-of circular dependencies. We have
segment-tar that depends on oak-core and then we have tools like oak-run
or oak-upgrade which depends on both oak-core and segment-tar.

this may not be an issue but in case of changes in the API, like for
1.5.12 we have the following situation. 1.5.12 has been released with
segment-tar 0.0.14 but this mix doesn't actually work on OSGi
environment as of API changes. On the other hand, in order to release
0.0.16 we need oak-core 1.5.12 with the changes.

Now oak-run and other tools may fail, or at least be in an unknown
situation.

All of this is my understanding and I may be wrong, so please correct me
if I'm wrong. I'm right, could adding an oak-core-api with independent
lifecycle solve the situation?

Davide