Re: Proposal about James modules merge

[Eric Charles]

On 2015-09-03 15:13, Matthieu Baechler wrote:


On 03/09/2015 11:48, Eric Charles wrote:



On 2015-09-03 11:04, Matthieu Baechler wrote:

Hi Eric,

On 03/09/2015 10:16, Eric Charles wrote:

I like Matthieu proposal (merge without mime4...), but this will open
the door to more refactoring that would maybe go against the initial
requirement of being able to embed some mailbox without the full
server.


Of course, as the mailbox API will probably change more often, it will
break potential mailbox-api direct users. It leads to two questions :
  - is there such users ?
  - do we expect alpha/beta software to be API stable ?

I really like the idea of merging things until 3.0 release happens then
decide if we split back or not.



If we merge, we should be sure this is the right thing to do before and
after 3.0.

Why would we split again after 3.0?


Because maybe when you have a 3.0 release, the project goal can change
from "releasing a great mail server" to "trying to grow a contributor
community" or anything else.

Stefano first talk about this idea:

"And maybe they could be merged until we get to a more stable solution,
and then splitted again once they are stable enough."

I don't see anything wrong in using the right process for a given goal.




Thx for the clarification.


Maybe we should write to guidelines we can refer when working in that
single repository, otherwise we will have endless discussions that
don't
occur for now simply because code live in separate projects.


I think maven dependencies capture the intent of module responsibility
very well. What would you want the guidelines to contain ? API stability
rules ? Anything else ?




Classes, Packages, Maven submodules and repositories all serve IMHO
segregation of responsibility and API.

For now, we have hard barrier that prevent someone to break this.


I don't see anything we would loose once the repositories are merged.
What prevent any commiter to add spring into data-api today ? Or to
introduce a cyclic dependency ?

Maybe I don't understand what you mean by "hard barrier", do you have
some examples ?



I will be (a bit) easier to refactor in a wrong way when everything will 
be in the same repository. But it is true that it is already possible to 
take a wrong road with the current structure.



I was thinking more about a documented diagram such as the one I started
on http://james.apache.org/server/3/dev.html to show the modules
interactions and boundaries.

A common understanding of such a representation will ease later
discussion.


Actually, I don't see how the merge impact this diagram. We can
definitively improve it, if it's what you mean, but is it really related
to the merge ?



It does not and the documentation should not be a requirement to merge.

+1 on the principle of your merge proposal.

On the details, I would rename 'james' by 'server', and 'backend' with 
something else ('persistence', 'common', 'util'...?, it depends what you 
anticipate to come into)



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Matthieu Baechler]

On 03/09/2015 11:48, Eric Charles wrote:



On 2015-09-03 11:04, Matthieu Baechler wrote:

Hi Eric,

On 03/09/2015 10:16, Eric Charles wrote:

I like Matthieu proposal (merge without mime4...), but this will open
the door to more refactoring that would maybe go against the initial
requirement of being able to embed some mailbox without the full server.


Of course, as the mailbox API will probably change more often, it will
break potential mailbox-api direct users. It leads to two questions :
  - is there such users ?
  - do we expect alpha/beta software to be API stable ?

I really like the idea of merging things until 3.0 release happens then
decide if we split back or not.



If we merge, we should be sure this is the right thing to do before and
after 3.0.

Why would we split again after 3.0?


Because maybe when you have a 3.0 release, the project goal can change 
from "releasing a great mail server" to "trying to grow a contributor 
community" or anything else.


Stefano first talk about this idea:

"And maybe they could be merged until we get to a more stable solution,
and then splitted again once they are stable enough."

I don't see anything wrong in using the right process for a given goal.



Maybe we should write to guidelines we can refer when working in that
single repository, otherwise we will have endless discussions that don't
occur for now simply because code live in separate projects.


I think maven dependencies capture the intent of module responsibility
very well. What would you want the guidelines to contain ? API stability
rules ? Anything else ?




Classes, Packages, Maven submodules and repositories all serve IMHO
segregation of responsibility and API.

For now, we have hard barrier that prevent someone to break this.


I don't see anything we would loose once the repositories are merged. 
What prevent any commiter to add spring into data-api today ? Or to 
introduce a cyclic dependency ?


Maybe I don't understand what you mean by "hard barrier", do you have 
some examples ?



I was thinking more about a documented diagram such as the one I started
on http://james.apache.org/server/3/dev.html to show the modules
interactions and boundaries.

A common understanding of such a representation will ease later discussion.


Actually, I don't see how the merge impact this diagram. We can 
definitively improve it, if it's what you mean, but is it really related 
to the merge ?


--
Matthieu Baechler

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Eric Charles]

On 2015-09-01 14:29, Benoit Tellier wrote:

(snip...)


  - Finally, there is the issue that started this thread. There might be
duplication between mailbox code and james-server-data-* one. In the
Cassandra example, we developed tools for creating tables, index, custom
types... That we want to use in both the cassandra-mailbox and
james-server-data-cassandra. We don't want to duplicate it, we don't
want dependencies between both projects. The only solution with
separated projects is to introduce an other separated project
introducing these tools ( what we started to develop ). This is not a
separated case : We can have uses of messages queue in several places :
mailqueue, mailbox event system, ... .



The mailbox-* serve the "Mailbox" persistence.
The james-server-data-* serve the "Mail Repository Store" persistence.

This is (not enough) explained on 
http://james.apache.org/server/3/feature-persistence.html


One idea for later (which we discussed some time ago) would be to 
migrate the james-server-data-* used for mails to the mailbox. The 
remaining persistence used for users/rrt/domains/... still need their 
own persistence.


To allow James 2.x migration to James 3.0, we need to keep those 
james-server-data-* as such.


Later, some kind of common infrastructure (persistence, queue,...) can 
come to life and serve other modules.





Merging modules together (mailbox, james, protocols and mpt for me )
solves all these issue elegantly and makes it easier to contribute to
James.

Le 01/09/2015 11:29, Stephen Brewin a écrit :

On 01/09/2015 08:18, Matthieu Baechler wrote:

Thank you for your answer Stephen. It looks like we agree one this
proposal.

Can I take your answer for a +1 ?

+1 for restructuring

We should discuss transitioning to GIT separately

--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



--
Eric Charles http://datalayer.io

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Eric Charles]

On 2015-09-03 11:04, Matthieu Baechler wrote:

Hi Eric,

On 03/09/2015 10:16, Eric Charles wrote:

I like Matthieu proposal (merge without mime4...), but this will open
the door to more refactoring that would maybe go against the initial
requirement of being able to embed some mailbox without the full server.


Of course, as the mailbox API will probably change more often, it will
break potential mailbox-api direct users. It leads to two questions :
  - is there such users ?
  - do we expect alpha/beta software to be API stable ?

I really like the idea of merging things until 3.0 release happens then
decide if we split back or not.



If we merge, we should be sure this is the right thing to do before and 
after 3.0.


Why would we split again after 3.0?


Maybe we should write to guidelines we can refer when working in that
single repository, otherwise we will have endless discussions that don't
occur for now simply because code live in separate projects.


I think maven dependencies capture the intent of module responsibility
very well. What would you want the guidelines to contain ? API stability
rules ? Anything else ?




Classes, Packages, Maven submodules and repositories all serve IMHO 
segregation of responsibility and API.


For now, we have hard barrier that prevent someone to break this.

I was thinking more about a documented diagram such as the one I started 
on http://james.apache.org/server/3/dev.html to show the modules 
interactions and boundaries.


A common understanding of such a representation will ease later discussion.

API stability is another concern to be addressed once we have the 
architecture documented.


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Matthieu Baechler]

Hi Eric,

On 03/09/2015 10:16, Eric Charles wrote:

I like Matthieu proposal (merge without mime4...), but this will open
the door to more refactoring that would maybe go against the initial
requirement of being able to embed some mailbox without the full server.


Of course, as the mailbox API will probably change more often, it will 
break potential mailbox-api direct users. It leads to two questions :

 - is there such users ?
 - do we expect alpha/beta software to be API stable ?

I really like the idea of merging things until 3.0 release happens then 
decide if we split back or not.



Maybe we should write to guidelines we can refer when working in that
single repository, otherwise we will have endless discussions that don't
occur for now simply because code live in separate projects.


I think maven dependencies capture the intent of module responsibility 
very well. What would you want the guidelines to contain ? API stability 
rules ? Anything else ?



--
Matthieu Baechler

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Eric Charles]

I like Matthieu proposal (merge without mime4...), but this will open 
the door to more refactoring that would maybe go against the initial 
requirement of being able to embed some mailbox without the full server.


Maybe we should write to guidelines we can refer when working in that 
single repository, otherwise we will have endless discussions that don't 
occur for now simply because code live in separate projects.


On 2015-09-01 19:19, Stephen Brewin wrote:

Hi Benoit

There appears to be consensus that our project layout should be
refactored along the lines suggested by Matthieu. You seem to be
suggesting that we go further, which I believe we should hold off on.

With Matthieu's refactored structure a mulit-module Maven build that
uses a BOM (a bill-of-materials POM) will solve the majority of
problems. Then we can look at how to solve the remaining edge cases.

Personally I use Eclipse while you use IntelliJ, both of which integrate
with Maven and support multi-module projects. In my day job we have much
more complex multi-module projects than is proposed for James and have
no problems developing using Eclipse/Maven and with CI using Jenkins.

There should be no need for manual builds and installation of .jars.
These are signs of a Maven anti-pattern fixable with the appropriate
Maven multi-module structure or a misunderstanding of the Maven
development process.

Certainly I am not in favour of merging modules at this stage. As
explained in an earlier post, this breaks an intent of the James
project, plus it has yet to be shown to be beneficial under Matthieu's
refactored structure.

Sometimes it is best to take baby steps. I believe this is the case here.

Cheers
--Steve

On 01/09/2015 13:29, Benoit Tellier wrote:

For me this is a +1.

I think there is several issue with today organization :



 - Some projects are not really separated. For instance, if I want to
add QUOTA support, I will modify Mailbox interfaces, but also change
things in protocols.



 - Having separated modules that are eavily changed makes build hard.
Today, you can either :
  - build manually each projects in the correct order and install
the given jar in your .m2 repository. This is long, error prone, and
not documented.
  - develop or rely on third party projects for aggregating this (
what we have done in james-parent ). This is not part of the Apache
project so relying on it seems strange to me. Moreover this tool is
designed for our needs ( our James with a Cassandra back-end ), with
tools we like ( docker ).
   I think both of these options are not the good ones. We easily see
that modules like protocols, james, mailbox and mpt can't be built
separately.



 - Work-flow is hard. Before starting working with Antoine and
Matthieu I was working on a PoC for IMAP quota, modifying mailbox and
protocols project on separated intelliJ instances. Each time I
modified something in mailbox, I needed to compile it, install it with
maven and reload maven in the protocol project, slowing me down a lot.



 - Apache CI is broken. Quite normal as building James is not trivial.



 - Library version can easily diverge between projects. It may be
unseen at compile time and with unit test but can cause serious
problems at runtime. Each time we modify a library version, we have to
modify it on each other James projects. This is error prone.



 - Finally, there is the issue that started this thread. There might
be duplication between mailbox code and james-server-data-* one. In
the Cassandra example, we developed tools for creating tables, index,
custom types... That we want to use in both the cassandra-mailbox and
james-server-data-cassandra. We don't want to duplicate it, we don't
want dependencies between both projects. The only solution with
separated projects is to introduce an other separated project
introducing these tools ( what we started to develop ). This is not a
separated case : We can have uses of messages queue in several places
: mailqueue, mailbox event system, ... .



Merging modules together (mailbox, james, protocols and mpt for me )
solves all these issue elegantly and makes it easier to contribute to
James.

Le 01/09/2015 11:29, Stephen Brewin a écrit :

On 01/09/2015 08:18, Matthieu Baechler wrote:

Thank you for your answer Stephen. It looks like we agree one this
proposal.

Can I take your answer for a +1 ?

+1 for restructuring

We should discuss transitioning to GIT separately

--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@j

Re: Proposal about James modules merge

[Eric Charles]

On 2015-08-27 11:11, Stephen Brewin wrote:

Hi

As I recall, the intent of having separate projects for many of the
components developed under the James umbrella was to satisfy the
requirement that they should be independent of James Server. While this
remains a requirement, separate repositories are needed for each project
to allow separate release versions and schedules. It also influences our
maven module layout and how dependencies might be better managed.



I can not talk about the drivers for separate projects, but I understand 
those projects as a way to be able to use them without the full james 
server. It is also a way to force developers to think in terms of API 
contracts between those projects.



Before proceeding with a discussion of how to simplify the development
workflow, we need to decide if the original requirement still holds.
Prospective solutions will be quite different depending on this answer.

Cheers
--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Eric Charles]

On 2015-08-28 20:40, Stephen Brewin wrote:

Hi Vincenzo and Ioan

While Mattiheu's proposal does mention switching to GIT and I agree that
GIT is superior to SVN and I support this, the most important part is
the restructuring of our projects.



James structure should be independent of any versioning system (git, 
svn...).


But yes, git has more features than svn, typical git project are managed 
in a single repository.



As I have explained in an earlier post the proposed change abandons the
façade that components, such as mailbox and protocols, are independent
of James server. My view is that the restructuring of our projects would
be beneficial. What are yours?

The proposed restructuring should not be confabulated with a switch to
GIT as it is achievable with SVN. A switch to GIT should be dealt with
as a separate issue.



Any migration to git is orthogonal to the restructure.
The restructure may ease the migration.


Cheers
--Steve

On 28/08/2015 09:08, Vincenzo Gianferrari Pini wrote:

Hi all,

sorry for not having been active at all in the last period.

Anyway, I agree with Ioan that using GIT is *much* more productive than
using SVN, so I cast here my +1.

Regards,
Vincenzo

Il giorno lun 24 ago 2015 alle ore 21:51 Ioan Eugen Stan <
stan.ieu...@gmail.com> ha scritto:


Hi,

Yes, the work flow is not the best with SVN. There is an option to
migrate James to git hosting and personally I think it will be a good
thing.

In order to make this a reality we have to raise a vote and raise a JIRA
issue to Apache Infra. The vote has to run for 72h.

You have my +1.

p.s. One thing to have in mind is that we need to we need to take care
of the site publishing also. But it's doable.

Regards,

--

*Ing. Vincenzo Gianferrari Pini*
Chairman & CTO - GoCloud
Sede legale: Via Larga 15, 20122 Milano (MI)
Sede op. ed amm.: Via Stefanardo da Vimercate 28, 20128 Milano (MI)
Cel. +39-3939837493
tel: +39-02-87250672 (dir.)
tel: +39-02-25514300 (cent.)
fax: +39-02-25514302

https://google.com/+VincenzoGianferrariPiniGoCloud
https://www.linkedin.com/in/vgianferrari
http://www.gocloud.eu
​http://www.mapadore.com​






-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Stephen Brewin]

Hi Benoit

There appears to be consensus that our project layout should be 
refactored along the lines suggested by Matthieu. You seem to be 
suggesting that we go further, which I believe we should hold off on.


With Matthieu's refactored structure a mulit-module Maven build that 
uses a BOM (a bill-of-materials POM) will solve the majority of 
problems. Then we can look at how to solve the remaining edge cases.


Personally I use Eclipse while you use IntelliJ, both of which integrate 
with Maven and support multi-module projects. In my day job we have much 
more complex multi-module projects than is proposed for James and have 
no problems developing using Eclipse/Maven and with CI using Jenkins.


There should be no need for manual builds and installation of .jars. 
These are signs of a Maven anti-pattern fixable with the appropriate 
Maven multi-module structure or a misunderstanding of the Maven 
development process.


Certainly I am not in favour of merging modules at this stage. As 
explained in an earlier post, this breaks an intent of the James 
project, plus it has yet to be shown to be beneficial under Matthieu's 
refactored structure.


Sometimes it is best to take baby steps. I believe this is the case here.

Cheers
--Steve

On 01/09/2015 13:29, Benoit Tellier wrote:

For me this is a +1.

I think there is several issue with today organization :



 - Some projects are not really separated. For instance, if I want to 
add QUOTA support, I will modify Mailbox interfaces, but also change 
things in protocols.




 - Having separated modules that are eavily changed makes build hard. 
Today, you can either :
  - build manually each projects in the correct order and install 
the given jar in your .m2 repository. This is long, error prone, and 
not documented.
  - develop or rely on third party projects for aggregating this ( 
what we have done in james-parent ). This is not part of the Apache 
project so relying on it seems strange to me. Moreover this tool is 
designed for our needs ( our James with a Cassandra back-end ), with 
tools we like ( docker ).
   I think both of these options are not the good ones. We easily see 
that modules like protocols, james, mailbox and mpt can't be built 
separately.




 - Work-flow is hard. Before starting working with Antoine and 
Matthieu I was working on a PoC for IMAP quota, modifying mailbox and 
protocols project on separated intelliJ instances. Each time I 
modified something in mailbox, I needed to compile it, install it with 
maven and reload maven in the protocol project, slowing me down a lot.




 - Apache CI is broken. Quite normal as building James is not trivial.



 - Library version can easily diverge between projects. It may be 
unseen at compile time and with unit test but can cause serious 
problems at runtime. Each time we modify a library version, we have to 
modify it on each other James projects. This is error prone.




 - Finally, there is the issue that started this thread. There might 
be duplication between mailbox code and james-server-data-* one. In 
the Cassandra example, we developed tools for creating tables, index, 
custom types... That we want to use in both the cassandra-mailbox and 
james-server-data-cassandra. We don't want to duplicate it, we don't 
want dependencies between both projects. The only solution with 
separated projects is to introduce an other separated project 
introducing these tools ( what we started to develop ). This is not a 
separated case : We can have uses of messages queue in several places 
: mailqueue, mailbox event system, ... .




Merging modules together (mailbox, james, protocols and mpt for me ) 
solves all these issue elegantly and makes it easier to contribute to 
James.


Le 01/09/2015 11:29, Stephen Brewin a écrit :

On 01/09/2015 08:18, Matthieu Baechler wrote:

Thank you for your answer Stephen. It looks like we agree one this
proposal.

Can I take your answer for a +1 ?

+1 for restructuring

We should discuss transitioning to GIT separately

--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Stefano Bagnara]

In my opinion this decision belongs to active developers: they are the
users of the source tree and the build tools, so they are entitled to
make the changes to feel confortable.

So, if active developers prefer to have a single branch, then you have
my +1 for this.

IIRC Robert was the main supporter of the split, so maybe he has
something to say...

And maybe they could be merged until we get to a more stable solution,
and then splitted again once they are stable enough.

The important thing is that a clear dependency is kept between modules
and you don't introduce cyclic dependencies.

Stefano

PS: I'm happy to read there is something new on the james 3 codebase!

On 1 September 2015 at 14:29, Benoit Tellier  wrote:
> For me this is a +1.
>
> I think there is several issue with today organization :
>
>
>
>  - Some projects are not really separated. For instance, if I want to add
> QUOTA support, I will modify Mailbox interfaces, but also change things in
> protocols.
>
>
>
>  - Having separated modules that are eavily changed makes build hard. Today,
> you can either :
>   - build manually each projects in the correct order and install the
> given jar in your .m2 repository. This is long, error prone, and not
> documented.
>   - develop or rely on third party projects for aggregating this ( what
> we have done in james-parent ). This is not part of the Apache project so
> relying on it seems strange to me. Moreover this tool is designed for our
> needs ( our James with a Cassandra back-end ), with tools we like ( docker
> ).
>I think both of these options are not the good ones. We easily see that
> modules like protocols, james, mailbox and mpt can't be built separately.
>
>
>
>  - Work-flow is hard. Before starting working with Antoine and Matthieu I
> was working on a PoC for IMAP quota, modifying mailbox and protocols project
> on separated intelliJ instances. Each time I modified something in mailbox,
> I needed to compile it, install it with maven and reload maven in the
> protocol project, slowing me down a lot.
>
>
>
>  - Apache CI is broken. Quite normal as building James is not trivial.
>
>
>
>  - Library version can easily diverge between projects. It may be unseen at
> compile time and with unit test but can cause serious problems at runtime.
> Each time we modify a library version, we have to modify it on each other
> James projects. This is error prone.
>
>
>
>  - Finally, there is the issue that started this thread. There might be
> duplication between mailbox code and james-server-data-* one. In the
> Cassandra example, we developed tools for creating tables, index, custom
> types... That we want to use in both the cassandra-mailbox and
> james-server-data-cassandra. We don't want to duplicate it, we don't want
> dependencies between both projects. The only solution with separated
> projects is to introduce an other separated project introducing these tools
> ( what we started to develop ). This is not a separated case : We can have
> uses of messages queue in several places : mailqueue, mailbox event system,
> ... .
>
>
>
> Merging modules together (mailbox, james, protocols and mpt for me ) solves
> all these issue elegantly and makes it easier to contribute to James.
>
>
> Le 01/09/2015 11:29, Stephen Brewin a écrit :
>>
>> On 01/09/2015 08:18, Matthieu Baechler wrote:
>>>
>>> Thank you for your answer Stephen. It looks like we agree one this
>>> proposal.
>>>
>>> Can I take your answer for a +1 ?
>>
>> +1 for restructuring
>>
>> We should discuss transitioning to GIT separately
>>
>> --Steve
>>
>> -
>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-dev-h...@james.apache.org
>>
>
> -
> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
> For additional commands, e-mail: server-dev-h...@james.apache.org
>

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Benoit Tellier]

For me this is a +1.

I think there is several issue with today organization :



 - Some projects are not really separated. For instance, if I want to 
add QUOTA support, I will modify Mailbox interfaces, but also change 
things in protocols.




 - Having separated modules that are eavily changed makes build hard. 
Today, you can either :
  - build manually each projects in the correct order and install 
the given jar in your .m2 repository. This is long, error prone, and not 
documented.
  - develop or rely on third party projects for aggregating this ( 
what we have done in james-parent ). This is not part of the Apache 
project so relying on it seems strange to me. Moreover this tool is 
designed for our needs ( our James with a Cassandra back-end ), with 
tools we like ( docker ).
   I think both of these options are not the good ones. We easily see 
that modules like protocols, james, mailbox and mpt can't be built 
separately.




 - Work-flow is hard. Before starting working with Antoine and Matthieu 
I was working on a PoC for IMAP quota, modifying mailbox and protocols 
project on separated intelliJ instances. Each time I modified something 
in mailbox, I needed to compile it, install it with maven and reload 
maven in the protocol project, slowing me down a lot.




 - Apache CI is broken. Quite normal as building James is not trivial.



 - Library version can easily diverge between projects. It may be 
unseen at compile time and with unit test but can cause serious problems 
at runtime. Each time we modify a library version, we have to modify it 
on each other James projects. This is error prone.




 - Finally, there is the issue that started this thread. There might be 
duplication between mailbox code and james-server-data-* one. In the 
Cassandra example, we developed tools for creating tables, index, custom 
types... That we want to use in both the cassandra-mailbox and 
james-server-data-cassandra. We don't want to duplicate it, we don't 
want dependencies between both projects. The only solution with 
separated projects is to introduce an other separated project 
introducing these tools ( what we started to develop ). This is not a 
separated case : We can have uses of messages queue in several places : 
mailqueue, mailbox event system, ... .




Merging modules together (mailbox, james, protocols and mpt for me ) 
solves all these issue elegantly and makes it easier to contribute to James.


Le 01/09/2015 11:29, Stephen Brewin a écrit :

On 01/09/2015 08:18, Matthieu Baechler wrote:

Thank you for your answer Stephen. It looks like we agree one this
proposal.

Can I take your answer for a +1 ?

+1 for restructuring

We should discuss transitioning to GIT separately

--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Stephen Brewin]

On 01/09/2015 08:18, Matthieu Baechler wrote:
Thank you for your answer Stephen. It looks like we agree one this 
proposal.


Can I take your answer for a +1 ?

+1 for restructuring

We should discuss transitioning to GIT separately

--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Matthieu Baechler]

Thank you for your answer Stephen. It looks like we agree one this proposal.

Can I take your answer for a +1 ?

Eric : you didn't gave your opinion yet, WDYT ?

--
Matthieu Baechler

On 27/08/2015 20:02, Stephen Brewin wrote:

My previous post corrected for copy/paste issues on phone!

On 27/08/2015 17:17, Stephen Brewin wrote:

On 27/08/2015 10:24, Matthieu Baechler wrote:

Hi Steve,

Thanks for your feedback.

On 27/08/2015 11:11, Stephen Brewin wrote:

Hi

As I recall, the intent of having separate projects for many of the
components developed under the James umbrella was to satisfy the
requirement that they should be independent of James Server. While this
remains a requirement, separate repositories are needed for each
project
to allow separate release versions and schedules. It also influences
our
maven module layout and how dependencies might be better managed.

Before proceeding with a discussion of how to simplify the development
workflow, we need to decide if the original requirement still holds.
Prospective solutions will be quite different depending on this answer.




I think there's always a balance to find between flexibility and
simplicity and we should never loose flexibility when there's nothing
to gain.

That's why I think we should not talk of requirements for "James
Server components" as a whole but to weight for each component what
we can gain and what we would loose.

It's what I tried to do by excluding jdkim, jsieve, jspf and mime4j
from the merge candidates as there's very little to gain IMO (I might
be wrong, of course).

I would really like the community to tell me what are the cases where
the other components could benefit from being separated, that would
help answering the question "do we want to change the requirements" ?

Cheers,


Hi Matthieu

At the moment the separate projects are only semi-independent which
gives rise to workflow and dependency problems. They should either be:

  * Entirely independent, maintained separately from James server and
James server should treat them no differently to other 3rd party
artefacts
  * Sub-modules of James server, sharing the same repository, version
numbers and dependency library (via a BOM), thereby easing the
workflow and dependency problems


/If we switch to the sub-module approach the separate maven artefacts
would still be available to 3rd parties, which I believe captures the
original intent of the requirement./

Theoretically, separate projects allow each to follow a more nimble
release cycle than James server, but recent years have shown little
evidence in support of this.

Cheers
--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Matthieu Baechler]

Thank you Stephen for clarifying this, I totally agree but didn't find 
time to answer, you just beat me at answering !


Cheers,
--
Matthieu Baechler

On 28/08/2015 20:40, Stephen Brewin wrote:

Hi Vincenzo and Ioan

While Mattiheu's proposal does mention switching to GIT and I agree that
GIT is superior to SVN and I support this, the most important part is
the restructuring of our projects.

As I have explained in an earlier post the proposed change abandons the
façade that components, such as mailbox and protocols, are independent
of James server. My view is that the restructuring of our projects would
be beneficial. What are yours?

The proposed restructuring should not be confabulated with a switch to
GIT as it is achievable with SVN. A switch to GIT should be dealt with
as a separate issue.

Cheers
--Steve

On 28/08/2015 09:08, Vincenzo Gianferrari Pini wrote:

Hi all,

sorry for not having been active at all in the last period.

Anyway, I agree with Ioan that using GIT is *much* more productive than
using SVN, so I cast here my +1.

Regards,
Vincenzo

Il giorno lun 24 ago 2015 alle ore 21:51 Ioan Eugen Stan <
stan.ieu...@gmail.com> ha scritto:


Hi,

Yes, the work flow is not the best with SVN. There is an option to
migrate James to git hosting and personally I think it will be a good
thing.

In order to make this a reality we have to raise a vote and raise a JIRA
issue to Apache Infra. The vote has to run for 72h.

You have my +1.

p.s. One thing to have in mind is that we need to we need to take care
of the site publishing also. But it's doable.

Regards,

--

*Ing. Vincenzo Gianferrari Pini*
Chairman & CTO - GoCloud
Sede legale: Via Larga 15, 20122 Milano (MI)
Sede op. ed amm.: Via Stefanardo da Vimercate 28, 20128 Milano (MI)
Cel. +39-3939837493
tel: +39-02-87250672 (dir.)
tel: +39-02-25514300 (cent.)
fax: +39-02-25514302

https://google.com/+VincenzoGianferrariPiniGoCloud
https://www.linkedin.com/in/vgianferrari
http://www.gocloud.eu
​http://www.mapadore.com​






-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Stephen Brewin]

Hi Vincenzo and Ioan

While Mattiheu's proposal does mention switching to GIT and I agree that 
GIT is superior to SVN and I support this, the most important part is 
the restructuring of our projects.


As I have explained in an earlier post the proposed change abandons the 
façade that components, such as mailbox and protocols, are independent 
of James server. My view is that the restructuring of our projects would 
be beneficial. What are yours?


The proposed restructuring should not be confabulated with a switch to 
GIT as it is achievable with SVN. A switch to GIT should be dealt with 
as a separate issue.


Cheers
--Steve

On 28/08/2015 09:08, Vincenzo Gianferrari Pini wrote:

Hi all,

sorry for not having been active at all in the last period.

Anyway, I agree with Ioan that using GIT is *much* more productive than
using SVN, so I cast here my +1.

Regards,
Vincenzo

Il giorno lun 24 ago 2015 alle ore 21:51 Ioan Eugen Stan <
stan.ieu...@gmail.com> ha scritto:


Hi,

Yes, the work flow is not the best with SVN. There is an option to
migrate James to git hosting and personally I think it will be a good
thing.

In order to make this a reality we have to raise a vote and raise a JIRA
issue to Apache Infra. The vote has to run for 72h.

You have my +1.

p.s. One thing to have in mind is that we need to we need to take care
of the site publishing also. But it's doable.

Regards,

--

*Ing. Vincenzo Gianferrari Pini*
Chairman & CTO - GoCloud
Sede legale: Via Larga 15, 20122 Milano (MI)
Sede op. ed amm.: Via Stefanardo da Vimercate 28, 20128 Milano (MI)
Cel. +39-3939837493
tel: +39-02-87250672 (dir.)
tel: +39-02-25514300 (cent.)
fax: +39-02-25514302

https://google.com/+VincenzoGianferrariPiniGoCloud
https://www.linkedin.com/in/vgianferrari
http://www.gocloud.eu
​http://www.mapadore.com​





-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Vincenzo Gianferrari Pini]

Hi all,

sorry for not having been active at all in the last period.

Anyway, I agree with Ioan that using GIT is *much* more productive than
using SVN, so I cast here my +1.

Regards,
Vincenzo

Il giorno lun 24 ago 2015 alle ore 21:51 Ioan Eugen Stan <
stan.ieu...@gmail.com> ha scritto:

> Hi,
>
> Yes, the work flow is not the best with SVN. There is an option to
> migrate James to git hosting and personally I think it will be a good
> thing.
>
> In order to make this a reality we have to raise a vote and raise a JIRA
> issue to Apache Infra. The vote has to run for 72h.
>
> You have my +1.
>
> p.s. One thing to have in mind is that we need to we need to take care
> of the site publishing also. But it's doable.
>
> Regards,
>
> --
*Ing. Vincenzo Gianferrari Pini*
Chairman & CTO - GoCloud
Sede legale: Via Larga 15, 20122 Milano (MI)
Sede op. ed amm.: Via Stefanardo da Vimercate 28, 20128 Milano (MI)
Cel. +39-3939837493
tel: +39-02-87250672 (dir.)
tel: +39-02-25514300 (cent.)
fax: +39-02-25514302

https://google.com/+VincenzoGianferrariPiniGoCloud
https://www.linkedin.com/in/vgianferrari
http://www.gocloud.eu
​http://www.mapadore.com​


-- 
Il presente messaggio ed i suoi allegati sono confidenziali in ragione 
della loro natura professionale e, in ogni caso, non accessibili al 
pubblico. Nel caso in cui il presente messaggio sia ricevuto da soggetto 
diverso dal suo diretto destinatario, si prega di darne immediata notizia 
all'indirizzo email *amministrazi...@gocloud.it 
* e di procedere immediatamente alla 
distruzione dello stesso e dei suoi allegati, da effettuarsi senza copiarne 
o rivelarne ad alcuno il contenuto. Grazie per la collaborazione.

This e-mail and any  attachments thereto are confidential and may be 
covered by legal professional privilege or otherwise protected from 
disclosure. If you are not the intended recipient, please notify us 
immediately by email to the address *amministrazi...@gocloud.it 
* and then immediately destroy this message and 
any attachments from your system, without copying or disclosing the 
contents to any other person. Thank you for your co-operation.

Re: Proposal about James modules merge

[Stephen Brewin]

My previous post corrected for copy/paste issues on phone!

On 27/08/2015 17:17, Stephen Brewin wrote:

On 27/08/2015 10:24, Matthieu Baechler wrote:

Hi Steve,

Thanks for your feedback.

On 27/08/2015 11:11, Stephen Brewin wrote:

Hi

As I recall, the intent of having separate projects for many of the
components developed under the James umbrella was to satisfy the
requirement that they should be independent of James Server. While this
remains a requirement, separate repositories are needed for each 
project
to allow separate release versions and schedules. It also influences 
our

maven module layout and how dependencies might be better managed.

Before proceeding with a discussion of how to simplify the development
workflow, we need to decide if the original requirement still holds.
Prospective solutions will be quite different depending on this answer.




I think there's always a balance to find between flexibility and 
simplicity and we should never loose flexibility when there's nothing 
to gain.


That's why I think we should not talk of requirements for "James 
Server components" as a whole but to weight for each component what 
we can gain and what we would loose.


It's what I tried to do by excluding jdkim, jsieve, jspf and mime4j 
from the merge candidates as there's very little to gain IMO (I might 
be wrong, of course).


I would really like the community to tell me what are the cases where 
the other components could benefit from being separated, that would 
help answering the question "do we want to change the requirements" ?


Cheers,


Hi Matthieu

At the moment the separate projects are only semi-independent which 
gives rise to workflow and dependency problems. They should either be:


  * Entirely independent, maintained separately from James server and
James server should treat them no differently to other 3rd party
artefacts
  * Sub-modules of James server, sharing the same repository, version
numbers and dependency library (via a BOM), thereby easing the
workflow and dependency problems


/If we switch to the sub-module approach the separate maven artefacts 
would still be available to 3rd parties, which I believe captures the 
original intent of the requirement./


Theoretically, separate projects allow each to follow a more nimble 
release cycle than James server, but recent years have shown little 
evidence in support of this.


Cheers
--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Stephen Brewin]

On 27/08/2015 10:24, Matthieu Baechler wrote:

Hi Steve,

Thanks for your feedback.

On 27/08/2015 11:11, Stephen Brewin wrote:

Hi

As I recall, the intent of having separate projects for many of the
components developed under the James umbrella was to satisfy the
requirement that they should be independent of James Server. While this
remains a requirement, separate repositories are needed for each project
to allow separate release versions and schedules. It also influences our
maven module layout and how dependencies might be better managed.

Before proceeding with a discussion of how to simplify the development
workflow, we need to decide if the original requirement still holds.
Prospective solutions will be quite different depending on this answer.




I think there's always a balance to find between flexibility and 
simplicity and we should never loose flexibility when there's nothing 
to gain.


That's why I think we should not talk of requirements for "James 
Server components" as a whole but to weight for each component what we 
can gain and what we would loose.


It's what I tried to do by excluding jdkim, jsieve, jspf and mime4j 
from the merge candidates as there's very little to gain IMO (I might 
be wrong, of course).


I would really like the community to tell me what are the cases where 
the other components could benefit from being separated, that would 
help answering the question "do we want to change the requirements" ?


Cheers,


Hi Matthieu

At the moment the separate projects are only semi-indendendant which 
gives rise to workflow and dependency problems. They should either be:


 * Entirely independent, maintained separately from James server and
   James server should treat them no differently to other 3rd party
   artefacts
 * Sub-modules of James server, sharing the same repository, version
   numbers and dependency library (via a BOM), thereby easing the
   workflow and dependency problems.

/If we switch to the sub-module approach the separate maven artefacts 
would still be available to 3rd parties, which I believe captures the 
original intent of the requirement. /


Theoretically, separate projects allow each to follow a more nimble 
release cycle than James server, but recent years have shown no evidence 
in support of this.


Cheers
--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Matthieu Baechler]

Hi Steve,

Thanks for your feedback.

On 27/08/2015 11:11, Stephen Brewin wrote:

Hi

As I recall, the intent of having separate projects for many of the
components developed under the James umbrella was to satisfy the
requirement that they should be independent of James Server. While this
remains a requirement, separate repositories are needed for each project
to allow separate release versions and schedules. It also influences our
maven module layout and how dependencies might be better managed.

Before proceeding with a discussion of how to simplify the development
workflow, we need to decide if the original requirement still holds.
Prospective solutions will be quite different depending on this answer.




I think there's always a balance to find between flexibility and 
simplicity and we should never loose flexibility when there's nothing to 
gain.


That's why I think we should not talk of requirements for "James Server 
components" as a whole but to weight for each component what we can gain 
and what we would loose.


It's what I tried to do by excluding jdkim, jsieve, jspf and mime4j from 
the merge candidates as there's very little to gain IMO (I might be 
wrong, of course).


I would really like the community to tell me what are the cases where 
the other components could benefit from being separated, that would help 
answering the question "do we want to change the requirements" ?


Cheers,

--
Matthieu Baechler

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Stephen Brewin]

Hi

As I recall, the intent of having separate projects for many of the 
components developed under the James umbrella was to satisfy the 
requirement that they should be independent of James Server. While this 
remains a requirement, separate repositories are needed for each project 
to allow separate release versions and schedules. It also influences our 
maven module layout and how dependencies might be better managed.


Before proceeding with a discussion of how to simplify the development 
workflow, we need to decide if the original requirement still holds. 
Prospective solutions will be quite different depending on this answer.


Cheers
--Steve

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Proposal about James modules merge

[Hassan Latif]

+1

On Mon, Aug 24, 2015 at 3:37 PM, Matthieu Baechler 
wrote:

> Hi all,
>
> For some months, Antoine Duprat, Benoit Tellier and myself are working
> daily on James 3.
>
> We tried hard to make our development workflow as simple as possible.
>
> One thing that's very annoying right now is that James is composed of
> several git repositories.
>
> We tried to use "git submodule" to aggregate all James modules into a
> parent project. You can see it by yourself here :
> https://github.com/linagora/james-parent
>
> Everything would be ok if James were mature enough to make changes in a
> single repository for most fixes and improvements.
>
> Our experience is that we almost always change several repositories for a
> task : adding a test to MPT tests often leads to some fixes in other
> modules, adding an implementation for an interface requires some spring
> configuration in the main repository, even a maven dependency in a given
> repository often breaks other modules when putting everything together.
>
> With the current layout, our workflow contains some tricky tasks :
>
>  x to create a "global branch", we create a branch for every repository, a
> branch for james-parent, then we have to commit everything for james-parent
> to point to the right version of submodules.
>
>  x for every single commit, we need to commit james-parent too in order to
> follow submodule revision
>
>  x We need several Pull-Requests for a single task, that makes it hard to
> reason about changes and it must be applied in the right order
>
>  x Branch merging is painful too
>
>
> While discussing in JIRA (https://issues.apache.org/jira/browse/JAMES-1588),
> Eric asked me to come with a proposal about merging some James repositories
> together and ask for feedback about Pros and Cons.
>
> To start, here is the repositories needed to build James :
>
> jdkim
> jsieve
> jspf
> mime4j
> james-project
> james
> mailbox
> mailet
> mpt
> protocols
>
> The first four are specialized modules and don't need to change with James
> development IMO, so they can live in their own git repository.
>
> The other 6 (7 if you count the JIRA-1588 proposed "backends" repository)
> are very tied together and could be merged into a single git repository
> with the following tree :
>
> james-project
> | james
> | mailbox
> | mailet
> | mpt
> | protocols
> | backends
>
> We should then make it clear on the website what is a "library" (jdkim,
> jsieve, jspf, mime4j) dependency and what is a "server module" (everything
> else).
>
> Here is my opinions about this new organization :
>
> Pro :
>
> + Faster dev for cross-module developments
> + Merge integration testing (MPT) to the main repository
> + Less build ordering issues when merging changes in two modules
> + Easier "Getting Start" (git clone, mvn compile)
> + Easier Tagging/Release
>
> Con :
>
> - Slower "mvn package"
> - Every modules have to evolve accordingly to prevent build failure
>
> Feel free to add your opinion to this proposal.
>
> --
> Matthieu Baechler
>
> -
> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
> For additional commands, e-mail: server-dev-h...@james.apache.org
>
>

Re: Proposal about James modules merge

[Ioan Eugen Stan]

Hi,

Yes, the work flow is not the best with SVN. There is an option to
migrate James to git hosting and personally I think it will be a good
thing.

In order to make this a reality we have to raise a vote and raise a JIRA
issue to Apache Infra. The vote has to run for 72h.

You have my +1.

p.s. One thing to have in mind is that we need to we need to take care
of the site publishing also. But it's doable.

Regards,



signature.asc
Description: OpenPGP digital signature