Re: Code organisation: makes guice application easier to discover
Hello Jean! On 08/06/2021 17:21, Jean Helou wrote: > Hi benoit ! > > On my crusade to reorganize James code related to Guice apps (and >> promote their adoption), I come to the next item of the tick list (after >> ZIP packaging, JIB packaging to enable distribution). >> > These were great improvements > > >> I would like to make those application easier to find in the source tree. >> > and this one promises to be too ! > > >> Here would be the principles I would like to enforce: >> - All server applications should be collocated under the same directory >> - Server application modules should be clearly separated from guice >> module declarations >> >> I do not intend to fully reorder James directories just now but rather >> take small steps in a globally consensual direction. As such I propose >> the following directory layout as a first step: >> > This first step would be nice and at least would make it easy to add a link > to all the provided james-server builds in the readme, The instructions for > running each provided build could then go in the readme of each build and > reduce the clutter in the primary readme, I'm all for it :D > > But if you have a `first step` then you must also have a goal, it would be > easier to assess the usefulness of the first step if we shared the vision > of the goal. The goal would be that the directory structure : - reflects the project goals - is honest toward the progress toward these goal - allow easily understanding the project architecture - and allow to easily find the project deliveries, tests, etc... - [add your goals here] I am starting with this last one, likely the easiest and less controversial to start with. > > I am partly aware of the history of the project and that some of the top > level directories are projects which used to live in different repositories > and have been merged back. so james git is essentially a mono-repo and > that's fine > > According to the website (which seems to have lost part of its colorful > landing page by the way) the primary components are : > - Server +1 > - Mailets +1, intended to be a generic API just like servlets are. Their use was intended outside of James. (source Dany Angus at ACEU 2019) This goal and its results could be re-assessed but I am not wishing to do this. > - Mailbox Regarding that one I do think mailbox do not make sense without a server to use it. I would be in favor to move it in the /server directory. A topic for another day. This module is tightly coupled with the protocols consuming it, underwent heavy changes during the JMAP implementations, and overall is a critical server component we need full control upon (eg performance). > - Protocols I do think protocols without a hard dependency on mailbox or server could be considered top level, but protocols like IMAP with a hard dependency on mailbox likely can't. > - MPT MPT is a weird thing. It mixes a set of tools along side with a test suite exercising other components. I think mpt/impl would deserve to be relocated. Maybe to /server/protocols/imap-tests /server/protocols/smtp-tests & /server/protocols/managesive-tests ? > > So I would expect these to be the top level directories in the project, > along with a couple supporting directories such as > - docs > - shared-libraries > maybe > - examples > > I included shared-libraries because I assume a reason for some of the > directories being top level is that they are used by multiple top level > modules : > my *guess* would be > - json > - core > - metrics > - testing/base > > Others sound like they should be reintegrated in the server directory tree > (guesstimate again) : > - backends-common Project with this dependency would also need to be in the server directory tree. This match the move of mailbox into server. > - event-bus > - event-sourcing > - benchmarks > - dockerfiles With the exception that it now mostly contains project related docker (compilation environment, websites) thus it can stands at "top level" in my opinion. > - grafana-reporting - third-party > > I have not taken the time to do a proper dependency analysis, this is just > how I expect to find the dependencies so I may be wrong :D We pretty much agree. > > If this organization is close to what you have in mind then the first step > is indeed the best. > > However If the goal is to push server in the spotlight then we should move > the apps directory to the top level and move mpt, mailets, mailbox, > protocols and mpt etc inside server > > Note that this doesn't prevent documenting them, nor publishing the > artifacts nor enforcing proper dependencies > > jean Benoit - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: What's the reason for having Username's constructor private?
Hello Andreas, On 08/06/2021 21:56, Andreas Joseph Krogh wrote: > På tirsdag 08. juni 2021 kl. 16:02:22, skrev Jean Helou > mailto:jean.he...@gmail.com>>: > > Hello Andreas, > > We have an environment where usernames can be *any* character thus the > > rules enforced by Username does not apply to us. We were > thinking about > > subclassing it, until we saw it has a private constructor > signaling that it > > isn't meant for extending. > > > > Do you mean org.apache.james.core.Username ? > > > > Yes. > > > > If I understand correctly, the only character restriction there is you > can't put an @ in the local part ... even > org.apache.james.core.Username#fromLocalPartWithoutDomain[1] won't > bypass > that ( and of course the usual not null, not empty) > As you can see in > > https://github.com/apache/james-project/blob/master/core/src/main/java/org/apache/james/core/Username.java#L132, > a Username has a mapping to a org.apache.james.core.MailAddress > and this > mapping is used in multiple places. Allowing @ in the username > would break > that conversion. > > Can you provide a more detailed picture of what you are trying to > achieve > and why you would want @ as part of the username ? > > > > We're building a custom IMAP-server, based on JAMES, where users > provide their IMAP-username on the format @, > where the username may be anything (except blank, ofcourse). The > (provided as part of the USERNAME as far as JAMES/IMAP > knows, parsed as "anything after the last '@'-character") is used for > knowing which database-mapping to use. It seems JAMES is hardcoded > to the concept of usernames being an email-address, or part of it, and > that is not always the case for us/our users. In that sense it seems > JAMES is intended for use as an application, not a library, is this > correct? > > In other words; We need the username to be "whatever", and > case-sensitive, so it would be preferable if Username was less > restrictive, or an interface. Without this we'll have to base32-encode > (because of case-insensitivity in Username) the username parsed with > our custom LoginCommandParser, and decode where needed. > > > Thanks. > I do not get why you need to rely on the Username POJO in your custom IMAP stack. Maybe you can introduce some abstractions that suites your needs and have the logic to convert it to the James POJOs in places that are needed. Also turning off virtual hosting enables to pass any string (except `@`) in the local part. Maybe this could be sufficient to easily solve your issue? Benoit > -- > *Andreas Joseph Krogh* > CTO / Partner - Visena AS > > > > (I'll admit that the current check is overly restrictive as it doesn't > account for quoting in the local part [2] which allows using @ in > a quoted > string or escaping @ using a \) > > [1] > > https://github.com/apache/james-project/blob/master/core/src/main/java/org/apache/james/core/Username.java#L71 > [2] https://datatracker.ietf.org/doc/html/rfc3696#section-3 > > Cheers, > Jean > > > > - > To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > For additional commands, e-mail: server-dev-h...@james.apache.org
Re: What's the reason for having Username's constructor private?
På tirsdag 08. juni 2021 kl. 16:02:22, skrev Jean Helou mailto:jean.he...@gmail.com>>: Hello Andreas, We have an environment where usernames can be *any* character thus the > rules enforced by Username does not apply to us. We were thinking about > subclassing it, until we saw it has a private constructor signaling that it > isn't meant for extending. > Do you mean org.apache.james.core.Username ? Yes. If I understand correctly, the only character restriction there is you can't put an @ in the local part ... even org.apache.james.core.Username#fromLocalPartWithoutDomain[1] won't bypass that ( and of course the usual not null, not empty) As you can see in https://github.com/apache/james-project/blob/master/core/src/main/java/org/apache/james/core/Username.java#L132, a Username has a mapping to a org.apache.james.core.MailAddress and this mapping is used in multiple places. Allowing @ in the username would break that conversion. Can you provide a more detailed picture of what you are trying to achieve and why you would want @ as part of the username ? We're building a custom IMAP-server, based on JAMES, where users provide their IMAP-username on the format @, where the username may be anything (except blank, ofcourse). The (provided as part of the USERNAME as far as JAMES/IMAP knows, parsed as "anything after the last '@'-character") is used for knowing which database-mapping to use. It seems JAMES is hardcoded to the concept of usernames being an email-address, or part of it, and that is not always the case for us/our users. In that sense it seems JAMES is intended for use as an application, not a library, is this correct? In other words; We need the username to be "whatever", and case-sensitive, so it would be preferable if Username was less restrictive, or an interface. Without this we'll have to base32-encode (because of case-insensitivity in Username) the username parsed with our custom LoginCommandParser, and decode where needed. Thanks. -- Andreas Joseph Krogh CTO / Partner - Visena AS (I'll admit that the current check is overly restrictive as it doesn't account for quoting in the local part [2] which allows using @ in a quoted string or escaping @ using a \) [1] https://github.com/apache/james-project/blob/master/core/src/main/java/org/apache/james/core/Username.java#L71 [2] https://datatracker.ietf.org/doc/html/rfc3696#section-3 Cheers, Jean - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: What's the reason for having Username's constructor private?
Hello Andreas, We have an environment where usernames can be *any* character thus the > rules enforced by Username does not apply to us. We were thinking about > subclassing it, until we saw it has a private constructor signaling that it > isn't meant for extending. > Do you mean org.apache.james.core.Username ? If I understand correctly, the only character restriction there is you can't put an @ in the local part ... even org.apache.james.core.Username#fromLocalPartWithoutDomain[1] won't bypass that ( and of course the usual not null, not empty) As you can see in https://github.com/apache/james-project/blob/master/core/src/main/java/org/apache/james/core/Username.java#L132, a Username has a mapping to a org.apache.james.core.MailAddress and this mapping is used in multiple places. Allowing @ in the username would break that conversion. Can you provide a more detailed picture of what you are trying to achieve and why you would want @ as part of the username ? (I'll admit that the current check is overly restrictive as it doesn't account for quoting in the local part [2] which allows using @ in a quoted string or escaping @ using a \) [1] https://github.com/apache/james-project/blob/master/core/src/main/java/org/apache/james/core/Username.java#L71 [2] https://datatracker.ietf.org/doc/html/rfc3696#section-3 Cheers, Jean > > > I propose to change this constructor to be protected so one may extend > this class with custom validation-logic for the userName. How are others > handling this problem? > > -- > *Andreas Joseph Krogh* > CTO / Partner - Visena AS > > - > To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > For additional commands, e-mail: server-dev-h...@james.apache.org
What's the reason for having Username's constructor private?
We have an environment where usernames can be any character thus the rules enforced by Username does not apply to us. We were thinking about subclassing it, until we saw it has a private constructor signaling that it isn't meant for extending. I propose to change this constructor to be protected so one may extend this class with custom validation-logic for the userName. How are others handling this problem? -- Andreas Joseph Krogh CTO / Partner - Visena AS - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Created] (JAMES-3595) Spooler processing starts before mailetContainer initialisation
Benoit Tellier created JAMES-3595:
-
Summary: Spooler processing starts before mailetContainer
initialisation
Key: JAMES-3595
URL: https://issues.apache.org/jira/browse/JAMES-3595
Project: James Server
Issue Type: Bug
Affects Versions: 3.6.0
Reporter: Benoit Tellier
Fix For: 3.7.0
I got the following log:
{code:java}
{
"_index": "emaily-2021.06.08",
"_type": "_doc",
"_id": "gByI6nkBU6_e7iSzPRK3",
"_score": 1,
"_source": {
"@timestamp": "2021-06-08T07:31:07.562Z",
"log":
"{\"timestamp\":\"2021-06-08T07:31:07.560Z\",\"level\":\"ERROR\",\"thread\":\"elastic-54\",\"logger\":\"org.apache.james.mailetcontainer.lib.AbstractStateCompositeProcessor\",\"message\":\"MailProcessor
'transport' could not be found. Processing
7d01c8f0-c82b-11eb-aa8a-9fabe7587cf3 to 'error'
instead\",\"context\":\"default\"}\n",
"stream": "stdout",
"time": "2021-06-08T07:31:07.562707651Z",
"log_processed": {
"timestamp": "2021-06-08T07:31:07.560Z",
"level": "ERROR",
"thread": "elastic-54",
"logger":
"org.apache.james.mailetcontainer.lib.AbstractStateCompositeProcessor",
"message": "MailProcessor 'transport' could not be found. Processing
7d01c8f0-c82b-11eb-aa8a-9fabe7587cf3 to 'error' instead",
"context": "default"
},
"k8s:pod_name": "james-jmap-764cc444bd-mjv79",
"k8s:namespace_name": "tenant-xx",
"k8s:pod_id": "424e3114-8611-4ba0-8cf1-752de8fc9bba",
"k8s:annotations": {
"cni_projectcalico_org/podIP": "10.2.0.169/32",
"kubectl_kubernetes_io/restartedAt": "2021-04-20T16:18:12+07:00"
},
"k8s:host": "node-a991ce2a-d56c-4728-9971-97abf73e0567",
"k8s:container_name": "james-jmap",
"k8s:docker_id":
"50140c8de5b57fde58878fd30204c7764b0eb7d96181101cdb755a65459e65e9",
"k8s:container_hash":
"chibenwa/openpaas-james-distributed-es6-backport@sha256:ef0325451bfe9cd862e1abccf358b683f7449e7c79ee8aa5f9a12ce49c1960f2",
"k8s:container_image":
"chibenwa/openpaas-james-distributed-es6-backport:unboundid2",
"k8s:app": "james",
"k8s:instance": "james-jmap",
"k8s:pod-template-hash": "764cc444bd"
},
"fields": {
"k8s:annotations.kubectl_kubernetes_io/restartedAt": [
"2021-04-20T09:18:12.000Z"
],
"log_processed.timestamp": [
"2021-06-08T07:31:07.560Z"
],
"@timestamp": [
"2021-06-08T07:31:07.562Z"
],
"time": [
"2021-06-08T07:31:07.562Z"
]
}
}
{code}
Clearly some processing was attempted before the startup finished...
I expect mail to be processed only when the mailetcontainer is ready for it...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Code organisation: makes guice application easier to discover
Hi benoit ! On my crusade to reorganize James code related to Guice apps (and > promote their adoption), I come to the next item of the tick list (after > ZIP packaging, JIB packaging to enable distribution). > These were great improvements > I would like to make those application easier to find in the source tree. > and this one promises to be too ! > Here would be the principles I would like to enforce: > - All server applications should be collocated under the same directory > - Server application modules should be clearly separated from guice > module declarations > > I do not intend to fully reorder James directories just now but rather > take small steps in a globally consensual direction. As such I propose > the following directory layout as a first step: > This first step would be nice and at least would make it easy to add a link to all the provided james-server builds in the readme, The instructions for running each provided build could then go in the readme of each build and reduce the clutter in the primary readme, I'm all for it :D But if you have a `first step` then you must also have a goal, it would be easier to assess the usefulness of the first step if we shared the vision of the goal. I am partly aware of the history of the project and that some of the top level directories are projects which used to live in different repositories and have been merged back. so james git is essentially a mono-repo and that's fine According to the website (which seems to have lost part of its colorful landing page by the way) the primary components are : - Server - Mailets - Mailbox - Protocols - MPT So I would expect these to be the top level directories in the project, along with a couple supporting directories such as - docs - shared-libraries maybe - examples I included shared-libraries because I assume a reason for some of the directories being top level is that they are used by multiple top level modules : my *guess* would be - json - core - metrics - testing/base Others sound like they should be reintegrated in the server directory tree (guesstimate again) : - backends-common - event-bus - event-sourcing - benchmarks - dockerfiles - grafana-reporting I have not taken the time to do a proper dependency analysis, this is just how I expect to find the dependencies so I may be wrong :D If this organization is close to what you have in mind then the first step is indeed the best. However If the goal is to push server in the spotlight then we should move the apps directory to the top level and move mpt, mailets, mailbox, protocols and mpt etc inside server Note that this doesn't prevent documenting them, nor publishing the artifacts nor enforcing proper dependencies jean
