On 03.11.21 04:45, btell...@apache.org wrote:
> Hello Otto,
>
It is actually Karsten :-)
> This sounds very promising! Thanks for sharing your feedback as well as
> these enhancements.
>
> I will give my thoughts on each one of these propositions, inlined in
> the email.
>
> Best regards,
>
> Benoit TELLIER
>
> On 02/11/2021 23:41, Otto, Karsten Andreas wrote:
>> Dear James Community,
>>
>> Over the last two years, we at Akquinet have developed an email solution
>> for the medical and healthcare sector. We chose Apache James for this
>> project because it provides
>> - a robust clustering solution out of the box,
>> - a comprehensive WebAdmin REST interface for integration with other
>> product components,
>> - a flexible Mailet architecture easily adapting to our specific
>> requirements,
>> - an open source solution with an active community and commercial
>> support where needed.
>>
>> Our deployment uses the distributed-pop3 app variant, with multiple
>> James server instances running on top of a Cassandra cluster,
>> S3-compatible blob storage and RabbitMQ. The choice of POP3 may seem
>> strange, but our customers typically employ third party systems for
>> semi-automatic mail processing. For this use case, POP3 enables much
>> simpler systems integration than the more complex protocols.
>>
>> Despite Apache James being a very flexible solution, we encountered a
>> few situations during development where we had to change the original
>> codebase in order to meet our requirements. We believe these changes
>> might well be of interest to the James community at large, and in the
>> spirit of open source we would like to share them with you!
>>
>> Over the next few weeks we plan to create Jira tickes and pull requests
>> for the features below. Let us know what you think, and which you would
>> like to see first!
>>
>> (The list includes a complexity estimate in T-Shirt size, where S is
>> just a few / localized changes and L is a lot of code / all over the place.)
>>
>>
>> SECURITY ENHANCEMENTS
>>
>> # TLS authentication via client certificate (M)
>>
>> Add options to network server configurations to set certificate-based
>> authentication mode (none, want, need), and the associated trust store
>> to validate these client certificates. Useful to limit server access to
>> trusted partners/users.
> +1
>
> Here I guess we are speaking of additional options to
> AbstractConfigurableAsyncServer class family within
> server/protocols/protocols-library maven module. Correct?
>
Yes, excactly.
> The hardest part is likely (unit) testing of the developed code. Not
> sure what is the best strategy regarding this...
Same thought here, and there was no test for this class before. I went
with manual testing for now, but if anybody has suggestions for proper
unit testing, I'd appreciate them.
>> # Separate trust store for S3 (M)
>>
>> Extend blob store configuration to specify a trust store, which is used
>> to validate the S3 server certificate. Useful if also using TLS client
>> cert authentication (see above) to keep the security realms separate.
> +1
>
> Just like one can pass certs / truststore for Cassandra and
> ElasticSearch one should be able to tune TLS for S3.
>>
>> # Delay on authentication failure (S)
>>
>> Add an option verifyFailureDelay to usersrepository.xml to delay the
>> response if someone tries to authenticate with a non-existing user or
>> wrong password. Basic protection against people using James as a
>> password oracle for brute-force/dictionary attacks.
> +1
>
> On HTTP I would have been saying "just use an API gateway ;-)" but no
> such things exists for IMAP/SMTP / TCP protocols. Thus I think your
> proposal is beneficial.
Indeed, we needed a central point for enforcing this, regardless of
which protocol triggered the authentication.
>>
>> # Support password salting (M) *
> +1
>
> Historically I have only be seriously using the LDAP repository and the
> default (eg Cassandra) implementation is trivially exposed to hasing
> table dictionnary attacks.
>
> Salting would prevent this. This SHOULD become the default ;-)
>
> Also we need to take care of this for the JPA implementation that
> follows a slightly different code path.
I saw that during development, but since we do not use JPA there was no
time to implement this yet, unfortunately. If necessary I can take
another crack at it.
>>
>> Add extra hashingMode choices in usersrepository.xml ("salted",
>> "legacy_salted") to include the user name in the password hash. Basic
>> protection against rainbow table cracking if someone ever manages to
>> steal the password database.
> As a side note:
> - We are writing something new, thus I do not think adding a
> legacy_salted new hashing mode makes any sense as we don't want to
> maintain something that nobody is using...
> - I would be happy to specify the hash algorithm used in combination
> with the salt. Eg salted_sha512, salted_sha256...
>
Salting is implemented as an extra accessor in Algorithm, isSalted(),
next to isLegacy(). So legacy_salted mode is kind of an artifact of that.
I believe Akquinet is the only current user of legacy_salted, as we
introduced the salting feature *before* we discovered we were using
legacy mode. That said, it is certainly discouraged for new users, and
maybe should just skip it in the documentation.
> Maybe we do not need to implement a generic salted_{hashalg}, but I
> would prefer to include the hashing algorithm in the name of this new
> option (so salted => salted_sha512)
>
In the current Cassandra implementation there is an extra column
"hashingmode" next to "algorithm". It is a very simple migration this
way, just need to add the column once (documented in
upgrade-instructions.md), and configure the default if it is not
set/null. I am not opposed to combining the two columns, but that would
require a more complex schema migration implementation.
>>
>> # Gradual migration of password hash settings (L) *
> +1
>
> The simplest way to achieve this is likely to specify the hash algorithm
> along side the new password in User::setPassword. This should then make
> it easy to implement.
We extended the DefaultUser constructor so that it takes two Algorithm
parameters: The current algorithm from the user database, used in
verifyPassword(), and the preferred algorithm from the configuration,
used in setPassword(). This way we avoided changing the parameters of
these methods, which would have had a much wider impact on the code base.
>>
>> Add a hashingMode column to the user table, use it together with the
>> algorithm column to verify password hashes. But use the configured
>> algorithm and hashingMode from usersrepository.xml when updating the
>> password. This way, the user database can gradually migrate to a
>> different (hopefully stronger) security setting. Useful to get rid of
>> the legacy hashing mode, and to introduce password salting (see above).
>>
>> *NOTE: Currently only works with Cassandra/Memory, but should be
>> possible for JPA as well if someone more experienced with this server
>> variant would like to tackle it.
>>
>>
>> POP3 ENHANCEMENTS
>>
>> # Fix WebAdmin routes for POP3 (S) - already submitted
>>
>> Include WebAdmin routes and example configuration for managing the
>> DeletedMessagesVault in the distributed-pop3 server variant.
>>
>> # Add Glowroot instrumentation for POP3 (S)
>
> +1
>
> Glowroot is useful for debugging, investigating slow calls,
> understanding Cassandra query flow.
>
>
> Bare in mind that Glowroot have a ~25% CPU costs, especially visible on
> Cassandra intensive mail requests.
>>
>> Minor refactoring of POP3 command handlers, and matching glowroot
>> configurations. Better diagnostics on what is going on in POP3 handling.
>>
>> # Configure restore location for deleted messages (S)
> +0.5 I guess it do not hurt
>
> (and I'm happy to see the deleted message vault is used :-) )
>>
>> Add an option to specify which mailbox receives any restored messages
>> from the DeletedMessagesVault, e.g. "INBOX" instead of the default
>> "Restored-Messages". Required since POP3 cannot access other mailboxes
>> than INBOX.
It does not hurt in the general case, but is required to make POP3 work
with DMV. And obviously, "Restored-Messages" is the default if there is
no configuration given.
>>
>>
>> GENERAL FEATURES
>>
>> # Extra system properties (S)
>>
>> Read a properties file immediately at server startup and set them as
>> system properties. Useful to keep command line clutter down and manageable.
> +1
>
> We had a recurring debate with @mbaechler on the use of system
> properties within James.
>
> On the one side there is the ease of use (no need to read a
> configuration file, transform it in a POJO, and rely on abstract
> factories etc to customise stuff).
>
> On the other hand there are concerns around the discoverability of such
> properties.
>
> CF https://issues.apache.org/jira/browse/JAMES-3487
>
> Maybe having this file would be a good place to document all those
> properties (and make the use of System properties an implementation detail).
>
> Would this suite you @mbaechler?
>
> @Otto maybe we could have a list of the system properties you did find
> useful? I think the community have limited knowledge regarding system
> property usage with James... At least me...
>
We mostly use this for TLS tweaks, in particular javax.net.ssl.keyStore
and friends to be used by Java Mail API. This is the client side for
mutually authenticated TLS connections, and worked thios way out of the
box. It might make sense to move this to a more suitable James
configuration file, but I am not sure where this would have to go.
>>
>> # Check user credentials via WebAdmin (M)
>>
>> Provide a WebAdmin route to check a username/password combination.
>> Reports 204 on success and 401 on failure. Useful for integrating James
>> with 3rd party services, e.g. a web admin GUI for users.
> +1
>>
>> # Generalized vacation handling, including via WebAdmin (L)
>>
>> Refactor the vacation handling code out of JMAP space and make it a
>> general James feature. Add a WebAdmin route to query and change a user's
>> vacation notice. Useful for integrating James with 3rd party services,
>> e.g. a web admin GUI.
> +0.1
>
> I do not have oppositions regarding this though we need to keep in mind
> Vacations are tightly coupled to the JMAP spec
> (https://jmap.io/spec-mail.html#vacation-response)
>
> I understand working with a nice JSON format is easier than aweful Sieve
> scripts, I understand using vacations pulls in the all pack of JMAP
> dependencies, thus I agree with such a refactoring.
>
>>
>>
>> Regards,
>> Karsten Otto
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
> For additional commands, e-mail: server-dev-h...@james.apache.org
>
