> > So from a user perspective adding a user would always succeed. But would > it > > succeed by doing nothing (the current behaviour in silencing the > > AlreadyExist exception) or would it succeed by effectively overwriting > the > > user (in a last write wins manner) ? > > Webadmin so far overwrite the user (and its password) in a last write > win manner. >
That sounds really scary > - Either we need to distinguish "create" from "update" within the > webadmin API > Well that would definitely have my vote : as an admin operator I *never* want to accidentally overwrite an existing user when trying to create a new one (with the possible exception of retrying a create operation that just timeouted, in which case my first reflex would be to execute a read to try and make sure that the operation that just failed hasn't actually succeeded) > Moreover what you say regarding conflict is very interesting. It suggest > strong consistency might not be needed. A simple "read before your > write" would be enough to see if a long standing user would be updated, > while not dealing with conflict for newly created user. > Rereading that paragraph, I realize I am biased towards human time scale. That's an option if the api/system is only operated by a human because computers and networks are relatively fast in the human time scale. If the api is used for automation that's not so true anymore as consensus propagation is slow in the computer time scale. With that said, user driven write operations on users should be relatively rare : creation, some updates and eventually deletion should probably trade off speed for consistency. if there are write operations triggered by internal events these may not require full consistency To illustrate a bit with actual use cases : As an admin of the server, I trigger a user creation, since I'm a sloppy admin, I already communicated the credentials and email address to 3rd parties which may start trying to use them while the creation is in progress *before* the create command completes I wouldn't expect logins to succeed or sent emails to be delivered though it wouldn't be an issue if they were *after* the command completes I expect all logins and all email delivery to succeed This implies full consistency on the write operation, but inconsistent reads are fine As a user I change my password , I know the new password since I provided it to the update command. *before* the command completes I would expect logins with the new credentials to fail and I would expect logins with the old credentials to succeed *after* the command completes I expect all logins with the new credentials to succeed and all logins with the old credentials to fail this implies full consistency on the corresponding read and writes (for password updates) you can probably infer the delete expectations are the reverse of the creation :) as far as I can tell the user only bears username and password and here I assume that username are immutable but updating the username mutability would require the same consistency constraints as updating the password > Also for ACLs isn't eventual consistency acceptable ? > My take is that it is. My first shot was to do just that. > > Howver the current code enforces a "transaction like" behavior, with an > ACLDiff fired on the mailbox event system. Maintaining denormalization > was also an issue in the face of reset (requiring to know previously > stored data). > > A full rewritte of the context ACLs are run in would have been needed. > And could have been controversial. > > > using transactions to > > avoid non serial writes but accepting stale reads ? > > Could be too. This could be a quick-win to enhance existing situation, > while waiting for stronger decisions and What you describe could be a very good short term solution for message > flags (I think we will need to challenge CONDSTORE/QRSYNC anyway!) > Why do you say this would only be a good short term solution ? My understanding is that currently - most reads and writes use LWT and that using LWT implies slow writes (to something akin to a lock I suppose) for both read and write operations - the amount of reads far outweighs the number of writes (at least for the listed use cases of ACLs, users and domains, it sounds like the UID/Flags stuff may not be so clear cut) In such a context dropping read consistency while keeping write consistency sounds like it would already be a huge gain :) The assertions in a number of tests will likely have to be updated to rely on `Awaitility.await()` :) On Wed, Dec 9, 2020 at 2:40 AM Tellier Benoit <[email protected]> wrote: > Sorry for repost, > > I sent that response before but it was lost. > > Maybe the unfamous text/html format issue. > > Le 07/12/2020 à 04:33, Jean Helou a écrit : > > Hello, > > > > I'm currently trying to increase overall efficiency of the Distributed > >> James server. > >> > > > > I have some concerns but i feel imposterish for posting them as they most > > likely come from my own lack of knowledge, i'll still try just in case > some > > of points are valid :) > > Thank you very much to dare to do so! > > You are likely not the only one to lack this knowledge, hopefully > discussions will clarify that. > > I also sometime have problems to express myself clearly, thus > explanation are normal. > > > > > - `users` we rely on LWT for throwing "AlreadyExist" exceptions. LWT > >> are likely unnecessary as the webadmin > >> presentation layer is offering an idempotent API (and silents the > >> AlreadyExist exceptions). Only the CLI > > > > (soon to be deprecated for Guice products) makes this distinction. > > > > > > So from a user perspective adding a user would always succeed. But would > it > > succeed by doing nothing (the current behaviour in silencing the > > AlreadyExist exception) or would it succeed by effectively overwriting > the > > user (in a last write wins manner) ? > > Webadmin so far overwrite the user (and its password) in a last write > win manner. > > > This is a completely different > > behaviour which is not necessarily desirable. > > this can be further divided into 2 different cases : > > - there are concurrent attempts to create the same user (in which case > the > > user data is very likely the same or very close, and has possibly never > > been exposed to a human) in which case the LWW behaviour may be > acceptable > > - A user has existed for a long time (definition of long to be defined > but > > I would say above a few seconds :) ) in which cas overwriting is most > > likely not acceptable > > > > My take is that we need to make a choice: > > - Either we need to distinguish "create" from "update" within the > webadmin API > - Or we relax the condition downstream. > > I would be in favor of handling conflict as part of the WebAdmin API. > > Moreover what you say regarding conflict is very interesting. It suggest > strong consistency might not be needed. A simple "read before your > write" would be enough to see if a long standing user would be updated, > while not dealing with conflict for newly created user. > > > > >> - `domains` we rely on LWT for throwing "AlreadyExist" exceptions. LWT > >> are likely unnecessary as the webadmin > >> presentation layer is offering an idempotent API (and silents the > >> AlreadyExist exceptions). Only the CLI > >> (soon to be deprecated for Guice products) makes this distinction. > >> Discussions have started on the topic and a proof of > >> concept is available. > >> > > > > same as above > > > > > No not really, as domains cary no other information (like user carry a > password). > > You don't have the risk to grant access to somebody else account by > mistake. > > > Why it would be ok to drop LWT for ACL updates only to replace it by > > eventsourcing when you write: > >> LWT are required for `eventSourcing`. As event sourcing usage is limited > > to low-usage use cases, the performance degradations are not an issue. > > Doesn't that mean that ACLs would still rely on LWT but within an > > additional layer ? > > Yes. Writes are resolved against event sourcing system using LWT. > > Reads are resolved against a projection, free of LWT, maintaines via > subscribers to the event sourcing system. > > > Also for ACLs isn't eventual consistency acceptable ? > > My take is that it is. My first shot was to do just that. > > Howver the current code enforces a "transaction like" behavior, with an > ACLDiff fired on the mailbox event system. Maintaining denormalization > was also an issue in the face of reset (requiring to know previously > stored data). > > A full rewritte of the context ACLs are run in would have been needed. > And could have been controversial. > > > using transactions to > > avoid non serial writes but accepting stale reads ? > > Could be too. This could be a quick-win to enhance existing situation, > while waiting for stronger decisions and > > Here we might need a distinction between reads made as part of the > update process (that are required to be up to date, so need to be > SERIAL!) and regular reads that are acceptable to be stale (and can be > made using QUORUM). > > What you describe could be a very good short term solution for message > flags (I think we will need to challenge CONDSTORE/QRSYNC anyway!) > > > > > That's the limit of my understanding : all the flags/UID/IMAP concerns > are > > beyond my current knowledge but I'll enjoy reading the comments :) > > > > jean > > > > Cheers, > > Benoit > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
