On 1/23/07, Stefano Bagnara <[EMAIL PROTECTED]> wrote:
robert burrell donkin wrote:
> commands that rely on or change message ID must be serialized and
> executed in the order that the client has sent them.
Ok now I understand the issue. I started from the, now I understand
wrong, assumption that IMAP commands have to be executed synchronously
and sequentially as they are received.
Do you think that we need more that one processing thread for a single
client?
to give me anywhere need adequate performance for small mailbox i'm
currently running with JAMES, i need need at least 20 threads peak
usage.
Can you make a common case example where the need of multiple concurrent
processors is needed for a single client? (if I have a concrete case I
remember better the issue in later discussions)
good timing (i was about the add an example above :-)
when evolution connects to an IMAP server, it sends commands to
discover the status of every mailbox. these need to execute in
parrallel. if you interested, i'll collate some statistics but IIRC i
see around 100 commands most of which could be executed concurrently.
>> Does it mean we have to take care to correctly coordinate/sort commands
>> for different protocols?
>
> i think so
>
> for example, the SMPT server may add a new mail into the mailbox. IMHO
> this should not be executed concurrently with IMAP commands that rely
> on or change message ID.
If messageids are in-memory in the session it would work.
but which session ;-)
it may be tricky to ensure the right semantics if updates of in-memory
session data were event driven. i think it would be safer and easier
to use scheduling to ensure that commands relying on or changing
message id were executed in serial.
What about multiple IMAP clients accessing the same imap folder? Are
message identifiers different or equal on the two clients?
unsure: would need to read the specification in detail
IIRC they are allowed to be different on different clients
in terms of sharing, the problem is whether deleted (but not expunged)
mails can be seen by other clients. maybe someone who knows the
protocol better could jump in here...
In pop3 we have to keep an in-session representation of the current
mailbox because the changes are only applied when you quit, so anyone
else trying to work on the mailbox should see the messages until the
client sends the quit.
What about imap?
>> Do you think it is better to manage sorting of commands instead of
>> simply using locks? Aren't locks enough for an IMAP implementation?
>
> it can be done with locks but locks are inefficient
>
> there are a number of IMAP commands that the specification implies are
> fine to be run concurrently with an SMPT add operation. these should
> not be blocked.
Is this concurrency allowed only for IMAP vs SMTP (other mailbox
spooling) operations or also for multiple IMAP commands from the same
client? What about IMAP commands from multiple IMAP clients: do we need
to synchronize them in any way?
IMO the concurrent demands of all the different protocols from all
different clients all need to be taken into account and managed by a
single processor per mailbox. i think that a scheduler would be the
easiest way to do this.
In pop3 we only should care that each mailbox is opened by a single client,
nothing else.
ok
> of course, this could be done by locking and i think that blocking
> mailboxes will be necessary but IMHO scheduling offers better
> performance, more flexibility and a cleaner architecture.
Do you already have some ideas about the scheduling?
just stuff i've been thinking about as this thread has gone on
SEDA is a good match but i'll draw up some UML
>> before IMAP we simply had MailRepository and MailRepository manage Locks
>> over Mail objects. MailRepository provide easy services and is
>> synchronized, everyone (pop3/smtp/spoolmanager/remotemanager) currently
>> uses the same repository interface.
>
> i've been collecting usage statistics on my local IMAP server and IMHO
> this kind of locking will ensure that james will never have reasonable
> IMAP performance.
>
> - robert
Do you think that a step-by-step refactoring is possible or do we need a
full rewrite to increase performances?
if we move towards messenging APIs then it should definitely possible
to go step-by-step
if the scheduler is factored out then a scheduler that executes using
the IO thread should give the current behaviour. so, getting the API
right would be the first step.
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
