Stephen,
Your description of what you (CF stuff) are doing is really not what I'm
looking for (where the loop starts and ends and POP3 operations within), but
lets continue for now.
Well, I think you have made an incorrect assumption in having the loop
structure you have now! During the connection, IMail will not renumber the
messages (but deleting UID while connection open, may have that and other
effects!). Also, that 'no such message' means that IMail did get the DELE
command and has marked that message for deletion (see below for more).
If your program does 'RETR 1, DELE 1', then increment both (or do all the
RETR #, then all the DELE #), you will be able to get and delete each
message in turn (just like a real POP3 client would do!), and only when you
disconnect (QUIT is the _only_ valid command), will IMail actually
(according to RFCs) perform all the DELE operations and remove all the
deleted email and renumber the remaining. In that one session, the number of
each message cannot change (again, by RFC). If a new message arrived during
the time of the session, STAT should show a bigger number than upon first
connection and you could get them (RETR X+1, etc...)
I don't know exactly how long this housekeeping takes (probably dependant on
size of mailbox and time to complete disk operations on the files, then
remove locks), but if we assume reasonable message size (a few to 10 K) and
some modest number of messages (a few hundred), I bet it is normally just
seconds (or less). Of course, system resources available, and larger
mailboxes, can also have some effects. Mailboxes of many megabytes may take
much longer to complete and will place bigger loads on the system and HD.
DELE time is sort of dependant upon the connection time, too. As long as
that connection is still valid, the DELE operation is really just a 'mark
for deletion' and will not be performed (a RETR # won't return the message,
just the error) until the connection is correctly terminated. I say this
again because you seem to think DELE should be done when issued, but that is
just not the case (see the POP3 RFC!). Seen from your perspective, yes it
would be 'slow' because it does not happen when you think it does. Issue a
RSET command before connection termination and you will now see that all
DELE messages can now be read by RETR #!
Glad to see you got a chuckle out of my earlier suggestion. And what you
report seeing is a results of your looping and multiple connections (so I
get one, too!).
If I assume your program is the only one that will have POP3 access to the
users account (why else would you be putting email into DB), you should try
the loop method above (or below). Of course, you will have to check that the
connections was really closed (verify the 'Bye' message was received) at the
server and then you can wait a short time (seconds) and reconnect (if you
think you need to). If at that time STAT shows zero messages, then you got
them all. I even bet you could do this many times faster than your current
method and not lose a message, nor worry about any renumbering (which now
has to happen after each connection close [which may or may not be
happening, I don't know, but expect it is not, that is why you get the 'no
such message' error in the subsequent connection] using my methods!).
I have to agree with you about deleting the UID file. I would never
recommend it except under very specific conditions and only when ALL
connections to that mailbox have been closed. You may be forcing a
renumbering by deleting the uid (and then establishing another connection)
and that could be contributing to your difficulties (or even affecting when
and what IMail is supposed to do!). If the connection is still open and a
renumber occurs, then you cannot continue with X+1 message, but have to go
back to 1. Sorry, I'm not exactly sure what would happen given these
circumstances, but I would avoid this. Better to abort the connection (that
will leave the maibox as it was before the connection), establish a new one
and start from the beginning.
So how about keeping it simple, Connect (once), issue STAT (to find current
number of message for a variable, X), RETR 1 to X, DELE 1 to X, STAT
(variable Y, to see if same number, if not connect again later or RETR X+1
to Y, DELE X+1 to Y) then QUIT (verifying 'Bye' message returned, indicating
a completed session). This should get all messages, mark them for deletion,
and upon close, perform the DELE operations. Any future connection will show
only new or nonDELEted messages. This would more closely duplicate the
typical order of operations of a standard POP3 client that is set to
'download message' (not leave copy on server).
Heck, if the only reason you think you have problems is the error message,
do all the RETR # first (so you can be sure you got them all) and only at
the very end, do the DELE #. Now if you try RETR # and get the error msg,
you can be sure IMail has marked each for deletion. Then close connection
(again check 'Bye' message') and you and the mailbox, should be in perfect
shape.
Sorry if this got long, but it was hard to make shorter and still say things
in a coherent (I hope!) way. Also, I can be a bit long winded, but hopefully
you now have the correct understanding of POP3 operations.
Whewww.....
Daniel Donnelly
________________________________________________________
----- Original Message -----
From: <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Thursday, November 30, 2000 2:53 PM
Subject: Re: [IMail Forum] missing "message #1"!
>
> Daniel Donnelly's response is pretty close on to what's happening, and I
> have a workaround (very poor workaround, but one none the less) to assist
> for the time being. Let me see if I can go farther:
>
> >Without seeing exactly what POP3/IMAP4 commands you
> >(or your 'looped query') are issuing, I cannot say exactly what
> >you are doing and if that is 'correct'.
>
> Briefly: running Cold Fusion middleware, standard CFLOOP with proper
> transaction code, error catching, and query locking. If I trust the code -
> having gone over it a LOT does not mean it's trustworthy :) - It should be
> completing all actions on it's end, waiting for completion before running
> the next iteration of the loop.
>
> Running a third-party (non-Cold Fusion) java-mail code to replace CF's
CFPOP
> command (highly flaky and processor intensive).
>
> >However, if one disconnects incorrectly (just aborting the connection)
> >IMail will see that as a bad POP3 session and will _not_ perform the
'DELE
> >1' operation,
>
> This is what I assume happens, but with the following observation: the
"will
> _not_ perform" occurs after several interactions of the loop (which I've
had
> to break down to run 35 every five minutes to slow down the corruption) -
> say iteration 8 through to 35 of the thirty-five. What this leads me to
> believe that there is - for lack of knowing how to phrase this - something
> slow in the DELE function on the Imail side - or something takes a long
time
> to close - and it allows the loop to issue a new RETR command, creating
for
> Imail and improper disconnect (or something similar).
>
> Running one query a minute over the mail (I have the folder backed up and
> have been running the queries against the EXACT SAME emails) - will cause
> none of the "Missing Message #1" - problems to occur.
>
> There just seems (and again, I can easily be wrong) to be something that
> isn't moving quickly on the Imail side to keep the process in time to the
> requests. Is there a timing issue of some sort that we are not aware of?
>
> So, yup - I'm connecting with another POP3 session when the code I'm
running
> has completed the transaction, and gone back again for more! ("Please sir,
> can I have some more?").
>
> If I have to give Imail time to complete the process - exactly how much
time
> is that (milliseconds, seconds, etc.)? Does anyone know?
>
> >So, I suspect you are not disconnecting correctly or
> >connecting multiple times to the mailbox,
> >with your 'looped query' and 'causing' the problem.
>
> Connecting multiple times. But I loop because:
> I'm pulling emails into a database. I want to make sure I grab each and
> every email - so I go grab one at a time. I don't want a case where I
delete
> message ID #1, which shifts all other message ID's down one, so by
deleting
> Message ID #2 I'm actually deleting Message ID #3, and Message #2 has
become
> #1 - and has been missed.
>
> >So the mailbox is not really corrupt,
> Well, I called it corrupt because by deleting them main.uid file - the
> system is forced to re-index and recognize the deletion. Now, with the
scale
> of the project - this becomes very cumbersome to code into the system.
> Managing one email account works, but hundreds or thousands? Right now -
> based on the information I get from GetAll, message one - if it's corrupt
I
> catch the error, delete the .uid and continue on in the loop. That does
> correct the situation.
>
> But cumbersome, messy, and - well - a performance issue that would be best
> addressed at the Imail level, not through middleware.
>
>
> >Also, if you used a 'DELE 2' command, instead of 'DELE 1', you probably
> >would find that you never have problems with message 1.
>
> Ha ha ha ha hah aha. ROTFL. I HAVE done that. And all I get is "no such
> message" error created after a while for message 2. Indeed, I tried
looping
> that on error I would ignore all messages up to that error. So, slowly I
> grew from no errors, to message 1, to message 1 &2, to, well, I went up to
> message 1 though 4 before I pulled the plug.
>
> Mr. Donnelly, thank you for your response - I think it's highly accurate
as
> to what and why things are happening - I just don't know of a good
solution
> (as in one I would trust under enterprise level abuse). I don't believe
> catching the error and deleting the .uid to be a good solution. There has
to
> be a solution closer to the source, and that means something with the
Imail
> engine. (I think!).
>
> Stephen R. Cassady
> President, Ububik new media
> [EMAIL PROTECTED]
> http://www.ububik.com
> 403.271.0468
>
>
>
>
> Please visit http://www.ipswitch.com/support/mailing-lists.html
> to be removed from this list.
>
> An Archive of this list is available at:
> http://www.mail-archive.com/imail_forum%40list.ipswitch.com/
>
Please visit http://www.ipswitch.com/support/mailing-lists.html
to be removed from this list.
An Archive of this list is available at:
http://www.mail-archive.com/imail_forum%40list.ipswitch.com/
