Re: qresync changes still not entirely correct?

2016-11-09 Thread Arnt Gulbrandsen

NSS Ltd writes:

Arnt, any idea why the modseq value becomes -1 ?  Hints will same me
time tracking it.


Fixed, I expect.

Aox generally uses a 'nextmodseq', ie. the next value of modseq to be 
assigned, while IMAP uses 'highestmodseq', which is either the highest 
modseq of any existing message or the highest modseq of any message ever, 
depending on context. (Aox uses 'next' because that context-dependent 
variation overstrains certain people's brains.)


If a variable contains a nextmodseq and is used to call something that 
takes a highestmodseq, subtracting 1 is necessary, and I incorrectly 
thought it was necessary in one case, which ended up being 0-1 quite often.


Arnt



Re: qresync changes still not entirely correct?

2016-09-19 Thread Arnt Gulbrandsen

NSS Ltd writes:

Next bit is to see why modseq is -1 at all, but if you can try with this
patch, I think you may end up with a more benign situation (but possible
big table updates with modseq>-1).  It is possible -1 is valid but Arnt
is the one to confirm there.  It's also possible something to do with
your mail client/setup.


The -1 isn't valid, using 0 is valid.

I guess there's a decrement too many... I tried for a bit of bug 
compatibility (sending slightly too many updates in order to counter 
clients that ask for slightly too few) and triggered a bug of my own.


I agree with the fromNumber() change, but I also want to fix it highter up 
in the call stack, and make sure modseq never goes negative.


Thank you SO MUCH. This made my day.

Arnt



Re: qresync changes still not entirely correct?

2016-09-19 Thread NSS Ltd
Hi Arnt,

Do you want me to sort this into a patch for the git repo?  Technically
it corrects the handling for -ve int64's but isn't related to whether
limit in fetch.cpp should be -1 or not.  My instinct is that it's
possible -1 is correct here but you're probably in a better position to
know.  If the -1 value is a problem, that's really a separate issue, so
I think the patch would still be good.

TIA

Jim



On 19/09/2016 18:53, Tom Ivar Helbekkmo wrote:
> NSS Ltd  writes:
>
>> A patch to handle negative numbers in estring.cpp
> That seems to have very nicely fixed it.  Everything is behaving well,
>



Re: qresync changes still not entirely correct?

2016-09-19 Thread Tom Ivar Helbekkmo
NSS Ltd  writes:

> A patch to handle negative numbers in estring.cpp

That seems to have very nicely fixed it.  Everything is behaving well,
with the only change seeming to be that the occurrences of -1 are being
passed on as -1, and PostgreSQL no longer complains.

-tih
-- 
Elections cannot be allowed to change anything.  --Dr. Wolfgang Schäuble



Re: qresync changes still not entirely correct?

2016-09-19 Thread NSS Ltd
Right, indeed negative int64 numbers are not properly handled.  Also
looks like this happened in the past with int which was patched but not
trickled into int64 handling.  As I look at it, the fromNumber stuff
looks like it may help from a refactor but I'll not go there just now.

A patch to handle negative numbers in estring.cpp (be careful of
formatting changes in email):


--- estring-orig2016-09-16 15:30:53.719083773 -0400
+++ estring-patched2016-09-19 13:06:48.826113327 -0400
@@ -947,7 +947,15 @@
 EString EString::fromNumber( int64 n, uint base )
 {
 EString r;
-r.appendNumber( n, base );
+if (0 == n) {
+r.append("0");// Short-circuit, 0 is 0 in any base, no
need for extra processing
+} else {
+if (n < 0) {
+n = -n;// Negate, otherwise negative numbers get
messed up
+r.append("-");// But we remember we need a - sign
+}
+r.appendNumber( n, base );
+}
 return r;
 }

Next bit is to see why modseq is -1 at all, but if you can try with this
patch, I think you may end up with a more benign situation (but possible
big table updates with modseq>-1).  It is possible -1 is valid but Arnt
is the one to confirm there.  It's also possible something to do with
your mail client/setup.

Jim




On 19/09/2016 17:46, NSS Ltd wrote:
> OK, I can now reproduce the output 'V' from fn.  -1 throws it off.  I'll
> dig around (the estring entries were supposed to track the stack on it,
> but I have a test harness I can use from here).
>
> I'll report back.
>
> Arnt, any idea why the modseq value becomes -1 ?  Hints will same me
> time tracking it.
>
> Thanks
> Jim
>
>



Re: qresync changes still not entirely correct?

2016-09-19 Thread NSS Ltd

OK, I can now reproduce the output 'V' from fn.  -1 throws it off.  I'll
dig around (the estring entries were supposed to track the stack on it,
but I have a test harness I can use from here).

I'll report back.

Arnt, any idea why the modseq value becomes -1 ?  Hints will same me
time tracking it.

Thanks
Jim




On 19/09/2016 14:32, Tom Ivar Helbekkmo wrote:
> Tom Ivar Helbekkmo  writes:
>
> Turns out (as Jim rightly guessed) that the extreme amount of logging
> was bogging everything down.  With just the part of the patch in place
> that modified fetch.cpp to log, I see output like this while Gnus is
> starting up, and scanning mailboxes:
>
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 99507
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99507 ) responds with 99507
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 99507
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99507 ) responds with 99507
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 188057
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 188057 ) responds with 188057
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 188057
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 188057 ) responds with 188057
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 168534
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 168534 ) responds with 168534
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 168534
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 168534 ) responds with 168534
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is -1
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( -1 ) responds with V
> Sep 19 13:21:58 barsoom Archiveopteryx: 23033/2/1/489/1 PostgreSQL Server: 
> invalid input syntax for integer: "V"
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 99112
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99112 ) responds with 99112
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 99112
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99112 ) responds with 99112
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is -1
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( -1 ) responds with V
> Sep 19 13:21:58 barsoom Archiveopteryx: 23033/2/1/491/1 PostgreSQL Server: 
> invalid input syntax for integer: "V"
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is -1
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( -1 ) responds with V
> Sep 19 13:21:58 barsoom Archiveopteryx: 23033/2/1/492/1 PostgreSQL Server: 
> invalid input syntax for integer: "V"
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 99139
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99139 ) responds with 99139
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 99139
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99139 ) responds with 99139
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 99320
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99320 ) responds with 99320
> Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 99320
> Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99320 ) responds with 99320
>
> Likewise, while receiving email from outside:
>
> Sep 19 15:27:08 barsoom postfix/qmgr[1695]: EA4AB1C7133: 
> from=, size=227461, nrcpt=1 (queue active)
> Sep 19 13:27:08 barsoom Archiveopteryx: 23033/4/1 Connected: SMTP server 
> 127.0.0.1:2026 connected to client 127.0.0.1:55718, on fd 10 (7 connections)
> Sep 19 13:27:09 barsoom Archiveopteryx: 23033/4/1/5/1 Injecting message 
> <264013603.288.1474291058616@nett-ewapp> into /users/tih/INBOX
> Sep 19 15:27:09 barsoom postfix/lmtp[1872]: EA4AB1C7133: 
> to=, relay=127.0.0.1[127.0.0.1]:2026, delay=541, 
> delays=540/0.03/0.02/0.55, dsn=2.1.5, status=sent (250 2.1.5 
> t...@hamartun.priv.no: OK)
> Sep 19 15:27:09 barsoom postfix/qmgr[1695]: EA4AB1C7133: removed
> Sep 19 13:27:11 barsoom Archiveopteryx: modseq limit appended is -1
> Sep 19 13:27:11 barsoom Archiveopteryx: fn( -1 ) responds with V
> Sep 19 13:27:11 barsoom Archiveopteryx: 23033/2/2/6/2 PostgreSQL Server: 
> invalid input syntax for integer: "V"
>
> -tih



Re: qresync changes still not entirely correct?

2016-09-19 Thread Tom Ivar Helbekkmo
Tom Ivar Helbekkmo  writes:

> NSS Ltd  writes:
>
>> OK, rather than implementing a roman numeral interface, are you able to
>> try patching with these 2 diffs :
>
> Yeah, but I didn't get anywhere with it (even after cleaning up your
> patch, the formatting of which was screwed up in transmission), because
> I was unable to connect to the IMAP server from Gnus afterwards.
> Nothing was logged, everything just hung.  Didn't take the time to debug
> this; just restarted my aox without changes.  Some species of deadlock?

Turns out (as Jim rightly guessed) that the extreme amount of logging
was bogging everything down.  With just the part of the patch in place
that modified fetch.cpp to log, I see output like this while Gnus is
starting up, and scanning mailboxes:

Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 99507
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99507 ) responds with 99507
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 99507
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99507 ) responds with 99507
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 188057
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 188057 ) responds with 188057
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 188057
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 188057 ) responds with 188057
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 168534
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 168534 ) responds with 168534
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 168534
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 168534 ) responds with 168534
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is -1
Sep 19 13:21:58 barsoom Archiveopteryx: fn( -1 ) responds with V
Sep 19 13:21:58 barsoom Archiveopteryx: 23033/2/1/489/1 PostgreSQL Server: 
invalid input syntax for integer: "V"
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 99112
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99112 ) responds with 99112
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 99112
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99112 ) responds with 99112
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is -1
Sep 19 13:21:58 barsoom Archiveopteryx: fn( -1 ) responds with V
Sep 19 13:21:58 barsoom Archiveopteryx: 23033/2/1/491/1 PostgreSQL Server: 
invalid input syntax for integer: "V"
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is -1
Sep 19 13:21:58 barsoom Archiveopteryx: fn( -1 ) responds with V
Sep 19 13:21:58 barsoom Archiveopteryx: 23033/2/1/492/1 PostgreSQL Server: 
invalid input syntax for integer: "V"
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 99139
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99139 ) responds with 99139
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 99139
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99139 ) responds with 99139
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit appended is 99320
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99320 ) responds with 99320
Sep 19 13:21:58 barsoom Archiveopteryx: modseq limit inserted is 99320
Sep 19 13:21:58 barsoom Archiveopteryx: fn( 99320 ) responds with 99320

Likewise, while receiving email from outside:

Sep 19 15:27:08 barsoom postfix/qmgr[1695]: EA4AB1C7133: 
from=, size=227461, nrcpt=1 (queue active)
Sep 19 13:27:08 barsoom Archiveopteryx: 23033/4/1 Connected: SMTP server 
127.0.0.1:2026 connected to client 127.0.0.1:55718, on fd 10 (7 connections)
Sep 19 13:27:09 barsoom Archiveopteryx: 23033/4/1/5/1 Injecting message 
<264013603.288.1474291058616@nett-ewapp> into /users/tih/INBOX
Sep 19 15:27:09 barsoom postfix/lmtp[1872]: EA4AB1C7133: 
to=, relay=127.0.0.1[127.0.0.1]:2026, delay=541, 
delays=540/0.03/0.02/0.55, dsn=2.1.5, status=sent (250 2.1.5 
t...@hamartun.priv.no: OK)
Sep 19 15:27:09 barsoom postfix/qmgr[1695]: EA4AB1C7133: removed
Sep 19 13:27:11 barsoom Archiveopteryx: modseq limit appended is -1
Sep 19 13:27:11 barsoom Archiveopteryx: fn( -1 ) responds with V
Sep 19 13:27:11 barsoom Archiveopteryx: 23033/2/2/6/2 PostgreSQL Server: 
invalid input syntax for integer: "V"

-tih
-- 
Elections cannot be allowed to change anything.  --Dr. Wolfgang Schäuble



Re: qresync changes still not entirely correct?

2016-09-18 Thread Tom Ivar Helbekkmo
NSS Ltd  writes:

> OK, rather than implementing a roman numeral interface, are you able to
> try patching with these 2 diffs :

Yeah, but I didn't get anywhere with it (even after cleaning up your
patch, the formatting of which was screwed up in transmission), because
I was unable to connect to the IMAP server from Gnus afterwards.
Nothing was logged, everything just hung.  Didn't take the time to debug
this; just restarted my aox without changes.  Some species of deadlock?

-tih
-- 
Elections cannot be allowed to change anything.  --Dr. Wolfgang Schäuble



Re: qresync changes still not entirely correct?

2016-09-18 Thread NSS Ltd

OK, rather than implementing a roman numeral interface, are you able to
try patching with these 2 diffs :

Syslog some detail from estring :

--- estring-orig2016-09-16 15:30:53.719083773 -0400
+++ estring.cpp2016-09-16 15:31:22.560537709 -0400
@@ -8,7 +8,7 @@
 #include 
 // strlen
 #include 
-
+#include 
 
 /*! \class EStringData estring.h
 
@@ -437,6 +437,7 @@
 
 void EString::appendNumber( int n, int base )
 {
+syslog(LOG_USER | LOG_DEBUG, "In void EString::appendNumber( int n
(%d), int base (%d) )\n", n, base);
 if ( n < 0 ) {
 append( '-' );
 n = -n;
@@ -446,6 +447,7 @@
 
 void EString::appendNumber( uint n, int base )
 {
+syslog(LOG_USER | LOG_DEBUG, "In void EString::appendNumber( uint n
(%u), int base (%d) )\n", n, base);
 
 EString s( fromNumber( n, base ) );
 
@@ -946,6 +948,7 @@
 
 EString EString::fromNumber( int64 n, uint base )
 {
+syslog(LOG_USER | LOG_DEBUG, "In void EString::fromNumber( int64 n
(%lld), uint base (%u) )\n", n, base);
 EString r;
 r.appendNumber( n, base );
 return r;
@@ -961,6 +964,7 @@
 
 void EString::appendNumber( int64 n, uint base )
 {
+syslog(LOG_USER | LOG_DEBUG, "In void EString::appendNumber( int64 n
(%lld), uint base (%u) )\n", n, base);
 int64 top = 1;
 while ( top * base <= n )
 top = base * top;


And then to add some detail in the imap process :


--- fetch-orig2016-09-16 15:35:39.369493459 -0400
+++ fetch.cpp2016-09-18 13:30:01.502632438 -0400
@@ -31,7 +31,7 @@
 #include "map.h"
 #include "utf.h"
 
-
+#include 
 
 static const char * legalAnnotationAttributes[] = {
 "value",
@@ -186,6 +186,9 @@
  ( c->state() == Command::Blocked ||
c->state() == Command::Finished ||
c->state() == Command::Executing ) ) {
+syslog(LOG_USER | LOG_DEBUG, "modseq limit inserted is %lld", limit);
+EString theString = fn(limit);
+syslog(LOG_USER | LOG_DEBUG, "fn( %lld ) responds with %s", limit,
theString.cstr());
 log( EString("Inserting flag update for modseq>") + fn( limit ) +
  " and UIDs " + set.set() + " before " +
  c->tag() + " " + c->name() );
@@ -194,6 +197,9 @@
 setGroup( c->group() );
 }
 else {
+syslog(LOG_USER | LOG_DEBUG, "modseq limit appended is %lld", limit);
+EString theString = fn(limit);
+syslog(LOG_USER | LOG_DEBUG, "fn( %lld ) responds with %s", limit,
theString.cstr());
 log( "Appending flag update for modseq>" + fn( limit ) +
  " and UIDs " + set.set() );
 i->commands()->append( this );

This will throw a fair bit of extra syslog entries, especially on a busy
server; if you can do this on something handling only your problematic
connection, that will be best as there will be more signal to noise. 
Once you capture the syslog data, I recommend putting the code back the
way it was for now.

Send the syslog lines from this patch (both good and bad results); I'm
curious what values are behind the issue (sometimes things pop out).  It
will at least give something to try and reproduce with.


Thanks

Jim





On 16/09/2016 16:56, Tom Ivar Helbekkmo wrote:
> No, it seems to be exactly when my MUA asks aox for messages in a
> mailbox newer than a specified message, that aox erroneously formats
> that message number as 'V' in its query to pgsql.  Now, if PostgreSQL
> would accept Roman numerals, I guess I wouldn't see an error message...
>



Re: qresync changes still not entirely correct?

2016-09-16 Thread Tom Ivar Helbekkmo
NSS Ltd  writes:

> When you say it happens all the time, do you mean every call has messed
> up (non numeric) results or is this just on particular uses?  (e.g.
> getting a particular message rather than total failure)

No, it seems to be exactly when my MUA asks aox for messages in a
mailbox newer than a specified message, that aox erroneously formats
that message number as 'V' in its query to pgsql.  Now, if PostgreSQL
would accept Roman numerals, I guess I wouldn't see an error message...

Everything works, so I suspect aox is falling back to reading each whole
mailbox every time I ask it to check for new mail.

-tih
-- 
Elections cannot be allowed to change anything.  --Dr. Wolfgang Schäuble



Re: qresync changes still not entirely correct?

2016-09-15 Thread NSS Ltd
Tom,

When you say it happens all the time, do you mean every call has messed
up (non numeric) results or is this just on particular uses?  (e.g.
getting a particular message rather than total failure)

Depending on which situation, I can give you some test harness code to
try and track this down.  It's not reproducible for me (I'm on Ubuntu
rather than netbsd) but if it is for you, this may be a way forward.

Jim


On 12/09/2016 21:42, Arnt Gulbrandsen wrote:
> Tom Ivar Helbekkmo writes:
>> Actually, I guess that's pretty good -- it indicates that this isn't an
>> optimization problem after all, right?
>
> I also have a debian box where it doesn't occur and one where it
> occurs depending on compiler switches. Now I know it happens on netbsd
> always. So it might depend on the gcc vagaries and object alignment...
> something.
>
> The code pattern where it happens is one of the most common in all of
> the aox source. Variants of
>
>  Estring d = "a" + fn(42) + "c";
>
> are used hundreds of times.
>
> Arnt
>



Re: qresync changes still not entirely correct?

2016-09-15 Thread Arnt Gulbrandsen
Yes, Dovecot is really fast so long as you do what it expects. You can 
construct IMAP requests it handles slowly, but most (all?) of the common 
requests are ones it's been optimized for and will handle really fast.


Arnt



Re: qresync changes still not entirely correct?

2016-09-12 Thread Arnt Gulbrandsen

Tom Ivar Helbekkmo writes:

Actually, I guess that's pretty good -- it indicates that this isn't an
optimization problem after all, right?


I also have a debian box where it doesn't occur and one where it occurs 
depending on compiler switches. Now I know it happens on netbsd always. So 
it might depend on the gcc vagaries and object alignment... something.


The code pattern where it happens is one of the most common in all of the 
aox source. Variants of


 Estring d = "a" + fn(42) + "c";

are used hundreds of times.

Arnt



Re: qresync changes still not entirely correct?

2016-09-12 Thread Tom Ivar Helbekkmo
Arnt Gulbrandsen  writes:

> If you can look at which of the files make a difference, that would be
> great.

Not as great as you think, I'm afraid.  By default, everything is
compiled -O3, so I tried building just the ones you mentioned -O0, which
didn't help.  Then I built the whole package -O0, and it still makes the
same mistake.

Actually, I guess that's pretty good -- it indicates that this isn't an
optimization problem after all, right?

Oh, and this is with GCC 5.4.0 on NetBSD/amd64-current.

-tih
-- 
Elections cannot be allowed to change anything.  --Dr. Wolfgang Schäuble



Re: qresync changes still not entirely correct?

2016-09-12 Thread Arnt Gulbrandsen

Tom Ivar Helbekkmo writes:

Ouch!  Those bugs are a pain.  I just tried compiling estring.cpp (and,
for good measure, estringlist.cpp) with -O0, but that didn't make any
difference...?


I don't remember whether I changed from -O0 to -O3 or the other way (I've 
been AFK for a month, and what I did in July has receded a little in my 
mind). postgres.cpp and allocator.cpp might also make a difference.


If you can look at which of the files make a difference, that would be 
great.


Arnt



Re: qresync changes still not entirely correct?

2016-09-12 Thread Tom Ivar Helbekkmo
Arnt Gulbrandsen  writes:

> The problem is somewhere inside EString::fromNumber(), not in qresync,
> and changing the optmisation flags makes the problem go away. I'm not
> sure how to attack that.

Ouch!  Those bugs are a pain.  I just tried compiling estring.cpp (and,
for good measure, estringlist.cpp) with -O0, but that didn't make any
difference...?

-tih
-- 
Elections cannot be allowed to change anything.  --Dr. Wolfgang Schäuble



Re: qresync changes still not entirely correct?

2016-09-12 Thread Arnt Gulbrandsen
The problem is somewhere inside EString::fromNumber(), not in qresync, and 
changing the optmisation flags makes the problem go away. I'm not sure how 
to attack that.


Arnt



Re: qresync changes still not entirely correct?

2016-09-11 Thread Info Akra
Hi,

I’ve been seeing the same ‘V' error recently, with:
- Archiveopteryx version 3.2.0
- postgresql 9.3
- Ubuntu 14.04.3 LTS

I’m in the process of migrating ~175 Gb of emails (~20 accounts) from a dovecot 
system to sox using imapsync, with many more issues than I had expected.  The 
imapsync program has complained often of losing connection to sox, or have 
dropped connections after waiting 120s for a reply.
Also when doing an account-to-account content comparison with a python script, 
I’ve noticed that the dovecot server (v2.2.9) is approximately 2x faster at 
responding to a 
"FETCH BODY[HEADERS]" than Archiveopteryx, all that with both software running 
on same machine/hard disks.

  Regards,
Hugo

> On 12 Sep, 2016, at 13:08, Tom Ivar Helbekkmo  wrote:
> 
> Yesterday, I noticed some strange error messages in my PostgreSQL and
> Archiveopteryx logs.  I don't know how long it's been like this, but I
> upgraded both to fresh versions (aox to HEAD of git "master" branch,
> pgsql to 9.5.3), with no change in behavior.
> 
> I use Gnus to read my email, and the problem occurs once per mailbox
> whenever I ask Gnus to refresh its view of them.  Here's what it looks
> like with verbose logging:
> 
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/287/1 Updating 1 (of 0) 
> session(s) on /users/tih/uninett for modseq [1,99115>, UID [1,198>
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1 Starting session on mailbox 
> /users/tih/uninett
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/287/1 Dequeueing query 
> "select first_recent from mailboxes where id=$1" ($1='148') on backend 1 
> (with 1 rows)
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/287/1 Dequeueing query 
> "select mm.uid, mm.modseq from mailbox_messages mm where mm.mailbox=$1 and 
> mm.uid<$2" ($1='148',$2='198') on backend 3 (with 197 rows)
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/497/1 Appending flag update 
> for modseq>V and UIDs
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/287 Dequeueing query 
> "begin" on backend 3
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/497/1 Dequeueing query 
> "begin" on backend 2
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/287 Dequeueing query 
> "select uid from mailbox_messages mm where mailbox=$1 and not seen order by 
> uid limit 1" ($1='148') on backend 3 (with 0 rows)
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/497/1 PostgreSQL Server: 
> invalid input syntax for integer: "V"
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/497/1 Execution time 5ms
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/287 Dequeueing query 
> "commit" on backend 3
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/287 Execution time 11ms
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/287 91019 OK [READ-ONLY] 
> done
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/288 Fetch <=197 messages:  
> flags
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/497/1 Dequeueing query 
> "commit" on backend 2
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/288 Dequeueing query 
> "select f.uid, fn.name from flags f join flag_names fn on (f.flag=fn.id) 
> where f.mailbox=$1 and f.uid=any($2)" ($1='148',$2='{1,2,3,4,5,6'..\
> . (681b)) on backend 3 (with 13 rows)
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/288 Dequeueing query 
> "select uid, seen, deleted from mailbox_messages where mailbox=$1 and 
> uid=any($2)" ($1='148',$2='{1,2,3,4,5,6'... (681b)) on backend 1 (with\
> 197 rows)
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/288 Execution time 10ms
> Sep 12 04:45:33 barsoom Archiveopteryx: 22609/2/1/288 91020 OK done
> 
> So, where does that V come from?  :)
> 
> -tih
> -- 
> Elections cannot be allowed to change anything.  --Dr. Wolfgang Schäuble
>