Re: Malformed "D" message received.
At 2021-04-08 09:26:22 +0200, t...@hamartun.priv.no wrote:
>
> Something changed in NetBSD-current last year, to make it no longer
> safe to assume that communication over a Unix domain stream socket
> would make the entire contents of a write() available to read() on
> the other side in one go.
I don't think the code means to assume that, though:
case Read:
while ( d->active && hasMessage( readBuffer() ) ) {
/* We call a function to process every message we receive.
This function is expected to parse and remove a message
from the readBuffer, throwing an exception for malformed
messages, and setting d->unknownMessage for messages that
it can't or won't handle. */
char msg = (*readBuffer())[0];
try {
if ( d->startup ) {
if ( !d->authenticated )
authentication( msg );
else
backendStartup( msg );
}
else {
process( msg );
}
if ( d->unknownMessage )
unknown( msg );
}
catch ( PgServerMessage::Error e ) {
error( "Malformed " + EString( , 1 ).quoted() +
" message received." );
}
}
…and hasMessage() returns false if we don't have the required number of
bytes available (every message starts with a type byte, followed by a
four-byte length):
static bool hasMessage( Buffer *b )
{
if ( b->size() < 5 ||
b->size() < 1+( (uint)((*b)[1]<<24)|((*b)[2]<<16)|
((*b)[3]<<8)|((*b)[4]) ) )
return false;
return true;
}
So we should clearly be parsing a 'D' message (or any message) only when
all of the bytes in the message are available, no matter how many reads
that takes.
-- Abhijit
Re: Malformed "D" message received.
At 2021-04-08 09:26:22 +0200, t...@hamartun.priv.no wrote: > > If I'm right, perhaps instead of closing the connection to PostgreSQL > when this happens, it should just push back the incomplete data, and > wait for more, with a suitable timeout? Thanks for the analysis. I'll have a look to see how difficult it will be to implement the solution you recommend above. -- Abhijit
Re: Malformed "D" message received.
At 2021-02-06 18:46:53 +0100, t...@hamartun.priv.no wrote: > > > On a hunch, I made the following change, and, so far, it seems to work: > > Spoke too soon. It's still happening. Can you get a tcpdump of the failing session, perhaps? I'd like to see how exactly the message is "malformed". -- Abhijit
Re: Postgres version check
At 2018-05-31 21:58:08 +0200, a...@gulbrandsen.priv.no wrote: > > Patch most welcome. db/postgres.cpp line near line 693. I pushed a couple of patches to address this. -- ams
Re: Failure to build with modern Clang
At 2018-04-05 14:20:21 -0500, f...@feld.me wrote: > > core/md5.cpp:55:5: error: 'register' storage class specifier is deprecated > and incompatible with C++17 [-Werror,-Wdeprecated-register] > register uint32 t; > ^ I think we can do without those 'register' specifiers, so I removed them. (When you compile MD5 code under C++17, it gets replaced with SHA-256 anyway, right? ;-) -- ams
Re: build failure with modern clang
At 2017-02-15 11:20:51 +0100, axel@chaos1.de wrote: > > Any chance to see this patch on github? Hm, odd that it didn't get pushed already. Pushed now. -- Abhijit
Re: build failure with modern clang
At 2017-01-09 12:36:23 -0600, f...@feld.me wrote: > > C++ build/clang/server/tlsthread.o > server/tlsthread.cpp:100:19: error: implicit conversion from 'long' to > 'int' changes value from 2168458239 to -2126509057 > [-Werror,-Wconstant-conversion] > | SSL_OP_NO_SSLv2 > ^ > 1 error generated. That's a bug. SSL_CTX_set_options() is documented to take a long, so I've changed 'options' from int to long. Thanks for the report. -- ams
Re: setannotate/getannotate
At 2014-07-07 14:33:56 +0100, nss...@networksystemssolutions.co.uk wrote: I can gz and attach it if that's OK. Please attach as text, no need to gzip. Also, I don't think there's any problem with posting it to the list (for review). Just mention in the Subject that it's a work in progress. -- ams
Re: [Diverted: post from non-subscriber] Flushing queue with 5.x.y error / status failed
This was the only message I found that was rejected by the list software while it was… indisposed. If anyone else posted in the last couple of days and didn't see the message on-list, please repost. Sorry for the inconvenience. -- ams
Re: Commit 8 january
At 2013-01-10 21:54:00 +0100, j...@rilk.com wrote: I have a problem to compile the version from the 8 january HEAD seems to build correctly. pull again? -- ams
Fwd: Re: web mail for masses - scaling archiveopteryx - active development?
[This was sent off-list and forwarded to me to resend to the list. It wasn't signed, but I assume from context that it's from Usman.] Thank you for your feedback. Arnt, Mark, Martin, Backup is not my concern for not storing attachments in db. IMHO dbs are not meant for storing most importantly serving files for websites that have huge traffic. Concurrency is the main issue, I don't know about postgresql but with mysql, we typically set it to not more than 512 on a beefy machine. HTTPD (specially nginx) is very robust for handling static files. Storage is also a concerns because on average attachment size actual mail size. Duplicate logic can still be applied if aox was to store file paths. I don't know the exact details about the logic it uses, but aox can download a chunk of file for dup-comparision. Ideally this should be done via job queue. Arnt, Regarding your statement: I think your design makes a very great deal of sense. In fact it's about the same as what I now wish Abhijit and I had pursued in 2006. - Is it too late to pursue it now in 2012? :) More questions: #1 This is not ideal not related to aox, but a postgresql question: is it possible to specificity a different directory for attachments table? I am thinking about a remote mounted directly, but like I said, not ideal - just curious. #2 Do you know what would be good identifier for shading: based on users, storage, activity? This question is vague, but if you can tell me how aox performs on some of your servers than I can derive a conclusion. What is your mailbox size like, avg. load on server, avg queries/sec, how much ram etc? #3 If I were to manually enter a user in system, which tables should I be entering the user id (besides users table)? Is it possible to get a list of such related tables (and preferably sql statements too)? #4 Expanding #3, it should be possible, but to confirm - can I add a user with a user_id (PK) of own choice? The reason for this is that I wouldn't like to avoid having duplicate user_id across aox servers. #5 Where can I find out about caching implementations of aox? Any plans to use memcache/redis etc for future? Is the project even active for development? It was very hard to find about aox. There are almost no results on google about it. The only other alternative is dbMail, but I have found aox's architecture to be more efficient documentations to be more helpful. Thanks
Re: git master doesn't seem to build
At 2012-10-28 08:50:44 +, ja...@mansionfamily.plus.com wrote: Abhijit, Is there any update on this? No, but I'm working my way through the changes. I'm afraid it's taking me some time to wrap my head around them. For the moment, I've reverted the commit in master and pushed a new fork-children branch. master should work now, and if I commit any more breakage, it'll be in the new branch. Sorry for the trouble. -- ams
Re: git master doesn't seem to build
(hey Arnt, if you're busy, post some notes and maybe someone else can finish it) I'm looking at it now. I don't think it'll be finished quickly, but I'll try to get HEAD building again soon, at least. After I review the code, I'll post an explanation here and we can discuss it. -- ams
warning: HEAD may be broken for a while
This is just a notice that we're going to land some major changes soon, and that git HEAD may be broken for a while. I'll write again when it's safe to test. -- ams
Re: Patch for md5.cpp compile problem
At 2012-07-31 22:50:43 +0200, por...@froglogic.com wrote: The attached patch is absolutely NOT tested. Is there any test suite for these classes? 334275 aspects of 178098 tests in 80 contexts; 0 failures (Thanks, applied.) -- ams
