On Fri, Feb 19, 2010 at 7:54 PM, Heikki Linnakangas
wrote:
> Heikki Linnakangas wrote:
>> Magnus Hagander wrote:
>>> Well, it's going to make the process that reads the WAL cause actual
>>> physical I/O... That'll take a chunk out of your total available I/O,
>>> which is likely to push you to the
Heikki Linnakangas wrote:
> Magnus Hagander wrote:
>> Well, it's going to make the process that reads the WAL cause actual
>> physical I/O... That'll take a chunk out of your total available I/O,
>> which is likely to push you to the limit of your I/O capacity much
>> quicker.
>
> Right, doesn't s
Magnus Hagander wrote:
> 2010/2/18 Fujii Masao :
>> On Thu, Feb 18, 2010 at 7:04 PM, Heikki Linnakangas
>> wrote:
>>> Magnus Hagander wrote:
O_DIRECT helps us when we're not going to read the file again, because
we don't waste cache on it. If we are, which is the case here, it
shoul
2010/2/18 Fujii Masao :
> On Thu, Feb 18, 2010 at 7:04 PM, Heikki Linnakangas
> wrote:
>> Magnus Hagander wrote:
>>> O_DIRECT helps us when we're not going to read the file again, because
>>> we don't waste cache on it. If we are, which is the case here, it
>>> should be really bad for performance
On Thu, Feb 18, 2010 at 7:04 PM, Heikki Linnakangas
wrote:
> Magnus Hagander wrote:
>> O_DIRECT helps us when we're not going to read the file again, because
>> we don't waste cache on it. If we are, which is the case here, it
>> should be really bad for performance, since we actually have to do a
Magnus Hagander wrote:
> O_DIRECT helps us when we're not going to read the file again, because
> we don't waste cache on it. If we are, which is the case here, it
> should be really bad for performance, since we actually have to do a
> physical read.
>
> Incidentally, that should also apply to ge
2010/2/18 Heikki Linnakangas :
> Fujii Masao wrote:
>> On Thu, Feb 18, 2010 at 5:28 AM, Heikki Linnakangas
>> wrote:
>>> If I'm reading the patch correctly, when wal_sync_method is 'open_sync',
>>> walreceiver nevertheless opens the WAL file without the O_DIRECT flag.
>>> When it later flushes it
Fujii Masao wrote:
> On Thu, Feb 18, 2010 at 5:28 AM, Heikki Linnakangas
> wrote:
>> If I'm reading the patch correctly, when wal_sync_method is 'open_sync',
>> walreceiver nevertheless opens the WAL file without the O_DIRECT flag.
>> When it later flushes it in XLogWalRcvFlush() by issue_xlog_fsy
On Thu, Feb 18, 2010 at 5:28 AM, Heikki Linnakangas
wrote:
> If I'm reading the patch correctly, when wal_sync_method is 'open_sync',
> walreceiver nevertheless opens the WAL file without the O_DIRECT flag.
> When it later flushes it in XLogWalRcvFlush() by issue_xlog_fsync(),
> issue_xlog_fsync()
Fujii Masao wrote:
> On Wed, Feb 17, 2010 at 6:00 PM, Fujii Masao wrote:
>> On Wed, Feb 17, 2010 at 4:07 PM, Fujii Masao wrote:
>>> On Wed, Feb 17, 2010 at 3:03 PM, Magnus Hagander
>>> wrote:
In that case, O_DIRECT would be counterproductive, no? It maps to
FILE_FLAG_NOI_BUFFERING, wh
On Wed, Feb 17, 2010 at 6:00 PM, Fujii Masao wrote:
> On Wed, Feb 17, 2010 at 4:07 PM, Fujii Masao wrote:
>> On Wed, Feb 17, 2010 at 3:03 PM, Magnus Hagander wrote:
>>> In that case, O_DIRECT would be counterproductive, no? It maps to
>>> FILE_FLAG_NOI_BUFFERING, which makes sure it doesn't go i
On Wed, Feb 17, 2010 at 4:07 PM, Fujii Masao wrote:
> On Wed, Feb 17, 2010 at 3:03 PM, Magnus Hagander wrote:
>> In that case, O_DIRECT would be counterproductive, no? It maps to
>> FILE_FLAG_NOI_BUFFERING, which makes sure it doesn't go into the
>> cache. So the read in the startup proc is actua
On Wed, Feb 17, 2010 at 3:27 PM, Tom Lane wrote:
> Magnus Hagander writes:
>> On Wed, Feb 17, 2010 at 06:55, Fujii Masao wrote:
>>> 2. Straightforwardly observe the alignment rule. Since the received WAL
>>> data might start at the middle of WAL block, walreceiver needs to keep
>>> the last
On Wed, Feb 17, 2010 at 3:03 PM, Magnus Hagander wrote:
> In that case, O_DIRECT would be counterproductive, no? It maps to
> FILE_FLAG_NOI_BUFFERING, which makes sure it doesn't go into the
> cache. So the read in the startup proc is actually guaranteed to
> reuqire a physical read - of something
Magnus Hagander writes:
> On Wed, Feb 17, 2010 at 06:55, Fujii Masao wrote:
>> 2. Straightforwardly observe the alignment rule. Since the received WAL
>> data might start at the middle of WAL block, walreceiver needs to keep
>> the last half-written WAL block for alignment. OTOH since the rec
On Wed, Feb 17, 2010 at 06:55, Fujii Masao wrote:
> On Wed, Feb 17, 2010 at 6:28 AM, Magnus Hagander wrote:
>> If you send me your amazon id, I can get you premissions on my private
>> image. I plan to clean it up and make it public, just haven't gotten
>> around to it yet...
>
> Thanks for your
On Wed, Feb 17, 2010 at 6:28 AM, Magnus Hagander wrote:
> If you send me your amazon id, I can get you premissions on my private
> image. I plan to clean it up and make it public, just haven't gotten
> around to it yet...
Thanks for your concern! I'll send the ID when I complete the preparation.
2010/2/16 Fujii Masao :
> On Tue, Feb 16, 2010 at 7:20 PM, Magnus Hagander wrote:
>> 2010/2/16 Fujii Masao :
>>> On Tue, Feb 16, 2010 at 12:37 AM, Magnus Hagander
>>> wrote:
With the libpq fixes, I get further (more on that fix later, btw), but
now I get stuck in this. When I do someth
2010/2/16 Fujii Masao :
> On Tue, Feb 16, 2010 at 1:33 AM, Magnus Hagander wrote:
>> 2010/2/15 Tom Lane :
>>> Magnus Hagander writes:
I changed your patch to this, because I find it a lot simpler. The
change is in the checking in pgwin32_recv - there is no need to ever
call waitfor
On Tue, Feb 16, 2010 at 7:20 PM, Magnus Hagander wrote:
> 2010/2/16 Fujii Masao :
>> On Tue, Feb 16, 2010 at 12:37 AM, Magnus Hagander
>> wrote:
>>> With the libpq fixes, I get further (more on that fix later, btw), but
>>> now I get stuck in this. When I do something on the master that
>>> gene
2010/2/16 Fujii Masao :
> On Tue, Feb 16, 2010 at 12:37 AM, Magnus Hagander wrote:
>> With the libpq fixes, I get further (more on that fix later, btw), but
>> now I get stuck in this. When I do something on the master that
>> generates WAL, such as insert a record, and then try to query this on
>
On Tue, Feb 16, 2010 at 12:37 AM, Magnus Hagander wrote:
> With the libpq fixes, I get further (more on that fix later, btw), but
> now I get stuck in this. When I do something on the master that
> generates WAL, such as insert a record, and then try to query this on
> the slave, the walreceiver p
On Tue, Feb 16, 2010 at 1:33 AM, Magnus Hagander wrote:
> 2010/2/15 Tom Lane :
>> Magnus Hagander writes:
>>> I changed your patch to this, because I find it a lot simpler. The
>>> change is in the checking in pgwin32_recv - there is no need to ever
>>> call waitforsinglesocket, we can just exit
2010/2/15 Tom Lane :
> Magnus Hagander writes:
>> I changed your patch to this, because I find it a lot simpler. The
>> change is in the checking in pgwin32_recv - there is no need to ever
>> call waitforsinglesocket, we can just exit out early.
>
>> Do you see any issue with that?
>
> This defini
Magnus Hagander writes:
> I changed your patch to this, because I find it a lot simpler. The
> change is in the checking in pgwin32_recv - there is no need to ever
> call waitforsinglesocket, we can just exit out early.
> Do you see any issue with that?
This definitely looks cleaner, but is ther
2010/2/15 Fujii Masao :
> On Sun, Feb 14, 2010 at 11:52 PM, Magnus Hagander wrote:
>> Remember that the win32 code *always* puts the socket in non-blocking
>> mode. So we can't just "teach the layer about it". We need some way to
>> pass the information down that this is actually something we want
With the libpq fixes, I get further (more on that fix later, btw), but
now I get stuck in this. When I do something on the master that
generates WAL, such as insert a record, and then try to query this on
the slave, the walreceiver process crashes with:
PANIC: XX000: could not write to log file 0
On Sun, Feb 14, 2010 at 11:52 PM, Magnus Hagander wrote:
> Sorry about the delay in responding to this.
Thanks for the response.
> Remember that the win32 code *always* puts the socket in non-blocking
> mode. So we can't just "teach the layer about it". We need some way to
> pass the information
2010/2/8 Fujii Masao :
> On Mon, Jan 18, 2010 at 11:46 PM, Magnus Hagander wrote:
From what I can tell, this indicates that pq_getbyte_if_available() is
not working - because it's supposed to never block, right?
>>>
>>> Right, it's not supposed to block.
>>>
This could be because th
On Mon, Jan 18, 2010 at 11:46 PM, Magnus Hagander wrote:
>>> From what I can tell, this indicates that pq_getbyte_if_available() is
>>> not working - because it's supposed to never block, right?
>>
>> Right, it's not supposed to block.
>>
>>> This could be because the win32 socket emulation layer
Tom Lane wrote:
> Magnus Hagander writes:
>> 2010/1/24 Joe Conway :
>>> Sorry for being thick -- I'm still missing something. I don't understand
>>> why any user program using libpq/PQexec running on Windows does not have
>>> the same issue. Or to put it another way, why does this only apply to
>>
Magnus Hagander writes:
> 2010/1/24 Joe Conway :
>> Sorry for being thick -- I'm still missing something. I don't understand
>> why any user program using libpq/PQexec running on Windows does not have
>> the same issue. Or to put it another way, why does this only apply to
>> libpq calls from back
2010/1/24 Joe Conway :
> On 01/21/2010 11:19 PM, Heikki Linnakangas wrote:
>> Joe Conway wrote:
>>> OK, so now I see why we want this fixed for dblink and walreceiver, but
>>> doesn't this approach leave every other WIN32 libpq client out in the
>>> cold? Is there nothing that can be done for the g
On 01/21/2010 11:19 PM, Heikki Linnakangas wrote:
> Joe Conway wrote:
>> OK, so now I see why we want this fixed for dblink and walreceiver, but
>> doesn't this approach leave every other WIN32 libpq client out in the
>> cold? Is there nothing that can be done for the general case, or is it a
>> SM
Marko Kreen wrote:
> On 1/22/10, Dimitri Fontaine wrote:
>> Heikki Linnakangas writes:
>> > The problem only applies to libpq calls from the backend. Client apps
>> > are not affected, only backend modules. If there's any other modules out
>> > there that use libpq, then yes, those have a prob
On 1/22/10, Dimitri Fontaine wrote:
> Heikki Linnakangas writes:
>
> > Joe Conway wrote:
> >> OK, so now I see why we want this fixed for dblink and walreceiver, but
> >> doesn't this approach leave every other WIN32 libpq client out in the
> >> cold? Is there nothing that can be done for the
Heikki Linnakangas writes:
> Joe Conway wrote:
>> OK, so now I see why we want this fixed for dblink and walreceiver, but
>> doesn't this approach leave every other WIN32 libpq client out in the
>> cold? Is there nothing that can be done for the general case, or is it a
>> SMOP?
>
> The problem o
Joe Conway wrote:
> OK, so now I see why we want this fixed for dblink and walreceiver, but
> doesn't this approach leave every other WIN32 libpq client out in the
> cold? Is there nothing that can be done for the general case, or is it a
> SMOP?
The problem only applies to libpq calls from the ba
On 01/21/2010 10:33 PM, Heikki Linnakangas wrote:
> Joe Conway wrote:
>> I have not been really following this thread, but why can't we put the
>> "#ifdef WIN32" and special definition of these functions into libpq. I
>> don't understand why we need special treatment for dblink.
>
> The problem is
Joe Conway wrote:
> +#ifdef WIN23
> ^
> I assume you meant WIN32 here ;-)
Yeah. I admit I haven't tested this on Windows, I just commented out
those #ifdef's and tested on Linux. Will need to verify that this
actually solves the problem on Windows before committing.
> +#define PQexec(
On 01/21/2010 04:46 AM, Heikki Linnakangas wrote:
> Heikki Linnakangas wrote:
>> Magnus Hagander wrote:
>>> 2010/1/17 Heikki Linnakangas :
We could replace the blocking PQexec() calls with PQsendQuery(), and use
the emulated version of select() to wait.
>>> Hmm. That would at least theor
Heikki Linnakangas wrote:
> Magnus Hagander wrote:
>> 2010/1/17 Heikki Linnakangas :
>>> We could replace the blocking PQexec() calls with PQsendQuery(), and use
>>> the emulated version of select() to wait.
>> Hmm. That would at least theoretically work, but aren't there still
>> places we may en
Magnus Hagander wrote:
> 2010/1/17 Heikki Linnakangas :
>> We could replace the blocking PQexec() calls with PQsendQuery(), and use
>> the emulated version of select() to wait.
>
> Hmm. That would at least theoretically work, but aren't there still
> places we may end up blocking further down? Or
2010/1/17 Heikki Linnakangas :
> Magnus Hagander wrote:
>> Which shows one potentially big problem - since we're calling select()
>> from inside libpq, it's not calling our "signal emulation layer
>> compatible select()". This means that at this point, walreceiver is
>> not interruptible. Which als
2010/1/18 Tom Lane :
> Magnus Hagander writes:
>> Which shows one potentially big problem - since we're calling select()
>> from inside libpq, it's not calling our "signal emulation layer
>> compatible select()". This means that at this point, walreceiver is
>> not interruptible.
>
> Ugh.
Indeed.
On Mon, Jan 18, 2010 at 6:40 PM, Magnus Hagander wrote:
> SSL_read calls into pqwin32_recv(), so you have the same problem. (see
> my_sock_read() and my_sock_write() in be-secure.c)
Oh, I confirmed that. Thanks!
Can we prevent SSL_read from being blocked in the renegotiation case
if we use poll/
On Mon, Jan 18, 2010 at 10:30, Fujii Masao wrote:
> On Mon, Jan 18, 2010 at 5:22 AM, Heikki Linnakangas
> wrote:
>>> This could be because the win32 socket emulation layer simply wasn't
>>> designed to deal with non-blocking sockets. Specifically, it actually
>>> *always* sets the socket to non-b
On Mon, Jan 18, 2010 at 5:22 AM, Heikki Linnakangas
wrote:
>> This could be because the win32 socket emulation layer simply wasn't
>> designed to deal with non-blocking sockets. Specifically, it actually
>> *always* sets the socket to non-blocking mode, and then uses that to
>> properly emulate ho
Magnus Hagander writes:
> Which shows one potentially big problem - since we're calling select()
> from inside libpq, it's not calling our "signal emulation layer
> compatible select()". This means that at this point, walreceiver is
> not interruptible.
Ugh.
> Which also shows itself if I shut d
Magnus Hagander wrote:
> Which shows one potentially big problem - since we're calling select()
> from inside libpq, it's not calling our "signal emulation layer
> compatible select()". This means that at this point, walreceiver is
> not interruptible. Which also shows itself if I shut down the sys
I'm trying to figure out why streaming replication doesn't work on
win32. Here is what I have so far: It starts up fine, and outputs:
LOG: starting archive recovery
LOG: standby_mode = 'on'
LOG: primary_conninfo = 'host=localhost port=5432'
LOG: starting streaming recovery at 0/200
After t
51 matches
Mail list logo
