Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Hi I am testing this feature on relative complex schema (38619 tables in db) and I got deadlock [pavel@localhost bin]$ /usr/local/pgsql/bin/vacuumdb test2 -fz -j 4 vacuumdb: vacuuming database "test2" vacuumdb: vacuuming of database "test2" failed: ERROR: deadlock detected DETAIL: Process 24689

Re: [HACKERS] Parallel Seq Scan

Robert Haas writes: > On Wed, Jan 28, 2015 at 9:12 AM, Thom Brown wrote: >> On 28 January 2015 at 14:03, Robert Haas wrote: >>> The problem here, as I see it, is that we're flying blind. If there's >>> just one spindle, I think it's got to be right to read the relation >>> sequentially. But i

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

On Wed, Jan 28, 2015 at 1:13 PM, Jim Nasby wrote: > > My $0.01: > > While I sympathize with Noah's sentiments, the only thing that makes sense > to me is that a JSON text field is treated the same way as we treat text. > Right now, that means NUL is not allowed, period. > > If no one has bitched

Re: [HACKERS] pg_upgrade and rsync

> So, for my 2c, I'm on the fence about it. On the one hand, I agree, > it's a bit of a complex process to get right. On the other hand, it's > far better if we put something out there along the lines of "if you > really want to, this is how to do it" than having folks try to fumble > through to

[HACKERS] Exposing the stats snapshot timestamp to SQL

In a previous thread Tom Lane said: (I'm also wondering if it'd make sense to expose the stats timestamp > as a callable function, so that the case could be dealt with > programmatically as well. But that's future-feature territory.) (http://www.postgresql.org/message-id/27251.1421684...@sss.pg

Re: [HACKERS] pg_upgrade and rsync

* Josh Berkus (j...@agliodbs.com) wrote: > On 01/28/2015 02:28 PM, Josh Berkus wrote: > > On 01/28/2015 02:10 PM, Josh Berkus wrote: > >> So 390MB were transferred out of a possible 474MB. That certainly seems > >> like we're still transferring the majority of the data, even though I > >> verified

Re: [HACKERS] pg_upgrade and rsync

On 01/28/2015 02:28 PM, Josh Berkus wrote: > On 01/28/2015 02:10 PM, Josh Berkus wrote: >> So 390MB were transferred out of a possible 474MB. That certainly seems >> like we're still transferring the majority of the data, even though I >> verified that the hard links are being sent as hard links.

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

Stephen Frost writes: > * David Johnston (david.g.johns...@gmail.com) wrote: >> Fair enough but "reset" to what? I don't know the internal mechanics but >> if the session default is "warning" and a local change sets it to "notice" >> then an unconditional reset would not get us back to the intend

Re: [HACKERS] Possible typo in create_policy.sgml

Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > [There's also a typo further down -- "filter out the records which are > visible", should be "not visible"] I agree, that's not really worded quite right. I've reworded this along the lines of what you suggested (though not exactly- if you

Re: [HACKERS] Possible typo in create_policy.sgml

* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 7, 2015 at 3:06 PM, Stephen Frost wrote: > > If I'm following correctly, Peter's specifically talking about: > > > > [ USING ( expression ) ] > > [ WITH CHECK ( > class="parameter">check_expression ) ] > > > > Where the USING para

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

On Wednesday, January 28, 2015, Stephen Frost wrote: > * David G Johnston (david.g.johns...@gmail.com ) wrote: > > Jerry Sievers-3 wrote > > > Hackers; I noticed this trying to import a large pg_dump file with > > > warnings supressed. > > > > > > It seems loading pgq sets client_min_messages to

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

* David Johnston (david.g.johns...@gmail.com) wrote: > On Wednesday, January 28, 2015, Stephen Frost wrote: > > Ehh.. Shouldn't we try to take a bit more care that we reset things > > after a CREATE EXTENSION is run? Not really sure how much effort we > > want to put into it, but I see a bit of

Re: [HACKERS] Possible typo in create_policy.sgml

* Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Jan 9, 2015 at 3:46 PM, Stephen Frost wrote: > >A policy permits SELECT, INSERT, UPDATE or DELETE commands to access rows > >in a table that has row level security enabled. Access to existing table > >rows is granted if they match

Re: [HACKERS] Parallel Seq Scan

On Wed, Jan 28, 2015 at 8:27 PM, Stephen Frost wrote: > I feel like one of us is misunderstanding the numbers, which is probably > in part because they're a bit piecemeal over email, but the seqscan > speed in this case looks pretty close to dd performance for this > particular test, when things a

Re: [HACKERS] Missing updates at few places for row level security

Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: > There is a new column added in pg_authid (rolbypassrls) > and the updation for same is missed in below places: > > a. System catalog page for pg_authid > http://www.postgresql.org/docs/devel/static/catalog-pg-authid.html > b. Do we want to ad

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

* David G Johnston (david.g.johns...@gmail.com) wrote: > Jerry Sievers-3 wrote > > Hackers; I noticed this trying to import a large pg_dump file with > > warnings supressed. > > > > It seems loading pgq sets client_min_messages to warning and leaves it > > this way which defeats an attempt to chan

Re: [HACKERS] Possible typo in create_policy.sgml

Peter, * Peter Geoghegan (p...@heroku.com) wrote: > I also don't see this behavior documented (this is from process_policies()): [...] > But is that really the right place for it? Does it not equally well > apply to FOR UPDATE policies, that can on their own have both barriers > quals and WITH CHE

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

Jerry Sievers-3 wrote > Hackers; I noticed this trying to import a large pg_dump file with > warnings supressed. > > It seems loading pgq sets client_min_messages to warning and leaves it > this way which defeats an attempt to change the setting prior and have > it stick. > > I tested with severa

[HACKERS] Small bug on CREATE EXTENSION pgq...

Hackers; I noticed this trying to import a large pg_dump file with warnings supressed. It seems loading pgq sets client_min_messages to warning and leaves it this way which defeats an attempt to change the setting prior and have it stick. I tested with several other extensions in same DB and only

Re: [HACKERS] Proposal: two new role attributes and/or capabilities?

Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: > On 12/23/14 12:52 PM, Stephen Frost wrote: > >Autovacuum can certainly run vacuum/analyze on a few tables every 12 > >hours, so I'm not really following where you see autovacuum being unable > >to cope. I agree that there*are* such cases, but

Re: [HACKERS] compiler warnings in copy.c

* Andres Freund (and...@2ndquadrant.com) wrote: > On 2015-01-28 15:05:11 -0500, Stephen Frost wrote: > > > Also, you seem to have pushed these commits with a date more than two > > > weeks in the past. Please don't do that! > > > > Oh, wow, sorry about that. I had expected a rebase to update the

Re: [HACKERS] Parallel Seq Scan

Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: > On 1/28/15 9:56 AM, Stephen Frost wrote: > >Such i/o systems do exist, but a single RAID5 group over spinning rust > >with a simple filter isn't going to cut it with a modern CPU- we're just > >too darn efficient to end up i/o bound in that case

Re: [HACKERS] File based Incremental backup v7

Hi Marco, 2015-01-27 19:04 GMT+01:00 Marco Nenciarini : > I've done some test and it looks like that FSM nodes always have > InvalidXLogRecPtr as LSN. > > Ive updated the patch to always include files if all their pages have > LSN == InvalidXLogRecPtr > > Updated patch v7 attached. > > Regards, >

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

On Mon, Jan 26, 2015 at 11:21 PM, Andreas Karlsson wrote: > Do you also think the SQL functions should be named numeric_int128_sum, > numeric_int128_avg, etc? Some quick review comments. These apply to int128-agg-v5.patch. * Why is there no declaration of the function numeric_int16_stddev_intern

Re: [HACKERS] pg_upgrade and rsync

On 01/28/2015 02:10 PM, Josh Berkus wrote: > So 390MB were transferred out of a possible 474MB. That certainly seems > like we're still transferring the majority of the data, even though I > verified that the hard links are being sent as hard links. No? Looks like the majority of that was pg_xlo

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

Hi, attached is v9 of the patch, modified along the lines of Tom's comments: 1) uses alen=64 for cases with private context, 8 otherwise 2) reverts removal of element_type from initArrayResultArr() When element_type=InvalidOid is passed to initArrayResultArr, it performs lookup using get_

Re: [HACKERS] pg_upgrade and rsync

Bruce, Stephen, etc.: So, I did a test trial of this and it seems like it didn't solve the issue of huge rsyncs. That is, the only reason to do this whole business via rsync, instead of doing a new basebackup of each replica, is to cut down on data transfer time by not resyncing the data from the

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

On 1/28/15 12:46 AM, Haribabu Kommi wrote: >Also, what happens if someone reloads the config in the middle of running >the SRF? hba entries are reloaded only in postmaster process, not in every backend. So there shouldn't be any problem with config file reload. Am i missing something? Ahh, goo

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

Andres Freund wrote: > On 2015-01-28 15:32:15 +, Kevin Grittner wrote: >> Andres Freund wrote: >>> ISTM that the check is just overzelous and/or needs to be moved into >>> ImportSnapshot(). There it then could be made to check if the exporting >>> xact was also deferrable. >> >> That would be

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

Kevin Grittner writes: > Kevin Grittner wrote: >> Having pg_dump use repeatable read transactions for the processes >> that import the snapshot would work fine, as long as they are >> reading a snapshot which was captured by a serializable read only >> deferrable transaction. > It looks like the

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

On 2015-01-28 15:32:15 +, Kevin Grittner wrote: > Andres Freund wrote: > > ISTM that the check is just overzelous and/or needs to be moved into > > ImportSnapshot(). There it then could be made to check if the exporting > > xact was also deferrable. > > That would be great if ImportSnapshot h

Re: [HACKERS] Make hba available to client code

David Fetter writes: > On Wed, Jan 28, 2015 at 04:10:42PM -0500, Tom Lane wrote: >> How exactly would exporting those functions help anything client-side? > Right now, pgbouncer, and aspirational things like it--other > connection poolers, maybe distributed transaction managers, etc.--can > fairl

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

Kevin Grittner wrote: > Having pg_dump use repeatable read transactions for the processes > that import the snapshot would work fine, as long as they are > reading a snapshot which was captured by a serializable read only > deferrable transaction. It looks like the attached patch does it (althou

Re: [HACKERS] Make hba available to client code

On 01/28/2015 04:26 PM, David Fetter wrote: On Wed, Jan 28, 2015 at 04:10:42PM -0500, Tom Lane wrote: David Fetter writes: While investigating another project, namely adding pg_hba.conf support to pgbouncer, I ran into a stumbling block others probably will, too: the hba code is backend-only,

Re: [HACKERS] Make hba available to client code

On Wed, Jan 28, 2015 at 04:10:42PM -0500, Tom Lane wrote: > David Fetter writes: > > While investigating another project, namely adding pg_hba.conf support > > to pgbouncer, I ran into a stumbling block others probably will, too: > > the hba code is backend-only, which means that if I were to do t

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

Jim Nasby writes: > While I sympathize with Noah's sentiments, the only thing that makes sense to > me is that a JSON text field is treated the same way as we treat text. Right > now, that means NUL is not allowed, period. > If no one has bitched about this with text, is it really that big a pr

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

On 1/28/15 11:36 AM, Tom Lane wrote: Andrew Dunstan writes: It's not clear to me how we should represent a unicode null. i.e. given a json of '["foo\ubar"]', I get that we'd store the element as 'foo\x00bar', but what is the result of (jsonb '["foo\ubar"')->>0 It's defined to

Re: [HACKERS] Make hba available to client code

David Fetter writes: > While investigating another project, namely adding pg_hba.conf support > to pgbouncer, I ran into a stumbling block others probably will, too: > the hba code is backend-only, which means that if I were to do this > as-is, I would be cooking a batch of very unappetizing copyp

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

On 28.1.2015 21:28, Tom Lane wrote: > Jeff Davis writes: >> On Tue, Jan 20, 2015 at 6:44 AM, Tom Lane wrote: >>> Uh, sorry, I've not been paying any attention to this thread for awhile. >>> What's the remaining questions at issue? > >> This patch is trying to improve the array_agg case where the

[HACKERS] Make hba available to client code

Folks, While investigating another project, namely adding pg_hba.conf support to pgbouncer, I ran into a stumbling block others probably will, too: the hba code is backend-only, which means that if I were to do this as-is, I would be cooking a batch of very unappetizing copypasta. I'm allergic to

[HACKERS] pg_dump issue - push useless statements REVOKE, GRANT

Hi I have a dump with thousands objects.I found often pattern in dump, that has not any sense. These operations has zero sense, but it decrease a database restore. It is expected behave? REVOKE ALL ON TABLE zobjrozp FROM PUBLIC; REVOKE ALL ON TABLE zobjrozp FROM sitkhaso; GRANT ALL ON TABLE zobj

Re: [HACKERS] Parallel Seq Scan

On 1/28/15 9:56 AM, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 28, 2015 at 10:40 AM, Tom Lane wrote: I thought the proposal to chunk on the basis of "each worker processes one 1GB-sized segment" should work all right. The kernel should see that as sequential

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

Jeff Davis writes: > On Tue, Jan 20, 2015 at 6:44 AM, Tom Lane wrote: >> Uh, sorry, I've not been paying any attention to this thread for awhile. >> What's the remaining questions at issue? > This patch is trying to improve the array_agg case where there are > many arrays being constructed simul

Re: [HACKERS] compiler warnings in copy.c

On 2015-01-28 15:05:11 -0500, Stephen Frost wrote: > > Also, you seem to have pushed these commits with a date more than two > > weeks in the past. Please don't do that! > > Oh, wow, sorry about that. I had expected a rebase to update the date. It updates the committer, but not the author date.

Re: [HACKERS] compiler warnings in copy.c

* Robert Haas (robertmh...@gmail.com) wrote: > My compiler is unhappy with the latest changes to copy.c: > > gcc -Wall -Wmissing-prototypes -Wpointer-arith > -Wdeclaration-after-statement -Wendif-labels > -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing > -fwrapv -fexcess-precisio

[HACKERS] compiler warnings in copy.c

My compiler is unhappy with the latest changes to copy.c: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -O2 -Wall -Werror -I../../../src/include -D

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

=?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= writes: > But I'm thinking about this patch and would not be interesting to have a > FDW to manipulate the hba file? Imagine if we are able to manipulate the > HBA file using INSERT/UPDATE/DELETE. Since the HBA file is fundamentally order-dependent, while

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

On Wed, Jan 28, 2015 at 4:46 AM, Haribabu Kommi wrote: > > On Wed, Jan 28, 2015 at 9:47 AM, Jim Nasby wrote: > > On 1/27/15 1:04 AM, Haribabu Kommi wrote: > >> > >> Here I attached the latest version of the patch. > >> I will add this patch to the next commitfest. > > > > > > Apologies if this wa

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

Andres Freund writes: > On 2015-01-28 13:38:51 -0500, Tom Lane wrote: >> #define BUFFERDESC_PADDED_SIZE (SIZEOF_VOID_P == 8 ? 64 : 32) > Hm, did you intentionally put a 32in there or was that just the logical > continuation of 64? Because there's no way it'll ever fit into 32 bytes > in the

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

On Wed, Jan 28, 2015 at 12:47 PM, Tom Lane wrote: > Merlin Moncure writes: >> ...hm, I spoke to soon. So I deleted everything, and booted up a new >> instance 9.4 vanilla with asserts on and took no other action. >> Applying the script with no data activity fails an assertion every >> single tim

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

Merlin Moncure writes: > ...hm, I spoke to soon. So I deleted everything, and booted up a new > instance 9.4 vanilla with asserts on and took no other action. > Applying the script with no data activity fails an assertion every > single time: > TRAP: FailedAssertion("!(flags & 0x0010)", File: "d

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

On 2015-01-28 13:38:51 -0500, Tom Lane wrote: > Andres Freund writes: > >> I personally still think that a comment above sbufdesc's definition > >> would be sufficient for now. But whatever. I'll enforce 64byte padding > >> on 64bit platforms, and do nothing on 32bit platforms. > > > Patch doing

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

Andres Freund writes: >> I personally still think that a comment above sbufdesc's definition >> would be sufficient for now. But whatever. I'll enforce 64byte padding >> on 64bit platforms, and do nothing on 32bit platforms. > Patch doing that attached. Surely the sizeof() in BufferShmemSize nee

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

David G Johnston writes: > Am I missing something or has there been no consideration in this "forbid" > plan on whether users will be able to retrieve, even if partially > incorrectly, any jsonb data that has already been stored? Data that's already been stored would look like the six characters

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

On Wed, Jan 28, 2015 at 8:05 AM, Merlin Moncure wrote: > On Thu, Jan 22, 2015 at 3:50 PM, Merlin Moncure wrote: >> I still haven't categorically ruled out pl/sh yet; that's something to >> keep in mind. > > Well, after bisection proved not to be fruitful, I replaced the pl/sh > calls with dummy c

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

On 2015-01-28 17:08:46 +0100, Andres Freund wrote: > I just have no idea whether it'd be beneficial to use more space on > 32bit to pad the individual entries. Since this mostly is beneficial on > multi-socket, highly concurrent workloads, I doubt it really matter. > I personally still think that

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

Tom Lane-2 wrote > Andrew Dunstan < > andrew@ > > writes: >> On 01/27/2015 02:28 PM, Tom Lane wrote: >>> Well, we can either fix it now or suffer with a broken representation >>> forever. I'm not wedded to the exact solution I described, but I think >>> we'll regret it if we don't change the rep

Re: [HACKERS] PQgetssl() and alternative SSL implementations

Heikki Linnakangas writes: > Right, that was the idea. I wanted it to include the word "OpenSSL", to > make it clear in the callers that it's specific to OpenSSL. And SSL, > because that's the name of the struct. I agree it looks silly, though. > One idea is to have two separate arguments: the

Re: [HACKERS] Sequence Access Method WIP

On 28/01/15 18:09, Heikki Linnakangas wrote: On 01/23/2015 02:34 AM, Petr Jelinek wrote: On 22/01/15 17:02, Petr Jelinek wrote: The new version (the one that is not submitted yet) of gapless sequence is way more ugly and probably not best example either but does guarantee gaplessness (it store

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

Andrew Dunstan writes: > It's not clear to me how we should represent a unicode null. i.e. given > a json of '["foo\ubar"]', I get that we'd store the element as > 'foo\x00bar', but what is the result of > (jsonb '["foo\ubar"')->>0 > It's defined to be text so we can't just shove a

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

Noah Misch writes: > On Tue, Jan 27, 2015 at 03:56:22PM -0500, Tom Lane wrote: >> So at this point I propose that we reject \u when de-escaping JSON. > I would have agreed on 2014-12-09, and this release is the last chance to make > such a change. It is a bold wager that could pay off, but -

Re: [HACKERS] PQgetssl() and alternative SSL implementations

* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > What bothers me about this is that it punts SSL work to the application > > and requires that they be coded to work with both OpenSSL and whatever > > else we implement (eg: GnuTLS) to do anything but the most simple > > checks. T

Re: [HACKERS] PQgetssl() and alternative SSL implementations

On 01/28/2015 06:58 PM, Stephen Frost wrote: >Although I think "OpenSSL SSL" is a little bit duplicatively >redundant. Why not just "OpenSSL"? I wondered also, but figured it was probably because it's OpenSSL's "ssl" structure, which then made sense. Right, that was the idea. I wanted it to i

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

On 01/28/2015 12:50 AM, Noah Misch wrote: On Tue, Jan 27, 2015 at 03:56:22PM -0500, Tom Lane wrote: Andrew Dunstan writes: On 01/27/2015 02:28 PM, Tom Lane wrote: Well, we can either fix it now or suffer with a broken representation forever. I'm not wedded to the exact solution I described,

Re: [HACKERS] Sequence Access Method WIP

On 01/23/2015 02:34 AM, Petr Jelinek wrote: On 22/01/15 17:02, Petr Jelinek wrote: The new version (the one that is not submitted yet) of gapless sequence is way more ugly and probably not best example either but does guarantee gaplessness (it stores the last value in it's own value table). So

Re: [HACKERS] PQgetssl() and alternative SSL implementations

Stephen Frost writes: > What bothers me about this is that it punts SSL work to the application > and requires that they be coded to work with both OpenSSL and whatever > else we implement (eg: GnuTLS) to do anything but the most simple > checks. That's a problem because people are *not* going to

Re: [HACKERS] PQgetssl() and alternative SSL implementations

* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 28, 2015 at 10:13 AM, Heikki Linnakangas > wrote: > > Here's a patch to implement the above scheme. It adds four functions to > > libpq, to interrogate the SSL status: > > > > int PQsslInUse(const PGconn *conn) > > Returns true (1) if the

Re: [HACKERS] PQgetssl() and alternative SSL implementations

On Wed, Jan 28, 2015 at 10:13 AM, Heikki Linnakangas wrote: > Here's a patch to implement the above scheme. It adds four functions to > libpq, to interrogate the SSL status: > > int PQsslInUse(const PGconn *conn) > Returns true (1) if the connection uses SSL, false (0) if not. > > const char *PQss

Re: [HACKERS] WITH CHECK and Column-Level Privileges

Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 27 January 2015 at 22:45, Stephen Frost wrote: > > Here's the latest set with a few additional improvements (mostly > > comments but also a couple missed #include's and eliminating unnecessary > > whitespace changes). Unless there are i

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

On 2015-01-28 10:58:40 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2015-01-28 10:35:28 -0500, Tom Lane wrote: > >> Even if you didn't have plans like that, it would be entire folly to > >> imagine that buffer headers will be exactly 64 bytes without some forcing > >> function for that. >

Re: [HACKERS] Parallel Seq Scan

* Stephen Frost (sfr...@snowman.net) wrote: > Such i/o systems do exist, but a single RAID5 group over spinning rust > with a simple filter isn't going to cut it with a modern CPU- we're just > too darn efficient to end up i/o bound in that case. err, to *not* end up i/o bound. Thanks,

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

Andres Freund writes: > On 2015-01-28 10:35:28 -0500, Tom Lane wrote: >> Even if you didn't have plans like that, it would be entire folly to >> imagine that buffer headers will be exactly 64 bytes without some forcing >> function for that. > Meh. The 128 byte additionally used by the alignment d

Re: [HACKERS] Parallel Seq Scan

* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 28, 2015 at 10:40 AM, Tom Lane wrote: > > I thought the proposal to chunk on the basis of "each worker processes > > one 1GB-sized segment" should work all right. The kernel should see that > > as sequential reads of different files, issu

Re: [HACKERS] Parallel Seq Scan

Robert Haas writes: > On Wed, Jan 28, 2015 at 10:40 AM, Tom Lane wrote: >> I thought the proposal to chunk on the basis of "each worker processes >> one 1GB-sized segment" should work all right. The kernel should see that >> as sequential reads of different files, issued by different processes;

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

On 2015-01-28 10:35:28 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2015-01-26 21:13:31 -0500, Robert Haas wrote: > >> So maybe we should also do something like what LWLocks do, and make a > >> union between the actual structure and an appropriate array of padding > >> bytes - say either

Re: [HACKERS] Parallel Seq Scan

On Wed, Jan 28, 2015 at 10:40 AM, Tom Lane wrote: > Robert Haas writes: >> The problem here, as I see it, is that we're flying blind. If there's >> just one spindle, I think it's got to be right to read the relation >> sequentially. But if there are multiple spindles, it might not be, >> but it

Re: [HACKERS] Parallel Seq Scan

Robert Haas writes: > The problem here, as I see it, is that we're flying blind. If there's > just one spindle, I think it's got to be right to read the relation > sequentially. But if there are multiple spindles, it might not be, > but it seems hard to predict what we should do. We don't know

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

On Wed, Jan 28, 2015 at 10:35 AM, Tom Lane wrote: > Andres Freund writes: >> On 2015-01-26 21:13:31 -0500, Robert Haas wrote: >>> So maybe we should also do something like what LWLocks do, and make a >>> union between the actual structure and an appropriate array of padding >>> bytes - say either

Re: [HACKERS] Parallel Seq Scan

On Wed, Jan 28, 2015 at 9:12 AM, Thom Brown wrote: > On 28 January 2015 at 14:03, Robert Haas wrote: >> The problem here, as I see it, is that we're flying blind. If there's >> just one spindle, I think it's got to be right to read the relation >> sequentially. But if there are multiple spindle

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

Andres Freund writes: > On 2015-01-26 21:13:31 -0500, Robert Haas wrote: >> So maybe we should also do something like what LWLocks do, and make a >> union between the actual structure and an appropriate array of padding >> bytes - say either 64 or 128 of them. > Hm. That's a bit bigger patch. I'm

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

Andres Freund wrote: > Kevin Grittner wrote: >>> Alexander Korotkov wrote: >>> Could we start snapshot-importing transaction with repeatable read isolation level? >> If you are talking about having pg_dump acquire a safe snapshot and >> have cooperating processes in the same pg_dump

Re: [HACKERS] Parallel Seq Scan

On Wed, Jan 28, 2015 at 7:46 AM, Robert Haas wrote: > > > All that aside, I still can't account for the numbers you are seeing. > When I run with your patch and what I think is your test case, I get > different (slower) numbers. And even if we've got 6 drives cranking > along at 400MB/s each, tha

Re: [HACKERS] PQgetssl() and alternative SSL implementations

On 08/20/2014 12:58 AM, Heikki Linnakangas wrote: On 08/19/2014 10:31 PM, Robert Haas wrote: On Tue, Aug 19, 2014 at 3:16 PM, Magnus Hagander wrote: On Tue, Aug 19, 2014 at 9:09 PM, Tom Lane wrote: Robert's got a point though: there is always going to be somebody who wants something we fail

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

On 2015-01-28 14:54:15 +, Kevin Grittner wrote: > Kevin Grittner wrote: > > Alexander Korotkov wrote: > > >> Could we start snapshot-importing transaction with repeatable > >> read isolation level? > > > > You can if you don't use the option which specifies that you want > > serializable beh

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

Kevin Grittner wrote: > Alexander Korotkov wrote: >> Could we start snapshot-importing transaction with repeatable >> read isolation level? > > You can if you don't use the option which specifies that you want > serializable behavior. Why specify --serializable-deferrable if > you don't? > >> A

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

Alexander Korotkov wrote: > when pg_dump is run with both --serializable-deferrable and -j > options to pg_dump, it returns errors: > > pg_dump: [archiver (db)] query failed: ERROR: a snapshot-importing > transaction must not be READ ONLY DEFERRABLE > pg_dump: [archiver (db)] query failed: ERRO

Re: [HACKERS] Safe memory allocation functions

On Tue, Jan 27, 2015 at 5:34 PM, Andres Freund wrote: > On 2015-01-27 17:27:53 +0900, Michael Paquier wrote: >> Alvaro Herrera wrote: >> >> So how about something like >> >> >> >> #define ALLOCFLAG_HUGE 0x01 >> >> #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 >> >> void * >> >> Mem

Re: [HACKERS] Parallel Seq Scan

On 28 January 2015 at 14:03, Robert Haas wrote: > The problem here, as I see it, is that we're flying blind. If there's > just one spindle, I think it's got to be right to read the relation > sequentially. But if there are multiple spindles, it might not be, > but it seems hard to predict what

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

On Thu, Jan 22, 2015 at 3:50 PM, Merlin Moncure wrote: > I still haven't categorically ruled out pl/sh yet; that's something to > keep in mind. Well, after bisection proved not to be fruitful, I replaced the pl/sh calls with dummy calls that approximated the same behavior and the problem went awa

Re: [HACKERS] Parallel Seq Scan

On Wed, Jan 28, 2015 at 2:08 AM, Heikki Linnakangas wrote: > OTOH, spreading the I/O across multiple files is not a good thing, if you > don't have a RAID setup like that. With a single spindle, you'll just induce > more seeks. > > Perhaps the OS is smart enough to read in large-enough chunks that

Re: [HACKERS] pg_upgrade and rsync

Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jan 27, 2015 at 09:36:58AM -0500, Stephen Frost wrote: > > The example listed works, but only when it's a local rsync: > > > > rsync --archive --hard-links --size-only old_dir new_dir remote_dir > > > > Perhaps a better example (or addit

Re: [HACKERS] pg_upgrade and rsync

* Bruce Momjian (br...@momjian.us) wrote: > Interesting problem, but doesn't rsync use sub-second accuracy? No. Simple test will show: touch xx/aa ; rsync -avv xx yy ; sleep 0.5 ; touch xx/aa ; rsync -avv xx yy Run that a few times and you'll see it report "xx/aa is uptodate" sometimes, dependi

Re: [HACKERS] pg_upgrade and rsync

* Jim Nasby (jim.na...@bluetreble.com) wrote: > On 1/27/15 9:29 AM, Stephen Frost wrote: > >>My point is that Bruce's patch suggests looking for "remote_dir" in > >>>the rsync documentation, but no such term appears there. > >Ah, well, perhaps we could simply add a bit of clarification to this: > >

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

On 2015-01-26 21:13:31 -0500, Robert Haas wrote: > On Mon, Jan 26, 2015 at 9:08 PM, Robert Haas wrote: > >> Contrary opinions? Robert? > > > > I'm totally OK with further aligning just that one allocation. > > Of course, now that I think about it, aligning it probably works > mostly because the s

Re: [HACKERS] TABLESAMPLE patch

On 28/01/15 08:23, Kyotaro HORIGUCHI wrote: Hi, I took a look on this and found nice. By the way, the parameter for REPEATABLE seems allowing to be a expression in ParseTableSample but the grammer rejects it. It wasn't my intention to support it, but you are correct, the code is generic enou

Re: [HACKERS] TABLESAMPLE patch

On 28/01/15 09:41, Kyotaro HORIGUCHI wrote: As an issue related to size esmation, I got a explain result as following, =# explain (analyze on, buffers on) select a from t1 tablesample system(10) where a < 5; QUERY PLAN ---

Re: [HACKERS] EvalPlanQual behaves oddly for FDW queries involving system columns

On 2015/01/19 17:10, Etsuro Fujita wrote: Attached is an updated version of the patch. I'll add this to the next CF. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsq

Re: [HACKERS] WITH CHECK and Column-Level Privileges

On 27 January 2015 at 22:45, Stephen Frost wrote: > Here's the latest set with a few additional improvements (mostly > comments but also a couple missed #include's and eliminating unnecessary > whitespace changes). Unless there are issues with my testing tonight or > concerns raised, I'll push th

Re: [HACKERS] TABLESAMPLE patch

Hello, > On 19/01/15 07:08, Amit Kapila wrote: > > On Sun, Jan 18, 2015 at 12:46 AM, Petr Jelinek > > wrote: > > No issues, but it seems we should check other paths where > > different handling could be required for tablesample scan. > > In set_rel_size(), it uses nor

Re: [HACKERS] ya comment typo

On 01/28/2015 10:16 AM, Erik Rijkers wrote: 'the the' in bufmgr.c Fixed, thanks. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

  1   2   >