Re: [HACKERS] concurrent COPY performance

Merlin Moncure wrote: On Tue, Jun 16, 2009 at 12:47 PM, Stefan Kaltenbrunner wrote: Hi! I have been doing some bulk loading testing recently - mostly with a focus on answering why we are "only" getting a (max of) cores/2(up to around 8 cores even less with more) speedup using parallel restore.

Re: [HACKERS] concurrent COPY performance

Andrew Dunstan wrote: > If a table is created or truncated in the same transaction that does > the load, and archiving is not on, the COPY is not WALed. Slightly off topic, but possibly relevant to the overall process: those are the same conditions under which I would love to see the rows inse

Re: [HACKERS] concurrent COPY performance

Merlin Moncure wrote: On Tue, Jun 16, 2009 at 12:47 PM, Stefan Kaltenbrunner wrote: Hi! I have been doing some bulk loading testing recently - mostly with a focus on answering why we are "only" getting a (max of) cores/2(up to around 8 cores even less with more) speedup using parallel rest

Re: [HACKERS] concurrent COPY performance

On Tue, Jun 16, 2009 at 12:47 PM, Stefan Kaltenbrunner wrote: > Hi! > > I have been doing some bulk loading testing recently - mostly with a focus > on answering why we are "only" getting a (max of) cores/2(up to around 8 > cores even less with more) speedup using parallel restore. > What I found i

Re: [HACKERS] machine-readable explain output

On 06/16/2009 09:51 PM, Robert Haas wrote: On Tue, Jun 16, 2009 at 2:12 PM, Tom Lane wrote: The main point here is that we have a pretty good idea of what general-purpose client code is likely to want to do with the data, and in a lot of cases that does not translate to having to know each node

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 2:12 PM, Tom Lane wrote: > The main point here is that we have a pretty good idea of what > general-purpose client code is likely to want to do with the data, and > in a lot of cases that does not translate to having to know each node > type explicitly, so long as it can be

Re: [HACKERS] GRANT ON ALL IN schema

Petr Jelinek wrote: Anyway back to my thoughts about this patch. First of all I see problem with the proposed syntax. For this syntax I think TABLES (FUNCTIONS, SEQUENCES, etc) would have to be added to keywords which is problematic because there are views named tables, sequences, views in inf

Re: [HACKERS] machine-readable explain output

Robert Haas writes: > On Tue, Jun 16, 2009 at 1:21 PM, Tom Lane wrote: >> As a concrete example of what I'm thinking about, I'd hope that PgAdmin >> would be able to display a graphical summary of a plan tree, and then >> pop up measurements associated with one of the nodes when you >> right-click

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

> -Original Message- > From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- > ow...@postgresql.org] On Behalf Of Stephen Frost > Sent: Tuesday, June 16, 2009 5:47 AM > To: Greg Stark > Cc: Robert Haas; Jeremy Kerr; ; Alvaro > Herrera; Stefan Kaltenbrunner; Gurjeet Singh > Subject

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 1:21 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jun 16, 2009 at 12:04 PM, Tom Lane wrote: >>> FWIW, I like Greg's idea of subdividing the available data this way. > >> I like it too, but I'd like to see us come up with a design that >> allows it to be used for all

Re: [HACKERS] machine-readable explain output

Robert Haas writes: > On Tue, Jun 16, 2009 at 12:04 PM, Tom Lane wrote: >> FWIW, I like Greg's idea of subdividing the available data this way. > I like it too, but I'd like to see us come up with a design that > allows it to be used for all of the output formats (text, XML, and > JSON). I think

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 12:04 PM, Tom Lane wrote: > Greg Stark writes: >> I'm picturing adding a new tag, such as , or actually I was >> thinking of . If we have separate tags for all the estimates >> and actual timings then any tags which come with the or >> option would just get mixed up with

[HACKERS] concurrent COPY performance

Hi! I have been doing some bulk loading testing recently - mostly with a focus on answering why we are "only" getting a (max of) cores/2(up to around 8 cores even less with more) speedup using parallel restore. What I found is that on some fast IO-subsystem we are CPU bottlenecked on concurren

Re: [HACKERS] postmaster recovery and automatic restart suppression

"Czichy, Thoralf (NSN - FI/Helsinki)" writes: > I am working together with Harald on this issue. Below some thoughts on > why we think it should be possible to disable the postmaster-internal > recovery attempt and instead have faults in the processes started > by postmaster escalated to postma

Re: [HACKERS] machine-readable explain output

Tom Lane wrote: But I'd be just as happy with a naming convention, like versus , etc. I don't know enough about XML usage to understand the benefits and costs of different ways of providing that kind of structure. FYI, you probably don't want this. the ':' is not just another character,

Re: [HACKERS] machine-readable explain output

Greg Stark writes: > I'm picturing adding a new tag, such as , or actually I was > thinking of . If we have separate tags for all the estimates > and actual timings then any tags which come with the or > option would just get mixed up with the estimates and timing > info. FWIW, I like Greg's id

Re: [HACKERS] postmaster recovery and automatic restart suppression

hi, I am working together with Harald on this issue. Below some thoughts on why we think it should be possible to disable the postmaster-internal recovery attempt and instead have faults in the processes started by postmaster escalated to postmaster-exit. [Our typical "embedded" situation]

[HACKERS] GRANT ON ALL IN schema

Hi all, I am thinking about implementing GRANT ON ALL TABLES IN schema TODO item. I saw many people sending proposals to the list but nobody seemed to actually do anything. I have few questions and problems to iron out before I can start the implementation. I would also like to note that I am

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

Jeremy Kerr wrote: Hi Tom, Speaking of which, what about some performance numbers? Like Heikki, I'm quite suspicious of whether there is any real-world gain to be had from this approach. Will send numbers tomorrow, with the reworked patch. I can easily redo my testing as well if required.

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 1:53 PM, Andres Freund wrote: > While that also looks sensible the more structured variant makes it easier > to integrate additional stats which may not easily be pressed in the > 'attribute' format. As a fastly contrived example you could have io > statistics over time like

Re: [HACKERS] machine-readable explain output

Andres Freund wrote: Anyway, I think what this discussion points out is that we actually need a formal XML Schema for this output. Agreed. If helpful I can create a schema for the current format. That will give us a useful starting point. cheers andrew -- Sent via pgsql-hackers mailing

Re: [HACKERS] machine-readable explain output

Tom Lane wrote: Andres Freund writes: On 06/16/2009 04:32 PM, Robert Haas wrote: Note that even in this case we DON'T rely on the ordering of the nodes. The inner nodes have child nodes which contain their relationship to the parent. Not in the case of Append nodes, bu

[HACKERS] Uninstallation error

Hi, I've found the uninstallation error... # make uninstall : n/man7/truncate.7 /usr/local/pgsql/share/man/man7/unlisten.7 /usr/local/pgsql/share/man/man7/update.7 /usr/local/pgsql/share/man/man7/vacuum.7 /usr/local/pgsql/share/man/man7/values.7 rm: cannot remove `/usr/local/pgsql/share/man/ma

Re: [HACKERS] machine-readable explain output

Andres Freund writes: > On 06/16/2009 04:32 PM, Robert Haas wrote: >> Note that even in this case we DON'T rely on the ordering of the >> nodes. The inner nodes have child nodes which contain their >> relationship to the parent. > Not in the case of Append nodes, but I fail to see a problem the

Re: [HACKERS] Synch Rep: communication between backends and walsender

Greg Stark writes: > On Tue, Jun 16, 2009 at 12:50 PM, Fujii Masao wrote: >> On some platforms, a signal doesn't interrupt sleep (i.e. poll or select >> system call) > say what? Yup, what he said. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 10:59 AM, Andrew Dunstan wrote: > > > Robert Haas wrote: >>> >>> If there is a relationship between the items then that needs to be >>> expressed >>> in the XML structure, either by use of child nodes or attributes. Relying >>> on >>> the sequence of nodes, if that's what yo

Re: [HACKERS] machine-readable explain output

Hi, On 06/16/2009 04:32 PM, Robert Haas wrote: On Tue, Jun 16, 2009 at 10:30 AM, Andres Freund wrote: How would you model something like: ... ... ... otherwise? There are potentially unlimited number of child nodes - AppendNode for example can have any number of them. Sure, you ca

Re: [HACKERS] machine-readable explain output

Robert Haas wrote: If there is a relationship between the items then that needs to be expressed in the XML structure, either by use of child nodes or attributes. Relying on the sequence of nodes, if that's what you're doing, is not a good idea, and I'm not doing that. Period, full stop

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

Hi Tom, > Speaking of which, what about some performance numbers? Like Heikki, > I'm quite suspicious of whether there is any real-world gain to be > had from this approach. Will send numbers tomorrow, with the reworked patch. Cheers, Jeremy -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Synch Rep: communication between backends and walsender

On Tue, Jun 16, 2009 at 12:50 PM, Fujii Masao wrote: > On some platforms, a signal doesn't interrupt sleep (i.e. poll or select > system call) say what? -- Gregory Stark http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 10:30 AM, Andres Freund wrote: > How would you model something like: > >   ... >   ... >  ... > > otherwise? > > There are potentially unlimited number of child nodes - AppendNode for > example can have any number of them. Sure, you can give each node a > 'offset=' id,

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 9:45 AM, Andrew Dunstan wrote: > Robert Haas wrote: >> >> 3. We have existing precedent for this design pattern in, e.g. >> table_to_xml >> >> http://www.postgresql.org/docs/current/interactive/functions-xml.html > > Tables are flat, explain output is not. > > If there is a

Re: [HACKERS] machine-readable explain output

On 06/16/2009 03:45 PM, Andrew Dunstan wrote: >> 3. We have existing precedent for this design pattern in, e.g. >> table_to_xml >> http://www.postgresql.org/docs/current/interactive/functions-xml.html > Tables are flat, explain output is not. Comparing Greg's approach with Robert's it seems to me

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

Greg Stark writes: > On some architectures like intel accessing unaligned ints is just > slow. On others (Alpha and PPC iirc?) it is an immediate bus error. To a first approximation, Intel is the *only* popular architecture that doesn't bus-error on unaligned accesses. (And I'm sure their chip d

Re: [HACKERS] machine-readable explain output

Robert Haas wrote: 3. We have existing precedent for this design pattern in, e.g. table_to_xml http://www.postgresql.org/docs/current/interactive/functions-xml.html Tables are flat, explain output is not. If there is a relationship between the items then that needs to be expressed in t

Re: [HACKERS] machine-readable explain output

On 06/16/2009 03:22 PM, Robert Haas wrote: Well, one problem with this is that the actual values are not costs, but times, and the estimated values are not times, but costs. The planner estimates the cost of operations on an arbitrary scale where the cost of a sequential page fetch is 1.0. Whe

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

Hi all, > That's correct. To check the alignment you would have to look at the > actual pointer. I would suggest using the existing macros to handle > alignment. Hm, though the only one I see offhand which is relevant is > the moderately silly PointerIsAligned(). Still it would make the code > cle

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 8:53 AM, Andres Freund wrote: > On 06/16/2009 02:14 PM, Greg Stark wrote: >> >> On Tue, Jun 16, 2009 at 12:19 PM, Andres Freund >>  wrote: >>> >>> 1710.98 >>> 1710.98 >>> 72398 >>> 4 >>> 136.595 >>> 136.595 >>> 72398 >>> 1 >> >> XML's not really my thing currently but it sur

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

Robert Haas wrote: Ooh, good point. I still don't like the 0x20 thing, but using uint32 instead of int or long is the main point, unless we support any platforms where 0x20 != ' '. All our server encodings are strictly ASCII supersets. So 0x20 is always the space character. cheers a

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

On Tue, Jun 16, 2009 at 8:38 AM, Greg Stark wrote: > On Tue, Jun 16, 2009 at 1:03 PM, Robert Haas wrote: >> I see that... but I don't think the test in the first loop is correct. >>  It's based on the value of i % 4, but I'm not convinced that you know >> anything about the alignment at the point w

Re: [HACKERS] machine-readable explain output

On 06/16/2009 02:14 PM, Greg Stark wrote: On Tue, Jun 16, 2009 at 12:19 PM, Andres Freund wrote: 1710.98 1710.98 72398 4 136.595 136.595 72398 1 XML's not really my thing currently but it sure seems strange to me to have *everything* be a separate tag like this. Doesn't XML do attributes too?

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

On Tue, Jun 16, 2009 at 1:41 PM, Stephen Frost wrote: > > Ah, you may be half right there (see below).  It does appear to be > assuming that char *s (or s[i == 0]) is aligned, which isn't a > guarentee (in fact, it might never be right..).  If having it actually > aligned is an important bit (as op

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

* Greg Stark (gsst...@mit.edu) wrote: > There are two points here that kind of cancel each other out :) Thanks for the insight. :) Stephen signature.asc Description: Digital signature

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

On Tue, Jun 16, 2009 at 1:03 PM, Robert Haas wrote: > >> On the flip side, I am curious as to if the arguments to a stored >> procedure are always aligned or not.  Never had a case to care before, >> but if palloc() is always going to return an aligned chunk of memory >> (per MemSetAligned in c.h)

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

* Robert Haas (robertmh...@gmail.com) wrote: > I see that... but I don't think the test in the first loop is correct. > It's based on the value of i % 4, but I'm not convinced that you know > anything about the alignment at the point where i == 0. Ah, you may be half right there (see below). It

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

On Tue, Jun 16, 2009 at 1:03 PM, Robert Haas wrote: > I see that... but I don't think the test in the first loop is correct. >  It's based on the value of i % 4, but I'm not convinced that you know > anything about the alignment at the point where i == 0. That's correct. To check the alignment you

Re: [HACKERS] machine-readable explain output

On Tue, Jun 16, 2009 at 12:19 PM, Andres Freund wrote: > 1710.98 > 1710.98 > 72398 > 4 > 136.595 > 136.595 > 72398 > 1 XML's not really my thing currently but it sure seems strange to me to have *everything* be a separate tag like this. Doesn't XML do attributes too? I would have thought to use ch

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

On Tue, Jun 16, 2009 at 6:30 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> As I look at this, another problem is that it seems to me that you're >> assuming that VARDATA_ANY() will return an aligned pointer, which >> isn't necessarily the case (see src/include/postgres.

[HACKERS] Synch Rep: communication between backends and walsender

http://archives.postgresql.org/pgsql-hackers/2008-12/msg00448.php One of the major complaints about the current synch rep patch is that signals are used for communication between backends and walsender. On some platforms, a signal doesn't interrupt sleep (i.e. poll or select system call), which wo

Re: [HACKERS] [PATCH] backend: compare word-at-a-time in bcTruelen

* Robert Haas (robertmh...@gmail.com) wrote: > As I look at this, another problem is that it seems to me that you're > assuming that VARDATA_ANY() will return an aligned pointer, which > isn't necessarily the case (see src/include/postgres.h). I believe you need to look at it more carefully. I do

Re: [HACKERS] machine-readable explain output

Hi, On 06/12/2009 07:15 AM, Robert Haas wrote: If you don't like the syntax, please argue about that on the "generic explain options v2" thread. Let's try to use this thread to discuss the output format, about which I spent a good deal of time agonizing. I spent some time playing around with th

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

(back from vacation) Tom Lane wrote: I wrote: Another interesting question is why successive vacuums aren't causing the index reltuples counts to go to zero. Shouldn't a partial vacuum result in *all* pages of the relation being marked as not needing to be examined by the next vacuum? I figu

Re: [HACKERS] question about meaning of character varying without length

Konstantin Izmailov wrote: Here you go: from Mail Delivery Subsystem > to pgf...@gmail.com dateMon, Jun 15, 2009 at 9:16 PM subject Delivery Status Notification (Failure) hide details 9:16 PM (31 minutes ago)