Re: [HACKERS] Foreign table permissions and cloning

2011-04-15 Thread Thom Brown
On 15 April 2011 04:26, Tom Lane wrote: > Why is this a documentation issue and not a code issue?  IMO we should > flat out reject both NOT NULL and DEFAULT declarations on foreign > tables, until such time as we're prepared to do something useful with > them.  Reasons: If the removal the redunda

[HACKERS] MMAP Buffers

2011-04-15 Thread Radosław Smogura
Hello, If I may, I want to share some concept to use mmap in PG. It's far, far away from perfect, but it's keeps WAL before data. As well I crated table, with index, inserted few values, and I done vacuum full on this table. Db inits welcome from orginal sources. Performance of read (if back

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Heikki Linnakangas
On 15.04.2011 13:32, Radosław Smogura wrote: If I may, I want to share some concept to use mmap in PG. It's far, far away from perfect, but it's keeps WAL before data. As well I crated table, with index, inserted few values, and I done vacuum full on this table. Db inits welcome from orginal sour

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Heikki Linnakangas
On 14.04.2011 23:02, Tom Lane wrote: Heikki Linnakangas writes: There's one very low-hanging fruit here, though. I profiled the pgbench case, with -M prepared, and found that like in Greg Smith's profile, hash_seq_search pops up quite high in the list. Those calls are coming from LockReleaseAll

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Radosław Smogura
On Fri, 15 Apr 2011 14:33:37 +0300, Heikki Linnakangas wrote: On 15.04.2011 13:32, Radosław Smogura wrote: If I may, I want to share some concept to use mmap in PG. It's far, far away from perfect, but it's keeps WAL before data. As well I crated table, with index, inserted few values, and I do

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Andrew Dunstan
On 04/15/2011 08:12 AM, Radosław Smogura wrote: The patch is quite hard to read because of random whitespace changes and other stylistic issues, but I have a couple of high-level questions on the design: Yes, but, hmm... in Netbeans I had really long gaps (probably 8 spaces, from tabs), so d

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Noah Misch
On Thu, Apr 14, 2011 at 05:10:41PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: > >> It shouldn't be > >> terribly difficult to come up with some kind of hash function based > >> on, say, the first two characters of the keyword that w

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread David Fetter
On Thu, Apr 14, 2011 at 11:16:03PM -0400, Tom Lane wrote: > David Fetter writes: > > On Thu, Apr 14, 2011 at 10:43:16AM -0400, Tom Lane wrote: > >> ... I think a lot of this ultimately traces to the extensible, > >> data-type-agnostic design philosophy. The fact that we don't > >> know what an in

Re: [HACKERS] Windows build issues

2011-04-15 Thread Magnus Hagander
On Thu, Apr 14, 2011 at 00:14, Brar Piening wrote: > On Wed, 13 Apr 2011 10:45:25 +0200, Magnus Hagander > wrote: >> >> So per your experience, all we really need to do is to define what the >> *max* level of the Windows SDK we can use is, to make sure people >> don't get the VS2010 compiler inst

Re: [HACKERS] Windows build issues

2011-04-15 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of vie abr 15 10:35:44 -0300 2011: > On Thu, Apr 14, 2011 at 00:14, Brar Piening wrote: > > The problem with VS 2010 (and the associated Windows SDK's) is the fact that > > MS dropped VCBuild.exe and now uses MSBuild.exe to build Visual C++ Projects > > so

Re: [HACKERS] Windows build issues

2011-04-15 Thread Magnus Hagander
On Fri, Apr 15, 2011 at 15:57, Alvaro Herrera wrote: > Excerpts from Magnus Hagander's message of vie abr 15 10:35:44 -0300 2011: >> On Thu, Apr 14, 2011 at 00:14, Brar Piening wrote: > >> > The problem with VS 2010 (and the associated Windows SDK's) is the fact >> > that >> > MS dropped VCBuild

Re: [HACKERS] WAL, xl_heap_insert and tuple oid mystry

2011-04-15 Thread Alvaro Herrera
Excerpts from jagan's message of jue abr 14 17:30:17 -0300 2011: > In general, what is the best way to unpack buffers containing tuples? > I came across "heap_form_tuple" but not sure if that is most optimal > way to go about. Is there some documentation on how tuples are > internally stored? Un

Re: [HACKERS] Foreign table permissions and cloning

2011-04-15 Thread Robert Haas
On Thu, Apr 14, 2011 at 8:26 PM, Tom Lane wrote: > 1. Accepting non-functional constraint declarations is something we've > been heard to ridicule mysql for.  With good reason. > > 2. It probably won't be too long before the planner makes optimization > decisions that assume NOT NULL declarations

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Tom Lane
Noah Misch writes: > On Thu, Apr 14, 2011 at 05:10:41PM -0400, Tom Lane wrote: >> FWIW, mysql used to use gperf for this purpose, but they've abandoned it >> in favor of some homegrown hashing scheme. > Do you have any details, like when mysql did this? With a quick look, I'm > failing to find c

Re: [HACKERS] [COMMITTERS] pgsql: Rename pg_regress option --multibyte to --encoding

2011-04-15 Thread Tom Lane
Peter Eisentraut writes: > Rename pg_regress option --multibyte to --encoding > Also refactor things a little bit so that the same methods for setting > test locale and encoding can be used everywhere. The buildfarm shows that this patch fails on Windows --- the contrib/unaccent regression test i

Re: [HACKERS] pg_dump --binary-upgrade vs. ALTER TYPE ... DROP ATTRIBUTE

2011-04-15 Thread Noah Misch
Robert, Thanks for the review. On Wed, Apr 13, 2011 at 08:01:17PM -0700, Robert Haas wrote: > I think you only need an AccessShareLock on InheritsRelationId, since > you are only selecting from it. True; fixed. > If we adopt the elsewhere-proposed approach of forbidding the use of > rowtypes to

Re: [HACKERS] [COMMITTERS] pgsql: Rename pg_regress option --multibyte to --encoding

2011-04-15 Thread Andrew Dunstan
On 04/15/2011 11:18 AM, Tom Lane wrote: Peter Eisentraut writes: Rename pg_regress option --multibyte to --encoding Also refactor things a little bit so that the same methods for setting test locale and encoding can be used everywhere. The buildfarm shows that this patch fails on Windows ---

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Joshua Berkus
Radoslaw, 10% improvement isn't very impressive from a switch to mmap. What workload did you test with? What I'd really like to see is testing with databases which are 50%, 90% and 200% the size of RAM ... that's where I'd expect the greatest gain from limiting copying. > Netbeans is possib

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Joshua Berkus
All, While it would be nice to improve our performance on this workload, let me point out that it's not a very important workload from the point of view of real performance challenges. Yes, there are folks out there with 100MB databases who only run one-liner select queries. But frankly, Val

Re: [HACKERS] WAL, xl_heap_insert and tuple oid mystry

2011-04-15 Thread jagan
Hi, > > In general, what is the best way to unpack buffers > > containing tuples? > Unpack buffers containing tuples?  You don't do > that.  If you have a page that comes from a table, you grab the tuple offset > from the line pointer, and start decoding there (probably > heap_deform_tuple is wh

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Joshua D. Drake
On 04/15/2011 09:59 AM, Joshua Berkus wrote: All, While it would be nice to improve our performance on this workload, let me point out that it's not a very important workload from the point of view of real performance challenges. Yes, there are folks out there with 100MB databases who only r

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Radosław Smogura
Joshua Berkus Friday 15 April 2011 18:55:04 > Radoslaw, > > 10% improvement isn't very impressive from a switch to mmap. What workload > did you test with? What I'd really like to see is testing with databases > which are 50%, 90% and 200% the size of RAM ... that's where I'd expect > the great

[HACKERS] Broken HOT chains in system catalogs

2011-04-15 Thread Tom Lane
I've been able to reproduce the syndrome reported here and here: http://archives.postgresql.org/pgsql-performance/2011-02/msg00497.php http://archives.postgresql.org/pgsql-general/2011-04/msg00487.php It's a bit tricky to do it reliably, but once you get into the right state it's reproducible. Yo

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Tom Lane
Joshua Berkus writes: > Certainly any sacrifice of functionality in order to be faster at that kind > of trivial workload would be foolhardy. Yeah, and I'd further say that any sacrifice of maintainability would be equally foolhardy. In particular I'm repelled by David's proposal of a whole par

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-15 Thread Kevin Grittner
Tom Lane wrote: > This amounts to assuming that no new indexes get added to system > catalogs after initdb, or at least not during concurrent > operations wherein indcheckxmin would be important. Sounds reasonable, but can we enforce it through locks rather than assuming, or isn't there a clea

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-15 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> This amounts to assuming that no new indexes get added to system >> catalogs after initdb, or at least not during concurrent >> operations wherein indcheckxmin would be important. > Sounds reasonable, but can we enforce it through locks rather than

Re: [HACKERS] Broken HOT chains in system catalogs

2011-04-15 Thread Kevin Grittner
Tom Lane wrote: > "Kevin Grittner" writes: >> can we enforce it through locks [?] > Well, we already enforce it through the allow_system_table_mods > mechanism: Good enough for me. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subsc

[HACKERS] regression test client encoding

2011-04-15 Thread Peter Eisentraut
Using pg_regress --encoding sets both the server encoding of the test database and the client encoding. (The choice of server encoding is further constrained by locale, but that's a different issue.) Looking at the expected variants of the pesky plpython_unicode test plpython_unicode.out

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Greg Smith
I did some more research into the memory allocation hotspots found in the profile, in regards to what MySQL has done in those areas. (And by "research" I mean "went to the bar last night and asked Baron about it") The issue of memory allocation being a limiter on performance has been nagging

Re: [HACKERS] regression test client encoding

2011-04-15 Thread Tom Lane
Peter Eisentraut writes: > What I'd suggest is that we take out the bit of code in pg_regress.c > that overrides the client encoding. That doesn't seem like a particularly good idea in view of the recent changes in psql to try to intuit a default encoding from its locale environment. If I say --

Re: [HACKERS] [COMMITTERS] pgsql: Rename pg_regress option --multibyte to --encoding

2011-04-15 Thread Peter Eisentraut
On Fri, 2011-04-15 at 12:28 -0400, Andrew Dunstan wrote: > Yeah, what is the point of this?: > > -# Adjust REGRESS_OPTS because we need a UTF8 database > -REGRESS_OPTS = --dbname=$(CONTRIB_TESTDB) --multibyte=UTF8 --no-locale > +# We need a UTF8 database +ENCODING = UTF8 +NO_LOCALE = 1

Re: [HACKERS] regression test client encoding

2011-04-15 Thread Peter Eisentraut
On Fri, 2011-04-15 at 16:09 -0400, Tom Lane wrote: > Peter Eisentraut writes: > > What I'd suggest is that we take out the bit of code in pg_regress.c > > that overrides the client encoding. > > That doesn't seem like a particularly good idea in view of the recent > changes in psql to try to intu

Re: [HACKERS] regression test client encoding

2011-04-15 Thread Tom Lane
Peter Eisentraut writes: > On Fri, 2011-04-15 at 16:09 -0400, Tom Lane wrote: >> That doesn't seem like a particularly good idea in view of the recent >> changes in psql to try to intuit a default encoding from its locale >> environment. If I say --encoding in the command line, that means I want

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-15 Thread Kevin Grittner
Greg Smith wrote: > I'm not sure exactly what is useful to be learned from that > specific work. But it does suggest two things: one, this is far > from an easy thing to fix. Two, the only reason MySQL does so > well on it is because there was some focused work on it, taking a > quite a while

Re: [HACKERS] WAL, xl_heap_insert and tuple oid mystry

2011-04-15 Thread Greg Stark
On Fri, Apr 15, 2011 at 6:29 PM, jagan wrote: > Thanks for the pointer. I will try that route of recreating tuple > descriptors and using heap_deform_tuple, although I suspect that > "heap_deform_tuple" > cannot be used by a stand-alone application (as it is not externally visible). > So, I will

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Greg Smith
Joshua Berkus wrote: Guys, can we *please* focus on the patch for now, rather than the formatting, which is fixable with sed? Never, and that's not true. Heikki was being nice; I wouldn't have even slogged through it long enough to ask the questions he did before kicking it back as unusa

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Robert Haas
On Apr 16, 2011, at 1:48 AM, Greg Smith wrote: > P.S. You know what else I feel should earn an automatic rejection without any > reviewer even looking at the code? Greg is absolutely right. And to the two he listed, let me add another of my own gripes: failing to provide submission notes that e

Re: [HACKERS] MMAP Buffers

2011-04-15 Thread Tom Lane
Greg Smith writes: > Reading one's own diff and reflecting on what you've changed is one of > the extremely underappreciated practices of good open-source software > development. Minimizing the size of that diff is perhaps the most > important thing someone can do in order to make their change