Re: [HACKERS] Deprecating RULES

2012-10-12 Thread Greg Stark
On Fri, Oct 12, 2012 at 7:55 AM, Simon Riggs wrote: > AFAICS all RULEs can be re-expressed as Triggers or Views. This is a bizarre discussion. Firstly this isn't even close to true. The whole source of people's discontentment is that triggers are *not* equivalent to rules. If they were then they

Re: [HACKERS] Truncate if exists

2012-10-12 Thread Greg Stark
On Tue, Oct 9, 2012 at 9:04 PM, Robert Haas wrote: > I've been a big proponent of adding "IF EXISTS" support to CREATE > TABLE and ALTER TABLE but I'm having a hard time getting excited about > this one. I can't imagine that many people would use it The reason CREATE IF NOT EXISTS and DROP IF EX

Re: [HACKERS] Successor of MD5 authentication, let's use SCRAM

2012-10-13 Thread Greg Stark
On Wed, Oct 10, 2012 at 11:41 AM, Heikki Linnakangas wrote: > 1. Salt length. Greg Stark calculated the odds of salt collisions here: > http://archives.postgresql.org/pgsql-hackers/2004-08/msg01540.php. It's not > too bad as it is, and as Greg pointed out, if you can eavesdrop it&

Re: [HACKERS] Potential autovacuum optimization: new tables

2012-10-14 Thread Greg Stark
On Sat, Oct 13, 2012 at 3:13 AM, Stephen Frost wrote: > Josh's concern is about autovacuum causing lots of stats churn, which is > understandable, we don't want it constantly rescanning a table I don't think rescanning the table is a big concern. autovacuum will only scan as often as it feels lik

Re: [HACKERS] Deprecating RULES

2012-10-14 Thread Greg Stark
On Sun, Oct 14, 2012 at 9:30 AM, Simon Riggs wrote: > On 12 October 2012 19:48, Greg Stark wrote: >> On Fri, Oct 12, 2012 at 7:55 AM, Simon Riggs wrote: >>> AFAICS all RULEs can be re-expressed as Triggers or Views. >> >> This is a bizarre discussion. Firstly

Re: [HACKERS] Deprecating RULES

2012-10-15 Thread Greg Stark
On Mon, Oct 15, 2012 at 8:00 AM, Simon Riggs wrote: > Please can anyone show me the SQL for a rule that cannot be written as > a view or a trigger? I do not believe such a thing exists and I will > provide free beer to the first person that can prove me wrong. Being written as a view doesn't help

Re: [HACKERS] Truncate if exists

2012-10-15 Thread Greg Stark
On Mon, Oct 15, 2012 at 3:26 PM, Robert Haas wrote: > To be perfectly frank, I think that's exactly where we ought to be > going. Oracle and Microsoft both did it, so why are we convinced it's > a bad idea? One of the huge problems with PL/pgsql is that every SQL > expression in there has to be

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-10-16 Thread Greg Stark
On Tue, Oct 16, 2012 at 9:47 PM, Peter Geoghegan wrote: > The patch will now been marked "ready for committer". Does this need > doc changes, in light of what is arguably a behavioural difference? > You only mentioned release notes. I'm happy to look at this one, probably next week at pgconf.eu.

Re: [HACKERS] [RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility

2012-10-17 Thread Greg Stark
On Wed, Oct 17, 2012 at 11:26 AM, Hannu Krosing wrote: > The simplest usage would be implementing "remote log tables" that is > tables, where you do INSERT on the master side, but it "inserts" only > a logical WAL record and nothing else. > > On subscriber side your replay process reads this WAL r

Re: [HACKERS] Deprecating RULES

2012-10-17 Thread Greg Stark
I dislike both of the explanations above which don't actually explain why people shouldn't use rules (Josh does say they're tricky which is a start). Just telling people we hate parts of the system doesn't really come off well and leaves them wondering why. I would suggest something like Warning:

Re: [HACKERS] Successor of MD5 authentication, let's use SCRAM

2012-10-22 Thread Greg Stark
On Sun, Oct 21, 2012 at 5:49 PM, Tom Lane wrote: > Magnus Hagander writes: >> I don't see a problem at all with providing the snakeoil cert. In >> fact, it's quite useful. > >> I see a problem with enabling it by default. Because it makes people >> think they are more secure than they are. > > I

Re: [HACKERS] ToDo: KNN Search should to support DISTINCT clasuse?

2012-10-22 Thread Greg Stark
On Mon, Oct 22, 2012 at 4:57 PM, Tom Lane wrote: > Don't hold your breath. There are two ways the system could implement > the DISTINCT clause: either sort and uniq, or hashaggregate. > hashaggregate will destroy any input ordering, so there's no value in > using the index as input. sort and uni

Re: [HACKERS] [RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility

2012-10-22 Thread Greg Stark
On Wed, Oct 17, 2012 at 7:48 PM, Christopher Browne wrote: > Well, replication is arguably a relevant case. > > For Slony, the origin/master node never cares about logged changes - that > data is only processed on replicas. Now, that's certainly a little weaselly > - the log data (sl_log_*) has g

Re: [HACKERS] Logical to physical page mapping

2012-10-28 Thread Greg Stark
On Sat, Oct 27, 2012 at 8:41 PM, Heikki Linnakangas wrote: > If the pointers are stored as simple 4-byte integers, you probably could > assume that they're atomic, and won't be torn. Actually I think any fixed-size data structure would be fine. We're worried about storage on disk here, not in mem

Re: [HACKERS] September 2012 commitfest

2012-10-30 Thread Greg Stark
On Mon, Oct 29, 2012 at 3:27 PM, Alvaro Herrera wrote: > * tuplesort memory usage: grow_memtuples > Greg Stark signed up for this I'll commit this later this week. I looked at it briefly at the conference but I think it actually does need some minor tweaks. > * Trim trailin

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-02 Thread Greg Stark
On Fri, Nov 2, 2012 at 1:19 AM, Greg Smith wrote: >> The idea at the time was to use the include *directory* functionality, >> for say a "config.d" directory in pgdata. The builtin one would then >> use a predictable filename in this directory, so that the DBA who >> prefers it can drop files both

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2012-11-15 Thread Greg Stark
On Thu, Nov 15, 2012 at 7:36 PM, Peter Geoghegan wrote: > On 15 November 2012 19:16, Robert Haas wrote: >> So what's next here? Do you want to work on these issue some more? >> Or does Jeff? I'd like to see this go in, but I'm not sure I have the >> bandwidth to do the legwork myself. > > I'll

Re: [HACKERS] Personal Copyright Notices

2010-02-02 Thread Greg Stark
So based on our discussion of last week my understanding is that as long as these people are content to release the code under the same license then these statements don't change anything since they're included in the Postgresql global development group anyways. greg On 2 Feb 2010 19:39, "Bruce M

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-03 Thread Greg Stark
On Tue, Feb 2, 2010 at 7:45 PM, Robert Haas wrote: > I think you're probably right, but it's not clear what the new name > should be until we have a comment explaining what the function is > responsible for. So I wrote some comments but wasn't going to repost the patch with the unchanged name wit

Re: [HACKERS] Personal Copyright Notices

2010-02-05 Thread Greg Stark
On Fri, Feb 5, 2010 at 3:24 AM, Bruce Momjian wrote: > Bruce Momjian wrote: >> The intagg copyright is on a _Makefile_: >> >>       # Makefile for integer aggregator >>       # Copyright (C) 2001 Digital Music Network. >>       # by Mark L. Woodward >>       # $PostgreSQL: pgsql/contrib/intagg/Mak

Re: [HACKERS] Strange heuristic in analyze.c

2010-02-08 Thread Greg Stark
On Fri, Feb 5, 2010 at 8:53 PM, Bruce Momjian wrote: > Do you want a C comment to document this problem? Well I would rather a better heuristic :) We really need some statistics nerds in this group who can pipe up when these kinds of issues come up. There must be a good way to estimate the proba

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-08 Thread Greg Stark
On Mon, Feb 8, 2010 at 4:53 AM, Robert Haas wrote: > On Sun, Feb 7, 2010 at 10:09 PM, Alvaro Herrera >> Yeah, it seems there are two patches here -- one is the addition of >> fsync_fname() and the other is the fsync_prepare stuff. Sorry, I'm just catching up on my mail from FOSDEM this past weeke

[HACKERS] Some belated patch review for "Buffers" explain analyze patch

2010-02-09 Thread Greg Stark
I was recently experimenting with explain analyze and I realized there are two things arguably wrong with the "Buffers" output in explain analyze: Firstly, it's printing out a number of buffers. We spent a lot of effort making all GUC variables use units of memory like "kB" and "MB" so the user sh

Re: [HACKERS] Some belated patch review for "Buffers" explain analyze patch

2010-02-09 Thread Greg Stark
On Tue, Feb 9, 2010 at 10:42 PM, Robert Haas wrote: >> A more important point is that it would be a nontrivial change, both as >> to code and documentation, and it's too late for such in 9.0.  So what >> we ought to be confining the discussion to right now is what 9.0 should >> print here. > > It'

Re: [HACKERS] Some belated patch review for "Buffers" explain analyze patch

2010-02-09 Thread Greg Stark
On Wed, Feb 10, 2010 at 12:32 AM, Tom Lane wrote: > The reason that EXPLAIN prints things the way it does is so that actual > costs/times are comparable to estimated costs. Oh, that was a thought I had along the way but forgot to mention in my email: since the buffer usage isn't related to the co

Re: [HACKERS] Some belated patch review for "Buffers" explain analyze patch

2010-02-10 Thread Greg Stark
On Wed, Feb 10, 2010 at 3:59 AM, Robert Haas wrote: > Yes.  We could add every bell and whistle imaginable to the text > format and it still would not begin to approach the verbosity of the > machine-readable formats.  Have you looked at them on a complex plan? > They are really, really long, and

Re: [HACKERS] Some belated patch review for "Buffers" explain analyze patch

2010-02-10 Thread Greg Stark
On Wed, Feb 10, 2010 at 1:26 PM, Robert Haas wrote: > For example, EXPLAIN (VERBOSE, FORMAT JSON) is often ridiculously > wide because each output list is printed on a single line. Perhaps this is just a terminology difference but it seems ridiculously *narrow* to me: QUERY PLAN -

Re: [HACKERS] knngist patch support

2010-02-11 Thread Greg Stark
On Thu, Feb 11, 2010 at 1:18 PM, Robert Haas wrote: > >> In my understanding >> this was always enough to submit code. User's documentation is depend on >> discussion and review and can be added later >> before releasing beta. > > Several people have said this lately, but it doesn't match what I'v

[HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL

2010-02-12 Thread Greg Stark
so I from by like having the server doing the cleanup because it down by necessarily have the while picture. it down nt know of it is the only replica reading these log files our if the site policy is to keep them for disaster recovery purposes. I like having this as an return val command though.

Re: [HACKERS] Streaming Replication docs

2010-02-13 Thread Greg Stark
On Fri, Feb 12, 2010 at 9:14 AM, Heikki Linnakangas wrote: > One glaring issue with the current documentation layout is that the > documentation for the various options in recovery.conf is split in two > places. I've been trying to explain to someone how to set this all up and they keep asking me

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Greg Stark
On Fri, Feb 12, 2010 at 3:49 PM, Robert Haas wrote: > Greg Stark, have you managed to get your access issues sorted out?  If Yep, will look at this today. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Greg Stark
On Sun, Feb 14, 2010 at 2:03 PM, Greg Stark wrote: > On Fri, Feb 12, 2010 at 3:49 PM, Robert Haas wrote: >> Greg Stark, have you managed to get your access issues sorted out?  If > > Yep, will look at this today. So I think we have a bigger problem than just copydir.c. It seems

Re: [HACKERS] Optimizing GetConflictingVirtualXIDs()

2010-02-14 Thread Greg Stark
On Sun, Feb 14, 2010 at 2:59 PM, Simon Riggs wrote: > Optimize GetConflictingVirtualXIDs() in roughly the same manner we > optimize TransactionIdIsInProgress(). > > Views? EINSUFFICIENTEXPLANATION :) -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Greg Stark
On Sun, Feb 14, 2010 at 8:57 PM, Robert Haas wrote: > On a pragmatic note, if this does turn out to be a problem, it's a > bug: and we can and do fix bugs whenever we discover them.  But the > other part of this patch - to speed up createdb - is a feature - and > we are very rapidly running out of

[HACKERS] Explain buffers display units.

2010-02-14 Thread Greg Stark
So this is what I did about my two complaints earlier about the explain buffer patch. a) Changed the line description to "Total Buffer Usage" which at least hints that it's something more akin to the "Total runtime" listed at the bottom than the "actual time". b) Used units of memory -- I formatt

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 10:02 AM, Andres Freund wrote: > Hi Marcin, > > Sounds rather unlikely to me. Its likely handled at an upper layer (vfs in > linux' case) and only overloaded when an optimized implementation is > available. > Which os do you see implementing that only on a part of the fil

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 11:34 AM, marcin mank wrote: > LOG:  could not link file "pg_xlog/xlogtemp.2367" to > "pg_xlog/0001" (initialization of log file 0, > This is not related -- it seems your filesystem doesn't support hard links. I thought we used "junctions" on versions o

[HACKERS] Regression failure on pika caused by CLUSTER rewrite

2010-02-15 Thread Greg Stark
In looking through the build farm wreckage caused by fsyncing directories I noticed this interesting failure on pika: == pgsql.13659/src/test/regress/regression.diffs === *** /home/pgbuildfarm/workdir/HEAD/pgsql.13659/src/test/regress/expected/cluster.out Wed F

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 11:50 AM, Andres Freund wrote: > If I understood him correctly marcin seems to mount a windows share on linux > via some vbox-proprietary pseudo filesystem. That wont get detected and thus > no junctions will be used... (I have doubts you even can create them via > vboxfs

Re: [HACKERS] Explain buffers display units.

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 2:22 PM, Robert Haas wrote: >> a) Changed the line description to "Total Buffer Usage" which at least >> hints that it's something more akin to the "Total runtime" listed at >> the bottom than the "actual time". >> >> b) Used units of memory -- I formatted them with 3 signi

Re: [HACKERS] Explain buffers display units.

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 6:05 PM, Robert Haas wrote: >> Well there was a 30+ message thread almost a week ago where there >> seemed to be some contention over the issue of whether the numbers >> should be averages or totals. But were there was no dispute over the >> idea of printing in memory units

Re: [HACKERS] Explain buffers display units.

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 7:58 PM, Robert Haas wrote: >>>  To me, buffers seem like discrete (and unitless) >>> entities, and we handle them that way elsewhere in the system (see, >>> e.g. pg_stat_database, pg_statio_all_tables).  I don't know that it's >>> a good idea to display that same informati

Re: [HACKERS] Explain buffers display units.

2010-02-16 Thread Greg Stark
On Tue, Feb 16, 2010 at 2:48 AM, Robert Haas wrote: > Multiplying by the block size makes it sound as if all the > memory was read or used, which is simply not the case - especially for > things like buffer hits, which don't actually read or allocate any > memory at all. In which case it represen

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-16 Thread Greg Stark
On Mon, Feb 15, 2010 at 7:51 PM, Jeroen Vermeulen wrote: > AFAIC a statement could go to "re-planning mode" if the shortest execution > time for the generic plan takes at least 10x longer than the longest > planning time.  That gives us a decent shot at finding statements where > re-planning is a

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-16 Thread Greg Stark
On Mon, Feb 15, 2010 at 7:11 PM, Bruce Momjian wrote: > 1. Why do we only do bind-level planning for anonymous wire-level queries? > > 2. I realize we did anonymous-only because that was the only way we had > in the protocol to _signal_ bind-time planning, but didn't we think of > this when we wer

[HACKERS] Re: [COMMITTERS] pgsql: Remove old-style VACUUM FULL (which was known for a little while

2010-02-16 Thread Greg Stark
On Tue, Feb 16, 2010 at 2:04 PM, Bruce Momjian wrote: > The MOVE_* bits go away after a while by vacuum and there is an easy > solution for 9.1 --- vacuum everything in 9.0.  Where things really get > hard is when we have to support two page formats or two data formats in > the same database.  You

Re: [HACKERS] Explain buffers display units.

2010-02-16 Thread Greg Stark
On Tue, Feb 16, 2010 at 3:54 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Greg Stark escribió: >>> Oops. Well, I would like to know if I'm in the minority and have to >>> roll this back before I fix that. > >> My personal opinion is that displaying n

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-16 Thread Greg Stark
On Tue, Feb 16, 2010 at 8:17 PM, Bruce Momjian wrote: >> Incidentally, can you have two active anonymous portals at the same time? > > No, the first one is deleted when the second is created, i.e., our docs > have: > >        An unnamed prepared statement lasts only until the next Parse statement

Re: [HACKERS] CommitFest Status Summary - 2010-02-14

2010-02-17 Thread Greg Stark
On Wed, Feb 17, 2010 at 6:39 AM, Tom Lane wrote: >> * Faster CREATE DATABASE by delaying fsync.  This is really two >> patches now, one of which is apparently to be backpatched. > > This one (both parts) seems to have crashed and burned on unexpected > portability issues :-(.  Do we have any expec

Re: [HACKERS] explain and PARAM_EXEC

2010-02-20 Thread Greg Stark
On Sat, Feb 20, 2010 at 4:33 AM, Tom Lane wrote: > It's really not much different from a function call with subplans as > functions. Perhaps it would be clearer to display the "(Subplan 1)" in a function call style format like Subplan1(b.oid) -- greg -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] scheduler in core

2010-02-20 Thread Greg Stark
On Sat, Feb 20, 2010 at 10:03 PM, Dimitri Fontaine wrote: > What would it take to have it included in core, so that it's not a > separate install to do? I'd love to have some support for running my > maintenance pl functions directly from the database. I mean without > installing, running and moni

Re: [HACKERS] parallelizing subplan execution (was: explain and PARAM_EXEC)

2010-02-21 Thread Greg Stark
On Sun, Feb 21, 2010 at 3:25 AM, Robert Haas wrote: > What kinds of things would be > sensible to hand off in this way?  Well, you'd want to find nodes that > are not likely to be repeatedly re-executed with different parameters, > like subplans or inner-indexscans, because otherwise you'll get >

Re: [HACKERS] A thought on Index Organized Tables

2010-02-22 Thread Greg Stark
On Mon, Feb 22, 2010 at 8:18 AM, Gokulakannan Somasundaram wrote: > a) IOT has both table and index in one structure. So no duplication of data > b) With visibility maps, we have three structures a) Table b) Index c) > Visibility map. So the disk footprint of the same data will be higher in > post

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-23 Thread Greg Stark
On Tue, Feb 23, 2010 at 5:37 AM, Tom Lane wrote: > So the problem is that fsync_fname is trying to fsync a file it's opened > O_RDONLY.  I don't know whether Windows is similarly picky, but we'll > soon find out. > Argh, now I feel silly. I had actually found that in my searches after the first b

[HACKERS] Assertion failure in walreceiver

2010-02-23 Thread Greg Stark
I tried to set up a simple master/slave setup and immediately ran into this assertion failure. The slave is just a cold copy of the database immediately after initdb. The first WAL segment hasn't been archived yet. It sees that the first archive fail hasn't been archived yet, starts up walreceiver

Re: [HACKERS] function side effects

2010-02-23 Thread Greg Stark
On Tue, Feb 23, 2010 at 4:52 PM, Kevin Grittner wrote: > Right, we all know it currently doesn't throw an error, but I can't > think of anywhere I'd like to have someone do that in a database for > which I have any responsibility.  Does anyone have a sane use case > for a non-volatile function to

Re: [HACKERS] function side effects

2010-02-23 Thread Greg Stark
On Tue, Feb 23, 2010 at 6:39 PM, Kevin Grittner wrote: >> Or somebody who uses the tsearch functions because they're >> planning to not change their dictionaries. > > I didn't realize tsearch functions were volatile.  Should they > really be so? Uhm, my mistake. They're stable. Ok, for that one I

Re: [HACKERS] Directory fsync and other fun

2010-02-23 Thread Greg Stark
On Wed, Feb 24, 2010 at 2:51 AM, Takahiro Itagaki wrote: > Also, I heard ext4 has a "feature" in that rename() might truncate the > renamed file to zero bytes on crash. The user data in the file might be > lost if the machine crashes just after rename(). In our case I think this is the one thing

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Greg Stark
On Wed, Feb 24, 2010 at 3:12 PM, Tom Lane wrote: > With an IOT I don't understand how you get out of index corruption > without data loss.  That's a showstopper for practical use, I think. That doesn't seem insurmountable to me. You could always allow a REINDEX to scan the index sequentially, ign

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Greg Stark
On Wed, Feb 24, 2010 at 4:12 PM, Gokulakannan Somasundaram wrote: > Sequential scans can be done on IOTs, just scan through the leaf pages. That doesn't work because when you split an index page any sequential scan in progress will either see the same tuples twice or will miss some tuples dependi

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Greg Stark
On Wed, Feb 24, 2010 at 5:46 PM, Tom Lane wrote: > "Kevin Grittner" writes: >> Greg Stark wrote: >>> That doesn't work because when you split an index page any >>> sequential scan in progress will either see the same tuples twice >>> or will

Re: [HACKERS] Assertion failure in walreceiver

2010-02-24 Thread Greg Stark
Did anyone see this? This seems like a pretty grave problem in streaming replication. On Tue, Feb 23, 2010 at 2:46 PM, Greg Stark wrote: > I tried to set up a simple master/slave setup and immediately ran into > this assertion failure. The slave is just a cold copy of the database > im

Re: [HACKERS] A thought on Index Organized Tables

2010-02-24 Thread Greg Stark
On Wed, Feb 24, 2010 at 8:04 PM, Gokulakannan Somasundaram wrote: > OK I think, i can think of a solution to achieve fast full index scan like > oracle. > a) Issue ids to every block that gets created inside the index. we are > already doing that > b) Now before the fast full index scan starts, no

Re: [HACKERS] Assertion failure in walreceiver

2010-02-25 Thread Greg Stark
On Thu, Feb 25, 2010 at 7:31 AM, Heikki Linnakangas wrote: > Committed removal of that and the assertion. You still can't use a copy > of the data directory taken right after initdb, because the initial > checkpoint record has the flag set indicating that archiving is not > enabled. While we're at

Re: [HACKERS] pg_stop_backup does not complete

2010-02-25 Thread Greg Stark
On Wed, Feb 24, 2010 at 11:14 PM, Josh Berkus wrote: > > Right.  I'm pointing out that production and "trying out 9.0 for the > first time" are actually different circumstances, and we need to be able > to handle both gracefully.  Since, if people have a bad experience > trying it out for the firs

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Greg Stark
On Thu, Feb 25, 2010 at 8:09 PM, Gokulakannan Somasundaram wrote: >   I think, somewhere things have been misunderstood. we only need 8 > bytes more per index entry. I thought Postgres has a 8 byte transaction id, > but it is only 4 bytes, so we only need to save the insertion and deletion

Re: [HACKERS] A thought on Index Organized Tables

2010-02-25 Thread Greg Stark
On Thu, Feb 25, 2010 at 9:25 PM, Tom Lane wrote: >  We've sweated > blood to get that struct down to where it is; there's no way to make it > smaller without giving up some really fundamental things, for example > the ability to do UPDATE :-( Oh, this is a tangent but I think there are some more

Re: [HACKERS] Assertion failure in walreceiver

2010-02-25 Thread Greg Stark
On Thu, Feb 25, 2010 at 7:31 AM, Heikki Linnakangas wrote: > While we're at it, the error message doesn't seem right: > > FATAL:  recovery connections cannot start because the > recovery_connections parameter is disabled on the WAL source server > > recovery_connections is on by default, the real

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 8:33 AM, Greg Smith wrote: > > I'm not sure what you might be expecting from the above combination, but > what actually happens is that many of the SELECT statements on the table > *that isn't even being updated* are canceled.  You see this in the logs: Well I proposed tha

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 4:47 AM, Tom Lane wrote: >> I feel the other one is easy. To store the hint bits inside the ItemId, in >> the place of size. > > No, we're not going there.  That breaks the fundamental page content > manipulation algorithms, and falls down for tuples not yet stored in a > p

Re: [HACKERS] pg_stop_backup does not complete

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 9:41 AM, Bernd Helmle wrote: > > > --On 24. Februar 2010 16:01:02 -0500 Tom Lane wrote: > >> One objection to this is that it's not very clear to the user when >> pg_stop_backup has finished with actual work and is just waiting for the >> archiver, ie when is it safe to hi

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 2:38 PM, Gokulakannan Somasundaram wrote: >   I have never written much code in C, and even if write it, i am > sure i will receive the comment that it is a unmaintainable code.(eg: Thick > index code and trailing nulls code) I definitely think thick indexes were t

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 4:43 PM, Richard Huxton wrote: > Let's see if I've got the concepts clear here, and hopefully my thinking it > through will help others reading the archives. > > There are two queues: I don't see two queues. I only see the one queue of operations which have been executed o

Re: [HACKERS] A thought on Index Organized Tables

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 6:30 PM, Gokulakannan Somasundaram wrote: > http://archives.postgresql.org/pgsql-hackers/2008-03/msg00682.php > I think, the buy-in became difficult because of the code quality. > Er, yeah. That's something we need to work on a bit. You should probably expect your first fe

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 7:16 PM, Tom Lane wrote: > I don't see a "substantial additional burden" there.  What I would > imagine is needed is that the slave transmits a single number back > --- its current oldest xmin --- and the walsender process publishes > that number as its transaction xmin in

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 8:30 PM, Tom Lane wrote: > How's it going to do that, when it has no queries at the instant > of startup? > Why shouldn't it have any queries at walreceiver startup? It has any xlog segments that were copied from the master and any it can find in the archive, it could easi

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 9:19 PM, Tom Lane wrote: > There's *definitely* not going to be enough information in the WAL > stream coming from a master that doesn't think it has HS slaves. > We can't afford to record all that extra stuff in installations for > which it's just useless overhead.  BTW, h

Re: [HACKERS] Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 11:56 PM, Greg Smith wrote: > This is also the reason why the whole "pause recovery" idea is a fruitless > path to wander down.  The whole point of this feature is that people have a > secondary server available for high-availability, *first and foremost*, but > they'd like

Re: [HACKERS] Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Sat, Feb 27, 2010 at 1:53 AM, Greg Smith wrote: > Greg Stark wrote: >> >> Well you can go sit in the same corner as Simon with your high >> availability servers. >> >> I want my ability to run large batch queries without any performance >> or r

Re: [HACKERS] Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Sat, Feb 27, 2010 at 2:43 AM, Greg Smith wrote: > > But if you're running the 8 hour report on the master right now, aren't you > already exposed to a similar pile of bloat issues while it's going?  If I > have the choice between "sometimes queries will get canceled" vs. "sometimes > the master

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-26 Thread Greg Stark
On Fri, Feb 26, 2010 at 9:44 PM, Tom Lane wrote: > Greg Stark writes: > >> What extra entries? > > Locks, just for starters.  I haven't read enough of the code yet to know > what else Simon added.  In the past it's not been necessary to record > any transient

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-27 Thread Greg Stark
On Sun, Feb 28, 2010 at 5:28 AM, Greg Smith wrote: > The idea of the workaround is that if you have a single long-running query > to execute, and you want to make sure it doesn't get canceled because of a > vacuum cleanup, you just have it connect back to the master to keep an open > snapshot the

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Stark
On Sun, Feb 28, 2010 at 6:07 AM, Greg Smith wrote: > Not forced to--have the option of.  There are obviously workloads where you > wouldn't want this.  At the same time, I think there are some pretty common > ones people are going to expect HS+SR to work on transparently where this > would obvious

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Greg Stark
On Sun, Feb 28, 2010 at 6:02 AM, Gokulakannan Somasundaram wrote: > So just with a addition of 8 bytes per tuple, we can have the snapshot > stored with the index. Can someone please comment on this? The transaction information on tuples take 18 bytes plus several info bits. It's possible just st

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Greg Stark
On Fri, Feb 26, 2010 at 4:01 AM, Robert Haas wrote: >> It's not going to be easier to implement.  Yeah, it would be easy to >> provide a global switch via a GUC setting, but that's not going to be >> helpful, because this is the sort of thing that really needs to be >> managed per-query.  Almost a

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-28 Thread Greg Stark
On Tue, Feb 23, 2010 at 3:38 PM, Tom Lane wrote: >> The plan I was thinking of was to pass a flag indicating if it's a >> directory to fsync_fname() and open it RD_ONLY if it's a directory and >> RDRW if it's a file. Then ignore any error returns (or at least EBADF >> and EINVAL) iff the flag indi

[HACKERS] Re: pgsql: add EPERM to the list of return codes to expect from opening

2010-03-01 Thread Greg Stark
This isn't working. The Windows ports are all saying "permission denied" but apparently that's not because errno is set to EPERM. Anyone know how to detect "permission denied" errors from open() on windows? On Mon, Mar 1, 2010 at 12:04 AM, Greg Stark wrote: >

Re: [HACKERS] Re: pgsql: add EPERM to the list of return codes to expect from opening

2010-03-01 Thread Greg Stark
So fwiw Narwhal says EACCESS is working. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-03-01 Thread Greg Stark
On Mon, Mar 1, 2010 at 5:50 PM, Josh Berkus wrote: > I don't think that defer_cleanup_age is a long-term solution.  But we > need *a* solution which does not involve delaying 9.0. So I think the primary solution currently is to raise max_standby_age. However there is a concern with max_standby_a

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-03-01 Thread Greg Stark
On Mon, Mar 1, 2010 at 7:21 PM, Josh Berkus wrote: > Completely aside from that, how many users are going to be happy with a > slave server which is constantly 5 minutes behind? > Uhm, well all the ones who are happy with our current warm standby setup for one? And all the ones who are looking f

[HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-03-01 Thread Greg Stark
josh, nobody is talking about it because it doesn't make sense. you could only retry if it was the first query in the transaction and only if you could prove there were no side-effects outside the database and then you would have no reason to think the retry would be any more likely to work. greg

Re: [HACKERS] Hung postmaster (8.3.9)

2010-03-02 Thread Greg Stark
We should probably also check and prohibit including directories as files. On Tuesday, March 2, 2010, Tom Lane wrote: > In the meantime, it seems like we ought to take two defensive steps: -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] pg_stop_backup does not complete

2010-03-02 Thread Greg Stark
On Tue, Mar 2, 2010 at 9:48 AM, Simon Riggs wrote: >> > Setting archive_mode to a command that does nothing but return true, e.g. >> > /bin/true, >> >> "return true" seems ambiguous for me. How about writing clearly >> "return a zero exit status" instead? > > Docs are already quite clear on that

Re: [HACKERS] Anyone know if Alvaro is OK?

2010-03-02 Thread Greg Stark
On Mon, Mar 1, 2010 at 11:11 PM, Chris Browne wrote: > Nobody really notices the carnage on the highways, because, > stochastically, there are such a large number of events, both positive > and negative (e.g. - millions of people making it home safely, and a > tiny number that don't) that it's dif

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Greg Stark
On Thu, Mar 4, 2010 at 12:11 PM, Fujii Masao wrote: > There is no post about this for over a month. Can I remove this > from TODO item of SR for 9.0? Thought? Objection? > Does smart shutdown still fail to shut down a slave? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Greg Stark
On Thu, Mar 4, 2010 at 3:56 PM, Robert Haas wrote: > On Thu, Mar 4, 2010 at 10:17 AM, Fujii Masao wrote: >> >> Yes. More precisely, smart shutdown during recovery does not complete >> until recovery ends. > > Well, I don't think we should let smart shutdown just never terminate > when standby_mod

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-03-10 Thread Greg Stark
On Wed, Mar 10, 2010 at 6:29 AM, Josh Berkus wrote: > Then I increased vacuum_defer_cleanup_age to 10, which represents > about 5 minutes of transactions on the test system.  This eliminated all > query cancels for the reporting query, which takes an average of 10s. > > Next is a database bloa

Re: [HACKERS] [patch] build issues on Win32

2010-03-11 Thread Greg Stark
2010/3/10 David Fetter : >> > --disable-shared, as previously mentioned. >> >> Oh.  Well, we don't really support that, and there is a proposal on >> the table to remove it altogether from the configure script.  I >> don't think we're going to contort our source code in order to make >> a marginal

Re: [HACKERS] operator exclusion constraints

2010-03-11 Thread Greg Stark
On Thu, Mar 11, 2010 at 5:29 AM, Tom Lane wrote: >> Indexes: >>     "foo_pkey" PRIMARY KEY, btree (f1), tablespace "ts1" >>     "foo_f2_exclusion" btree (f2), tablespace "ts1" >>     "foo_f3_exclusion" btree (f3) DEFERRABLE INITIALLY DEFERRED >> Exclusion constraints: >>     "foo_f2_exclusion" EXC

Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE

2010-03-11 Thread Greg Stark
On Wed, Mar 10, 2010 at 11:37 PM, Tom Lane wrote: > My conclusion is that this is probably a compiler bug.  Both buildfarm > animals appear to be using Sun Studio, although on different > architectures which weakens the compiler-bug theory a bit.  Even though > we are not seeing the failure in lat

<    2   3   4   5   6   7   8   9   10   11   >