Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
On Monday 08 February 2010 05:53:23 Robert Haas wrote: > On Sun, Feb 7, 2010 at 10:09 PM, Alvaro Herrera > > wrote: > > Andres Freund escribió: > >> I personally think the fsync on the directory should be added to the > >> stable branches - other opinions? > >> If wanted I can prepare patches for that. > > > > Yeah, it seems there are two patches here -- one is the addition of > > fsync_fname() and the other is the fsync_prepare stuff. > > Andres, you want to take a crack at splitting this up? Will do. Later today or tomorrow morning. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
On Mon, 2010-02-08 at 12:25 +1100, Andrew McNamara wrote: > For now, ocpgdb has no Python 3 support (I don't foresee any real > problems, however). Encoding issues are the big one. There are a couple gotchas, and I provided the details here: http://wiki.postgresql.org/wiki/Driver_development#Text_Encoding Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Largeobject Access Controls (r2460)
(2010/02/05 13:53), Takahiro Itagaki wrote: > > KaiGai Kohei wrote: > >>> default:both contents and metadata >>> --data-only:same >>> --schema-only: neither >> >> However, it means only large object performs an exceptional object class >> that dumps its owner, acl and comment even if --data-only is given. >> Is it really what you suggested, isn't it? > > I wonder we still need to have both "BLOB ITEM" and "BLOB DATA" > even if we will take the all-or-nothing behavior. Can we handle > BLOB's owner, acl, comment and data with one entry kind? The attached patch was a refactored one according to the suggestion. In the default or --data-only, it dumps data contents of large objects and its properties (owner, comment and access privileges), but it dumps nothing when --schema-only is given. default:both contents and metadata --data-only:same --schema-only: neither It replaced existing "BLOBS" and "BLOB COMMENTS" sections by the new "LARGE OBJECT" section which is associated with a certain large object. Its section header contains OID of the large object to be restored, so the pg_restore tries to load the specified large object from the given archive. _PrintTocData() handlers were modified to support the "LARGE OBJECT" section that loads the specified large object only, not whole of the archived ones like "BLOBS". It also support to read "BLOBS" and "BLOB COMMENTS" sections, but never write out these legacy sections any more. The archive file will never contain "blobs.toc", because we can find OID of the large objects to be restored in the section header, without any special purpose files. It also allows to omit _StartBlobs() and _EndBlobs() method in tar and file format. Basically, I like this approach more than the previous combination of "BLOB ITEM" and "BLOB DATA". However, we have a known issue here. The ACL section is categorized to REQ_SCHEMA in _tocEntryRequired(), so we cannot dump them when --data-only options, even if large object itself is dumped out. Of course, we can solve it with putting a few more exceptional treatments, although it is not graceful. However, it seems to me the matter comes from that _tocEntryRequired() can only returns a mask of REQ_SCHEMA and REQ_DATA. Right now, it is not easy to categorize ACL/COMMENT section into either of them. I think we should consider REQ_ACL and REQ_COMMENT to inform caller whether the appeared section to be dumped now, or not. Any idea? Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei pgsql-fix-pg_dump-blob-privs.5.patch Description: application/octect-stream -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
On Sun, Feb 7, 2010 at 10:09 PM, Alvaro Herrera wrote: > Andres Freund escribió: >> I personally think the fsync on the directory should be added to the stable >> branches - other opinions? >> If wanted I can prepare patches for that. > > Yeah, it seems there are two patches here -- one is the addition of > fsync_fname() and the other is the fsync_prepare stuff. Andres, you want to take a crack at splitting this up? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Knngist for 8.5
2010/2/7 Oleg Bartunov : > I understand your complaints. I think, the real problem is that some of us > live in the part of word with long holidays in December, while we in Russia > have very long holidays in January. So, about a month we couldn't > synchronize developers and reviewers. I'm not sure if we took this into > account. Yeah, that definitely made things harder. I had the feeling when I started looking at this stuff over Christmas that it was going to take a really determined and non-stop effort to get it all done, and we haven't quite had that, either on the reviewing end or on your end. Your holidays slowed things down, but we also had a quite small pool of round-robin reviewers for this CF, and I couldn't get anyone to sign on for knngist. Mark Cave-Ayland eventually volunteered but that was relatively late, and then he hasn't posted anything yet because he got involved in helping with rbtree (which by the way isn't quite done; we should really try to finish that up). So I think it was a combination of things. By the way, I wish I had your holiday schedule! Can you send me a few of those? > In regard to the knngist patch I want to claim, that I and Teodor are here > and willing to answer any questions. I really hope that Mark (or someone else) will post a review before this CommitFest is over. I believe it is out of reach to get this committed for this CF, but it would sure be nice to see it get at least some review. I would like to review it myself at some point, but I think right now I need to focus on things that are a little further along and have a better chance of getting in. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Feb 7, 2010 at 3:37 PM, Josh Berkus wrote: >> As between the two, I get the feeling that there is more interest in >> writeable CTEs. But that impression might be wrong, since it's an >> unscientific recollection of discussions on -hackers; which are >> themselves not representative of anything. > > Writeable CTE is definitely the bigger feature. Effectively, it allows > people to do in a single query data-transformation operations which > would have taken a stored procedure before. Think of it as comparable > to the introduction of callbacks in Perl for coolness. Now if I knew what callbacks in Perl were, I'd probably be impressed. You mean closures? >> I have not looked at the window functions patch at all, and I haven't >> looked at the latest version of writeable CTEs, either. I will try to >> spend some time on it in the next couple of days. My feeling about >> the last version is that it lacked a lot in the documentation >> department, and also in the comments department. Since I don't know >> that code very well, that made it hard for me to assess technical >> correctness. > > Hmmm, that's potentially lethal. David Fetter has been doing a lot of > presentations on the feature; surely he could turn them into some > documentation? David? I would be 100% in favor of some more help on the documentation. I do plan to reread this patch, but I don't know that I can cover the amount of work that needs to be done myself, and as you say, lack of adequate documentation could very well kill this patch. In fact, I'll go so far as to say it's one of the most likely reasons why this patch might not get in. So any resources we can bring to bear on that issue would be well spent. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Provide rowcount for utility SELECTs
On Sun, Feb 7, 2010 at 12:46 PM, Boszormenyi Zoltan wrote: >> 1. Looks like you've falsified the last comment block in PortalRunMulti(). > You mean the "fake something up" part? Will fix the comment. Yes. >> 2. I don't like the duplication of code in PortalRun() between the >> first and second branches of the switch statement. > The PORTAL_ONE_SELECT and PORTAL_ONE_RETURNING/PORTAL_UTIL_SELECT > cases differ only in that the latter case runs this below everything else: > if (!portal->holdStore) > FillPortalStore(portal, isTopLevel); > Would it be desired to unify these cases? This way there would be > no code duplication. Something like: > if (portal->strategy != PORTAL_ONE_SELECT && !portal->holdStore) > FillPortalStore(portal, isTopLevel); > ... (everything else) I thought about that but wasn't sure. I recommend that you give it a try that way and we'll see how it looks. >> 3. You've falsified the comment preceding that code, too. > > Or this one? > /* we know the query is supposed to set > the tag */ > if (completionTag && portal->commandTag) > ... > The condition and the comment still seems to be true. This is the one I was referring to. I guess "falsified" was too strong, but I don't think that comment describes the function of that code too well any more. Maybe it didn't before either, but I think it's worse now. >> 4. Is there any reason to use pg_strcasecmp() instead of plain old strcmp()? > I don't have any particular reason, strcmp() would do. OK, please change it. >> Someone who knows the overall structure of the code better than I do >> will have to comment on whether there are any code paths to worry >> about that do not go through PortalRun(). >> >> A general concern I have is that this what we're basically doing here >> is handling the most common case in ProcessQuery() and then installing >> fallback mechanisms to pick up any stragglers: but the fallback >> mechanisms only guarantee that we'll add a number to the command tag, >> not that it will be meaningful. Unfortunately, my limited imagination >> can't quite figure out in what cases we'll get a SELECT command tag >> back other than (1) plain old SELECT, (2) SELECT INTO, and (3) CTAS, >> so I'm not sure what to go test. Any thoughts on these issues, anyone? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Feb 7, 2010 at 4:54 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> On Sun, Feb 7, 2010 at 4:03 PM, Dimitri Fontaine >> wrote: >>> In case I'm not clear, what I'm saying is that I think we can consider >>> the writable CTE patch ready for commit even though we still have to >>> decide what its impacts on documentation should be. >> >> Whether a patch is ready to commit will be up to the committer > > "Ready for Committer" is what I though but failed to type. *shrug* Same issue, to some degree. For a patch of this size, the difference between "Needs Review" and "Ready for Committer" is maybe somewhat less than normal. My point is just that I think there is work that can be usefully done on this patch by people other than Tom, even though I believe that ultimately Tom will have to make the call on whether it goes in. I don't think that should cause Tom to put off looking at it himself, but neither do I think that anyone else should feel like we've accomplished something by labelling it Ready for Committer. I'm disappointed that we marked this RfC so early in the cycle without catching the docs issue; Marko could have started working on that much sooner if we'd given him that feedback. Let's not take our eye off the ball again. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Andres Freund escribió: > I personally think the fsync on the directory should be added to the stable > branches - other opinions? > If wanted I can prepare patches for that. Yeah, it seems there are two patches here -- one is the addition of fsync_fname() and the other is the fsync_prepare stuff. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bugs in b-tree dead page removal
I wrote: > And there's another problem: _bt_pagedel is designed to recurse > in certain improbable cases, but I think this is flat out wrong > when doing WAL replay --- if the original process did recurse > then it will have emitted a WAL record for each deleted page, > meaning replay would try to delete twice. No, scratch that, I misread it: _bt_pagedel isn't invoked during WAL replay, but for cleanup of incomplete deletions at termination of WAL replay. So any recursing it has to do also corresponds to actions that weren't in WAL. So that's OK. I'm still concerned about the interlock against read-only transactions though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Bugs in b-tree dead page removal
Whilst looking around for stuff that could be deleted thanks to removing old-style VACUUM FULL, I came across some code in btree that seems rather seriously buggy. For reasons explained in nbtree/README, we can't physically recycle a "deleted" btree index page until all transactions open at the time of deletion are gone --- otherwise we might re-use a page that an existing scan is about to land on, and confuse that scan. (This condition is overly strong, of course, but it's what's designed in at the moment.) The way this is implemented is to label a freshly-deleted page with the current value of ReadNewTransactionId(). Once that value is older than RecentXmin, the page is presumed recyclable. I think this was all right when it was designed, but isn't it rather badly broken by our subsequent changes to have transactions not take out an XID until/unless they write something? A read-only transaction could easily be much older than RecentXmin, no? The odds of an actual problem seem not very high, since to be affected a scan would have to be already "in flight" to the problem page when the deletion occurs. By the time RecentXmin advances and we feed the page to the FSM and get it back, the scan's almost surely going to have arrived. And I think the logic is such that this would not happen before the next VACUUM in any case. Still, it seems pretty bogus. Another issue is that it's not clear what happens in a Hot Standby slave --- it doesn't look like Simon put any interlocking in this area to protect slave queries against having the page disappear from under them. The odds of an actual problem are probably a good bit higher in an HS slave. And there's another problem: _bt_pagedel is designed to recurse in certain improbable cases, but I think this is flat out wrong when doing WAL replay --- if the original process did recurse then it will have emitted a WAL record for each deleted page, meaning replay would try to delete twice. That last problem is easy to fix, but I'm not at all sure what to do about the scan interlock problem. Thoughts? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
On Sunday 07 February 2010 19:27:02 Andres Freund wrote:
> On Sunday 07 February 2010 19:23:10 Robert Haas wrote:
> > On Sun, Feb 7, 2010 at 11:24 AM, Tom Lane wrote:
> > > Greg Smith writes:
> > >> This is turning into yet another one of those situations where
> > >> something simple and useful is being killed by trying to generalize
> > >> it way more than it needs to be, given its current goals and its lack
> > >> of external interfaces. There's no catversion bump or API breakage
> > >> to hinder future refactoring if this isn't optimally designed
> > >> internally from day one.
> > >
> > > I agree that it's too late in the cycle for any major redesign of the
> > > patch. But is it too much to ask to use a less confusing name for the
> > > function?
> >
> > +1. Let's just rename the thing, add some comments, and call it good.
>
> Will post a updated patch in the next hours unless somebody beats me too
> it.
Here we go.
I left the name at my suggestion pg_fsync_prepare instead of Tom's
prepare_for_fsync because it seemed more consistend with the naming in the
rest of the file. Obviously feel free to adjust.
I personally think the fsync on the directory should be added to the stable
branches - other opinions?
If wanted I can prepare patches for that.
Andres
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index 7ffa2eb..bc5753a 100644
*** a/src/backend/storage/file/fd.c
--- b/src/backend/storage/file/fd.c
*** pg_fdatasync(int fd)
*** 320,325
--- 320,361
}
/*
+ * pg_fsync_prepare --- try to make a later fsync on the same file faster
+ *
+ * A call to this function does not guarantee anything!
+ *
+ * The idea is to tell the kernel to write out its cache so that a
+ * fsync later on has less to write out synchronously. This allows
+ * that write requests get reordered more freely.
+ *
+ * In the current implementation this has the additional effect of
+ * dropping the cache in that region and thus can be used to avoid
+ * cache poisoning. This may or may not be wanted.
+ *
+ * XXX: Ideally this API would use sync_file_range (or similar on
+ * platforms other than linux) and a seperate one for cache
+ * control. We are not there yet.
+ *
+ * Look at the thread below 200912282354.51892.and...@anarazel.de in
+ * pgsql-hackers for a longer discussion.
+ */
+ int
+ pg_fsync_prepare(int fd, off_t offset, off_t amount)
+ {
+ if (enableFsync)
+ {
+ #if defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_DONTNEED)
+ return posix_fadvise(fd, offset, amount, POSIX_FADV_DONTNEED);
+ #else
+ return 0;
+ #endif
+ }
+ else
+ return 0;
+ }
+
+
+ /*
* InitFileAccess --- initialize this module during backend startup
*
* This is called during either normal or standalone backend start.
diff --git a/src/include/storage/fd.h b/src/include/storage/fd.h
index 21cb024..b1a4b49 100644
*** a/src/include/storage/fd.h
--- b/src/include/storage/fd.h
*** extern int pg_fsync_no_writethrough(int
*** 99,104
--- 99,106
extern int pg_fsync_writethrough(int fd);
extern int pg_fdatasync(int fd);
+ extern int prepare_for_fsync(int fd, off_t offset, off_t amount);
+
/* Filename components for OpenTemporaryFile */
#define PG_TEMP_FILES_DIR "pgsql_tmp"
#define PG_TEMP_FILE_PREFIX "pgsql_tmp"
diff --git a/src/port/copydir.c b/src/port/copydir.c
index 72fbf36..eef3cfb 100644
*** a/src/port/copydir.c
--- b/src/port/copydir.c
***
*** 37,42
--- 37,43
static void copy_file(char *fromfile, char *tofile);
+ static void fsync_fname(char *fname);
/*
*** copydir(char *fromdir, char *todir, bool
*** 64,69
--- 65,73
(errcode_for_file_access(),
errmsg("could not open directory \"%s\": %m", fromdir)));
+ /*
+ * Copy all the files
+ */
while ((xlde = ReadDir(xldir, fromdir)) != NULL)
{
struct stat fst;
*** copydir(char *fromdir, char *todir, bool
*** 89,96
--- 93,127
else if (S_ISREG(fst.st_mode))
copy_file(fromfile, tofile);
}
+ FreeDir(xldir);
+
+ /*
+ * Be paranoid here and fsync all files to ensure we catch problems.
+ */
+ xldir = AllocateDir(fromdir);
+ if (xldir == NULL)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open directory \"%s\": %m", fromdir)));
+
+ while ((xlde = ReadDir(xldir, fromdir)) != NULL)
+ {
+ if (strcmp(xlde->d_name, ".") == 0 ||
+ strcmp(xlde->d_name, "..") == 0)
+ continue;
+ snprintf(tofile, MAXPGPATH, "%s/%s", todir, xlde->d_name);
+ fsync_fname(tofile);
+ }
FreeDir(xldir);
+
+ /* It's important to fsync the destination directory itself as
+ * individual file fsyncs don't guarantee that the directory entry
+ * for the file is synced. Recent versions of ext4 have made the
+ * window much wider but it's been true for ext3 and other
+ * filesyetems in the past
+ */
+ fsync_fname(todir);
}
/*
Re: [HACKERS] Confusion over Python drivers
>I added you into the list at http://wiki.postgresql.org/wiki/Python Thanks. >Can you check what I put in there, confirm Windows compatibility, and >comment on Python 3.X support? I haven't tried it under Windows and I haven't had any feedback either way from Windows users. For now, ocpgdb has no Python 3 support (I don't foresee any real problems, however). >I'd be curious to hear more about the escaping bugs you ran into as well. >We already have some notes on the TODO that pushing more of this work >toward the standard libpq routines would seem appropriate for things >passing between the driver and libpq. Were the issues you ran into on >that side, or more on the Python side of how things were being formatted? It was a while ago now and I can't remember the specific details - it was more a general feeling that the existing offerings were going about it the wrong way (with respect to parameter passing and escaping). I suspect this was a historical artifact (presumably libpq didn't provide escaping facilities or parameterised queries when the adapters were written). Essentially, I just wanted a pyPgSQL with a more modern implementation. Psycopg was (is?) also using Protocol 2. I felt that the way forward was to switch to the Protocol 3 API features, in particular, parameterised queries, and none of the existing Python adapters had done that (I got the impression while writing my module that nobody was exercising the new features). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add on_plperl_init and on_plperlu_init to plperl UPDATE 3 [PATCH]
Tim Bunce wrote: This is the third update to the fourth of the patches to be split out from the former 'plperl feature patch 1'. Changes in this patch: - Added plperl.on_plperl_init and plperl.on_plperlu_init GUCs Both are PGC_SUSET SPI functions are not available when the code is run. Errors are detected and reported as ereport(ERROR, ...) Corresponding documentation and tests for both. - Renamed plperl.on_perl_init to plperl.on_init - Improved state management of select_perl_context() An error during interpreter initialization will leave the state (interp_state etc) unchanged. - The utf8fix code has been greatly simplified. - More code comments re PGC_SUSET and no access to SPI functions. The docs on this patch need some cleaning up and expanding: * The possible insecurity of %_SHARED needs expanding. * The docs still refer to plperl.on_untrusted_init * plperl.on_plperl_init and plperl.on_plperlu_init can be documented together rather than repeating the same stuff almost word for word. * extra examples for these two, and an explanation of why one might want to use each of the three on*init settings would be good. I'll continue reviewing the patch, but these things at least need fixing. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Backup history file should be replicated in Streaming Replication?
On Sun, Feb 7, 2010 at 1:02 AM, Bruce Momjian wrote:
>> src/backend/access/transam/xlog.c
>> > else
>> > {
>> > XLogRecPtr InvalidXLogRecPtr = {0, 0};
>> > ControlFile->minRecoveryPoint = InvalidXLogRecPtr;
>> > }
>>
>> In my original patch, the above is for the problem discussed in
>> http://archives.postgresql.org/pgsql-hackers/2009-12/msg02039.php
>>
>> Since you've already fixed the problem, that code is useless.
>> How about getting rid of that code?
>
> Has this been addressed?
No. We need to obtain the comment about that from Heikki.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
Andrew McNamara wrote: I got sick of the constant stream of escaping bugs impacting on psycopg and pyPgSQL, and wrote my own DB-API driver, using the more modern libpq/binary/protocol 3 APIs where ever possible. The result is BSD licensed: http://code.google.com/p/ocpgdb/ I added you into the list at http://wiki.postgresql.org/wiki/Python Can you check what I put in there, confirm Windows compatibility, and comment on Python 3.X support? I'd be curious to hear more about the escaping bugs you ran into as well. We already have some notes on the TODO that pushing more of this work toward the standard libpq routines would seem appropriate for things passing between the driver and libpq. Were the issues you ran into on that side, or more on the Python side of how things were being formatted? -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
>Any other suggestions before I turn the above into a roadmap page on the >wiki? I got sick of the constant stream of escaping bugs impacting on psycopg and pyPgSQL, and wrote my own DB-API driver, using the more modern libpq/binary/protocol 3 APIs where ever possible. The result is BSD licensed: http://code.google.com/p/ocpgdb/ As well as using the newer APIs, I have attempted to keep the code as simple as possible, eschewing things like threading as adding too much complexity for too little gain (particularly true of Python threading), and I kept to just the code DB-API functionality. The C code exists mainly to present a pythonic view of libpq. I found that type conversion and marshalling could generally be done from python with more than acceptable performance (via the C-coded "struct" module in the standard library for common types). In my tests, ocpgdb has performed at least as well as pyPgSQL and psycopg, often a lot better, primarily due to the use of the libpq binary protocols, I think. I'm not proposing my module as your canonical implementation, although you're welcome to it if you like. Rather, it demonstrates another viable approach, minimal, and using newer libpq APIs. BTW, with respect to the discussion of the Python DB-API - I see it as specifying a lowest-common-denominator, or the subset of functionality that should be available from most databases without requiring contortions. Like eating at McDonalds, it does the job, but it's never going to delight or surprise. A PostGreSQL blessed adapter really should provide access to all the features in libpq, and I'm not sure this is directly compatible with DBAPI. Instead, the DBAPI-compliance should be layered on top. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, 7 Feb 2010, Robert Haas wrote: On Sun, Feb 7, 2010 at 1:38 AM, Josh Berkus wrote: I think it might be time to revisit this issue. SR is in, and we have a week left in the CF, and we have all of the above patches plus 5 small ones left to deal with. rbtree is close to being committable, I think; knngist has not been reviewed yet; you (Tom) have claimed the frame options patch but I haven't seen any update on it in a while; I doubt either of the other two are ready to commit but I'm not sure how far they have to go. I think, as previously discussed, that we should bounce knngist. It's a complex patch and nobody saw anything of it until Jan 15, so I don't feel bad about it. Mark Cave-Ayland was going to review it, but apparently felt that rbtree was the higher priority. Hey, I'm lost here, when we previously discussed, that knngist should be rejected ? knngist is a legal patch, submitted in time (and discussed in -hackers) and it's not our fault, people are busy doing other reviews. Knngist has some prerequisites, rbtree, for example, and it took a while, but now, when we're close to commit rbtree, people can review knngist. rbtree is a prerequisite for knngist. point_ops was too. I think we're going to get rbtree done yet, but the main patch seems out of reach. There's no way it's going to go in without both pre-commit and post-commit changes, and I don't think we have time for that now. We have a week to get review and do possible required pre-commit changes. Mark, you can download test data for knngist from http://www.sai.msu.su/~megera/wiki/2009-11-25 Regards, Oleg _ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Precedence of Postfix operators
Thanks Tom, for the explanation. Gokul. On Sun, Feb 7, 2010 at 10:14 PM, Tom Lane wrote: > Gokulakannan Somasundaram writes: > >Is there any reason why we have given lesser precedence for postfix > > operator compared to multiplication/division? Usually postfix operators > have > > more precedence than the binary operations. Is this some kind of work > around > > to provide user-defined operators? Can someone help me understand this? > > A bit of poking in the CVS logs for gram.y reveals > > 2001-01-23 17:39 tgl > >* src/backend/parser/gram.y: Give 'a_expr ::= a_expr Op' production >a slightly lower precedence than Op, so that the sequence 'a_expr >Op Op a_expr' will be parsed as a_expr Op (Op a_expr) not (a_expr >Op) Op a_expr as formerly. In other words, prefer treating >user-defined operators as prefix operators to treating them as >postfix operators, when there is an ambiguity. Also clean up a >couple of other infelicities in production priority assignment --- >for example, BETWEEN wasn't being given the intended priority, but >that of AND. > > There are several other nasty things that we've had to do in order to > keep supporting postfix operators at all. I thibk most people view them > as a legacy feature best avoided. > >regards, tom lane >
Re: [HACKERS] damage control mode
Robert Haas writes: > On Sun, Feb 7, 2010 at 4:03 PM, Dimitri Fontaine > wrote: >> In case I'm not clear, what I'm saying is that I think we can consider >> the writable CTE patch ready for commit even though we still have to >> decide what its impacts on documentation should be. > > Whether a patch is ready to commit will be up to the committer "Ready for Committer" is what I though but failed to type. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Feb 7, 2010 at 4:03 PM, Dimitri Fontaine wrote: > Marko Tiikkaja writes: >> The documentation has definitely improved from the last time Robert >> looked at it, but I fear it still needs some more work. I'm willing to >> do that work, but I need something concrete. > > It seems to me documentation is required to get into the source tree > before beta, and as we see with some other patches it's definitely the > case even with our newer procedures that some code gets in without its > documentation properly finished. I guess this amounts to the commiter > willing to fill up the docs later on. > > But here it's even better as we have the author willing to stay there > and write needed documentation as soon as community agrees on what that > is. > > In case I'm not clear, what I'm saying is that I think we can consider > the writable CTE patch ready for commit even though we still have to > decide what its impacts on documentation should be. Whether a patch is ready to commit will be up to the committer, and I am doubtful that anyone other than Tom is qualified to do this one. Others may feel differently, of course. The rest of us should focus our effort on improving the patch, rather than arguing about whether we would commit it as is. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby documentation
On Feb 7, 2010, at 12:35 PM, Josh Berkus wrote: >> I've always thought this feature was misnamed and nothing has happened >> to change my mind, but it's not clear whether I'm in the majority. > > I'm afraid force of habit is more powerful than correctness on this one. > It's going to be HS/SR whether that's perfectly correct or not. What would be correct? I thought HS/SR were pretty correct (as long as no one confuses SR with synchronous replication!). Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
Marko Tiikkaja writes: > The documentation has definitely improved from the last time Robert > looked at it, but I fear it still needs some more work. I'm willing to > do that work, but I need something concrete. It seems to me documentation is required to get into the source tree before beta, and as we see with some other patches it's definitely the case even with our newer procedures that some code gets in without its documentation properly finished. I guess this amounts to the commiter willing to fill up the docs later on. But here it's even better as we have the author willing to stay there and write needed documentation as soon as community agrees on what that is. In case I'm not clear, what I'm saying is that I think we can consider the writable CTE patch ready for commit even though we still have to decide what its impacts on documentation should be. There has to be a difference between last alpha and first beta, right? Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On 2010-02-07 22:37 +0200, Josh Berkus wrote: > Robert, >> I have not looked at the window functions patch at all, and I haven't >> looked at the latest version of writeable CTEs, either. I will try to >> spend some time on it in the next couple of days. My feeling about >> the last version is that it lacked a lot in the documentation >> department, and also in the comments department. Since I don't know >> that code very well, that made it hard for me to assess technical >> correctness. > > Hmmm, that's potentially lethal. David Fetter has been doing a lot of > presentations on the feature; surely he could turn them into some > documentation? David? The documentation has definitely improved from the last time Robert looked at it, but I fear it still needs some more work. I'm willing to do that work, but I need something concrete. Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Small Bug in pgstat display during recovery conflict resolution
Hi Simon, Hi all,
if (!logged && (wait_s > 0 || wait_us > 50))
{
const char *oldactivitymsg;
int len;
oldactivitymsg = get_ps_display(&len);
snprintf(waitactivitymsg, sizeof(waitactivitymsg),
"waiting for max_standby_delay (%u s)",
MaxStandbyDelay);
set_ps_display(waitactivitymsg, false);
if (len > 100)
len = 100;
memcpy(waitactivitymsg, oldactivitymsg, len);
pgstat_report_waiting(true);
logged = true;
}
..
if (logged)
{
set_ps_display(waitactivitymsg, false);
pgstat_report_waiting(false);
}
That doesnt work because get_ps_display returns the internal buffer. This
leads to the situation that after conflict resolution the
"waiting for max_standby_delay ..."
message is displayed until the next segment starts where its replaced
again by the
"... recovering ..." line.
Additionally the old code may print unintialized memory if get_ps_display
returns a string without a \0 terminator.
The attached patch fixes that.
Andres
From a0198eab28552b9af9a70298f49a17348077c6fd Mon Sep 17 00:00:00 2001
From: Andres Freund
Date: Sun, 7 Feb 2010 19:52:07 +0100
Subject: [PATCH] The stat reporting in ResolveRecoveryConflictWithVirtualXIDs had a
small bug - it tried to use the buffer returned by get_ps_display() as
temp. storage - which does not work.
---
src/backend/storage/ipc/standby.c | 17 +++--
1 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index 63a9c68..995a2dc 100644
*** a/src/backend/storage/ipc/standby.c
--- b/src/backend/storage/ipc/standby.c
*** ResolveRecoveryConflictWithVirtualXIDs(V
*** 161,166
--- 161,167
ProcSignalReason reason)
{
char waitactivitymsg[100];
+ char oldactivitymsg[101];
while (VirtualTransactionIdIsValid(*waitlist))
{
*** ResolveRecoveryConflictWithVirtualXIDs(V
*** 183,199
TimestampDifference(waitStart, now, &wait_s, &wait_us);
if (!logged && (wait_s > 0 || wait_us > 50))
{
! const char *oldactivitymsg;
int len;
! oldactivitymsg = get_ps_display(&len);
snprintf(waitactivitymsg, sizeof(waitactivitymsg),
"waiting for max_standby_delay (%u s)",
MaxStandbyDelay);
set_ps_display(waitactivitymsg, false);
- if (len > 100)
- len = 100;
- memcpy(waitactivitymsg, oldactivitymsg, len);
pgstat_report_waiting(true);
--- 184,204
TimestampDifference(waitStart, now, &wait_s, &wait_us);
if (!logged && (wait_s > 0 || wait_us > 50))
{
! const char *oldactivitymsgp;
int len;
! oldactivitymsgp = get_ps_display(&len);
!
! if (len > 100)
! len = 100;
!
! memcpy(oldactivitymsg, oldactivitymsgp, len);
! oldactivitymsg[len] = 0;
!
snprintf(waitactivitymsg, sizeof(waitactivitymsg),
"waiting for max_standby_delay (%u s)",
MaxStandbyDelay);
set_ps_display(waitactivitymsg, false);
pgstat_report_waiting(true);
*** ResolveRecoveryConflictWithVirtualXIDs(V
*** 223,229
/* Reset ps display */
if (logged)
{
! set_ps_display(waitactivitymsg, false);
pgstat_report_waiting(false);
}
--- 228,234
/* Reset ps display */
if (logged)
{
! set_ps_display(oldactivitymsg, false);
pgstat_report_waiting(false);
}
--
1.6.5.12.gd65df24
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
Robert, > As between the two, I get the feeling that there is more interest in > writeable CTEs. But that impression might be wrong, since it's an > unscientific recollection of discussions on -hackers; which are > themselves not representative of anything. Writeable CTE is definitely the bigger feature. Effectively, it allows people to do in a single query data-transformation operations which would have taken a stored procedure before. Think of it as comparable to the introduction of callbacks in Perl for coolness. > I have not looked at the window functions patch at all, and I haven't > looked at the latest version of writeable CTEs, either. I will try to > spend some time on it in the next couple of days. My feeling about > the last version is that it lacked a lot in the documentation > department, and also in the comments department. Since I don't know > that code very well, that made it hard for me to assess technical > correctness. Hmmm, that's potentially lethal. David Fetter has been doing a lot of presentations on the feature; surely he could turn them into some documentation? David? --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby documentation
> I've always thought this feature was misnamed and nothing has happened > to change my mind, but it's not clear whether I'm in the majority. I'm afraid force of habit is more powerful than correctness on this one. It's going to be HS/SR whether that's perfectly correct or not. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
Josh Berkus wrote: Anyway, I don't yet have a full diagnosis on the transaction control issue or I'd already have posted it to psycopg -- it may be a toxic interaction between Django and Psycopg2 rather than psycopg2 alone. I'd not have brought it up except for this discussion. I'm going to remove it from the list on the wiki then for now. I don't want to annoy the developers by adding a more speculative bug that might not even be in their software. If you get to where it's confirmed and info posted to their list, please add a link back into the page once it's reported, i.e. link to their mailing list archives or something like that. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Knngist for 8.5
Robert, I understand your complaints. I think, the real problem is that some of us live in the part of word with long holidays in December, while we in Russia have very long holidays in January. So, about a month we couldn't synchronize developers and reviewers. I'm not sure if we took this into account. In regard to the knngist patch I want to claim, that I and Teodor are here and willing to answer any questions. Oleg PS. I changed subject not to interfere with other topics. On Sun, 7 Feb 2010, Robert Haas wrote: 2010/2/7 Oleg Bartunov : On Sun, 7 Feb 2010, Robert Haas wrote: On Sun, Feb 7, 2010 at 1:38 AM, Josh Berkus wrote: I think it might be time to revisit this issue. SR is in, and we have a week left in the CF, and we have all of the above patches plus 5 small ones left to deal with. rbtree is close to being committable, I think; knngist has not been reviewed yet; you (Tom) have claimed the frame options patch but I haven't seen any update on it in a while; I doubt either of the other two are ready to commit but I'm not sure how far they have to go. I think, as previously discussed, that we should bounce knngist. It's a complex patch and nobody saw anything of it until Jan 15, so I don't feel bad about it. Mark Cave-Ayland was going to review it, but apparently felt that rbtree was the higher priority. Hey, I'm lost here, when we previously discussed, that knngist should be rejected ? Huh? Have you been reading -hackers for the last month? I first raised this issue on December 30th, and there has been lots more discussion of it since then. http://archives.postgresql.org/pgsql-hackers/2009-12/msg02329.php knngist is a legal patch, submitted in time (and discussed in -hackers) and it's not our fault, people are busy doing other reviews. I never said anything about fault. If there's not enough time to get something committed, then there isn't. That's not a punishment; it's just something that sometimes happens to patches submitted near the end of the cycle. We've been openly discussing this problem on -hackers for weeks. But since you brought it up, let's discuss what has happened so far and the likelihood that this patch is going to be committable in the next week. Knngist has some prerequisites, rbtree, for example, and it took a while, but now, when we're close to commit rbtree, people can review knngist. This patch is a group of three related patches: point_ops, rbtree, knngist. point_ops, the simplest, was initially submitted on November 23rd. An updated version was submitted on December 30th. I reviewed it on December 31st and made some minor suggestions for improvement, which Teodor accepted. It was committed on January 14th - so IOW, 1 review and 14 days from first review to commit. rbtree, which was more complex, was also submitted on November 23rd. I took a quick look on it on December 31st, a more complete review on January 10th, and a still more complete review on January 20th. I reviewed it again on January 25th and again on February 5th; and Mark Cave-Ayland reviewed it on January 29th. However, the questions that I asked yesterday and the suggestions I made for reworking it have yet to be acted on, so it's going to take at least one more round of reviewing before this is ready for commit. Discounting my quick look on December 31st as not being a real review, that means this patch will have had at least six rounds of review before commit over about 4 weeks. knngist is the final and most complex patch. We have 7 or 8 days left in the CommitFest. It has had zero reviews thus far. Are we going to accomplish six rounds of review in those 7 or 8 days? Or maybe more, since the patch is more complex and has far more interaction with the rest of the code than rbtree? I don't find that very realistic. I think the only way this is going to get committed in the next week is if we basically assume that everything is OK and commit it without a careful review, and I am not in favor of that. But perhaps someone else will advocate for it. Frankly, the politics of the end of the release cycle are a bit frustrating to me. If these patches had been submitted a few weeks sooner, they would have been reviewed in the 2009-11 CommitFest and we would be in much better shape right now. ...Robert Regards, Oleg _ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
Greg Smith wrote: Here's a full TODO page that includes everything mentioned here as best I could summarize it: http://wiki.postgresql.org/wiki/Python_PostgreSQL_Driver_TODO Looks like the first action item is to talk with the Psycopg people about their license. Oh: and I'm going to take care of this. License changes can be a very sensitive topic and I'm told that discussion probably needs to happy in Italian too; I can arrange that. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
>> I'm not a Python user myself, but I have trouble understanding how you >> can describe bugs in one of the Python drivers as off-topic to the >> Python driver situation. > > I thought the topic was "Confusion over Python drivers"? > > The only bug there was likely app one, or at least its not widespread > so off-topic. Rest are more like non-essential cool features, > so again off-topic. Not at all, except that maybe this discussion belongs on -advocacy rather than here. Driver quality is partly performance and stability, but also partly what features it implements and how well it implements them. *particularly* for PostgreSQL, which is a database with lots of cool features; if the driver doesn't support our cool features, then they don't matter to Python developers. Anyway, I don't yet have a full diagnosis on the transaction control issue or I'd already have posted it to psycopg -- it may be a toxic interaction between Django and Psycopg2 rather than psycopg2 alone. I'd not have brought it up except for this discussion. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Function to return whole relation path?
On Feb 7, 2010, at 1:30 PM, Tom Lane wrote: David Christensen writes: On Feb 7, 2010, at 11:04 AM, Tom Lane wrote: pg_relation_filepath(regclass) returns text which would expose the output of relpath(), ie, the $PGDATA-relative path name of the relation. Should this return multiple values (text[] or SETOF text) for tables which wrapped over the single file-limits (1GB, IIRC)? So: "pg_tblspc/ 48372/8.5_201002061/68483/172744", "pg_tblspc/ 48372/8.5_201002061/68483/172744.1", etc? No, I'm not inclined to go there. The set of actually existing segments seems too volatile; and anyone worried about this probably can add a star to the end of the pathname ... True, although it'd need to be more refined than just *, as you'd need to be careful to only pick up those related to the actual relid: "172744", "172744.1", etc, and not those with a common prefix: "1727441", "1727441.1", etc. (common prefix). If that needs to be someone else's problem, makes sense here. Regards, David -- David Christensen End Point Corporation da...@endpoint.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Function to return whole relation path?
David Christensen writes: > On Feb 7, 2010, at 11:04 AM, Tom Lane wrote: >> pg_relation_filepath(regclass) returns text >> which would expose the output of relpath(), ie, the $PGDATA-relative >> path name of the relation. > Should this return multiple values (text[] or SETOF text) for tables > which wrapped over the single file-limits (1GB, IIRC)? So: "pg_tblspc/ > 48372/8.5_201002061/68483/172744", "pg_tblspc/ > 48372/8.5_201002061/68483/172744.1", etc? No, I'm not inclined to go there. The set of actually existing segments seems too volatile; and anyone worried about this probably can add a star to the end of the pathname ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
2010/2/7 Oleg Bartunov : > On Sun, 7 Feb 2010, Robert Haas wrote: > >> On Sun, Feb 7, 2010 at 1:38 AM, Josh Berkus wrote: I think it might be time to revisit this issue. SR is in, and we have a week left in the CF, and we have all of the above patches plus 5 small ones left to deal with. rbtree is close to being committable, I think; knngist has not been reviewed yet; you (Tom) have claimed the frame options patch but I haven't seen any update on it in a while; I doubt either of the other two are ready to commit but I'm not sure how far they have to go. >>> >>> I think, as previously discussed, that we should bounce knngist. It's >>> a complex patch and nobody saw anything of it until Jan 15, so I don't >>> feel bad about it. Mark Cave-Ayland was going to review it, but >>> apparently felt that rbtree was the higher priority. > > Hey, I'm lost here, when we previously discussed, that knngist should be > rejected ? Huh? Have you been reading -hackers for the last month? I first raised this issue on December 30th, and there has been lots more discussion of it since then. http://archives.postgresql.org/pgsql-hackers/2009-12/msg02329.php > knngist is a legal patch, submitted in time (and discussed in > -hackers) and it's not our fault, people are busy doing other reviews. I never said anything about fault. If there's not enough time to get something committed, then there isn't. That's not a punishment; it's just something that sometimes happens to patches submitted near the end of the cycle. We've been openly discussing this problem on -hackers for weeks. But since you brought it up, let's discuss what has happened so far and the likelihood that this patch is going to be committable in the next week. > Knngist has some prerequisites, rbtree, for example, and it took a while, > but now, when we're close to commit rbtree, people can review knngist. This patch is a group of three related patches: point_ops, rbtree, knngist. point_ops, the simplest, was initially submitted on November 23rd. An updated version was submitted on December 30th. I reviewed it on December 31st and made some minor suggestions for improvement, which Teodor accepted. It was committed on January 14th - so IOW, 1 review and 14 days from first review to commit. rbtree, which was more complex, was also submitted on November 23rd. I took a quick look on it on December 31st, a more complete review on January 10th, and a still more complete review on January 20th. I reviewed it again on January 25th and again on February 5th; and Mark Cave-Ayland reviewed it on January 29th. However, the questions that I asked yesterday and the suggestions I made for reworking it have yet to be acted on, so it's going to take at least one more round of reviewing before this is ready for commit. Discounting my quick look on December 31st as not being a real review, that means this patch will have had at least six rounds of review before commit over about 4 weeks. knngist is the final and most complex patch. We have 7 or 8 days left in the CommitFest. It has had zero reviews thus far. Are we going to accomplish six rounds of review in those 7 or 8 days? Or maybe more, since the patch is more complex and has far more interaction with the rest of the code than rbtree? I don't find that very realistic. I think the only way this is going to get committed in the next week is if we basically assume that everything is OK and commit it without a careful review, and I am not in favor of that. But perhaps someone else will advocate for it. Frankly, the politics of the end of the release cycle are a bit frustrating to me. If these patches had been submitted a few weeks sooner, they would have been reviewed in the 2009-11 CommitFest and we would be in much better shape right now. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Function to return whole relation path?
On Feb 7, 2010, at 11:04 AM, Tom Lane wrote: In connection with the relation-mapping patch I proposed a function pg_relation_filenode(regclass) returns oid which client code would need to use instead of looking at pg_class.relfilenode, if it wants to get a number that will be meaningful for mapped system catalogs. I still think we need that, but while thinking about how it'd be used, I wondered if it wouldn't be far more useful to provide pg_relation_filepath(regclass) returns text which would expose the output of relpath(), ie, the $PGDATA-relative path name of the relation. So you'd get something like "base/58381/92034" or "pg_tblspc/48372/8.5_201002061/68483/172744". For clients that are actually trying to match up files with tables, this would avoid having to essentially duplicate the knowledge embedded in relpath(). In particular, the recent decision to include catversion in tablespace subdirectories is going to be a significant PITA for such clients, as there is no easy way to discover catversion by asking the backend. I don't see any security issue here, since this wouldn't give you any information that's not readily available (like absolute pathnames on the server) --- but it would avoid code duplication. Objections, better ideas? Should this return multiple values (text[] or SETOF text) for tables which wrapped over the single file-limits (1GB, IIRC)? So: "pg_tblspc/ 48372/8.5_201002061/68483/172744", "pg_tblspc/ 48372/8.5_201002061/68483/172744.1", etc? Regards, David -- David Christensen End Point Corporation da...@endpoint.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
On Sunday 07 February 2010 19:23:10 Robert Haas wrote: > On Sun, Feb 7, 2010 at 11:24 AM, Tom Lane wrote: > > Greg Smith writes: > >> This is turning into yet another one of those situations where something > >> simple and useful is being killed by trying to generalize it way more > >> than it needs to be, given its current goals and its lack of external > >> interfaces. There's no catversion bump or API breakage to hinder future > >> refactoring if this isn't optimally designed internally from day one. > > > > I agree that it's too late in the cycle for any major redesign of the > > patch. But is it too much to ask to use a less confusing name for the > > function? > > +1. Let's just rename the thing, add some comments, and call it good. Will post a updated patch in the next hours unless somebody beats me too it. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
On Sun, Feb 7, 2010 at 11:24 AM, Tom Lane wrote: > Greg Smith writes: >> This is turning into yet another one of those situations where something >> simple and useful is being killed by trying to generalize it way more >> than it needs to be, given its current goals and its lack of external >> interfaces. There's no catversion bump or API breakage to hinder future >> refactoring if this isn't optimally designed internally from day one. > > I agree that it's too late in the cycle for any major redesign of the > patch. But is it too much to ask to use a less confusing name for the > function? +1. Let's just rename the thing, add some comments, and call it good. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Provide rowcount for utility SELECTs
Robert Haas írta: > On Tue, Feb 2, 2010 at 4:03 AM, Boszormenyi Zoltan wrote: > >> Thanks for testing it, with the attached patch your test case also >> returns SELECT N. >> > > Thoughts: > > 1. Looks like you've falsified the last comment block in PortalRunMulti(). > You mean the "fake something up" part? Will fix the comment. > 2. I don't like the duplication of code in PortalRun() between the > first and second branches of the switch statement. > The PORTAL_ONE_SELECT and PORTAL_ONE_RETURNING/PORTAL_UTIL_SELECT cases differ only in that the latter case runs this below everything else: if (!portal->holdStore) FillPortalStore(portal, isTopLevel); Would it be desired to unify these cases? This way there would be no code duplication. Something like: if (portal->strategy != PORTAL_ONE_SELECT && !portal->holdStore) FillPortalStore(portal, isTopLevel); ... (everything else) > 3. You've falsified the comment preceding that code, too. > This one? /* * Set up global portal context pointers. * * We have to play a special game here to support utility commands like * VACUUM and CLUSTER, which internally start and commit transactions. * When we are called to execute such a command, CurrentResourceOwner will * be pointing to the TopTransactionResourceOwner --- which will be * destroyed and replaced in the course of the internal commit and * restart. So we need to be prepared to restore it as pointing to the * exit-time TopTransactionResourceOwner. (Ain't that ugly? This idea of * internally starting whole new transactions is not good.) * CurrentMemoryContext has a similar problem, but the other pointers we * save here will be NULL or pointing to longer-lived objects. */ I can't see why. Can you clarify? Or this one? /* we know the query is supposed to set the tag */ if (completionTag && portal->commandTag) ... The condition and the comment still seems to be true. Which comment are you talking about? > 4. Is there any reason to use pg_strcasecmp() instead of plain old strcmp()? > I don't have any particular reason, strcmp() would do. > Someone who knows the overall structure of the code better than I do > will have to comment on whether there are any code paths to worry > about that do not go through PortalRun(). > > A general concern I have is that this what we're basically doing here > is handling the most common case in ProcessQuery() and then installing > fallback mechanisms to pick up any stragglers: but the fallback > mechanisms only guarantee that we'll add a number to the command tag, > not that it will be meaningful. Unfortunately, my limited imagination > can't quite figure out in what cases we'll get a SELECT command tag > back other than (1) plain old SELECT, (2) SELECT INTO, and (3) CTAS, > so I'm not sure what to go test. > > ...Robert > > Best regards, Zoltán Böszörményi -- Bible has answers for everything. Proof: "But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil." (Matthew 5:37) - basics of digital technology. "May your kingdom come" - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig & Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_class has no toast table?
Simon Riggs writes: > On Sat, 2010-02-06 at 13:04 -0500, Tom Lane wrote: >> We might still want to consider toast-ifying pg_class if anyone ever >> complains about not having room for wide relacl values; but CLUSTER >> shouldn't be a forcing function for such decisions. > What failure do you get if you have too many relacls or too many > reloptions? We would want it to fail cleanly. Is it enough to mark those > columns as MAIN storage? You'd get a "tuple too large" error if the tuple still didn't fit on a page after compressing the wide columns. We don't need to do anything special for that. > Neither of those is worth worrying about a toast table for. Anybody with > that long a relacl hasn't thought about their admin structure enough. Yeah, that's my thought also. You'd likely start having performance problems with thousand-item ACL lists anyway. You should switch over to using groups long before that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Function to return whole relation path?
In connection with the relation-mapping patch I proposed a function pg_relation_filenode(regclass) returns oid which client code would need to use instead of looking at pg_class.relfilenode, if it wants to get a number that will be meaningful for mapped system catalogs. I still think we need that, but while thinking about how it'd be used, I wondered if it wouldn't be far more useful to provide pg_relation_filepath(regclass) returns text which would expose the output of relpath(), ie, the $PGDATA-relative path name of the relation. So you'd get something like "base/58381/92034" or "pg_tblspc/48372/8.5_201002061/68483/172744". For clients that are actually trying to match up files with tables, this would avoid having to essentially duplicate the knowledge embedded in relpath(). In particular, the recent decision to include catversion in tablespace subdirectories is going to be a significant PITA for such clients, as there is no easy way to discover catversion by asking the backend. I don't see any security issue here, since this wouldn't give you any information that's not readily available (like absolute pathnames on the server) --- but it would avoid code duplication. Objections, better ideas? BTW, I think both functions should return NULL for relations without storage. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_class has no toast table?
On Sat, 2010-02-06 at 13:04 -0500, Tom Lane wrote: > I wrote: > > Still fooling with VACUUM FULL on catalogs ... I find that a sanity > > check I put in is barfing on "VACUUM FULL pg_class", because the > > transient table is built with a toast table, whereas pg_class hasn't got > > one. It seems like it probably ought to have one, because either relacl > > or reloptions could in principle be too big to fit without toasting > > (which is exactly why AlterTableCreateToastTable thinks it should make > > one for the transient table). > > > I have a vague feeling that we intentionally omitted a toast table for > > pg_class, but I don't remember why. Comments? > BTW, I decided not to touch this issue in the current patch --- it turns > out there are quite a few system catalogs that lack a toast table but > AlterTableCreateToastTable's rules would say to create one. The one > that made me stop adding them was pg_largeobject. That has a bytea > column but there are usage rules that limit the possible width of the > column, and of course AlterTableCreateToastTable doesn't know that. > > So I tweaked the CLUSTER/VAC FULL logic to never add a toast table if > the original table hasn't got one. (It can still *remove* a toast > table, as might happen after dropping a wide column for instance.) > > We might still want to consider toast-ifying pg_class if anyone ever > complains about not having room for wide relacl values; but CLUSTER > shouldn't be a forcing function for such decisions. What failure do you get if you have too many relacls or too many reloptions? We would want it to fail cleanly. Is it enough to mark those columns as MAIN storage? Neither of those is worth worrying about a toast table for. Anybody with that long a relacl hasn't thought about their admin structure enough. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby and DROP DATABASE
On Sat, 2010-02-06 at 17:32 +0100, Andres Freund wrote: > > > > So it seems at least the behavior is quite different from what the > > docs stats. Am I missing something here? > Its a small bug/typo in standby.c:ResolveRecoveryConflictWithDatabase > > The line: > CancelDBBackends(dbid, PROCSIG_RECOVERY_CONFLICT_TABLESPACE, > true); > > has to be > CancelDBBackends(dbid, PROCSIG_RECOVERY_CONFLICT_DATABASE, > true); Well spotted, thanks for the report and the analysis. The code for drop database worked when committed but it looks like the re-factoring of the code broke it. Will fix. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Precedence of Postfix operators
Gokulakannan Somasundaram writes: >Is there any reason why we have given lesser precedence for postfix > operator compared to multiplication/division? Usually postfix operators have > more precedence than the binary operations. Is this some kind of work around > to provide user-defined operators? Can someone help me understand this? A bit of poking in the CVS logs for gram.y reveals 2001-01-23 17:39 tgl * src/backend/parser/gram.y: Give 'a_expr ::= a_expr Op' production a slightly lower precedence than Op, so that the sequence 'a_expr Op Op a_expr' will be parsed as a_expr Op (Op a_expr) not (a_expr Op) Op a_expr as formerly. In other words, prefer treating user-defined operators as prefix operators to treating them as postfix operators, when there is an ambiguity. Also clean up a couple of other infelicities in production priority assignment --- for example, BETWEEN wasn't being given the intended priority, but that of AND. There are several other nasty things that we've had to do in order to keep supporting postfix operators at all. I thibk most people view them as a legacy feature best avoided. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Greg Smith writes: > This is turning into yet another one of those situations where something > simple and useful is being killed by trying to generalize it way more > than it needs to be, given its current goals and its lack of external > interfaces. There's no catversion bump or API breakage to hinder future > refactoring if this isn't optimally designed internally from day one. I agree that it's too late in the cycle for any major redesign of the patch. But is it too much to ask to use a less confusing name for the function? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reading deleted records - PageHeader v3
On Sat Feb 6 01:20 AM, Tom Lane wrote: > "Jonathan Bond-Caron" writes: > > I think part of my problem is I haven't really understood what 'Then > > make sure you have the right alignment' means. > > > My approach currently is: > > > After reading HeapTupleHeaderData (23 bytes), I advance another 4 > > bytes > > (hoff) and try to read a 32 bit integer (first attribute). > > No. First you start at the tuple beginning plus the number of bytes > indicated by hoff (which should be at least 24). Thanks, much appreciated! I was reading HeapTupleHeaderData as 23 bytes but it's 27 bytes in access/htup.h?rev=1.87. The hoff now makes sense with a 28 bytes value and I can start to read the user data. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby documentation
On Sun, Feb 7, 2010 at 4:41 AM, Markus Wanner wrote: > Bruce Momjian wrote: >> Do we want to call the feature "hot standby"? Is a read-only standby a >> "standby" or a "slave"? > > I think hot standby is pretty much the term, now. See here for the previous iteration of this discussion: http://archives.postgresql.org/pgsql-hackers/2009-08/msg00870.php I've always thought this feature was misnamed and nothing has happened to change my mind, but it's not clear whether I'm in the majority. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Feb 7, 2010 at 1:38 AM, Josh Berkus wrote: >> I think it might be time to revisit this issue. SR is in, and we have >> a week left in the CF, and we have all of the above patches plus 5 >> small ones left to deal with. rbtree is close to being committable, I >> think; knngist has not been reviewed yet; you (Tom) have claimed the >> frame options patch but I haven't seen any update on it in a while; I >> doubt either of the other two are ready to commit but I'm not sure how >> far they have to go. > > I think, as previously discussed, that we should bounce knngist. It's > a complex patch and nobody saw anything of it until Jan 15, so I don't > feel bad about it. Mark Cave-Ayland was going to review it, but > apparently felt that rbtree was the higher priority. rbtree is a prerequisite for knngist. point_ops was too. I think we're going to get rbtree done yet, but the main patch seems out of reach. There's no way it's going to go in without both pre-commit and post-commit changes, and I don't think we have time for that now. > Also, this one: > Provide rowcount for utility SELECTs > ... based on your last review does not look anywhere near ready. I wouldn't worry about that one. It's a small patch. It's not going to suck a lot of anyone's time; it'll either make it, or not. > We know the following because of endless discussion on -hackers; what > these patches seem to be suffering from is endless arguments over > relatively minor points, it just requires someone to make a decision on > implementation method: > Add on_trusted_init and on_untrusted_init to plperl I think this one is in pretty good shape. I think the ball is Andrew Dunstan's court to commit it, or not. > Faster CREATE DATABASE by delaying fsync Too much attempt to design an abstraction layer for things we can't abstract; not enough doing something that is good enough for now. Let's just put in something that's not particularly abstract and we can rearrange later. > For the rest, can we just have reviewer reports on readiness? > Writeable CTEs This was marked ready by the reviewer a long time ago; but that was considerably over-optimistic, as the recent comments on that thread attest. A major feature without documentation is by definition not ready. > Package namespace and Safe init cleanup for plperl I think this is close. Another one for Andrew Dunstan to make the call, I believe. > More frame options in window functions Not sure. > Fix large object support in pg_dump Doesn't matter because this one is an open item. I'm hoping Itagaki Takahiro is going to take the lead on this one because he committed the prior patch that created the situation we're now trying to fix. > Actually, looking at that list, I think we're in pretty darned good > shape. That's a pretty small list of things left to commit. Keep in > mind that last year at this time (week 3 of CF-last) we still had over a > dozen patches completely unreviewed! Unfortunately, we haven't been very successful this time in attracting non-committer reviewers. Only about 20% of the committed patches are listed as having been reviewed by a non-committer. Still, I agree with your overall conclusion. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Feb 7, 2010 at 2:05 AM, Tom Lane wrote: > Given that we have a week still to go in the CF, I feel fairly > confident of still getting the window frame patch in on time > (assuming that there are indeed no major problems with it). > I have not let go of it for that reason, and also because I doubt > anybody else is qualified to commit it --- AFAIR only Hitoshi-san > and I were really neck-deep in the original window patch. > > However, with the deadline fast approaching, I don't feel that I > can also promise to handle writeable CTEs, which is another one > that I'd really like to be the committer for. Maybe we had better > make a management decision about which of those two is higher > priority to get into 9.0. Also: I haven't been following either > one terribly closely lately. If there's reason to think that one is > more likely to be committable than the other, that ought to get > factored into the choice of which to go to first; but I'm not > sure whether that's the case. As between the two, I get the feeling that there is more interest in writeable CTEs. But that impression might be wrong, since it's an unscientific recollection of discussions on -hackers; which are themselves not representative of anything. I have not looked at the window functions patch at all, and I haven't looked at the latest version of writeable CTEs, either. I will try to spend some time on it in the next couple of days. My feeling about the last version is that it lacked a lot in the documentation department, and also in the comments department. Since I don't know that code very well, that made it hard for me to assess technical correctness. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sat, Feb 06, 2010 at 10:38:00PM -0800, Josh Berkus wrote: > > Add on_trusted_init and on_untrusted_init to plperl > Package namespace and Safe init cleanup for plperl Alex Hunsaker has marked the latest version of both of those as Ready for Committer. Tim. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Precedence of Postfix operators
Hi, Is there any reason why we have given lesser precedence for postfix operator compared to multiplication/division? Usually postfix operators have more precedence than the binary operations. Is this some kind of work around to provide user-defined operators? Can someone help me understand this? Thanks, Gokul.
Re: [HACKERS] Streaming replication in docs
On 7 February 2010 10:49, Heikki Linnakangas wrote: > Thom Brown wrote: >> On 7 February 2010 10:20, Thom Brown wrote: >>> The post about the dev docs needing more hot standby mentions prompted >>> me to have a look at how streaming replication is documented. Ignore >>> this if this has already been discussed (I couldn't find any posts), >>> but I couldn't find any mention of streaming replication except in the >>> write ahead log configuration. There doesn't seem to be anything in >>> the index linked to a description of this major feature, why you would >>> use it, or how it works. There's also no mention in the release >>> overview. >>> >>> This is probably due to it being a work in progress and hasn't been >>> written yet, but thought I'd raise it just in case. >> >> Okay, after some searching, I found some sections on streaming >> replication not referenced in the index: >> >> http://developer.postgresql.org/pgdocs/postgres/warm-standby.html#STREAMING-REPLICATION >> >> Streaming Replication Protocol: >> http://developer.postgresql.org/pgdocs/postgres/protocol-replication.html >> >> Main section about Streaming Replication: Maybe these should get >> indexed under streaming replication? > > Yeah, as you noticed, the documentation for streaming replication is > currently non-existent. That obviously needs to be fixed. I think we > need to move the existing sections around, and of course add a lot of > documentation for the new streaming replication and hot standby specific > stuff. I proposed a new layout here: > > http://archives.postgresql.org/message-id/4b50a600.3090...@enterprisedb.com > > which people seemed to agree with. If you want to help with that, > perhaps the best starting point would be to start writing text for the > new chapters of that new layout. > I'm wondering if I "gave it a go", I'd be more of a hindrance than a help, as I'm not really familiar enough with the workings of hot standby to provide this missing information. In fact that's why I looked for the documentation, because I wanted to learn how it works and how to use it. I'd be happy to review any doc changes to ensure enough information is available and that it's clear enough to end-users though. And I do prefer the proposed layout. Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
Marko, > I thought the topic was "Confusion over Python drivers"? > > The only bug there was likely app one, or at least its not widespread > so off-topic. Rest are more like non-essential cool features, > so again off-topic. > Those lack of "non-essential cool features" is right on topic - because what one developer may see as "non-essential" is the most important feature for the next developer. Exactly these kind of issues are the source of the confusion we now phase: developer B needing feature X, which was ignored by driver A. So I concur to put them on the discussion agenda; if they drop out of priority, at least it is documented WHY and will save other developers these thoughts. Harald -- GHUM Harald Massa persuadere et programmare Harald Armin Massa Spielberger Straße 49 70435 Stuttgart 0173/9409607 no fx, no carrier pigeon - %s is too gigantic of an industry to bend to the whims of reality
Re: [HACKERS] Streaming replication in docs
Thom Brown wrote: > On 7 February 2010 10:20, Thom Brown wrote: >> The post about the dev docs needing more hot standby mentions prompted >> me to have a look at how streaming replication is documented. Ignore >> this if this has already been discussed (I couldn't find any posts), >> but I couldn't find any mention of streaming replication except in the >> write ahead log configuration. There doesn't seem to be anything in >> the index linked to a description of this major feature, why you would >> use it, or how it works. There's also no mention in the release >> overview. >> >> This is probably due to it being a work in progress and hasn't been >> written yet, but thought I'd raise it just in case. > > Okay, after some searching, I found some sections on streaming > replication not referenced in the index: > > http://developer.postgresql.org/pgdocs/postgres/warm-standby.html#STREAMING-REPLICATION > > Streaming Replication Protocol: > http://developer.postgresql.org/pgdocs/postgres/protocol-replication.html > > Main section about Streaming Replication: Maybe these should get > indexed under streaming replication? Yeah, as you noticed, the documentation for streaming replication is currently non-existent. That obviously needs to be fixed. I think we need to move the existing sections around, and of course add a lot of documentation for the new streaming replication and hot standby specific stuff. I proposed a new layout here: http://archives.postgresql.org/message-id/4b50a600.3090...@enterprisedb.com which people seemed to agree with. If you want to help with that, perhaps the best starting point would be to start writing text for the new chapters of that new layout. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication in docs
On 7 February 2010 10:20, Thom Brown wrote: > The post about the dev docs needing more hot standby mentions prompted > me to have a look at how streaming replication is documented. Ignore > this if this has already been discussed (I couldn't find any posts), > but I couldn't find any mention of streaming replication except in the > write ahead log configuration. There doesn't seem to be anything in > the index linked to a description of this major feature, why you would > use it, or how it works. There's also no mention in the release > overview. > > This is probably due to it being a work in progress and hasn't been > written yet, but thought I'd raise it just in case. > > Thanks > > Thom Okay, after some searching, I found some sections on streaming replication not referenced in the index: http://developer.postgresql.org/pgdocs/postgres/warm-standby.html#STREAMING-REPLICATION Streaming Replication Protocol: http://developer.postgresql.org/pgdocs/postgres/protocol-replication.html Main section about Streaming Replication: Maybe these should get indexed under streaming replication? Thanks Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Streaming replication in docs
The post about the dev docs needing more hot standby mentions prompted me to have a look at how streaming replication is documented. Ignore this if this has already been discussed (I couldn't find any posts), but I couldn't find any mention of streaming replication except in the write ahead log configuration. There doesn't seem to be anything in the index linked to a description of this major feature, why you would use it, or how it works. There's also no mention in the release overview. This is probably due to it being a work in progress and hasn't been written yet, but thought I'd raise it just in case. Thanks Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
On 2/7/10, Robert Haas wrote: > On Sat, Feb 6, 2010 at 7:48 PM, Marko Kreen wrote: > > This is long-term todo item for psycopg, seems offtopic > > to the "driver situation". > > [...] > > > This is routine bug in either app or psycopg, we have no reason > > to touch it. The guy should report to appropriate lists. > > [...] > > > Long-term todo item for psycopg2, offtopic for "driver situation". > > [...] > > > psycopg2 has array support, I'd like to have tuple/record also. > > > > Minor todo item for psycopg, mostly but not completely offtopic > > for "driver situation". > > > I'm not a Python user myself, but I have trouble understanding how you > can describe bugs in one of the Python drivers as off-topic to the > Python driver situation. I thought the topic was "Confusion over Python drivers"? The only bug there was likely app one, or at least its not widespread so off-topic. Rest are more like non-essential cool features, so again off-topic. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusion over Python drivers
Marko Kreen wrote: I think we should concentrate on the PR problem and technical issues related to that, keep the other low-level and non-user-visible issues out. Or at least separate. (PsycopgTodo wiki page?) That's just a matter of prioritizing the issues. Put the big ones at the top, the trivia at the bottom, and if you knock stuff of there somewhere along the way you discover you've made enough progress that the PR stuff starts going away, because people are able to get their work done with less drama. Here's a full TODO page that includes everything mentioned here as best I could summarize it: http://wiki.postgresql.org/wiki/Python_PostgreSQL_Driver_TODO Looks like the first action item is to talk with the Psycopg people about their license. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby documentation
Bruce, Bruce Momjian wrote: Ah, I now realize it only mentions "warm" standby, not "hot", so I just updated the documentation to reflect that; you can see it here: Maybe the table below also needs an update, because unlike "Warm Standby using PITR", a hot standby accepts read-only queries and can be configured to not loose data on master failure. Do we want to call the feature "hot standby"? Is a read-only standby a "standby" or a "slave"? I think hot standby is pretty much the term, now. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Robert Haas wrote: Well it seems that what we're trying to implement is more like it_would_be_nice_if_you_would_start_syncing_this_file_range_but_its_ok_if_you_dont(), so maybe that would work. Anyway, is there something that we can agree on and get committed here for 9.0, or should we postpone this to 9.1? It seems simple enough that we ought to be able to get it done, but we're running out of time and we don't seem to have a clear vision here yet... This is turning into yet another one of those situations where something simple and useful is being killed by trying to generalize it way more than it needs to be, given its current goals and its lack of external interfaces. There's no catversion bump or API breakage to hinder future refactoring if this isn't optimally designed internally from day one. The feature is valuable and there seems at least one spot where it may be resolving the possibility of a subtle OS interaction bug by being more thorough in the way that it writes and syncs. The main contention seems to be over naming and completely optional additional abstraction. I consider the whole "let's make this cover every type of complicated sync on every platform" goal interesting and worthwhile, but it's completely optional for this release. The stuff being fretted over now is ultimately an internal interface that can be refactored at will in later releases with no user impact. If the goal here could be shifted back to finding the minimal level of abstraction that doesn't seem completely wrong, then updating the function names and comments to match that more closely, this could return to committable. That's all I thought was left to do when I moved it to "ready for committer", and as far as I've seen this expanded scope of discussion has just moved backwards from that point. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
