[HACKERS] docco on external storage?
Perhaps I am missing something, but the documentation on external storage appears remarkably sparse. It says it is better for substring searches, but otherwise doesn't say very much. I have a table with two modest-length bytea fields (around 9K and 2K respectively) of already compressed data. I was looking for a clue as to whether or not these should use external storage - I suspect they should but I didn't see anything that gave me a definite answer. I'd prepare doc updates on this if they were required but I don't have the requisite knowledge, nor time right now to go trawling endlessly to acquire it. cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] docco on external storage?
Thanks. We don't really expect DBAs to go hunting through the source code, though, do we? cheers andrew - Original Message - From: Joe Conway [EMAIL PROTECTED] To: Andrew Dunstan [EMAIL PROTECTED] Cc: Postgresql Hackers [EMAIL PROTECTED] Sent: Monday, November 03, 2003 9:50 PM Subject: Re: [HACKERS] docco on external storage? Andrew Dunstan wrote: I have a table with two modest-length bytea fields (around 9K and 2K respectively) of already compressed data. I was looking for a clue as to whether or not these should use external storage - I suspect they should but I didn't see anything that gave me a definite answer. From src/include/access/tuptoaster.h: - /* * These symbols control toaster activation. If a tuple is larger than * TOAST_TUPLE_THRESHOLD, we will try to toast it down to no more than * TOAST_TUPLE_TARGET bytes. Both numbers include all tuple header and * alignment-padding overhead. * * The numbers need not be the same, though they currently are. */ #define TOAST_TUPLE_THRESHOLD (MaxTupleSize / 4) #define TOAST_TUPLE_TARGET (MaxTupleSize / 4) And from src/include/access/htup.h: --- /* * MaxTupleSize is the maximum allowed size of a tuple, including header * and MAXALIGN alignment padding. Basically it's BLCKSZ minus the * other stuff that has to be on a disk page. The other stuff * includes access-method- dependent special space, which we assume * will be no more than MaxSpecialSpace bytes (currently, on heap pages * it's actually zero). * * NOTE: we do not need to count an ItemId for the tuple because * sizeof(PageHeaderData) includes the first ItemId on the page. */ #define MaxSpecialSpace 32 #define MaxTupleSize\ (BLCKSZ - MAXALIGN(sizeof(PageHeaderData) + MaxSpecialSpace)) And from src/include/pg_config_manual.h: #define BLCKSZ 8192 So, it looks like the threshold for external storage is somewhere around 2000 bytes. Joe ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Experimental patch for inter-page delay in VACUUM
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: What still needs to be addressed is the IO storm cause by checkpoints. I see it much relaxed when stretching out the BufferSync() over most of the time until the next one should occur. But the kernel sync at it's end still pushes the system hard against the wall. I have never been happy with the fact that we use sync(2) at all. Quite aside from the I/O storm issue, sync() is really an unsafe way to do a checkpoint, because there is no way to be certain when it is done. And on top of that, it does too much, because it forces syncing of files unrelated to Postgres. I would like to see us go over to fsync, or some other technique that gives more certainty about when the write has occurred. There might be some scope that way to allow stretching out the I/O, too. The main problem with this is knowing which files need to be fsync'd. The only idea I have come up with is to move all buffer write operations into a background writer process, which could easily keep track of every file it's written into since the last checkpoint. This could cause problems though if a backend wants to acquire a free buffer and there's none to be had --- do we want it to wait for the background process to do something? We could possibly say that backends may write dirty buffers for themselves, but only if they fsync them immediately. As long as this path is seldom taken, the extra fsyncs shouldn't be a big performance problem. Actually, once you build it this way, you could make all writes synchronous (open the files O_SYNC) so that there is never any need for explicit fsync at checkpoint time. The background writer process would be the one incurring the wait in most cases, and that's just fine. In this way you could directly control the rate at which writes are issued, and there's no I/O storm at all. (fsync could still cause an I/O storm if there's lots of pending writes in a single file.) Or maybe fdatasync() would be slightly more efficient - do we care about flushing metadata that much? cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] equal() perf tweak
Tom Lane wrote: snip excellent scheme for improving List /snip Just a thought - if we are messing with the List definition should we at the same time address the strict aliasing issues arising from Node's multiple personalities (I think it is the main offender). Or is the intention never to do this, or not any time soon? (Either is OK, although I suspect it will have to happen sometime as more compilers start relying on the C99 rules more aggressively). cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] [PATCHES] equal() perf tweak
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Just a thought - if we are messing with the List definition should we at the same time address the strict aliasing issues arising from Node's multiple personalities (I think it is the main offender). Or is the intention never to do this, or not any time soon? I have no intention of messing with the Node concept; it's built into the backend far too firmly to consider any significant change. I don't think we understand exactly what we'd have to avoid in order to enable strict aliasing, but if it requires getting rid of Node then it ain't happening. (I doubt that it does, anyway. I think the issues are probably quite localized. The main problem I see is that we don't have any trustworthy check to find out everyplace that strict aliasing could cause problems.) *nod* (I made the last point previously :-) ) I don't claim that my understanding is anything like complete. However, in the case of gcc it might be easier to fix than you think, without a lot of declaration rearranging, using gcc's new may_alias type attribute, (see http://gcc.gnu.org/onlinedocs/gcc-3.3.2/gcc/Type-Attributes.html ) which could probably be added in a very few places (e.g. Node's typedef) to fix things. Of course, that doesn't help when vendor foo's compiler starts doing type-based alias analysis. Still, to quote Airplane, that's not important right now. cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Open Sourcing pgManage
Marc G. Fournier wrote: On Tue, 4 Nov 2003, Joshua D. Drake wrote: Hello, If that is the case that is fine. I just wanted to throw it out there but doesn't that mean that psql would be separate as well? no new client applications BTW, Joshua, thanks for releasing this - all my client side work is currently Java (a Tomcat webapp in fact) so I'm very interested to see the shape of your app, as I'm sure others are. cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Experimental patch for inter-page delay in VACUUM
scott.marlowe wrote: On Tue, 4 Nov 2003, Tom Lane wrote: The main problem with this is knowing which files need to be fsync'd. Wasn't this a problem that the win32 port had to solve by keeping a list of all files that need fsyncing since Windows doesn't do sync() in the classical sense? If so, then could we use that code to keep track of the files that need fsyncing? according to the win32 page at http://momjian.postgresql.org/main/writings/pgsql/win32.html this is still to be done. I seem to recall Bruce saying that SRA had found the solution to this, something along these lines, but I am not sure the code is there yet (don't have access to that branch on this machine) cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Open Sourcing pgManage
Andreas Pflug wrote: Marc G. Fournier wrote: JSP, remotely, through Jakarta-Tomcat? One of the limitations of pgAdmin, as far as I'm concerned, is the fact that you can run it remotely ... if you could run pgManage under something like Jakarta-Tomcat as a JSP, that would be *really* cool ... D'oh, just clued into the 'java' aspect ... Joshua, will this run as a pgAdmin is designed for a good interactive experience, which isn't achievable using web technologies. SSL connection is supported, so on not-too-slow lines remote usage should be possible without security issues, or over a VPN (I'm working like this). For web access, phpPGadmin should be usable; haven't tried so far. I don't think any of this contradicts what Marc said. And, as Joshua pointed out it could with some work be made to run as an applet, which would be very cool for, say, an ISP to provide (nothing at all required for the user to install). Don't get me wrong - pgadmin is cool - I especially recommend it to my Windows oriented clients and colleagues who hate using command lines. I think there is room for lots of GUIs, though, and having a Java admin GUI would be cool too, as would having a servlet/JSP based admin client deployable as a web archive. (BTW, have a look at the phpPgAdmin screen shots at http://phppgadmin.sourceforge.net/?page=screenshots - they are quite nice, even though I am not a PHP fan). cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Open Sourcing pgManage
Dave Page wrote: -Original Message- From: Andrew Dunstan [mailto:[EMAIL PROTECTED] Don't get me wrong - pgadmin is cool - I especially recommend it to my Windows oriented clients and colleagues who hate using command lines. Why not your Linux or FreeBSD oriented colleagues? It runs just as well on those platforms. especially != only :-) I have it installed on the RH9 machine I use for development, and happily show it to people there. most *nix people I come into contact with are old fossils like me who prefer scripts and command lines for doing things. I don't use an IDE (unless you count emacs as an IDE) for development, and I rarely use control-panel-like apps. BTW, pgadmin could improve its Linux coverage somewhat by a) providing RPMs for versions of RedHat before 9, or at least providing SRPMs that can be built on such platforms, and b) getting pgadmin included in the fedora package set. Anyway, CommandPrompt have apparently done something cool and they are donating it and we should all be happy, no? :-) cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Performance features the 4th
Jan Wieck wrote: How portable is getrusage()? Could the postmaster issue that frequently for RUSAGE_CHILDREN and leave the result somewhere in the shared memory for whoever is concerned? SVr4, BSD4.3, SUS2 and POSIX1003.1, I believe. I also believe there is a M$ dll available that gives that functionality (psapi.dll). cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-www] [HACKERS] Changes to Contributor List
Bruce Momjian wrote: Tom Lane wrote: Josh Berkus [EMAIL PROTECTED] writes: B) What contributors are listed under Major Developers who haven't contributed any code since 7.1.0? I think we had agreed that formerly-listed contributors would not be deleted, but would be moved to a new section titled Contributors Emeritus or some such. Please make sure that Tom Lockhart and Vadim get listed that way, at least. I think the Emeritus word might be too hard for non-native English speakers, and even for less educated English speakers. Rupert Murdoch once sacked an editor (over the Hitler Diaries forgery fiasco) by giving him the title of Editor Emeritus. The editor asked what it meant and Murdoch is reported to have replied It's Latin, Frank. The 'e' means you're out and the 'meritus' means you deserve it. :-) cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Information Schema and constraint names not unique
Tom Lane wrote: The reason the spec defines these views this way is that it expects constraint names to be unique across a whole schema. We don't enforce that, and I don't think we want to start doing so (that was already proposed and shot down at least once). You are of course free to use constraint names that are distinct if you want to follow the spec's lead. Would a good halfway house be to ensure that generated names were unique within a schema (e.g. instead of generating $1 generate tablename$1)? I know this might make looking to see if something is a generated constraint mildly harder. It would have the advantage of a slightly more meaningful name on the constraint. Doing that we still wouldn't enforce the spec's requirements for uniqueness of constraint names within a schema (which are arguably silly), but wouldn't violate them ourselves. (I'm sure there are wrinkles I haven't thought of, though. Not sure about what it would do to backwards compatibility, for instance.) cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Information Schema and constraint names not unique
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: I don't think we really need a method to guarantee unique names. It would already help a lot if we just added the table name, or something that was until a short time before the action believed to be the table name, or even only the table OID, before (or after) the $1. I don't have a problem with switching from $1 to tablename_$1, or some such, for auto-generated constraint names. But if it's not guaranteed unique, does it really satisfy Philip's concern? He wouldn't see identical rows returned from his query any more, would he? My point was that doing this nothing would prevent the user creating duplicate constraint names but the system would not produce (or would be most unlikely to produce) duplicates. I read the thread from last year on Google at http://groups.google.com/groups?hl=enlr=ie=UTF-8threadm=18252.1025635125%40sss.pgh.pa.usrnum=1prev=/groups%3Fq%3Dunique%2Bconstraint%2Bnames%2Bgroup:comp.databases.postgresql.hackers%2Bgroup:comp.databases.postgresql.hackers%2Bgroup:comp.databases.postgresql.hackers%2Bgroup:comp.databases.postgresql.hackers%26hl%3Den%26lr%3D%26ie%3DUTF-8%26group%3Dcomp.databases.postgresql.hackers%26selm%3D18252.1025635125%2540sss.pgh.pa.us%26rnum%3D1 which was why I thought this would be a move in the right direction without encountering those problems. (I much prefer using tablename to OID, BTW) cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Bogus bind() warnings
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: When I start up with -i, I get the following log: LOG: could not bind IPv4 socket: Address already in use There is no other postmaster running anywhere. I suspect that this has to do with IPv6. This is a SuSE 8.something machine that is relatively fully IPv6 enabled. Is it possible that that kernel considers binding to an IPv6 port to conflict with binding to the same port number as an IPv4 port? IIRC that was the behavior we once expected would happen, but later found out that most kernels don't (yet?) act that way. The present design of trying to bind to both IPv6 and IPv4 sockets would be unnecessary if the kernels acted more rationally. I have seen this before, and reported it, but can't find the thread right now. On Linux with IP6 enabled, IP4 is tunnelled over IP6 - they *are* the same sockets, AFAIK. Didn't we put in a patch after lengthy discussion that fixes things from a pg_hba.conf POV exactly to handle this (i.e. to match an IP4 address in the file with the corresponding IP6 address: n.n.n.n/x - :::n.n.n.n/96+x )? I also recall someone saying this would change in later versions of Linux. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Bogus bind() warnings
Tom Lane wrote: Kurt Roeckx [EMAIL PROTECTED] writes: Should we just not give that error message, in case we already binded to AF_INET6 ::? Seems like a cure worse than the disease to me --- it could mask real problems. I suppose we could think about dropping it from LOG to DEBUG1 level, so that it wouldn't appear in a default setup; but I'm not sure I'm for that either. Given the, ahem, wide variety of behaviors that seem to be out there, I think we'd best be happy if we have a v4/v6 implementation that has no problems worse than spurious log messages ... I agree. Things that might be serious problems should not be hidden. Maybe it would be better to add a message that the error might be harmless if you have IPv6 turned on. cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Information Schema and constraint names not
Philip Warner wrote: At 03:37 AM 7/11/2003, Peter Eisentraut wrote: It would already help a lot if we just added the table name, or something that was until a short time before the action believed to be the table name, or even only the table OID, before (or after) the $1. Can we allow/bypass the pg_* restriction, and call it pg_table-oid_n, and for pedants like me, add a DB setting that says 'enforce unique constraints' ala the spec to avoid manually created constraints being non-unique? Alternatively, I would be happy *not* to enforce constraint name uniqueness (and break the spec) so long as we also break the spec and add table OID (or something else) to the information schema (table name would be OK so long as renaming the table won't break anything). The first seems impractical for reasons given by Tom. Why provide an option for behaviour we fear could deadlock etc.? Regarding the second option, I don't understand what virtue there is in breaking the spec more, rather than embedding the table name in the constraint name. (If we do that, if the table is renamed my instinct would be to rename constraints in the renamed table with autogenerated names, but I haven't looked into it). cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Information Schema and constraint names not unique
Bruce Momjian wrote:
Tom Lane wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
On Thu, Nov 06, 2003 at 11:42:13AM -0500, Tom Lane wrote:
I don't have a problem with switching from $1 to tablename_$1, or
some such, for auto-generated constraint names. But if it's not
guaranteed unique, does it really satisfy Philip's concern?
It certainly _is_ unique within a schema ...
(But what happens to the constraint name when the table is renamed?)
Exactly. Also consider manually-assigned constraint names that happen
to look like foo_$n --- these could cause trouble if table foo is
created later. To make a guarantee of uniqueness would require more
infrastructure than just a simple hack of the constraint name generator
logic.
BTW we also have some problems with auto-generated names for column
constraints; these generally look like tablename_columnname, and
that's not unique:
regression=# create table foo (f1 int check (f1 0) check (f1 10));
ERROR: check constraint foo_f1 already exists
Is this a TODO to fix?
I think there are several of them from this thread:
. make autogenerated column constraint names unique per table (by adding
_$n ?)
. add tableoid or tablename to information_schema.{check_constraints,
referential_constraints} (I think those are the only places where it
would be needed, from my quick skimming).
. add tableoid or tablename to autogenerated table constraint names
Is that a fair summary of discussion so far?
(My take)
Using tableoid instead of tablename avoids renaming problems, but makes
the names horribly opaque IMNSHO. I know I've been annoyed from an
aesthetic POV more than once by the $1 thing.
cheers
andrew
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Information Schema and constraint names not unique
Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
. add tableoid or tablename to information_schema.{check_constraints,
referential_constraints} (I think those are the only places where it
would be needed, from my quick skimming).
. add tableoid or tablename to autogenerated table constraint names
These are mutually exclusive --- I see no reason to do both.
In that case I vote for option 2 - it makes the names nicer and gets us
closer to compliance with the spec.
:-)
(Option 1 is trivially easy by comparison, though).
cheers
andrew
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [HACKERS] What do you want me to do?
Alvaro Herrera wrote: I don't have an opinion on the Win32 issue. I do :-) I think the most important thing for Win32 is for you to set the direction somewhat (i.e. in more detail than is on your win32 page) and then jump on Joshua's offer of a dedicated developer (possibly two) to work on it for 320 hours. cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] What do you want me to do?
Marc G. Fournier wrote: On Fri, 7 Nov 2003, Robert Treat wrote: I know most people have talked about using bugzilla, but is anyone familiar with GNATS? I'm currently rereading Open Sources and there's a paragraph or two mentioning it's use and the fact that it can be interfaced with completely by email. FreeBSD uses it almost exclusively and it supports email interaction with the database, but I don't think there are very many good GUI front ends for it (or, at least, not that I've seen) ... No. A few other thoughts: . the Samba team have apparently abandoned their own tool and moved to bugzilla . if we used bugzilla this might give some impetus to the bugzilla team's efforts to provide pg as a backend (maybe we could help with that) . it would seem slightly strange to me for an RDBMS project to use a bug tracking system that was not RDBMS-backed . developers are far more likely to be familiar with bugzilla . are there any active developers without web access? If not, why is pure email interaction important? Bugzilla is far from perfect. But it's getting better. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] What do you want me to do?
Christopher Browne wrote: [EMAIL PROTECTED] (Bruce Momjian) writes: The worst was my upstairs hallway that had no light fixtures, so late at night if no other lights were on in the house, you had to walk down the hallway with your hands out in front of you so you didn't bump into anything. We had a nightlight in the hallway, but that didn't help much. Here's the solution: http://www.nightvis.com/site/hm/pvs5/default.asp Excalibur's Dual Tube Goggle Systems are precision-manufactured night vision devices which provide near-daylight operational capabilities at night. A built-in Infrared Light Emitting Diode (IR LED) provides on-call covert illumination for close-up work in totally dark areas. In addition, the dual tube configuration improves reliability and aids in the physical coordination of tasks, such as traversing uneven terrain, or performing complicated tasks by hand. Or the low-tech solution: don't go upstairs late at night ... ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Experimental ARC implementation
Greg Stark wrote: Bruce Momjian [EMAIL PROTECTED] writes: I am not really aiming at removing sync() alltogether. We know already that open,fsync,close does not guarantee you flush dirty OS-buffers for which another process might so far only have done open,write. And you So for what it's worth, though the spec seems to indicate fsync is only guaranteed to sync writes to that file descriptor, in reality all known VFS's do not associated dirty buffers with particular file descriptors. At least I checked with people that NetBSD and Solaris do not. Both developers said they were unaware of any OS that kept dirty buffers per-fd and couldn't imagine anyone wanting to do that. It would be fairly easy to check Linux. All the others out there are fairly closely related to either NetBSD or Solaris. The Linux man page for fsync says this: fsync copies all in-core parts of a file to disk, and waits until the device reports that all parts are on stable storage. It also updates metadata stat information. which seems to imply similar behaviour on Linux as for other OSs. What I could understand of the source code in mm/filemap.c and fs/buffer.c appeared to confirm that. cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] bugzilla (Was: What do you want me to do?)
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Bugzilla is far from perfect. But it's getting better. FWIW, I would like to try a bugzilla-based tracking system for Postgres. Our last attempt at a tracking system failed miserably, but I think that was (a) because the software we tried was really unpolished, and (b) because we let anybody and his pet chihuahua enter bug reports, so the signal-to-noise ratio went to zero in no time. As long as we can restrict data entry to people who know what they're doing (not necessarily developers, but people who know PG well enough to tell bug from user error), I think it could work, and would beat the heck out of the way we do things now. . if we used bugzilla this might give some impetus to the bugzilla team's efforts to provide pg as a backend (maybe we could help with that) Red Hat has been using a PG-based version of bugzilla for some time. I'm not sure what the holdup is in getting that work merged back upstream, but I'd sure like to see it happen. Anyway we could start with using their version, rather than suffer the ignominy of using That Other Database to track our own bug reports ;-) The status of this can be seen at: http://bugzilla.mozilla.org/show_bug.cgi?id=98304 This item is listed on their Master Plan page at http://www.mozilla.org/projects/bugzilla/roadmap.html as being in the category Things we want in 2.18 but will get pushed to 2.20 if they're not completed by the time everything in the above list. I'd hate that to happen. The last comment on the bug page says: The Red Hat guys did a quick 'n dirty port. It works, but doesn't quite make use of the best of PostgreSQL. Also, their tarball is out of date with the current schema used by Bugzilla. My experience is that migrating to new versions of bugzilla is a major pain, so I'd hate to start out with something we suspect we would have to throw away later. The bug is actually assigned to David Lawrence at RedHat - maybe you'd like to get some status from him? :-) . are there any active developers without web access? If not, why is pure email interaction important? Bugzilla already does email output (ie, notify you of changes to bug entries you're interested in) well enough. We thought during the last go-round that it was important to have email input so we could allow mail to pgsql-bugs to go directly into the tracking system, but in hindsight that was a really bad idea. What we could use instead is for someone knowledgeable to commit to transferring *valid* emailed bug reports into the tracking system. Bruce could do that if he wants, but there are surely dozens of other people who would be qualified to handle this task. Actually, whatever software we pick to run the tracking system, my guess is that the experiment will not stand or fall on the software. What we need for success is one or two people who will take responsibility for housekeeping: putting in valid reports, spotting duplicate reports and doing the right cleanup, etc. Do we have any volunteers for that sort of thing? All good points. Bug triage is critical to success in my experience. You can take the suggested approach of trying to rule them out before they get into the system, or be aggressive about triage when they do get there - I've seen both work. RedHat allows anybody (with or without pooch) to sign up for an account and enter bugs, and I've had good responses myself from them for bugs I've filed. There is a certain niceness and openness about doing things that way, and I'm not sure the triage effort is any greater. Your housekeeper looks at today's list and either rules something not a bug or assigns it. For emailed bugs I agree doing triage before they get into the system makes sense. And, since I have argued for it I guess I should volunteer to help, although my knowledge of pg internals is still on the steep part of the learning curve. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] What do you want me to do?
Robert Treat wrote: On Fri, 2003-11-07 at 15:28, Andrew Dunstan wrote: Marc G. Fournier wrote: On Fri, 7 Nov 2003, Robert Treat wrote: I know most people have talked about using bugzilla, but is anyone familiar with GNATS? I'm currently rereading Open Sources and there's a paragraph or two mentioning it's use and the fact that it can be interfaced with completely by email. FreeBSD uses it almost exclusively and it supports email interaction with the database, but I don't think there are very many good GUI front ends for it (or, at least, not that I've seen) ... No. personal axe to grind? er, no. I was only agreeing with Marc about GUI interfaces. What axe to grind do you imagine I could have? Postgres is a fine product, and I have been very glad to find that its development process is very open in fact as well as in name. I want to see it succeed. To that end I want to free Bruce and Tom and everybody else from as much drudgery as possible and at the same time make finding out the state of things easier. That's all. I've never used it, but it's been around a long time, allows for interaction completely through email (which is how we do things now), has a web front end for anyone who wants to use it to use, and as i understand it has a tcl based desktop app for folks to use as well. seems it's being dismissed prematurely imho. Every person wishing to submit a bug will have to have send-pr installed or else we'll get lots of reports not broken up into fields. That doesn't sound like a recipe for success to me. A few other thoughts: . the Samba team have apparently abandoned their own tool and moved to bugzilla . if we used bugzilla this might give some impetus to the bugzilla team's efforts to provide pg as a backend (maybe we could help with that) . it would seem slightly strange to me for an RDBMS project to use a bug tracking system that was not RDBMS-backed we serve far more static pages on the website than we do database driven ones... *nod* but there has been talk of moving to bricolage, hasn't there? the software we distribute is housed on fileservers and sent via ftp, we dont expect people to store and retrieve it from a database... you're reaching now ... our mailing lists software actually uses another db product in fact... let's just get the right tool for the job... Yes. I agree. Bugs (including enhancements) strike me as a classic case of data that belongs in a database. . developers are far more likely to be familiar with bugzilla developers are far more likely to be familiar with windows and mysql as well... c'mon ... . are there any active developers without web access? If not, why is pure email interaction important? for the same reason mailing lists work better than message boards... it's just easier. i'm much more likely to read an email list the scroll through web forms, and if i am going to respond to a bug report, i'm much mroe likely to if i can hit reply and start typing than if i have to fire up a browser to do it. Tom explicitly said he *didn't* want a system where email poured straight into the bugtrack db. Yes, it is a different way of doing things, and it takes getting used to. Bugzilla is far from perfect. But it's getting better. don't get me wrong, i like bugzilla and all, but theres no need to put blinders on... I don't. But I do think the current processes can stand improvement. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] What do you want me to do?
Dave Cramer wrote: Jira is a fantastic bug tracking project management system and is available free of charge for open source projects. http://www.atlassian.com/software/jira/ Wow, that looks very cool indeed! And they are Aussies to boot! :-) cheers andreew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] What do you want me to do?
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: I don't think we need decicated bug transferrers. Typically, when someone reports a problem by email, the first step is that some developer or other expert responds (unless the reporter gets blown away by fellow users as clueless :-)). So the natural extension of this process would be that the person doing the analysis records the problem. Yeah, that sounds like it would work. I still think it would be a good idea to have one or two people actively in charge of the overall health of the bug repository --- cross-linking duplicate bugs, making sure fixed bugs get closed out, in general correcting misinformation when they find it. This wouldn't be a large time commitment AFAICS, but without somebody applying pressure in the right direction I think that the general quality of information in the database would inevitably slide downhill. You have described a good part of my professional life in the last 3 years ;-) I had a meeting every morning with product/project management to review/triage bugs and in turn I would spend hours asking my staff What is happening with bug xyz?. I lived off the bug system (bugzilla and/or ClearQuest). Getting developers used to it is still a hassle - I once had to send out an email that said in effect if you aren't working on a defect assigned to you then you aren't doing your job. In a volunteer project things work somewhat differently, of course, but the housekeeping functions are still essential. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] bugzilla (Was: What do you want me to do?)
Christopher Kings-Lynne wrote: The doesn't quite make the best use of PG quote is one of the best examples of buck-passing I've seen in awhile. If Bugzilla had been designed with some thought to DB independence to start with, we'd not be having this discussion. You have to laugh at an app that actually uses MySQL's replication to get around not having row locks!!! And it actually has a sanity check feature for 'checking' your referential integrity. I laughed so hard I cried. And yet we use it at work :P Bugzilla was put together by a guy who admitted he didn't know much about databases, IIRC. It worked and he was in a hurry. This is a classic story of a piece of software that is far more long lasting and far more dirty than was originally intended - I have seen it before many times and I expect to see it until I die. I could tell many similar stories that would make you laugh/cry even harder, but this isn't the place or time :-) Seriously, I have wondered if it might be a good idea to assemble a small hit team that would take some high profile open source projects and make sure they worked with Postgres. Bugzilla would be the most obvious candidate, but there are certainly others. I suspect that could be quite productive, though. Thoughts? andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [7.3.x] function does not exist ... ?
Marc G. Fournier wrote:
'k, this doesn't look right, but it could be that I'm overlooking
something ...
The function I created:
CREATE FUNCTION month_trunc (timestamp without time zone) RETURNS timestamp without
time zone
AS 'SELECT date_trunc(''month'', $1 )'
LANGUAGE sql IMMUTABLE;
The query that fails:
ams=# select * from traffic_logs where month_trunc(runtime) = month_trunc(now());
ERROR: Function month_trunc(timestamp with time zone) does not exist
Unable to identify a function that satisfies the given argument types
You may need to add explicit typecasts
The query that succeeds:
ams=# explain select * from traffic_logs where month_trunc(runtime) = '2003-10-01';
QUERY PLAN
---
Index Scan using tl_month on traffic_logs (cost=0.00..30751.90 rows=8211 width=36)
Index Cond: (month_trunc(runtime) = '2003-10-01 00:00:00'::timestamp without time
zone)
(2 rows)
I haven't mis-spelt anything that I can see ... is this something that is
known not to be doable?
Try casting now() to timestamp without time zone?
cheers
andrew
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [HACKERS] bugzilla (Was: What do you want me to do?)
Peter Eisentraut wrote: Andrew Dunstan writes: Seriously, I have wondered if it might be a good idea to assemble a small hit team that would take some high profile open source projects and make sure they worked with Postgres. Bugzilla would be the most obvious candidate, but there are certainly others. I suspect that could be quite productive, though. Good thought, but a hit team is not the right answer, because any project that would have been hit in this way will just go bad again the moment its database layer is changed. What would work better are consultants: people that hang around on the other project's mailing lists, offer advise on database layer modelling and implementation, do clean up tasks, check regularly if everything works with the PG development branch, be there when the developers of that other project have a question. I've been doing a bit of that, and my sensation is that most developers of database-backed applications are dying to have people like that at their disposal. Well, I didn't mean hit and run team ;-). I take your point, though. However, some projects will need a big effort up front - Bugzilla certainly will. I am doing some work on it but any help will be appreciated. Part of the problem is that there is no layering of the app, separating out the logical and physical views. So instead of a nice procedural layer where one could concentrate the creation of Db-specific SQL, it is littered throughout the app. Some of the issues include: . efficient replacement of the enumerated types . transactional safety . properly replacing the calles to replace into . full text searching . migration scripts As you can see, doing it properly is quite a big job. However, I also think that there is value in identifying those projects that will give the best bang for the buck for our project, and then trying to concentrate some resources on those. Your suggestion elsewhere of pick your second favourite app is likely to result in a more scattergun approach. Also, if it had the imprimatur of the PostgreSQL community to some extent appraoches to projects might be more welcome - Dear open-source-project-manager, on behalf of the PostgrSQL community we would like to offer you assistance in making sure your application works with PostgrSQL, the world's most advanced open-source database system cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] bugzilla (Was: What do you want me to do?)
Peter Eisentraut wrote: Andrew Dunstan writes: Your suggestion elsewhere of pick your second favourite app is likely to result in a more scattergun approach. Also, if it had the imprimatur of the PostgreSQL community to some extent appraoches to projects might be more welcome - Dear open-source-project-manager, on behalf of the PostgrSQL community we would like to offer you assistance in making sure your application works with PostgrSQL, the world's most advanced open-source database system The only way someone is going to get work done on a sustained basis is if he's got a personal interest, the so-called itch. You're not going to achieve anything, except possibly being ridiculed, if you start sending out form letters on behalf of the PostgreSQL community. If people already support PostgreSQL to some extent, go there and test it and send in patches with improvements. If people don't support PostgreSQL yet, get a good sense for what the feeling of the project maintainers toward database abstraction layers is, then throw out a design plan. But the key is to show results, not intentions. That is how open-source development works. *shrug* I'm not sending out anything. OpenSource works in lots of different ways, in my experience. Some projects welcome all comers, some are very exclusive, for example. Anyway, in relation to bugzilla, I am working on stuff to submit to them, so I won't be faced with show me the code challenges. I nearly have a db-independant table creation module ready, but that will be just a start. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] rpm support for 7.4 and beyond
Tom Lane wrote: Robert Treat [EMAIL PROTECTED] writes: Also (and maybe someone from Red Hat can weigh in here) are there any plans from Red Hat to release RHEL rpms for postgresql in the future, I can tell you that Red Hat is getting beat up regularly for having omitted Postgres (and MySQL!) from RHEL 3. If you are a paying RHEL customer, make sure you let them know you're not happy about it. Upper management keep changing their mind about how exactly they want to support these databases on RHEL --- well, okay, that's a business decision and outside my sphere as an engineer. But in the meantime the effective support is none at all, and people have got to hold their feet to the fire about it. I just spent 15 minutes searching on the RH web site trying to locate a complete list of packages in the various RHEL personalities, with conspicuous lack of success. How anyone can make decisions about it without knowing exactly what is in it is beyond me. To answer Robert's original question, I suspect that there will be many corporate and individual users of RH 7.x, 8 and 9 for quite some time, and providing RPMs for these will be highly desirable. I currently have access to one of each of these (fairly vanilla installations, too), and can help out if need be while that lasts - I have no plans to upgrade any of them right now, and would only want to do so if some critical issue came up that forced it - we are behind a firewall so even remote root exploits might not make me upgrade. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] rpm support for 7.4 and beyond
I wrote: I just spent 15 minutes searching on the RH web site trying to locate a complete list of packages in the various RHEL personalities, with conspicuous lack of success. How anyone can make decisions about it without knowing exactly what is in it is beyond me. However, you can see a list of its SRPMs here: http://distro.ibiblio.org/pub/Linux/distributions/redhat/enterprise/3/en/os/i386/SRPMS/ and presumably on other mirrors. it includes rh-postgresql-7.3.4-8.src.rpm (14.9Mb) and mysql-3.23.58-1.src.rpm (11.5 Mb) cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] cvs head? initdb?
and the problem seen here with the order of messages sent by initdb has been fixed by Tom in the head branch (which is the only place it would have occurred). cheers andrew Robert Treat wrote: They are no longer in sync. Jan has started committing some of his ARC work (though I think it might be backed out currently) and Tom has committed his cross data-type index work. Both have reported errors with initdb so it might not be completely stable right now. generally it should be, but not always, you can report errors if you want or wait until they sort it out. You should still be able to get an RC2 build with the appropriate tags applied. HTH Robert Treat On Wed, 2003-11-12 at 21:04, elein wrote: What is the status of CVS head? Isn't it in sync with 7.4.RC2? I just upgraded from CVS and rebuilt clean and initdb now gives this lovely informative initdb failed message. I haven't had trouble previously with cvs head or other versions. $ sudo make install ... PostgreSQL installation complete. /local/src/postgres-cvs/pgsql[elein:cookie] $ type initdb initdb is hashed (/local/pghead/bin/initdb) /local/src/postgres-cvs/pgsql[elein:cookie] $ initdb The database cluster will be initialized with locales COLLATE: C CTYPE:en_US MESSAGES: en_US MONETARY: en_US NUMERIC: en_US TIME: en_US initdb: failed initdb: removing contents of data directory /local/pghead/data creating directory /local/pghead/data ... /local/src/postgres-cvs/pgsql[elein:cookie] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] heads up -- subtle change of behavior of new initdb
The shell script said this: $ECHO_N fixing permissions on existing directory $PGDATA... $ECHO_C chmod go-rwx $PGDATA || exit_nicely There's no more rationale than that for this patch. I'm inclined to agree with you, though. cheers andrew Greg Stark wrote: + if (!chmod(pg_data,0700)) Out of curiosity, what was the rationale for using 0700? I know it was a pain for me when I had a script to monitor the tmp usage. Surely read access to privileged users isn't really a problem? I'm thinking more of loosening the paranoia check elsewhere rather than this default. Wouldn't at least 0750 be safe? That way putting a user in the postgres group would grant him access to be able to browse around and read the files in pg_data. Actually I should think 02750 would be better so that the group is inherited by subdirectories. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] heads up -- subtle change of behavior of new initdb
Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: I'm not suggesting making that the default setup, just loosening the paranoia check so that if an admin sets the directory to be group readable the database doesn't refuse to start up. In previous discussions of this point, paranoia was considered desirable. I don't think the situation has changed. Would it be worth having a command line option to relax the paranoia a bit, leaving the current paranoia setting as the default? I guess it would have to be on the command line because IIRC this is checked before we ever look at the config file. cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] oh dear ...
Marc G. Fournier wrote: On Fri, 14 Nov 2003, Bruce Momjian wrote: Tom Lane wrote: I said: This worked in 7.3: regression=# select '1999-jan-08'::date; ERROR: date/time field value out of range: 1999-jan-08 HINT: Perhaps you need a different datestyle setting. Setting DateStyle to YMD doesn't help, and in any case I'd think that this ought to be considered an unambiguous input format. This appears to be an oversight in the portions of the datetime code that we recently changed to enforce DateStyle more tightly. Specifically, DecodeNumber was rewritten without realizing that it was invoked in a special way when a textual month name appears in the input. DecodeDate actually makes two passes over the input, noting the textual month name in the first pass, and then calling DecodeNumber on only the numeric fields in the second pass. This means that when DecodeNumber is called for the first time, the MONTH flag may already be set. The rewrite mistakenly assumed that in this case we must be at the second field of an MM-DD-YY-order input. I propose the attached patch to fix the problem. It doesn't break any regression tests, and it appears to fix the cases noted in its comment. Opinions on whether to apply this to 7.4? I guess the question is whether we would fix this in a minor release, and I think the answer it yes, so we can fix it now. Ah, so we attempt to fix a bug that affects what appears to be a small % of configurations with quick testing and with the greater possibility of affecting a larger % of configurations ... instead of releasing what we has been reported as being stable on the large % of configurations, and fixing it for that small % of configuratiosn in a minor release? Sounds to me like a decision design to benefit the few at the risk of the many ... when documenting the known bug for those few would be safer ... I'm confused. My understanding from what Tom said is that it affects all configurations. cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] heads up -- subtle change of behavior of new initdb
darnit!
patch attached.
(Thinks - do we need to worry about suid sgid and sticky bits on data dir?)
andrew
Tom Lane wrote:
Joe Conway [EMAIL PROTECTED] writes:
I just noticed tonight that the new initdb introduced a subtle change of
behavior. I use a shell script to automate the process of
- rm old data directory
- mkdir new data directory
- initdb
- load from pgdumpall
Now, that second step is not needed, but as of today it produces an
installation that won't start due to improper permissions on data
That's a bug --- evidently the fix permissions path of control is
wrong; can you take a look?
? .deps
? initdb
Index: initdb.c
===
RCS file: /projects/cvsroot/pgsql-server/src/bin/initdb/initdb.c,v
retrieving revision 1.7
diff -c -w -r1.7 initdb.c
*** initdb.c13 Nov 2003 23:46:31 - 1.7
--- initdb.c14 Nov 2003 06:47:50 -
***
*** 2345,2350
--- 2345,2359
made_new_pgdata = true;
}
+ else
+ {
+ printf(fixing permissions on existing directory %s... ,pg_data);
+ fflush(stdout);
+ if (!chmod(pg_data,0700))
+ exit_nicely();
+ else
+ check_ok();
+ }
/* Create required subdirectories */
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] [HACKERS] heads up -- subtle change of behavior of
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: darnit! patch attached. Applied with correction (you got the return-value check backwards) and further work to deal reasonably with error conditions occurring in check_data_dir. darnit again. I'm taking a break - my head is swimming with Java, JavaScript, Perl, HTML and XML/XSL from my real (i.e. paying) work, and context switching is causing massive mental thrashing. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Connexions question
Er, that's a per-server limit, not a per-database limit (which is what he asked for), isn't it? cheers andrew Dann Corbit wrote: It's a command line option for the server. http://developer.postgresql.org/docs/postgres/app-postmaster.html -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Monday, November 17, 2003 11:21 AM To: pgsql-hackers list Subject: [HACKERS] Connexions question Hi all, I don't think it's a FAQ, Is it possible to limit then number of simultaneous connexions one can make to a particular databse. E.G: I have 128 connexions max of witch I wan't to restrict at most 60 to database x, leaving in the worst case 68 for all others... Am I clear? ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [pgsql-advocacy] Not 7.5, but 8.0 ?
Claudio Natoli wrote: I'm sorry if I'm being alow here - is there any problem with running a production server on cygwin's postgresql? Is the cygwin port of lesser quality, or otherwise inferior? Performance, performance, perfomance... and perfomance... it is (almost) always worse perfomance when we emulate something... and using Cygwin we are emulating U*nix... Absolutely. The DB throughput available to our application with postgresql under cygwin is about 1/3 of what we get under Linux with a similar spec machine/config. That, and, more importantly, the odd spurious cygipc lock up, precludes our use of postgresql/cygwin in a production setting. And not having postgresql available on all our target platforms (which includes Windows) precludes the use of it at all, as we desire a single DB solution. I don't imagine we are the only ones in this situation (and to all those who see a Windows port as uninteresting, please keep this in mind). You are far from alone. And there's one other factor: most large enterprises have quite strict policies about what can be installed on their data center servers. Getting Cygwin past those policies would often be difficult. That factor alone was enough to make my product manager rule Postgres out as a solution that we would bundle with our software. Hopefully, we can change this situation soon... Right. Here's the situation as I see it: . there have been lots of requests for a native Win32 port . this is important to some people and not important to others . the decision has long ago been made to do it, and some work has been done, and more is being done Isn't it time to move on? As for release numbering, ISTM that is not fundamentally very important. At my former company we had code names for branches and decided release names/numbers near release time in accordance with marketing requirements. Let's not get hung up on nominalism. A release number is just a tag and we can call it whatever seems good at the time. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [pgsql-advocacy] Not 7.5, but 8.0 ?
Bruce Momjian wrote: I am ready to work with anyone to make fork/exec happen. It requires we find out what globals are being set by the postmaster, and have the child run those same routines. I can show you examples of what I have done and walk you through areas that need work. If you look at the EXEC_BACKEND defines in CVS, you can see what I have done so far. We need to have EXEC_BACKEND working on Unix first, then we can add the CreateProcess call on Win32, so all this can be done on Unix first. How is EXEC_BACKEND going to be enabled? A configure option? A global define? cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [pgsql-advocacy] Not 7.5, but 8.0 ?
ow wrote: Have *never* seen ppl running Oracle or Sybase on Windows. Not sure about DB/2 or Informix, never worked with them, but I'd suspect the picture is the same. Then you need to get out more. I have seen Oracle, Sybase, DB2 (and probably Informix, I forget) all running on Windows in a number of large enterprise data centers. They may claim that they have win port but it's more of a marketing gimmick than a useful feature that affects real, not hypothetical, users. IMHO, core postgreSql development should not be sacrificed for the sake of win32 port. Nobody is sacrificing anything. As usual, people are working on the things that they want to work on. A Win32 port is clearly not important *to*you*. It is to others, and it's going to happen. You might dislike the decision but you need to get over it. If you feel other things are more important feel free to contribute to that work. I am sure the core team will make sure that the Win32 work does not break or degrade the product on Unix, so why the heck should you even care? I'm not a big Windows fan either, but I also live in the real world. I suspect that goes for most of us who want to see this work done. I still don't know why we are even having this discussion. andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [pgsql-www] [HACKERS] Release cycle length
Josh Berkus wrote: Guys, I agree with Neil ... it's not the length of the development part of the cycle, it's the length of the beta testing. I do think an online bug tracker (bugzilla or whatever) would help. I also think that having a person in charge of testing would help as well ... no biggie, just someone whose duty it is to e-mail people in the community and ask about the results of testing, especially on the more obscure ports. I think a few e-mail reminders would do a *lot* to speed things up. But I'm not volunteering for this job; managing the release PR is herding cats enough! Maybe some sort of automated distributed build farm would be a good idea. Check out http://build.samba.org/about.html to see how samba does it (much lighter than the Mozilla tinderbox approach). We wouldn't need to be as intensive as they appear to be - maybe a once or twice a day download and test run would do the trick, but it could pick up lots of breakage fairly quickly. That is not to say that more intensive testing isn't also needed on occasion. cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] Build farm
Marc G. Fournier wrote: On Tue, 18 Nov 2003, Andrew Dunstan wrote: Maybe some sort of automated distributed build farm would be a good idea. Check out http://build.samba.org/about.html to see how samba does it (much lighter than the Mozilla tinderbox approach). We wouldn't need to be as intensive as they appear to be - maybe a once or twice a day download and test run would do the trick, but it could pick up lots of breakage fairly quickly. That is not to say that more intensive testing isn't also needed on occasion. Check the archives on this, as its been hashed out already once at least ... I think the big issue/problem is that nobody seems able (or wants) to come up with a script that could be setup in cron on machines to do this ... something simple that would dump the output to a log file and, if regression tests failed, email'd the machine owner that it needs to be checked would do, I would think ... If there's general interest I'll try to cook something up. (This kind of stuff is right up my alley). I'd prefer some automated display of results, though. A simple CGI script should be all that's required for that. cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Build farm
Peter Eisentraut wrote: Andrew Dunstan writes: If there's general interest I'll try to cook something up. (This kind of stuff is right up my alley). I'd prefer some automated display of results, though. A simple CGI script should be all that's required for that. The real problem will be to find enough machines so that the build farm becomes useful. IMO, that would mean *more* machines than are currently lines in the supported-platforms table. Useful is probably subjective. That list would at least be a good place to start, though. What combinations of variables do you think we would need? This would be a fairly painless way for users to be helpful to the project, btw - the way I am envisioning things this would be fairly much a set and forget process. I'll have an example page available in a few days. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] initdb segfaults - latest cvs
strk wrote: Running initdb: creating template1 database in /pgroot-cvs/data/base/1 ... child process was terminated by signal 11 It is working fine for me (RH9). Can you provide more details? Platform? How you are calling initdb? cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Build farm
Peter Eisentraut wrote: Andrew Dunstan writes: Useful is probably subjective. That list would at least be a good place to start, though. What combinations of variables do you think we would need? First of all, I don't necessarily think that a large list of CPU/operation system combinations is going to help much. IIRC, this round of platform testing showed us two real problems, and both happened because the operating system version in question came out the previous day, so we could not have caught it. Much more problems arise when people use different versions of secondary packages, such as Tcl, Perl, Kerberos, Flex, Bison. So you would need to compile a large collection of these things. The problem again is that it is usually the brand-new or the odd intermediate version of such a tool that breaks things, so a set and forget build farm is not going to catch it. Another real source of problems are real systems. Weird combinations of packages, weird network setups, weird applications, custom kernels. These cannot be detected on out of the box setups. In fact, the regression tests might not detect them at all. Hence the open-source community approach. Closed-source development teams can do all the above, with great effort. But by throwing out the code and have real people test them on real systems with real applications, you can do much better. The fact that something doesn't find everything doesn't mean it is of no value. (Thinks of Scott Adams' nice example: Your theory of gravity doesn't prove why there are no unicorns, so it is wrong. ;-) ) I don't believe there is a single open source community approach - open source projects all have differing ways of handling problems. At least 2 very significant open source projects I know of run build farms, notwithstanding that your objections should apply equally to them. Mozilla's is fairly centralised and very complex and heavy, but gives fairly immediate feedback if anything gets broken. Samba's is much lighter, distributed, and they still apparently see good value in it. (Samba uses a torture test - perhaps we need one of those in addition to the regression tests.) Maybe it wouldn't be of great value to PostgreSQL. And maybe it would. I have an open mind about it. I don't think incompleteness is an argument against it, though. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Build farm
Peter Eisentraut wrote: The Samba build daemon suite is pretty good. We have a couple of those hosts in our office in fact. (I think they're building PostgreSQL regularly as well.) A tip: You might find that adopting the source code of the Samba suite to PostgreSQL is harder than writing a new one. Yes, I agree. I have looked at it for ideas, but not for code. I'm not using rsync or anything like that, for instance. I'm going for something very simple to start with. Essentially what I have is something like this pseudocode: cvs update check if there really was an update and if not exit configure; get config.log make 21 | make-filter makelog make check 21 | check-filter checklog (TBD) send config status, make status, check status, logfiles make distclean The send piece will probably be a perl script using LWP and talking to a CGI script. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 4 Clause license?
This whole thing is starting to make my head hurt. There has been more effort spent over this license issue than I would have spent if I hadn't taken the shortcut of using the FreeBSD code. I think maybe the simplest thing is for me to prepare a patch that rips that code out and replaces it with a (slightly simpler - less umask hacking required, I think) piece of code that I will write. I won't make the mistake of borrowing code again like this. (I naively thought using FreeBSD code would be safe, and was careful not to use any GPL code.) cheers andrew Terry Lambert wrote: Erik Trulsson wrote: On Mon, Nov 17, 2003 at 02:48:08PM -0500, Rod Taylor wrote: The PostgreSQL group has recently had a patch submitted with a snippet of code from FreeBSDs src/bin/mkdir/mkdir.c. http://www.freebsd.org/cgi/cvsweb.cgi/src/bin/mkdir/mkdir.c?annotate=1.27 Is this intentionally under the 4 clause license or does the copyright from the website (2 clause) applied to everything that is non-contrib? http://www.freebsd.org/copyright/freebsd-license.html That copyright notice on the website should apply to everything that is not under some other license. Different parts of the system is under different licenses and copyrights depending on who wrote it. The mkdir.c *was* under the 4 clause license. However all material that was part of the original BSDs and thus was copyrighted by The Regents of the University of California has had its license changed such that clause 3 (the advertising clause) no longer apply. People seem to frequently misunderstand what a license is, and more specifically, what the conversion from a 4 clause to a 3 clause license meant, in the case of the UCB License. This change does not apply to derivative works, only to the original code itself. So if you went back and grabbed the mkdir.c code off the BSD 4.4-Lite2 CDROM, and used that, fine. If you grabbed the mkdir.c off the FreeBSD sources, and even one line was modified by someone, then it's a derivative work, and, unless you can also get written permission from the contributor, it stays under the license from which it was derived. The announcement by the University only permits the change, it does not mandate the change, for this very reason: otherwise third party redistributed code would have sudddenly become legally questionable. By the same token, if you dual-license some code under th GPL and another license, and someone gets the GPL'ed version, and makes changes, unless thy specifically permit it, the code contributed back is only licensed under the GPL. This is why SGI licensing the XFS code under the GPL was a stupid move: a contributer contributing code back results in an improved code base that can only be used under the terms of the GPL, and not in SGI's commercial product offerings. I believe that SGI did not actually expect any significant or worthwhile bug fixes or enhancements to come from the GPL'ed code using community. In terms of getting written approval for the license change from other contributors, this is basically the role that the Regents of the University of California and the UCB CSRG were fulfilling: a legal entity to whom such representations could be made by contributors, and who could then legally forward those representations to another. FreeBSD has no such legal entity, at present. The closest you could come is perhaps the FreeBSD Foundation. Had there been a FreeBSD Foundation from day on, to whom rights could have been assigned by contributors (turning it into The FreeBSD Foundation and its Contributors), then the license would be capable of being modified after the fact. Without that, however, you must track down all of the individual contributors to get the license changed. My recommendation is to us the code off the 4.4 BSD-Lite2 CDROM, if you can, live with the 4 clause license if the code contains changes you need, if you can, or contact the contributors, if it is a small enough job. If none of those things will work for you, then start with the 4.4 BSD-Lite2 CDROM code, convert to the 3 clause license, as permitted by the university, and then hack out whatever modifications you ned on top of that for yourself. -- Terry ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Build farm
Peter Eisentraut wrote: Andrew Dunstan writes: Essentially what I have is something like this pseudocode: cvs update Be sure check past branches as well. check if there really was an update and if not exit OK. configure; get config.log Ideally, you'd try all possible option combinations for configure. Or at least enable everything. I have had in mind from the start doing multiple configurations and multiple branches. Right now I'm working only with everything/head, but will make provision for multiple sets of both. How many branches back do you think should we go? Right now I'd be inclined only to do REL7_4_STABLE and HEAD as a default. Maybe we could set the default to be gettable from the web server so that as new releases come along build farm members using the default wouldn't need to make any changes. However, everything would also be settable locally on each build farm member in an options file. make 21 | make-filter makelog make check 21 | check-filter checklog You could also try out make distcheck. It tries out the complete build, installation, uninstallation, regression test, and distribution building. OK. (TBD) send config status, make status, check status, logfiles OK. make distclean When I played around with this, always copied the CVS tree to a new directory and deleted that one at the end. That way, bugs in the clean procedure (known to happen) don't trip up the whole process. OK. We've also seen odd problems with cvs update, I seem to recall, but I'd rather avoid having to fetch the entire tree for each run, to keep bandwidth use down. (I believe cvs update should be fairly reliable if there are no local changes, which would be true in this instance). The send piece will probably be a perl script using LWP and talking to a CGI script. That will be the difficult part to organize, if it's supposed to be distributed and autonomous. sending the results won't be a huge problem - storing and displaying them nicely will be a bit more fun :-) Upload of results would be over authenticated SSL to prevent spurious results being fed to us - all you would need to join the build farm would be a username/password from the buildfarm admin. Thanks for your input cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 4 Clause license?
Rod Taylor wrote: I think maybe the simplest thing is for me to prepare a patch that rips that code out and replaces it with a (slightly simpler - less umask hacking required, I think) piece of code that I will write. The FreeBSD folks sorted it out for us. Everyones names should be in the copyright for the file. The licence portion should be the 3 clause version -- no advertising clause. Whose names? It's not easily discoverable from browsing the CVS tree. I would probably spend as much time, if not more, finding out as in rewriting the 40 or so lines of code required. I think borrowing should be encouraged, and now that we know what license / copyright we need to carry over, this can be done without worry. I'm a fan of borrowing, but not if it causes headaches. I'll hold off for a bit in case I've misunderstood something. trollOf course, now that SCO is claiming ownership of BSD code . /troll cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] 4 Clause license?
Marc G. Fournier wrote: On Thu, 20 Nov 2003, Joshua D. Drake wrote: Based on the below wouldn't they also have to go after Microsoft? Depends ... does MicroSoft use BSD TCP/IP, or did they write their own? I know that Linux is not using BSD TCP/IP (or, at least, they didn't in their first 3 incarnations of the stack) ... M$ have used BSD code in the past, I'm fairly sure, even if they no longer do. But they do have a license from SCO. OK, I know I shouldn't have raised this topic ... cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Build farm
Tom Lane wrote: I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. Yes, I seem to recall seeing several instances of things like you mean foonix version 97 1/2 has a bad frobnitz.h? over the last 6 months. Having that caught early is exactly the advantage, I believe. I'm for it ... I'm working on it :-) Regarding make distcheck that Peter suggested I use, unless I'm mistaken it carefully does its own configure, thus ignoring the configure options set in the original directory. Perhaps we need either to have the distcheck target pick up all the --with/--without and --enable/--disable options, or to have a similar target that does that. Thoughts? cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] initdb mkdir_p() doesn't work
Peter Eisentraut wrote: Here is what I get: peter ~$ pg-install/bin/initdb pg-install/var/data ... creating directory pg-install/var/data ... initdb: failed No points for details in the error message here either. If I create pg-install/var first, then it work. I will check it out. I know I spent quite some time making sure this worked, but I might have missed something obvious. I wonder if it is platform specific? cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] initdb mkdir_p() doesn't work
Tom Lane wrote: AFAICS mkdatadir() shouldn't consider subdir == NULL as a reason to fail rather than trying mkdir_p. Indeed, if anything the opposite: when subdir isn't NULL the immediately prior directory level should exist already. Right. In fact, I can't see any good reason to call mkdir and then mkdir_p at all. See my patch from this afternoon. I concur with Peter's gripe that a perror() or two wouldn't hurt here. Sure. Of course, the reason I put this on my web site and asked for eyeballs was to try to catch some of this sort of stuff before the program went into the tree :-) cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] initdb mkdir_p() doesn't work
Andrew Dunstan wrote:
Peter Eisentraut wrote:
Here is what I get:
peter ~$ pg-install/bin/initdb pg-install/var/data
...
creating directory pg-install/var/data ... initdb: failed
No points for details in the error message here either.
If I create pg-install/var first, then it work.
I will check it out. I know I spent quite some time making sure this
worked, but I might have missed something obvious. I wonder if it is
platform specific?
I don't remember why the code is the way it is. The failure appears to
be before we ever get to mkdir_p(). I can't see any reason right now why
we can't call mkdir_p() in all cases. The attached patch does that (and
makes the code slightly simpler as a result). I tested it with one
element and 2 element existant and nonexistant paths, and it appeared to
work for all of them.
cheers
andrew
? .deps
? initdb
Index: initdb.c
===
RCS file: /projects/cvsroot/pgsql-server/src/bin/initdb/initdb.c,v
retrieving revision 1.11
diff -c -w -r1.11 initdb.c
*** initdb.c17 Nov 2003 20:35:28 - 1.11
--- initdb.c23 Nov 2003 19:46:56 -
***
*** 797,803
mkdatadir(char *subdir)
{
char *path;
- int res;
path = xmalloc(strlen(pg_data) + 2 +
(subdir == NULL ? 0 : strlen(subdir)));
--- 797,802
***
*** 807,819
else
strcpy(path, pg_data);
! res = mkdir(path, 0700);
! if (res == 0)
! return true;
! else if (subdir == NULL || errno != ENOENT)
! return false;
! else
! return !mkdir_p(path, 0700);
}
--- 806,812
else
strcpy(path, pg_data);
! return (mkdir_p(path, 0700) == 0);
}
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] initdb mkdir_p() doesn't work
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: AFAICS mkdatadir() shouldn't consider subdir == NULL as a reason to fail rather than trying mkdir_p. Right. In fact, I can't see any good reason to call mkdir and then mkdir_p at all. See my patch from this afternoon. I'm unsure about that. I liked the original idea of only trying mkdir_p when plain mkdir() had failed with ENOENT. I am not convinced your proposed patch will behave desirably under all error cases. In particular, mkdir_p seems rather dependent on knowing just which errno codes will get returned --- which is okay for its heritage as BSD-only code, but how well will it port? Better to only invoke it when we have reason to think it can help. OK, then the simple thing to do would be either to change the test on subdir or to remove it altogether and just check for ENOENT. I'd be surprised if the code weren't fairly portable, though. Sure. Of course, the reason I put this on my web site and asked for eyeballs was to try to catch some of this sort of stuff before the program went into the tree :-) We have a whole development cycle to shake these issues out. Don't panic. alfred.e.newman-modeWhat, me panic?/alfred.e.newman-mode I started with 2 goals: have initdb work on Unix with make check, and have the Win32/Cygwin issues sorted out as far as possible, so when we get a working w32 postmaster we can actually use it :-). There are 2500 lines of C here - the odd bug isn't surprising. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Build farm
Jean-Michel POURE wrote: Le Vendredi 21 Novembre 2003 19:47, Tom Lane a crit : I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. Dear friends, We have a small build farm for pgAdmin covering Win32, FreeBSD and most GNU/ Linux systems. See http://www.pgadmin.org/pgadmin3/download.php#snapshots The advantage are immediate feedback and correction of problems. Also, in a release cycle, developers and translators are quite motivated to see their work published fast. Of course, it is always hard to mesure the real impact of a build farm. My opinion it that it is quite positive, as it helps tighten the links between people, which is free software is mostly about. Right. But I think we have been talking about using the build farm to do test builds rather than to provide snapshots. I'd be very wary of providing arbitrary snapshots of postgres, whereas I'd be prepared to try a snapshot of pgadmin3 under certain circumstances. (Also, building your own snapshot of postgres is somewhat easier than building your own snapshot of pgadmin3). cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Build farm
Bruce Momjian wrote: FYI, the HP testdrive farm, http://www.testdrive.hp.com, has shared directories for most of the machines, meaning you can CVS update once and telnet in to compile for each platform. As Peter pointed out, these machines are firewalled. But presumably one could upload a snapshot to them. What I had in mind was a more distributed system, though. Of course, these things are not mutually exclusive - using the HP testdrive farm looks like it might be nice. But it would be hard to automate, I suspect. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Build farm
Bruce Momjian wrote: Andrew Dunstan wrote: Bruce Momjian wrote: FYI, the HP testdrive farm, http://www.testdrive.hp.com, has shared directories for most of the machines, meaning you can CVS update once and telnet in to compile for each platform. As Peter pointed out, these machines are firewalled. But presumably one could upload a snapshot to them. What I had in mind was a more distributed system, though. Of course, these things are not mutually exclusive - using the HP testdrive farm looks like it might be nice. But it would be hard to automate, I suspect. I figured you could just upload once and telnet and build on each machine. What I'm working on (slowly - I'm quite busy right now, and about to be away from home for 5 days) is a system which would (or could) run from cron on every member of the farm, and upload its results to a central server where it could be displayed, in a somewhat similar way to the way the Samba build farm works - see http://build.samba.org/ - so we'd be able to see at a glance when something is broken and where and why. We could also incorporate email notification of breakage, as a refinement. I have a few pieces of this working but not a full suite yet - it will essentially be 3 perl scripts - one on the client (to run the update(s), build(s) and upload the results) and two on the central server (one for upload and one for display). When I get a demo page done I'll show it working with a couple of hosts. Of course, you can automate (almost) anything, including telnet, but right now I'm assuming the farm members will have internet connectivity. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] initdb should create a warning message [was Re:
Greg Stark wrote: Oliver Elphick [EMAIL PROTECTED] writes: Then it needs to be stated very prominently. But the place to put a sign saying Dangerous cliff edge is beside the path that leads along it. The only way to make this prominent would be a file with the *name* THIS DIRECTORY CONTAINS CRITICAL DATA. Not a README with that message inside. Renaming the directories is the only suggestion I've seen that makes sense. The others remind me of the warning that is now placed on coffee cup lids at fast food places: Caution, Contents May Be Hot. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] remaining patches?
What's happening to the remaining patches that were held over for 7.5, e.g. mine which does some logging enhancements? cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] [pgsql-hackers-win32] What packages from Mingw and MSys are needed?
Tony and Bryn Reina wrote: 1. Which parts of MSYS and Mingw are needed for the building a Win32 version of PostgreSQL? There are several packages listed on the Mingw website: MingGW-3.1.0-1.exe mingw-utils-0.2 mingw-runtime-3.2 msys-1.0.9.exe msysDTK-1.0.1.exe binutils gcc win32api-2.4 mingw32-make 2. Do I have to build OpenSSL from Mingw to compile --with-openssl? Or does Mingw already have it as a package? 3. Does the current PostgreSQL 7.4 compile with Mingw? Thanks. -Tony Reina see here for some info: http://archives.postgresql.org/pgsql-hackers/2003-09/msg00938.php I didn't try compiling with openssl. I believe we are still waiting for signalling and process management code to complete the Win32 port. cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Encoding problem with 7.4
E.Rodichev wrote: On Fri, 28 Nov 2003, Tom Lane wrote: E.Rodichev [EMAIL PROTECTED] writes: /e:2createdb test test | er | SQL_ASCII - Incorrect! (3 rows) Let's note than the last line is in fact completely incorrect. What's incorrect about it? You didn't ask for any other encoding than SQL_ASCII. It is incorrect, because database test is, really, in KOI8, NOT in SQL_ASCII in this example, as I explained in my mail. Best wishes, E.R. You can set the default encoding at initdb time, IIRC, but you didn't. You can set the default at initdb time, or per database at createdb time, but it has to be done explicitly. You seem to think it should be picked up from the environment, but this isn't so, you must use the -E|--encoding flag on either createdb or initdb, or if creating directly from SQL use the ENCODING option on create database to use something other than the default set by initdb. examples: [EMAIL PROTECTED] bin]$ ./initdb /tmp/enctry The files belonging to this database system will be owned by user andrew. This user must also own the server process. The database cluster will be initialized with locales: COLLATE: ru_RU.KOI8-R CTYPE:ru_RU.KOI8-R MESSAGES: en_US.iso885915 MONETARY: en_US.iso885915 NUMERIC: en_US.iso885915 TIME: en_US.iso885915 creating directory /tmp/enctry... ok creating directory /tmp/enctry/base... ok creating directory /tmp/enctry/global... ok creating directory /tmp/enctry/pg_xlog... ok creating directory /tmp/enctry/pg_clog... ok selecting default max_connections... 100 selecting default shared_buffers... 1000 creating configuration files... ok creating template1 database in /tmp/enctry/base/1... ok initializing pg_shadow... ok enabling unlimited row size for system tables... ok initializing pg_depend... ok creating system views... ok loading pg_description... ok creating conversions... ok setting privileges on built-in objects... ok creating information schema... ok vacuuming database template1... ok copying template1 to template0... ok Success. You can now start the database server using: ./postmaster -D /tmp/enctry or ./pg_ctl -D /tmp/enctry -l logfile start [EMAIL PROTECTED] bin]$ ./pg_ctl -D /tmp/enctry -l /tmp/enclog -o '-p 5433' start postmaster successfully started [EMAIL PROTECTED] bin]$ ./createdb -E KOI8-R -p 5433 testme CREATE DATABASE [EMAIL PROTECTED] bin]$ ./psql -p 5433 -l List of databases Name| Owner | Encoding ---++--- template0 | andrew | SQL_ASCII template1 | andrew | SQL_ASCII testme| andrew | KOI8 (3 rows) [EMAIL PROTECTED] bin]$ ./pg_ctl -D /tmp/enctry -o '-p 5433' stop waiting for postmaster to shut down..done postmaster successfully shut down [EMAIL PROTECTED] bin]$ rm -rf /tmp/enctry [EMAIL PROTECTED] bin]$ ./initdb -E KOI8-R /tmp/enctry The files belonging to this database system will be owned by user andrew. This user must also own the server process. The database cluster will be initialized with locales: COLLATE: ru_RU.KOI8-R CTYPE:ru_RU.KOI8-R MESSAGES: en_US.iso885915 MONETARY: en_US.iso885915 NUMERIC: en_US.iso885915 TIME: en_US.iso885915 creating directory /tmp/enctry... ok creating directory /tmp/enctry/base... ok creating directory /tmp/enctry/global... ok creating directory /tmp/enctry/pg_xlog... ok creating directory /tmp/enctry/pg_clog... ok selecting default max_connections... 100 selecting default shared_buffers... 1000 creating configuration files... ok creating template1 database in /tmp/enctry/base/1... ok initializing pg_shadow... ok enabling unlimited row size for system tables... ok initializing pg_depend... ok creating system views... ok loading pg_description... ok creating conversions... ok setting privileges on built-in objects... ok creating information schema... ok vacuuming database template1... ok copying template1 to template0... ok Success. You can now start the database server using: ./postmaster -D /tmp/enctry or ./pg_ctl -D /tmp/enctry -l logfile start [EMAIL PROTECTED] bin]$ ./pg_ctl -D /tmp/enctry -l /tmp/enclog -o '-p 5433' start postmaster successfully started [EMAIL PROTECTED] bin]$ ./createdb -p 5433 testme CREATE DATABASE [EMAIL PROTECTED] bin]$ ./psql -p 5433 -l List of databases Name| Owner | Encoding ---++-- template0 | andrew | KOI8 template1 | andrew | KOI8 testme| andrew | KOI8 (3 rows) [EMAIL PROTECTED] bin]$ cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Inside the Regex Engine
Tom Lane wrote: [EMAIL PROTECTED] (David Fetter) writes: While PL/Perl is great, it's not available everywhere, and I'd like to be able to grab atoms from a regex match in, say, a SELECT. Is there some way to get access to them? There's a three-parameter variant of substring() that allows extraction of a portion of a regex match --- unfortunately it uses SQL99's brain-dead notion of regex, which will not satisfy any Perl weenie :-( I think it'd be worth our while to define some comparable functionality that depends only on the POSIX regex engine ... substitute should be relatively straightforward, I guess; split and match maybe less so - what do you return? An array? Or you could require an explicit subscript to get a particular return value as in split_part(), which would be potentially inefficient if you want more than one (although I guess results could be cached). cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Encoding problem with 7.4
E.Rodichev wrote: On Wed, 3 Dec 2003, Stephan Szabo wrote: On Wed, 3 Dec 2003, E.Rodichev wrote: On Fri, 28 Nov 2003, Tom Lane wrote: E.Rodichev [EMAIL PROTECTED] writes: /e:2createdb test test | er | SQL_ASCII - Incorrect! (3 rows) Let's note than the last line is in fact completely incorrect. What's incorrect about it? You didn't ask for any other encoding than SQL_ASCII. It is incorrect, because database test is, really, in KOI8, NOT in SQL_ASCII in this example, as I explained in my mail. No, it isn't. As far as PostgreSQL is concerned the database is SQL_ASCII since you didn't override the default encoding at initdb time or at createdb time. You did choose LC_ values that seem to want KOI8, but locale and encoding are separate, if you want KOI8 encoding, you have to say so. Yes, it is! If db test is SQL_ASCII, AND all LC_* env are set to C, the sorting of ASCII characters is, for example, a A b B c C not A B C a b c (the first order is true for ru_RU.KOI8-R, the latter one - for C). To summarize shortly: - initdb _without_ -E flag, but with ru_RU.KOI8-R environment; - createdb with any environment; - psql indicates SQL_ASCII; - sorting and upper/lowercasing are in ru_RU.KOI8-R, even with LC_* environment is set to C. Where is the logic? Encoding and collation order are two different things. LC_* settings have no effect on encoding. see http://www.postgresql.org/docs/current/static/charset.html cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Encoding problem with 7.4
E.Rodichev wrote: On Wed, 3 Dec 2003, Stephan Szabo wrote: The locale settings depend on LC_* at initdb time only. When the postmaster starts it sets the locale based on the stored values from initdb, not on the current environment. With an SQL_ASCII database being accessed from a client with client_encoding set to SQL_ASCII (which it should be if you aren't setting it) the byte values of a string are passed along with no conversion for the encoding. This means that from within one environment you should get back what you put in, so it might *look* like it's KOI8-R if that's what you're in, but it's not because someone accessing it from say an ISO8859-1 system may see something different. As a result, the possibility to control encodings and locales looks as follows: initdb createdb psql Encoding: Y Y Y Locale:Y N N It seems that more natural scheme will be initdb createdb psql Encoding: Y Y Y Locale:Y Y Y Now the possibility to use different encodings for createdb and psql is a bit strange... Also, it is impossible to have different locales for different databases within one cluster, and it is impossible to use different locales with one database. The latter is even more critical. The reason is that the sorting under C locale is much more effective compared with one under another locales (10-50 times faster for some implementations!). Another reason is that for some applications it is _necessary_ to use different sort order for different tables. For example, I may have two tables: russian_persons and forein_persons, and i'd like to print the sorted list of persons. The russian_persons names must be sorted with ru_RU.KOI8-R locale, and the forein_persons - with C locale. see Multi-Language Support section on TODO list at http://developer.postgresql.org/todo.php - note that this specifies per-column locales rather than per-table, which should be even more useful. Most of these items have no names against them, meaning you could work on them ... cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Minor (very) feature request...
Marc G. Fournier wrote: run it through syslog? or set log_timestamp = true in postgresql.conf ? On Thu, 4 Dec 2003, Steve Wampler wrote: Would it be (is it?) possible to add timestamp to the log messages put out by postgresql? I've got several databases running in an environment where users have this annoying habit of coming up to me with (Oh yes, three days ago around 4pm our instrument had trouble writing to database X.). Having some way of telling which messages were output when would be helpful in such cases, since I'm not allowed to beat the users into submission... ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Double linked list with one pointer
Bruce Momjian wrote: Gaetano Mendola wrote: I don't think the article is available online, alas, but you can find some related source code demonstrating the technique at: http://www.semantics.org/tyr/tyr0_5/list.h That certainly is an amazing idea. You know the pointer you are coming from so you can XOR to find the next pointer. I agree with a Tom that we don't have much use for double-link lists, but is a nice idea. I must confess that it strikes me as a really really horrid and ugly hack - very likely to be error-prone and non-portable and undebuggable, and for almost no saving worth having. But maybe that's just me. In general I've been impressed with the quality of Pg code over the last 6 months or so - I'd hate to see it polluted by something like this. cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] IDENT and IPv6 (was Re: [GENERAL] pg_hba.conf
Kurt Roeckx wrote: On Sat, Dec 06, 2003 at 01:30:02PM -0600, Seum-Lim Gan wrote: Hi, The ident server we currently use is pidentd 3.0.16 from : http://www.lysator.liu.se/ or ftp://ftp.lysator.liu.se/pub/ident/servers The ChangeLog of it says: Solaris 8 (including IPv6) support added. But I have a feeling it's better supported in the 3.1 test versions. 3.0.16's KNOWNBUGS file says this: * In general - wait for Pidentd 3.1 befor using it with IPv6 systems. Of course, using ident for any sort of security mechanism is not good practice anyway, except possibly on the local host. Over a network it is totally untrustworthy. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] CVS HEAD compile failure
fresh checkout just compiled fine for me on Linux (RH8) with ssl enabled. Maybe it is your openssl installation? cheers andrew Bruce Momjian wrote: Attached is a compile failure I am seeing in CVS HEAD in bin/pg_dump. The offending lines are: gmake[3]: Leaving directory `/usr/var/local/src/gen/pgsql/CURRENT/pgsql/src/backend/parser' gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes -Wmissing-declarations -O1 -Wall -Wmissing-prototypes -Wmissing-declar ations -Wpointer-arith -Wcast-align -I../../../src/interfaces/libpq -I../../../src/include -I/usr/local/include/readline -I/us r/contrib/include -DBINDIR=\/usr/local/pgsql/bin\ -c -o common.o common.c In file included from /usr/contrib/include/openssl/comp.h:5, from /usr/contrib/include/openssl/ssl.h:171, from ../../../src/interfaces/libpq/libpq-fe.h:33, from pg_backup.h:30, from pg_backup_archiver.h:58, from common.c:21: /usr/contrib/include/openssl/crypto.h:358: syntax error before `free_func' /usr/contrib/include/openssl/crypto.h:358: syntax error before `)' /usr/contrib/include/openssl/crypto.h:363: syntax error before `free_func' /usr/contrib/include/openssl/crypto.h:363: syntax error before `)' In file included from /usr/contrib/include/openssl/rsa.h:62, from /usr/contrib/include/openssl/evp.h:137, from /usr/contrib/include/openssl/x509.h:67, from /usr/contrib/include/openssl/ssl.h:177, from ../../../src/interfaces/libpq/libpq-fe.h:33, from pg_backup.h:30, from pg_backup_archiver.h:58, from common.c:21: /usr/contrib/include/openssl/asn1.h:802: syntax error before `free_func' /usr/contrib/include/openssl/asn1.h:802: `d2i_ASN1_SET' declared as function returning a function /usr/contrib/include/openssl/asn1.h:803: syntax error before `int' /usr/contrib/include/openssl/asn1.h:907: syntax error before `free_func' /usr/contrib/include/openssl/asn1.h:907: syntax error before `)' In file included from /usr/contrib/include/openssl/evp.h:148, from /usr/contrib/include/openssl/x509.h:67, from /usr/contrib/include/openssl/ssl.h:177, from ../../../src/interfaces/libpq/libpq-fe.h:33, from pg_backup.h:30, from pg_backup_archiver.h:58, from common.c:21: as you can see, these are all related to free_func.The line in crypto.h is: int CRYPTO_set_locked_mem_functions(void *(*m)(size_t), void (*free_func)(void *)); I can't see what could be causing this failure. I thought maybe 'free_func' was defined in our code, but I don't see it. Also, there are earlier references to free_func in that file that aren't marked as failures.I looked at the recent changes to pg_dump/Makefile, but don't see anything. This is a clean build. I can compile 7.4.X CVS just fine. This failure began within the past few days. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS HEAD compile failure
Bruce Momjian wrote: Andrew Dunstan wrote: fresh checkout just compiled fine for me on Linux (RH8) with ssl enabled. Maybe it is your openssl installation? It is openssl 0.9.7c. 7.4 CVS compiles fine so I don't see how it can be my SSL install. I just tried with this version of openssl (on RH9), and it still compiled fine (CFLAGS=-H shows the right files being picked up). Can you give us the complete config settings? Maybe some interaction between them has found a bug somewhere. (BTW, the INSTALL file says you can use --with-openssl=/path but configure barfs on it - we need to fix one or the other). cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] CVS HEAD compile failure
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Where is the typedef here: int CRYPTO_set_locked_mem_functions(void *(*m)(size_t), void (*free_func)(void *)); size_t ... If there's a missing typedef shouldn't we see something like this: `size_t' undeclared (first use in this function) (assuming he is using gcc)? cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] CVS HEAD compile failure
Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
Andrew Dunstan wrote:
fresh checkout just compiled fine for me on Linux (RH8) with ssl
enabled. Maybe it is your openssl installation?
It is openssl 0.9.7c. 7.4 CVS compiles fine so I don't see how it can
be my SSL install.
I've been able to reproduce this on one of my machines, and it's nasty.
It's a conflict between other people's headers.
/usr/local/include/zlib.h (version 1.1.4) quoth:
typedef void (*free_func) OF((voidpf opaque, voidpf address));
/usr/local/ssl/include/openssl/crypto.h (0.9.7c) quoth:
int CRYPTO_set_locked_mem_functions(void *(*m)(size_t), void (*free_func)(void *));
So if this version of zlib.h is included before this version of
crypto.h, things break. Other way 'round works fine.
We can work around this by being more careful about inclusion order
of these headers. Ultimately it would be nice if the upstream library
authors could avoid the name conflict. I'm of the opinion that zlib
should not be typedef'ing a name as generic as free_func, but if
they've made that part of their exported API, it might be hard for them
to change.
In that case I'm confused about why this code compiles on my machine:
#include stdio.h
#include zconf.h
typedef voidpf (*alloc_func) OF((voidpf opaque, uInt items, uInt size));
typedef void (*free_func) OF((voidpf opaque, voidpf address));
int CRYPTO_set_locked_mem_functions(void *(*m)(size_t), void
(*free_func)(void *));
int main()
{
return 0;
}
Also, in compiling pg_dump/common.c, zlib.h (which has this typedef on
my machine) *is* included before openssl/crypto.h.
Either there is something I'm not getting (quite possible ;-) ), or the
problem lies deeper. (compiler version maybe?)
cheers
andrew
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] CVS HEAD compile failure
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: I've been able to reproduce this on one of my machines, and it's nasty. In that case I'm confused about why this code compiles on my machine: What compiler are you using? I'm using gcc 2.95.3 (on the machine that shows the failure), and I think Bruce is also using something less than the latest. It's possible that more recent gcc's are able to figure out that the reference to free_func should be interpreted as a parameter name and not a typedef. In fact, trying your test program fails here: $ gcc -Wall zzz.c zzz.c:8: parse error before `free_func' zzz.c:8: parse error before `)' $ so compiler difference definitely seems to be the answer. gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5) cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] PostgreSQL port to pure Java?
Frank Wiles wrote: On Tue, 9 Dec 2003 07:15:41 -0800 (PST) Ivelin Ivanov [EMAIL PROTECTED] wrote: Has this subject been discussed before? I did not find any references to it in the archives. I think that a co-bundle between an open source J2EE container like JBoss and a scalable database like PostgreSQL will be a blast. There are several well performing comercial Java dbs out there and there is Hypersonic which is free and fast, but supports only READ_UNCOMMITED and is build to grow up to ~200MB. This would be a huge undertaking, rewriting PostgreSQL entirely in Java. Very true. I just did a rough count and founs about 510,000 lines of code in .c and .h files in the source distribution. Not to mention it would kill PostgreSQL's current speedy performance! Maybe, maybe not. Modern JVMs have much better performance characteristics than was once the case. Also, some of the things that Java buys you (memory management, threading, for example) might actually enhance performance in some circumstances. A crude port wouldn't work, though - it would have to be done in such a way as to leverage the platform's strengths, just as we leverage the strengths of writing in C. The *big* problem would be keeping a Java port in sync with the base. That would make it almost impossible to do in a worthwhile way IMNSHO - the maintenance would be a nightmare. It would be an excellent student exercise, though :-) As a Java programmer, I do agree that having a pure Java RDBMS system would be a Good Thing (tm), and the PostgreSQL code base might be an excellent place to start creating such a monster :-). cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PostgreSQL port to pure Java?
Neil Conway wrote: Andrew Dunstan [EMAIL PROTECTED] writes: As a Java programmer, I do agree that having a pure Java RDBMS system would be a Good Thing (tm) Are there any advantages that this would provide that we could get without investing so much effort? For example, PL/Java seems like a reasonable approach to Java PG integration that doesn't involve rewriting hundreds of thousands of lines of code. 2 different things, ISTM. I don't think the PG group should touch a port to Java - it would be a huge distraction. If someone (say, Ivelin) wants to do it, good luck to them. PL/Java would be way cool, though, and have very significant appeal, and is very much worth doing, I believe. (Not that I have the time to do it.) cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PostgreSQL port to pure Java?
D'Arcy J.M. Cain wrote: On December 9, 2003 12:15 pm, Neil Conway wrote: P.S. While we're contemplating pies-in-the-sky, I'd personally love to rewrite PostgreSQL in Objective Caml. I vote for InterCal. :-) Pick your poison from this site: http://99-bottles-of-beer.ls-la.net/ (see especially the entry for make :-) ) Personally, I vote for Ada :-) cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PostgreSQL port to pure Java?
Dave Cramer wrote: Have a look at Axion for a pure java db http://axion.tigris.org/ Not as full featured,but still useful. Er, I take it that not as full featured is an example of meiosis :-) Here's what the web page says: Not (Yet) Supported Features * ALTER TABLE (other than add/drop constraint) * client/server mode * constraints/foreign keys (partial support is available) * GROUP BY/HAVING * stored procedures * sub-selects * triggers (note that default column values are supported) * user-level security -- This is basically a small embedded db engine, not an enterprise class RDBMS. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] pljava revisited
Thomas Hallgren wrote: Hi, I'm working on a new pl/java prototype that I hope will become production quality some time in the future. Before my project gets to far, I'd like to gather some input from other users. I've taken a slightly different approach than what seems to be the case for other attempts that I've managed to dig up. Here's some highlights in my approach: 1. A new Java VM is spawned for each connection. I know that this will give a performance hit when a new connection is created. The alternative however, implies that all calls becomes inter-process calls which I think is a much worse scenario. Especially since most modern environments today has some kind of connection pooling. Another reason is that the connections represents sessions and those sessions gets a very natural isolation using separate VM's. A third reason is that the current connection would become unavailable in a remote process (see #5). Maybe on-demand might be better - if the particular backend doesn't need it why incur the overhead? 2. There's no actual Java code in the body of a function. Simply a reference to a static method. My reasoning is that when writing (and debugging) java, you want to use your favorite IDE. Mixing Java with SQL just gets messy. Perhaps an example or two might help me understand better how this would work. 3. As opposed to the Tcl, Python, and Perl, that for obvious reasons uses strings, my pl/java will use native types wherever possible. A flag can be added to the function definition if real objects are preferred instead of primitives (motivated by the fact that the primitives cannot reflect NULL values). 4. The code is actually written using JNI and C++ but without any templates, no -style object references, no operator overloads, external class libraries etc. I use C++ simply to get better quality, readability and structure on the code. Other pl* (perl, python, tcl) languages have vanilla C glue code. Might be better to stick to this. If you aren't using advanced C++ features that shouldn't be too hard - well structured C can be just as readable as well structured C++. At the very lowest level, about the only things C++ buys you are the ability to declare variables in arbitrary places, and // style comments. 5. I plan to write a JDBC layer using JNI on top of the SPI calls to enable JDBC functionality on the current connection. Some things will be limited (begin/commit etc. will not be possible to do here for instance). Again. examples would help me understand better. Is there a web page for your project? cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] pljava revisited
Thomas Hallgren wrote: The JVM will be started on-demand. Although I realize that one JVM per connection will consume a fair amount of resources, I still think it is the best solution. The description of this system must of course make it very clear that this is what happens and ultimately provide the means of tuning the JVM's as much as possible. I advocate this solution because I think that the people that has the primary interest of a pl/java will be those who write enterprise systems using Java. J2EE systems are always equipped with connection pools. Yes, but as was pointed out even if I use connection pooling I would rather not have, say, 25 JVMs loaded if I can help it. But, I'm of course open for other alternatives. Let's say that there's a JVM with a thread-pool that the Postgress sessions will connect to using some kind of RPC. This implies that each call will have an overhead of at least 2 OS context switches. Compared to in-process calls, this will severely crippel the performance. How do you suggest that we circumvent this problem? Context switches are not likely to be more expensive that loading an extra JVM, I suspect. Depending on your OS/hw they can be incredibly cheap, in fact. Antother problem is that we will immeditately loose the ability to use the current connection provided by the SPI interfaces. We can of course establish a back-channel to the original process but that will incure even more performance hits. A third alternative is to establish brand new connections in the remote JVM. Problem then is to propagate the transaction context correctly. Albeit solvable, the performance using distributed transactions will be much worse than in-process. How do we solve this? We are theorising ahead of data, somewhat. My suggestion would be to continue in the direction you are going, and later, when you can, stress test it. Ideally, if you then need to move to a shared JVM this would be transparent to upper levels of the code. C++ or C is not a big issue. I might rewrite it into pure C. The main reason for C++ is to be able to use objects with virtual methods. I know how to do that in C too but I don't quite agree that its just as clean :-) Maybe not, but it's what is used in the core Pg distribution. Go with the flow :-) cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] CSV hack
You also need to quote values containing the separator. cheers andrew (who used to set creating CSV as a programming exercise - students almost never get it right) David Fetter wrote: Kind people, I've come up with yet another little hack, this time for turning 1-d arrays into CSV format. It's very handy in conjunction with the array_accum aggregate (can this be made a standard aggregate?) in http://developer.postgresql.org/docs/postgres/xaggr.html. Here 'tis... CREATE OR REPLACE FUNCTION csv(anyarray) RETURNS TEXT AS 'DECLARE in_array ALIAS FOR $1; temp_string TEXT; quoted_string TEXT; i INTEGER; BEGIN FOR i IN array_lower(in_array, 1)..array_upper(in_array, 1) LOOP IF in_array[i]::TEXT ~ THEN temp_string := || replace(in_array[i]::TEXT, , ) || ; ELSE temp_string := in_array[i]::TEXT; END IF; IF i = array_lower(in_array, 1) THEN quoted_string := temp_string; ELSE quoted_string := quoted_string || '','' || temp_string; END IF; END LOOP; RETURN quoted_string; END; ' LANGUAGE 'plpgsql'; Those DBD::Pg users among us who'd like to be able to bind_columns to postgresql arrays may have a leg up with Text::CSV_XS. Other middleware should be able to handle such things, too. :) Cheers, D ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 7.4 build problem on Linux Vserver
Tom Lane wrote: Dave Page [EMAIL PROTECTED] writes: gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes -Wmissing-declarations prod -I../../src/include -D_GNU_SOURCE -I/usr/include -c -o path.o path.c gcc: cannot specify -o with -c or -S and multiple compilations How is prod getting into that command line? I suspect it's coming from an environment variable like PROFILE or CFLAGS ... Is there a case for making Makefile/shell variables and defines we use have less generic names? I know I encountered a problem on Windows when doing initdb because it defines one of BINDIR/DATADIR (I forget which). It took me ages to find out what was going on, and I fixed it by prepending the variable (in this case a define) with PG cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] pljava revisited
Peter Eisentraut wrote: Thomas Hallgren wrote: What are your thoughts and ideas? Instead of making up your own stuff, there's a whole SQL standard that tells you how Java embedded in an SQL server should work. Of course that doesn't tell you about implementation details. Where can it be found? cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Use of 'cp -r' in CREATE DATABASE
Nigel J. Andrews said: On Thu, 11 Dec 2003, Alvaro Herrera wrote: On Thu, Dec 11, 2003 at 06:36:05PM -0500, Bruce Momjian wrote: Our dbcommands.c has for create database: snprintf(buf, sizeof(buf), cp -r '%s' '%s', src_loc, target_dir); [...] I think we should switch to -R in our code. But you will have to write special code for Win32, won't you? Maybe it would be better to avoid using system commands altogether and copy the whole thing using syscalls ... That was my immediate thought. Unfortunately that means reinventing the wheel; or grabbing it from BSD or somewhere and distributing it with postgresql. We need a consistent policy about it, I think. I grabbed some code for a recursive mkdir from NetBSD, and it is in initdb.c. But I also wrote a recursive rm, and Bruce replaced it with calls to the native utilities using system(), for understandable reasons. Maybe we need a small, portable, utility library. Or maybe just relying on system utilities is acceptable. cheers andrew ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Use of 'cp -r' in CREATE DATABASE
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: but my BSD/OS manual only documents 'cp -R' and mentions: I think we should switch to -R in our code. And break the code on who knows how many other systems? No thanks. If we want to do anything at all with this code, we should eliminate the use of system(cp) entirely in favor of doing the recursive copy logic ourselves. We already have the beginnings of same in the Windows port, and I think we'll be forced down that path anyway for tablespaces. That seems cleaner to me. IIRC we don't copy anything but plain files and directories - no special files, symlinks or fifos, so the -R/-r differences shouldn't affect us anyway, should they? Also, that should make the implementation of an internal recursive copy much simpler - far fewer cases to consider. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Resurrecting pg_upgrade
re Windows: pipes, yes, hard links, no (and no sane symlinks either) - also of course no (sane) shell - is this going to be a script or a C program? Maybe use an option which you would disable on Windows to copy the files instead of hardlinking them. Yes it would take lots more time and space, but copying raw files would surely still be a lot faster than loading the dump. cheers andew Tom Lane wrote: [snip whole lotta good stuff] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Resurrecting pg_upgrade
Matthew T. O'Connor wrote: On Fri, 2003-12-12 at 14:51, Andrew Dunstan wrote: re Windows: pipes, yes, hard links, no (and no sane symlinks either) Actually, NTFS does support hard links, there is just no support for it in any MS file management GUI. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnfiles/html/ntfs5.asp [snip] I learn something new every day! :-) I guess we *are* specifying that only NTFS will be supported on Windows? (I saw someone the other day running XP on FAT - I couldn't believe it!) Maybe use an option which you would disable on Windows to copy the files instead of hardlinking them. Yes it would take lots more time and space, but copying raw files would surely still be a lot faster than loading the dump. I think this would be a good feature even without hard link problems. If I am a paranoid admin, and I can afford the time and disk space required, I would want to keep a complete copy of my database, even after the new server is up and running. I agree. A little paranoia never hurt anyone. Of course, you could always back it up manually beforehand too. cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [PATCHES] fork/exec patch
[moved to hackers / win32] Claudio Natoli wrote: Or do people have strong leanings towards fix as you go along? Just feels like that way could see us getting bogged down making things perfect instead of advancing the port... w.r.t. Win32, I think the way to proceed is (in this order): . make it work . make it elegant . make it fast BTW, I agree with Bruce, you're doing excellent stuff. Now for the fun part (signals). cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] fork/exec patch
Bruce Momjian wrote: Andrew Dunstan wrote: Now for the fun part (signals). Actually, no. I thought fork/exec would be a real mess (as did Tom), but Claudio has done an excellent job of producing a minimal patch. The work isn't done yet, but this small patch has taken us much closer, so I assume signals will be even easier. Well, it's speculation on both our parts :-). ISTM we'll need an explicit event loop to check the shmem (or whatever we use to simulate signals) every so often - maybe that will be easy, I don't know - I'm interested to see what turns up. (Of course, if we were threaded we'd just need a thread to watch for the event ...) cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [PATCHES] fork/exec patch
Bruce Momjian wrote: Have you looked at the CONNX signal code on the Win32 page: http://momjian.postgresql.org/main/writings/pgsql/win32.html It uses shared memory and events. Yes, and I just did again. I guess I must be missing something, though - I don't see what in that code causes the signalled process to call the handler corresponding to the signal. Maybe I'm just a little brain dead today ... andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] [PATCHES] fork/exec patch
Magnus Hagander wrote: Bruce Momjian wrote: Have you looked at the CONNX signal code on the Win32 page: http://momjian.postgresql.org/main/writings/pgsql/win32.html It uses shared memory and events. Yes, and I just did again. I guess I must be missing something, though - I don't see what in that code causes the signalled process to call the handler corresponding to the signal. Maybe I'm just a little brain dead today ... Can't find that part either, but a few questions for the implementation regardless of wether that code is around somewhere: At what times do signals actually *need* to be delivered? And at what points do the calling process need to be notified? Actually interrupting a running process to execute a procedure in a thread can be pretty darn tricky - AFAIK you have to manually switch thread context and create an exception which you then catch, call handler, reset and continue. However, if it's acceptable to have delivery only when the thread is in alertable state this should not be necessary. Then you can basically take two approaches depending on when you need the response: If you just need a response that the receiving process has queued the thread handler, then create a separate thread that receives the signal and queues a user APC on the main thread. This will then execute when the thread enters alertable state. Actually, I see that in os-fix2.cpp there is code that sets up a thread that just polls for the event and then calls the corresponding handler. If you need a response once it has actually run, then the main thread needs to do signal polling now and then. This has the bad sideeffect that the main thread will block completely until the signal is delivered, which might be a while. I don't know what the semantics are for kill() on unix there? And if it is sync, does postgresql actually need that property? kill() should return success upon the signal being queued, as I understand it - i.e. no sync. All this kind of answers my original question, by pointing out the need to poll one way or another, which is why I suggested that signal emulation might be messy, and more complicated than the fork/exec case. In effect, the runtime in Unix provides the signal polling for you for free, which is why this method of IPC is so commonly used. cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-hackers-win32] [HACKERS] [PATCHES] fork/exec patch
Magnus Hagander wrote: Absolutely, but there are other signals to send, no? Or you might want to send a signal directly to a backend (to cancel for example), as you can do on Unix. In normal operation the only thing that should be signalling a backend is the postmaster. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [pgsql-hackers-win32] [HACKERS] [PATCHES] fork/exec patch
Neil Conway said: Andrew Dunstan [EMAIL PROTECTED] writes: In normal operation the only thing that should be signalling a backend is the postmaster. Oh? What about LISTEN/NOTIFY? er, yeah. *self-lart* + ... or another backend cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [PATCHES] Double Backslash example patch
Andreas Pflug wrote: Tom Lane wrote: I think the minimum amount of additional work that has to be done before we can apply it is to teach psql's input parser about $QUOTE$ --- without that, the feature is not only useless but counterproductive. IMHO it's not useless. pgAdmin already knows that quoting scheme... not yet tested, I'll have to check that patch. OTOH we sure want it for psql - being able to use this in scripts fed to psql is very important IMNSHO. I see that psql/mainloop.c contains this comment: * FIXME: rewrite this whole thing with flex Is now the time to do that? It would be a pity if substantial changes were made here only to be thrown away ... cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] Double Backslash example patch
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: I see that psql/mainloop.c contains this comment: * FIXME: rewrite this whole thing with flex Is now the time to do that? If you feel like giving it a shot, ain't nobody gonna stand in your way. MainLoop() is well past the level of unmaintainability IMHO (which is why I'd not tried to touch it in the original quick patch). I'm not totally convinced that flex can do the job, but in any case a ground-up redesign for legibility might be the only cure. I will take a look - no guarantees though. I agree that flex might well end up making things more obscure than less. Keeping track of open quote operators to make sure we have proper nesting and closing looks like it might be a bit messy, but doable. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TODO list
Jonathan Gardner wrote: Marko Zmak wrote: I've been usin psql for quite a long time and I found it to be very comfortable. Since some web providers in my country refuse to put psql while some thing are still in psql TODO list, I'm interested in following... I'd like to know when are you planning to deal with this TODO item: Allow limits on per-db/user connections I would appreciate if it was as soon as possible. Thanks. I know of no one working on this feature. Is this something a someone who is new to PostgreSQL development can attempt? Does someone already have a design planned out? Before we even get there we need agreement on exactly what it means :-) Seems like the best place for the settings might be pg_hba.conf. Say we have a new keyword limit there. Here are the possibilities (I think): case 1 - limit username's connections to dbname: limit dbname username n case2 - limit username's connections regardless of database: limit all username n case 3 - limit all users' connections to dbname: limit dbname all n case 4 - limit username's connections to any particular database: limit any username n case 5 - limit all users' connections to any particular database: limit any all n case 6 - limit all users' connections regardless of database: limit all all n Would we want to bother about cases 4 and 5? Are the semantics of case 6 what we want? Once the behaviour is agreed, (based on my recent experience) I think this is something that might well be attempted by someone new to Pg development. If you ask questions you will get plenty of help. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] TODO list
Tom Lane said: Andrew Dunstan [EMAIL PROTECTED] writes: case 6 - limit all users' connections regardless of database: limit all all n That's called max_connections. Don't think we need a redundant implementation of same ... no - this was intended to limit *each* user - max-connections limits total connections. Maybe I expressed it badly. (reinforces my point about needing to make sure we get the semantics straight first). Another little nitpick is that I don't like assuming that any and all are never going to be used as database or user names. (I know that pg_hba.conf already uses all this way, and IMHO that was a bogus decision. Something like * would have been less likely to collide.) I entirely agree. Let's change it. For a new major release people will have probably need to do an initdb anyway. On an implementation level, where are you thinking of enforcing this? pg_hba.conf would not be very appropriate for the most likely place to put it, which is in backend startup shortly after establishing a PGPROC entry (with the data about numbers of active connections obtained by scanning the PGPROC array for other backends connected to the same database or with the same userid). I think we've thrown away the PostmasterContext long before that, so we couldn't use cached pg_hba.conf data without some redesign of the startup sequence. Without digging deeply at all I thought probably in the postmaster. But I would defer to your good advice ;-) I'm not at all dogmatic about using pg_hba.conf - it just seemed similar to the info we carry there. cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] TODO list
Þórhallur Hálfdánarson said: I'd like to mention that administrators likely to use the this feature would probably like to be able to tune this without having to modify a file -- updating via SQL (= storing this in a system table) would be extremely nice... We set connection permissions in a config file - why would we not use the same mechanism for connection limits? I'm not objecting, but I think we should be consistent. cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] replace all with * in pg_hba.conf
If people are happy with Tom's suggestion of using '*' instead of 'all' in pg_hba.conf I will prepare a patch for it. (I will also replace the ugly long IP6 localhost netmask with a CIDR mask). cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
