Re: [HACKERS] docs on tsearch2
Check http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/ Oleg On Wed, 3 Mar 2004, [gb2312] wrote: hi all Are there any other docs about tsearch2 except for the 3 docs in \contrib\tsearch2\docs i want some tech docs on tsearch2 thank all---(end of broadcast)---TIP 8: explain analyze is your friend Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [pgsql-hackers-win32] [HACKERS] Tablespaces
Gavin Sherry wrote: I'm going to focus on implementing this on the system(s) I'm used to developing on (ie, those which support symlinks). Once that is done, I'll talk with the Win32 guys about what, if anything, we can do about getting this to work on Win32 (and possibly other non-symlink supporting OSs). If the design can allow the default tablespace to not require symlinks, then I think we're in good shape, as we can simply choose to introduce tablespaces under Win32 whenever it suits. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Tablespaces
Bruce Momjian writes: I just checked from the MinGW console and I see: [snip] It accepts ln -s, but does nothing with it. And even if it had worked, it wouldn't really matter, since we don't actually want to *run* the system under MinGW/msys, just build it. I think the idea of implementing in symlinks for non-compliant platforms in md.c has some merit. FWIW, looks like that is how cygwin implements symlinks... Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] log_line_info
Bruce Momjian said: Andrew Dunstan wrote: I haven't had any other feedback on this patch that I posted. However, I'm a bit dissatisfied with it for a couple of reasons: . when a connection is logged we don't yet know the user and database, because we haven't processed the initial packet yet. That causes %U and %D to produce empty strings, which looks mildly ugly. I'm inclined in this case to emit something like or [unknown] for these escapes. . we don't produce any output for postmaster, stats collector etc. processes. If we really want to get rid of log_pid and log_timestamp this needs to be dealt with, IMNSHO. We could handle that in a few ways: - have a separate GUC var (log_line_info_postmaster?) Not much gain over keeping the existing vars, though - have a special marker in the string (%X maybe) that says stop processing for postmaster here. Example: %T [%P]:%X [EMAIL PROTECTED](%C:%S %I line:%L - have a special marker where what follows is the postmaster variant, defaulting to the beginning if not found. Examples: %T [%P]: (everybody gets timestamp and pid) %T [%P]: [EMAIL PROTECTED](%C:%S %I line:%L %X%T [%P]: (same effect as example under previous point) - something else I haven't thought of ;-) Seems the cleanest would be to just print nothing for items that have no meaning for the postmaster. Not quite clean enough - I also want to be able to supress irrelevant literal characters. See the actual patch sent in on Feb 29th at http://archives.postgresql.org/pgsql-patches/2004-02/msg00266.php where I used the first variant I suggested above. cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [pgsql-hackers-win32] [HACKERS] Tablespaces
For tablespaces on OS's that don't support it, I think we will have to store the path name in the file and read it via the backend. Somehow we should cache those lookups. My feeling is that we need not support tablespaces on OS's without symlinks. To create symlinked directories on Win2k NTFS see: http://www.sysinternals.com/ntw2k/source/misc.shtml#junction I think Win2000 or XP would be a reasonable restriction for Win32 PG installations that want tablespaces. Andreas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Tablespaces
On Wed, 2004-03-03 at 04:59, Tom Lane wrote: What might make sense is some sort of marker file in a tablespace directory that links back to the owning $PGDATA directory. CREATE TABLESPACE should create this, or reject if it already exists. It will not be enough for the marker to list the path of the parent $PGDATA, since that path might get changed by system administration action. The marker should contain some sort of unique string which would match the same string somewhere in $PGDATA. Then, if either tablespace or $PGDATA were moved, it would be possible to tie the two back together. It wouldn't be an issue on most normal systems, but might be of crucial importance for an ISP running numerous separate clusters. -- Oliver Elphick [EMAIL PROTECTED] LFIX Ltd ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-hackers-win32] [HACKERS] Tablespaces
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: My feeling is that we need not support tablespaces on OS's without symlinks. Agreed, but are we going to support non-tablespace installs? I wasn't sure that was an option. A setup containing only the default tablespace cannot use any symlinks. That doesn't seem hard though. Yea, I think you are right. We just disable CREATE TABLESPACE and the rest should just work. Basically, pg_tablespace will only have one entry on those platforms. The initdb directory structure will have a single tablespace, but that doesn't use symlinks. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [pgsql-hackers-win32] [HACKERS] Tablespaces
Claudio Natoli wrote: Bruce Momjian writes: I just checked from the MinGW console and I see: [snip] It accepts ln -s, but does nothing with it. And even if it had worked, it wouldn't really matter, since we don't actually want to *run* the system under MinGW/msys, just build it. I think the idea of implementing in symlinks for non-compliant platforms in md.c has some merit. FWIW, looks like that is how cygwin implements symlinks... Why can't we use MS Win32 shortcut files to simulate symlinks? MinGW doesn't do it, so I suppose it isn't possible. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Thread safe connection-name mapping in ECPG. Is it
Should I add this to the TODO list?
---
Lee Kindness wrote:
Sort of related, I was thinking about adding some more thread-related
code such that if a connection wasn't explicitely specified then the
last connection SET or CONNECTed to for the current thread is used,
rather than just the last connection.
But yeah, specifying the connection by variable (be it string or
connection ptr) would be a definite step forward. Currently you cannot
write a generic function like:
int getit(char *using_connection)
{
EXEC SQL BEGIN DECLARE SECTION;
char *s_connection = using_connection;
int s_it;
EXEC SQL END DECLARE SECTION;
EXEC SQL AT :s_connection SELECT it INTO :s_it FROM some_table;
return( s_it );
}
which could be run concurrently by multiple threads.
L.
Shridhar Daithankar writes:
On Friday 27 February 2004 20:54, Michael Meskes wrote:
On Fri, Feb 27, 2004 at 04:22:33PM +0530, Shridhar Daithankar wrote:
How about, allowing 'connection *'? If somebody puts a 'connection *'
there it is used. If it is a string a name search is performed. Best of
both worlds.
How shall anyone put a pointer to a connection struct inside the SQL
statement?
It would help me a lot if you'd be able to give some examples.
EXEC SQL BEGIN DECLARE SECTION;
connect *connectionPtr;
EXEC SQL END DECLARE SECTION;
EXEC SQL CONNECT TO db AS connectionPtr;
EXEC SQL AT connectionPtr SELECT 1;
After all, it is matter of parsing some code and emitting equivalent C code,
isn't it?
Shridhar
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
Re: [pgsql-hackers-win32] [HACKERS] Tablespaces
I just checked from the MinGW console and I see: [snip] It accepts ln -s, but does nothing with it. And even if it had worked, it wouldn't really matter, since we don't actually want to *run* the system under MinGW/msys, just build it. I think the idea of implementing in symlinks for non-compliant platforms in md.c has some merit. FWIW, looks like that is how cygwin implements symlinks... Why can't we use MS Win32 shortcut files to simulate symlinks? MinGW doesn't do it, so I suppose it isn't possible. Shortcut files are *only* a GUI feature. If you do a dir in the filesystem they show up as a file. If you issue open() or similar on them (for example, by doing type on the command prompt), you will get the .lnk file, not the file it points to. //Magnus ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Thread safe connection-name mapping in ECPG. Is it
Oh.. By all means..Please do..
The reason I posted it because I didn't wanted to work on it if core is not
going to accept it on account of non-compliance with spec.
Is this fine?
* Allow a 'connection *' pointer to be specified instead of a string to denote
a connection.
I plan to work on it whenever possible. What I would like to do is eliminate the
locks around name-connection mapping as we would be directly using the
connection instead of a name.
I think we can also add the SQL-CA to connection structure so that each
connection gets it's own SQL-CA. That way ECPG is as thread-safe as the calling
application gets.
And on the plus side we don't have to worry about platform specific threading
models either.
Thoughts?
Shridhar
Bruce Momjian wrote:
Should I add this to the TODO list?
---
Lee Kindness wrote:
Sort of related, I was thinking about adding some more thread-related
code such that if a connection wasn't explicitely specified then the
last connection SET or CONNECTed to for the current thread is used,
rather than just the last connection.
But yeah, specifying the connection by variable (be it string or
connection ptr) would be a definite step forward. Currently you cannot
write a generic function like:
int getit(char *using_connection)
{
EXEC SQL BEGIN DECLARE SECTION;
char *s_connection = using_connection;
int s_it;
EXEC SQL END DECLARE SECTION;
EXEC SQL AT :s_connection SELECT it INTO :s_it FROM some_table;
return( s_it );
}
which could be run concurrently by multiple threads.
L.
Shridhar Daithankar writes:
On Friday 27 February 2004 20:54, Michael Meskes wrote:
On Fri, Feb 27, 2004 at 04:22:33PM +0530, Shridhar Daithankar wrote:
How about, allowing 'connection *'? If somebody puts a 'connection *'
there it is used. If it is a string a name search is performed. Best of
both worlds.
How shall anyone put a pointer to a connection struct inside the SQL
statement?
It would help me a lot if you'd be able to give some examples.
EXEC SQL BEGIN DECLARE SECTION;
connect *connectionPtr;
EXEC SQL END DECLARE SECTION;
EXEC SQL CONNECT TO db AS connectionPtr;
EXEC SQL AT connectionPtr SELECT 1;
After all, it is matter of parsing some code and emitting equivalent C code,
isn't it?
Shridhar
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-hackers-win32] [HACKERS] Tablespaces
Zeugswetter Andreas SB SD [EMAIL PROTECTED] writes: My feeling is that we need not support tablespaces on OS's without symlinks. To create symlinked directories on Win2k NTFS see: http://www.sysinternals.com/ntw2k/source/misc.shtml#junction I think Win2000 or XP would be a reasonable restriction for Win32 PG installations that want tablespaces. Oh, good --- symlinks for directories are all that we need for this design. I think that settles it then. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Tablespaces
Oliver Elphick [EMAIL PROTECTED] writes: It will not be enough for the marker to list the path of the parent $PGDATA, since that path might get changed by system administration action. The marker should contain some sort of unique string which would match the same string somewhere in $PGDATA. We have already added a notion of a unique installation identifier for PITR purposes (look in pg_control). So we could use that for this purpose if we wanted to. But I'm not sure how important it really is. AFAICS the behavior of CREATE TABLESPACE will be create marker file, if it already exists then abort. It has no need to actually look in the file and so there's no need for the contents to be unique. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Thread safe connection-name mapping in ECPG. Is it
Shridhar Daithankar [EMAIL PROTECTED] writes: The reason I posted it because I didn't wanted to work on it if core is not going to accept it on account of non-compliance with spec. When it comes to ecpg, Michael Meskes is the man you have to convince, not any of the core committee. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-hackers-win32] [HACKERS] Tablespaces
Zeugswetter Andreas SB SD wrote: To create symlinked directories on Win2k NTFS see: http://www.sysinternals.com/ntw2k/source/misc.shtml#junction I don't think we could use this s/w though, unless the author is prepared to relicense it. I'm sure implementing a clean room version of the relevant parts wouldn't be too hard, though. cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [pgsql-hackers-win32] [HACKERS] Tablespaces
To create symlinked directories on Win2k NTFS see: http://www.sysinternals.com/ntw2k/source/misc.shtml#junction I don't think we could use this s/w though, unless the author is prepared to relicense it. I'm sure implementing a clean room version of the relevant parts wouldn't be too hard, though. Definitly not (ooh, danger...) - it's a simple and well documented call to DeviceIOControl(). http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/ base/fsctl_set_reparse_point.asp is the one you want, I think. We can always reference to that utility as a good way to get information about the junctions pgsql has created... //Magnus ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] Client side copy functionality
Tom, with the since 7.4 deprecated copy support functions in libpq, is there any define or other way to figure out if compiling against a pre-7.4 or post-7.4 libpq? Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Client side copy functionality
Jan Wieck [EMAIL PROTECTED] writes: with the since 7.4 deprecated copy support functions in libpq, is there any define or other way to figure out if compiling against a pre-7.4 or post-7.4 libpq? Hm, don't think so. If you've autoconfiscated your client code, you can just put in a configure test for one of the newer libpq functions. [ sounds of diff'ing ... ] If you're desperate for an #ifdef solution, it looks like you could test for the existence of one of the PG_DIAG_XXX macros added to postgres_ext.h. This seems a bit circuitous though. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Tablespaces
Bruce Momjian [EMAIL PROTECTED] writes: Greg Stark wrote: I am expecting to hear some bleating about this from people whose preferred platforms don't support symlinks ;-). However, if we don't Well, one option would be to have the low level filesystem storage (md.c?) routines implement a kind of symlink themselves. Just a file with a special magic number followed by a path. On further contemplation it doesn't seem like using symlinks really ought to be necessary. It should be possible to drive everything off the catalog tables while avoidin having the low level filesystem code know anything about them. Instead of having the low level code fetch the pg_* records themselves, some piece of higher level code would do the query and call down to storage layer to inform it of the locations for everything. It would have to do this on database initialization and on any subsequent object creation. Basically maintain an in-memory hash table of oid - path, and call down to the low level code whenever that hash changes. (Or more likely oid-ts_id and a separate list of ts_id - path.) -- greg ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Thread safe connection-name mapping in ECPG. Is it
Tom Lane wrote: Shridhar Daithankar [EMAIL PROTECTED] writes: The reason I posted it because I didn't wanted to work on it if core is not going to accept it on account of non-compliance with spec. When it comes to ecpg, Michael Meskes is the man you have to convince, not any of the core committee. Michael doesn't own ecpg any more than the core owns the backend code. I would get an opinion from Michael and see what others think of the idea. You can always ask for a vote too. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] log_line_info
Andrew Dunstan wrote: defaulting to the beginning if not found. Examples: %T [%P]: (everybody gets timestamp and pid) %T [%P]: [EMAIL PROTECTED](%C:%S %I line:%L %X%T [%P]: (same effect as example under previous point) - something else I haven't thought of ;-) Seems the cleanest would be to just print nothing for items that have no meaning for the postmaster. Not quite clean enough - I also want to be able to supress irrelevant literal characters. See the actual patch sent in on Feb 29th at http://archives.postgresql.org/pgsql-patches/2004-02/msg00266.php where I used the first variant I suggested above. Yes, I understand there is no way to know if the literal characters are supposed to print for postmaster message, but is seems easier to just print them than to do the more complicated options you propose. One idea would be to suppress whitespace or perhaps special characters _after_ an option that is skipped --- not perfect, but this is only log output. Of course, if you do %option, you get the leading but not the . Maybe just go with suppressing trailing whitespace of a skipped option is best. Also, I think you should switch to lowercase like Peter suggested. You aren't supporting any of the printf() letters, so you might as well use lowercase. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Tablespaces
Greg Stark wrote: Bruce Momjian [EMAIL PROTECTED] writes: Greg Stark wrote: I am expecting to hear some bleating about this from people whose preferred platforms don't support symlinks ;-). However, if we don't Well, one option would be to have the low level filesystem storage (md.c?) routines implement a kind of symlink themselves. Just a file with a special magic number followed by a path. On further contemplation it doesn't seem like using symlinks really ought to be necessary. It should be possible to drive everything off the catalog tables while avoidin having the low level filesystem code know anything about them. Instead of having the low level code fetch the pg_* records themselves, some piece of higher level code would do the query and call down to storage layer to inform it of the locations for everything. It would have to do this on database initialization and on any subsequent object creation. Basically maintain an in-memory hash table of oid - path, and call down to the low level code whenever that hash changes. (Or more likely oid-ts_id and a separate list of ts_id - path.) The advantage of symlinks is that an administrator could see how things are laid out from the command line. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Tablespaces
Oliver Elphick wrote: On Wed, 2004-03-03 at 04:59, Tom Lane wrote: What might make sense is some sort of marker file in a tablespace directory that links back to the owning $PGDATA directory. CREATE TABLESPACE should create this, or reject if it already exists. It will not be enough for the marker to list the path of the parent $PGDATA, since that path might get changed by system administration action. The marker should contain some sort of unique string which would match the same string somewhere in $PGDATA. Then, if either tablespace or $PGDATA were moved, it would be possible to tie the two back together. It wouldn't be an issue on most normal systems, but might be of crucial importance for an ISP running numerous separate clusters. Taking this one step further would be to do something like Oracle does. Every datafile in Oracle (because the Oracle storage manager stores multiple objects inside datafiles, one could say there is some similarity between Oracle datafiles and the proposed pg tablespaces), has meta info that tells it which database instance it belongs to and the last checkpoint that occured (It might actually be more granular than checkpoint, such that on a clean shutdown you can tell that all datafiles are consistent with each other and form a consistent database instance). So Oracle on every checkpoint updates all datafiles with an identifier. Now you might ask why is this useful. Well in normal day to day operation it isn't, but it can be usefull in disaster recovery. If you loose a disk and need to restore the entire database from backups it can be difficult to make sure you have done it all correctly (do I have all the necessary files/directories? did I get the right ones from the right tapes?) Especially if you have directories spread across various different disks that might be backed up to different tapes. So by having additional information stored in each datafile Oracle can provide additional checks that the set of files that are being used when the database starts up are consistent and all belong together. Oracle also ensures that all the datafiles that are suposed to exist actually do as well. So what might this mean for postgres and tablespaces? It could mean that on startup the database checks to verify that all the tablespaces that are registered actually exist. And that the data in each tablespace is consistent with the current WAL status. (i.e. someone didn't restore a tablespace from backup while the database was down that is old and needs recovery. A lot of what I am talking about here become PITR issues. But since PITR and tablespaces are both potential features for 7.5, how they interact probably should be thought about in the designs for each. thanks, --Barry ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [pgsql-hackers-win32] [HACKERS] What's left?
Jan Wieck [EMAIL PROTECTED] writes: the point is that PostgreSQL is no GNU product, never has been and if someone intends to he shall do so after yanking out the contributions I made. Note that when you released your contributions you did so under a license that imposed no such conditions. If Microsoft wanted to release a Microsoft Postgresql under a completely proprietary license they would be free to do so. Likewise if someone wanted to release a GPL'd GNU Postgresql they could do it. And nobody could force either to yank anyone's code. -- greg ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [pgsql-hackers-win32] [HACKERS] What's left?
Greg Stark wrote: imposed no such conditions. If Microsoft wanted to release a Microsoft Postgresql under a completely proprietary license they would be free to do I have often wondered, in a completely off-topic and unproductive sort of way, if exactly that has not already been done by an unscrupulous or semi-scrupulous commercial vendor. This has been done in the past (a certain vendor's tcp/ip stack comes to mind), but I wonder if anyone ever pulled it with this project. Merlin ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-hackers-win32] [HACKERS] What's left?
Merlin Moncure [EMAIL PROTECTED] wrote: Greg Stark wrote: imposed no such conditions. If Microsoft wanted to release a Microsoft Postgresql under a completely proprietary license they would be free to do I have often wondered, in a completely off-topic and unproductive sort of way, if exactly that has not already been done by an unscrupulous or semi-scrupulous commercial vendor. This has been done in the past (a certain vendor's tcp/ip stack comes to mind), but I wonder if anyone ever pulled it with this project. There's nothing unscrupulous about including BSD's TCP/IP stack in a commercial product. In fact, the main reason we all use TCP/IP today is because of the BSD license. It was just another protocol at one time. If I remember correctly, Postgres was used as the basis for the Illustra commercial product, which was bought by Informix and merged into Informix Universal Server. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Out of space situation and WAL log pre-allocation (was Tablespaces)
Tom Lane [mailto:[EMAIL PROTECTED] Joe Conway [EMAIL PROTECTED] writes: Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: Maybe specify an archive location (that of course could be on a separate partition) that the external archiver should check in addition to the normal WAL location. At some predetermined interval, push WAL log segments no longer needed to the archive location. Does that really help? The panic happens when you fill the normal and archive partitions, how's that different from one partition? I see your point. But it would allow you to use a relatively modest local partition for WAL segments, while you might be using a 1TB netapp tray over NFS for the archive segments. Fair enough, but it seems to me that that sort of setup really falls in the category of a user-defined archiving process --- that is, the hook that Postgres calls will push WAL segments from the local partition to the NFS server, and then pushing them off NFS to tape is the responsibility of some other user-defined subprocess. Database panic happens if and only if the local partition overflows. I don't see that making Postgres explicitly aware of the secondary NFS arrangement will buy anything. Tom's last sentence there summarises the design I was working with. I had considered Joe's suggested approach (which was Oracle's also). However, the PITR design will come with a usable low-function program which can easily copy logs from pg_xlog to another archive directory. That's needed as a test harness anyway, so it may as well be part of the package. You'd be able to use that in production to copy xlogs to another larger directory as a staging area to tape/failover on another system: effectively Joe's idea is catered for in the basic package. Anyway I'm answering questions before publishing the design as stands...though people do keep spurring me to refine it as I'm writing it down! That's why its good to document it I guess. I guess if the archive partition fills up, I would err on the side of dropping archive segments on the floor. That should be user-scriptable policy, in my worldview. Hmmm. Very difficult that one. My experience is in commercial systems. Dropping archive segments on the floor is just absolutely NOT GOOD, if that is the only behaviour. The whole purpose of having a dbms is so that you can protect your business data, while using it. Such behaviour would most likely be a barrier to wider commercial adoption. [Oracle and other dbms will freeze when this situation is hit, rather than continue and drop archive logs.] User-selectable behaviour? OK. That's how we deal with fsync; I can relate to that. That hadn't been part of my thinking because of the importance I'd attached to the log files themselves, but I can go with that, if that's what was meant. So, if we had a parameter called Wal_archive_policy that has 3 settings: None = no archiving Optimistic = archive, but if for some reason log space runs out then make space by dropping the oldest archive logs Strict = if log space runs out, stop further write transactions from committing, by whatever means, even if this takes down dbms. That way, we've got something akin to transaction isolation level with various levels of protection. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] The Return of PITR
In the last few days, a number of issues have come up on [HACKERS] and [PERFORM] that have made important contribs to the PITR topic. Moreover, work such as tablespaces has sufficient cross-over with PITR to make it important to double post things, which defeats the object of having a separate list. AFAIKS, the experiment of having a separate PITR list has not worked, though it has successfully raised awareness and moved the topic forward. I suggest from here that PITR is discussed on the HACKERS list again, and will do this myself from now on. ...this is not a de-commitment from me! Best Regards, Simon Riggs ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Out of space situation and WAL log pre-allocation (was Tablespaces)
Joe Conway [EMAIL PROTECTED] writes: Simon Riggs wrote: O... and other dbms will freeze when this situation is hit, rather than continue and drop archive logs.] Been there, done that, don't see how it's any better. I hesitate to be real specific here, but let's just say the end result was restore from backup :-( It's hard for me to imagine a situation in which killing the database would be considered a more attractive option than dropping old log data. You may or may not ever need the old log data, but you darn well do need a functioning database. (If you don't, you wouldn't be going to all this work.) I think also that Simon completely misunderstood my intent in saying that this could be user-scriptable policy. By that I meant that the *user* could write the code to behave whichever way he liked. Not that we were going to go into a mad rush of feature invention and try to support every combination we could think of. I repeat: code that pushes logs into a secondary area is not ours to write. We should concentrate on providing an API that lets users write it. We have only limited manpower for this project and we need to spend it on getting the core functionality done right, not on inventing frammishes. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-hackers-win32] [HACKERS] What's left?
-Original Message- From: Merlin Moncure [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 03, 2004 12:28 PM To: Greg Stark Cc: [EMAIL PROTECTED] Subject: Re: [pgsql-hackers-win32] [HACKERS] What's left? Greg Stark wrote: imposed no such conditions. If Microsoft wanted to release a Microsoft Postgresql under a completely proprietary license they would be free to do I have often wondered, in a completely off-topic and unproductive sort of way, if exactly that has not already been done by an unscrupulous or semi-scrupulous commercial vendor. This has been done in the past (a certain vendor's tcp/ip stack comes to mind), but I wonder if anyone ever pulled it with this project. It would be pretty ridiculous for anyone to do that. After all, how painful is it to add the BSD notice? And lacking the notice, you would be in violation of the license. Not to say that it hasn't happened or won't happen. But it would be really stupid. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Out of space situation and WAL log pre-allocation (was
Simon Riggs wrote: User-selectable behaviour? OK. That's how we deal with fsync; I can relate to that. That hadn't been part of my thinking because of the importance I'd attached to the log files themselves, but I can go with that, if that's what was meant. So, if we had a parameter called Wal_archive_policy that has 3 settings: None = no archiving Optimistic = archive, but if for some reason log space runs out then make space by dropping the oldest archive logs Strict = if log space runs out, stop further write transactions from committing, by whatever means, even if this takes down dbms. That way, we've got something akin to transaction isolation level with various levels of protection. Yep, we will definately need something like that. Basically whenever the logs are being archived, you have to stop the database if you can't archive, no? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Out of space situation and WAL log pre-allocation (was
Simon Riggs wrote: Tom Lane [mailto:[EMAIL PROTECTED] That should be user-scriptable policy, in my worldview. O... and other dbms will freeze when this situation is hit, rather than continue and drop archive logs.] Been there, done that, don't see how it's any better. I hesitate to be real specific here, but let's just say the end result was restore from backup :-( So, if we had a parameter called Wal_archive_policy that has 3 settings: None = no archiving Optimistic = archive, but if for some reason log space runs out then make space by dropping the oldest archive logs Strict = if log space runs out, stop further write transactions from committing, by whatever means, even if this takes down dbms. That sounds good to me. For the Optimistic case, we need to yell loudly if we do find ourselves needing to drop segments. For the Strict case, we just need to be sure it works correctly ;-) Joe ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Thread safe connection-name mapping in ECPG. Is it
Shridhar, want to discuss this off list a bit to work through the various
options and then revent back to the list with a suitable to-do (for
discussion)?
L.
- Original Message -
From: Shridhar Daithankar [EMAIL PROTECTED]
To: Bruce Momjian [EMAIL PROTECTED]
Cc: PostgreSQL-development [EMAIL PROTECTED]
Sent: Wednesday, March 03, 2004 2:10 PM
Subject: Re: Thread safe connection-name mapping in ECPG. Is it
Oh.. By all means..Please do..
The reason I posted it because I didn't wanted to work on it if core is
not
going to accept it on account of non-compliance with spec.
Is this fine?
* Allow a 'connection *' pointer to be specified instead of a string to
denote
a connection.
I plan to work on it whenever possible. What I would like to do is
eliminate the
locks around name-connection mapping as we would be directly using the
connection instead of a name.
I think we can also add the SQL-CA to connection structure so that each
connection gets it's own SQL-CA. That way ECPG is as thread-safe as the
calling
application gets.
And on the plus side we don't have to worry about platform specific
threading
models either.
Thoughts?
Shridhar
Bruce Momjian wrote:
Should I add this to the TODO list?
--
-
Lee Kindness wrote:
Sort of related, I was thinking about adding some more thread-related
code such that if a connection wasn't explicitely specified then the
last connection SET or CONNECTed to for the current thread is used,
rather than just the last connection.
But yeah, specifying the connection by variable (be it string or
connection ptr) would be a definite step forward. Currently you cannot
write a generic function like:
int getit(char *using_connection)
{
EXEC SQL BEGIN DECLARE SECTION;
char *s_connection = using_connection;
int s_it;
EXEC SQL END DECLARE SECTION;
EXEC SQL AT :s_connection SELECT it INTO :s_it FROM some_table;
return( s_it );
}
which could be run concurrently by multiple threads.
L.
Shridhar Daithankar writes:
On Friday 27 February 2004 20:54, Michael Meskes wrote:
On Fri, Feb 27, 2004 at 04:22:33PM +0530, Shridhar Daithankar
wrote:
How about, allowing 'connection *'? If somebody puts a
'connection *'
there it is used. If it is a string a name search is performed.
Best of
both worlds.
How shall anyone put a pointer to a connection struct inside the
SQL
statement?
It would help me a lot if you'd be able to give some examples.
EXEC SQL BEGIN DECLARE SECTION;
connect *connectionPtr;
EXEC SQL END DECLARE SECTION;
EXEC SQL CONNECT TO db AS connectionPtr;
EXEC SQL AT connectionPtr SELECT 1;
After all, it is matter of parsing some code and emitting equivalent
C code,
isn't it?
Shridhar
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [HACKERS] Prepared queries and portals
Cyril VELTER [EMAIL PROTECTED] writes: so I've modified libpq to handle the case by adding to functions : PQexecPreparedPortal(conn,stmtName,portalName,nParams,paramValues,paramlengt h,paramFormats,resultFormat,maxrows); and PQfetchPortal(conn,portalName,maxrows) PQexecPreparedPortal is a PQexecPrepared clone but you can specify the portal in which the result should be put (which might be the unnamed one) and the maximum number of rows to retreive after the execution. PQfetchPortal fetch the next rows. This works nicely. This looks fairly messy to me. Seems like the exposed API ought to be three functions: set up a portal, fetch (up to) N rows from a portal, close down a portal. Your proposal confuses the first two and fails to provide the last. More generally it might be a good idea to provide lower-level access to other parts of the extended-query protocol, such as Describe and Sync. I did not do anything about that in 7.4 for lack of time, but it's still something that ought to be thought about. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Slony-I makes progress
On Wed, Mar 03, 2004 at 04:57:28PM -0500, Jan Wieck wrote: After some substantial progress on the Slony-I engine development, I'd like to give a little status report and invite everyone who wants to participate in this project to join the mailing list and the development team. Jan, thank you so much for your hard work on this project. Both, the provider change and the failover need a much more complex configuration than the current shell scripts can setup. The problem is that the configuration and administration tools are not designed yet. So here is a huge field for others to step up and take a key role in this project. So what are you looking for here? When I last built slony, things mostly worked, but a few niggling details were broken. I was going to submit a few patches, but when I talked to you, it seemed like you weren't quite ready for patches. Is the tree stable enough that I could do some work on it and expect it to remain relatively consistent over a few hours or even a day or two? Also, to get this out of the way (as it presently plagues erserver), do you have a particular language in mind? I'd like to avoid the dogmatic jihad by not submitting a perl tool if the eventual goal is to be end-to-end C (or java or tcl or whatever). Alex -- [EMAIL PROTECTED] Alex J. Avriette, Unix Systems Gladiator Something seems to have happened to the life support system , Dave. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Tablespaces
Bruce Momjian wrote: Greg Stark wrote: Bruce Momjian [EMAIL PROTECTED] writes: Greg Stark wrote: I am expecting to hear some bleating about this from people whose preferred platforms don't support symlinks ;-). However, if we don't Well, one option would be to have the low level filesystem storage (md.c?) routines implement a kind of symlink themselves. Just a file with a special magic number followed by a path. On further contemplation it doesn't seem like using symlinks really ought to be necessary. It should be possible to drive everything off the catalog tables while avoidin having the low level filesystem code know anything about them. Instead of having the low level code fetch the pg_* records themselves, some piece of higher level code would do the query and call down to storage layer to inform it of the locations for everything. It would have to do this on database initialization and on any subsequent object creation. Basically maintain an in-memory hash table of oid - path, and call down to the low level code whenever that hash changes. (Or more likely oid-ts_id and a separate list of ts_id - path.) The advantage of symlinks is that an administrator could see how things are laid out from the command line. That's a poor reason to require symlinks. The administrator can just as easily open up psql and query pg_tablespace to see that same information. Besides, the postgres doesn't know where to look on the filesystem for the /path/to/oid without a system catalog lookup. There doesn't seem to be any sensible reason to force a filesystem requirement when the core operations are diffferent to begin with. If a more global view of all databases is necessary, perhaps there ought to be a system wide view which could display all of that information at once: dbname, relation name, and physical location. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Tablespaces
Thomas Swan wrote: The advantage of symlinks is that an administrator could see how things are laid out from the command line. That's a poor reason to require symlinks. The administrator can just as easily open up psql and query pg_tablespace to see that same information. Besides, the postgres doesn't know where to look on the filesystem for the /path/to/oid without a system catalog lookup. There doesn't seem to be any sensible reason to force a filesystem requirement when the core operations are diffferent to begin with. If a more global view of all databases is necessary, perhaps there ought to be a system wide view which could display all of that information at once: dbname, relation name, and physical location. Who doesn't have symlinks these days, and is going to be using tablespaces? Even Win32 has them. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] 7.4.2 branding
I have caught up on my email and am working on the 7.4.2 branding now. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Comments on patch for date_trunc( 'week', ... );
Hey Bruce, I never saw the patch hit the hackers list. Did any of you smart folks take a look at it? Cheers, Rob When grilled further on (Wed, 3 Mar 2004 13:58:02 -0500 (EST)), Bruce Momjian [EMAIL PROTECTED] confessed: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches I will try to apply it within the next 48 hours. --- Robert Creager wrote: -- Start of PGP signed section. Per a brief conversation with Tom, I've created a patch for adding support for'week' within the date_trunc function. Within the patch I added a couple of test cases and associated target output, and changed the documentation to add 'week' appropriately. Comments? Should I of bypassed this list and subscribed/posted to the patch list? Cheers, Rob -- 17:07:39 up 15 days, 46 min, 4 users, load average: 2.18, 2.23, 2.27 Linux 2.4.21-0.13_test #60 SMP Sun Dec 7 17:00:02 MST 2003 [ Attachment, skipping... ] -- End of PGP section, PGP failed! -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 -- 20:21:06 up 18 days, 3:59, 2 users, load average: 1.52, 1.31, 1.25 Linux 2.4.21-0.13_test #60 SMP Sun Dec 7 17:00:02 MST 2003 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Comments on patch for date_trunc( 'week', ... );
Well, it must have hit the lists if I have put it in the patch queue, no? --- Hey Bruce, I never saw the patch hit the hackers list. Did any of you smart folks take a look at it? Cheers, Rob When grilled further on (Wed, 3 Mar 2004 13:58:02 -0500 (EST)), Bruce Momjian [EMAIL PROTECTED] confessed: Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches I will try to apply it within the next 48 hours. --- Robert Creager wrote: -- Start of PGP signed section. Per a brief conversation with Tom, I've created a patch for adding support for'week' within the date_trunc function. Within the patch I added a couple of test cases and associated target output, and changed the documentation to add 'week' appropriately. Comments? Should I of bypassed this list and subscribed/posted to the patch list? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Slony-I makes progress
Followup-To: Slony1-general ML Alex J. Avriette wrote: On Wed, Mar 03, 2004 at 04:57:28PM -0500, Jan Wieck wrote: After some substantial progress on the Slony-I engine development, I'd like to give a little status report and invite everyone who wants to participate in this project to join the mailing list and the development team. Jan, thank you so much for your hard work on this project. Both, the provider change and the failover need a much more complex configuration than the current shell scripts can setup. The problem is that the configuration and administration tools are not designed yet. So here is a huge field for others to step up and take a key role in this project. So what are you looking for here? When I last built slony, things mostly worked, but a few niggling details were broken. I was going to submit a few patches, but when I talked to you, it seemed like you weren't quite ready for patches. Is the tree stable enough that I could do some work on it and expect it to remain relatively consistent over a few hours or even a day or two? What I am looking for is a super-comfortable GUI application that makes planning and configuring a master-cascaded-multislave replication system doable by everyone who can identify a clickable button. Honestly, I personally can live with a sh+sed+m4 tool collection. But I guess only few would agree to that. So it's basically up to you and everyone else around here what the outcome of this is. What is required to fit into the data-center is a batch utility that can be called in a script and that causes a currently failing cluster to change the configuration (change the origin of data sets, change providers, drop nodes ... that kind of stuff). The same utility would ideally be able to setup new nodes etc. so that it can be used as an interims solution until the GUI wizzard is ready for prime time. The current CVS replicates fine and the test_?_pgbench scripts in the src/ducttape directory do it all at once. I have changed a couple of things in the autoconf stuff. The whole thing is now expected to be compiled and installed by the postgres user with --prefix pointing to the postgres home directory (the same as the --prefix for the PG installation from sources was). The problem here is, that if we ever want to create a single C function from a GUI tool on a remote box, its shared library better be in the PostgreSQL lib directory so it can be ... AS '$libdir/objfile' no matter where that is and what extension shared objects on that architecture have. Also, to get this out of the way (as it presently plagues erserver), do you have a particular language in mind? I'd like to avoid the dogmatic jihad by not submitting a perl tool if the eventual goal is to be end-to-end C (or java or tcl or whatever). For the production batch commandline utility I think it is C. Other than that ... I said the field is open. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] 7.4.2 branding
On Wednesday 03 March 2004 10:11 pm, Bruce Momjian wrote: I have caught up on my email and am working on the 7.4.2 branding now. Ack is it too late to submit a patch to fix the int overflow problem with pg_autovacuum? I'll send a patch to the patches list shortly, hopefully it will get included. Matthew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] 7.4.2 branding
On Wed, 3 Mar 2004, Bruce Momjian wrote: I have caught up on my email and am working on the 7.4.2 branding now. Just as a reminder, before ppl panic ... we are aiming for Monday for a 7.4.2 release ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] 7.4.2 branding
Matthew T. O'Connor [EMAIL PROTECTED] writes: Ack is it too late to submit a patch to fix the int overflow problem with pg_autovacuum? No; we're not planning to wrap 7.4.2 till (probably) Sunday. If you can get the patch in within the next day or so, we can include it. I thought we'd fixed that problem already though. Is there another? regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] 7.4.2 branding
On Thursday 04 March 2004 12:04 am, Tom Lane wrote: I thought we'd fixed that problem already though. Is there another? Don't know if it's another, but the problem posted by Cott Lang on 1/17 ( http://archives.postgresql.org/pgsql-bugs/2004-01/msg00111.php ) has not been resolved. pg_autovacuum.c was last updated on 1/8. In the referenced email, he included a fix that was working him, but in your follow up email, you suggested that there were more changes required. I have tried to follow your recommendations, but I'm not sure everything is done right. I'm going to submit my patch to the patches list shortly. Matthew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Tablespaces
Tom Lane wrote: Thomas Swan [EMAIL PROTECTED] writes: Bruce Momjian wrote: The advantage of symlinks is that an administrator could see how things are laid out from the command line. That's a poor reason to require symlinks. The administrator can just as easily open up psql and query pg_tablespace to see that same information. Something to keep in mind here is that one of the times you would most likely need that information is when the database is broken and you *can't* simply open up psql and inspect system catalogs. I like the fact that a symlink implementation can be inspected without depending on a working database. That's a sufficient argument, to allow for it. Recoverability would be one reason. If we were going to build a non-symlink implementation, I'd want the highlevel-to-lowlevel data transfer to take the form of a flat ASCII file that could be inspected by hand, rather than some hidden in-memory datastructure. But given the previous discussion in this thread, I cannot see any strong reason not to rely on symlinks for the purpose. We are not in the business of building replacements for OS features. I do like the flat file output at least for a record of what went where. Regardless of whether or not symlinks are used, the admin would need to know what directories/files/filesystems are to be backed up. I am concerned as to what extent different filesystems do when you back the directories up.Would NTFS containing symlinks be able to be backed up with a tar/zip command, or is something more elaborate needed? In the past, before upgrading, I have had to tar the pgdata directory with the postmaster shutdown to insure a quick restoration of the database in case an upgrade didn't proceed uneventfully. Also, in the event of a major version upgrade the restored information may or may not proceed uneventfully.I just wanted to point out something I thought might be an issue further down the road. Perhaps the system catalog / flat file approach would be a more solid approach, both of which would not involve replacing or duplicating OS features. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Thread safe connection-name mapping in ECPG. Is it
Lee Kindness wrote: Shridhar, want to discuss this off list a bit to work through the various options and then revent back to the list with a suitable to-do (for discussion)? I don't mind. Just for summary, I am listing the discussion/proposal so far on this issue.. - Dispose names of connectiong and replace them with a pointer. - Make SQL CA a member of connection structure rather than a thread local variable. - Update man pages and documentation - Update examples. Drop me a mail offline to take this further. Shridhar ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Out of space situation and WAL log pre-allocation (was
Tom Lane wrote: I think also that Simon completely misunderstood my intent in saying that this could be user-scriptable policy. By that I meant that the *user* could write the code to behave whichever way he liked. Not that we were going to go into a mad rush of feature invention and try to support every combination we could think of. I repeat: code that pushes logs into a secondary area is not ours to write. We should concentrate on providing an API that lets users write it. We have only limited manpower for this project and we need to spend it on getting the core functionality done right, not on inventing frammishes. Hmm... I totally agree. I think the backend could just offer a shared memory segment and a marker message to another process to allow copy from it. then it is the applications business to do the things. Of course there has to be a two way agreement about it but an API is a real nice thing rather than an application. Shridhar ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
[HACKERS] Regression tests on Nintendo Game Cube
Folks, Today Michael Steil and I have tested PostgreSQL 7.4.1 on Nintendo Game Cubes. All regression test (but stats - stats collector was off instead of on) have passed successfully. [EMAIL PROTECTED] root]# uname -a Linux 192.168.0.47 2.6.3 #20 Wed Mar 3 12:22:07 CET 2004 ppc unknown unknown GNU/Linux [EMAIL PROTECTED] root]# cat /proc/cpuinfo processor : 0 cpu : 745/755 revision: 50.20 (pvr 0008 3214) bogomips: 968.70 vendor : IBM machine : Nintendo GameCube It is damn hard to compile PostgreSQL on 24 mb ram + 16 mb swap (stolen from my sound buffer provided by Nintendo). gram.c and preproc.c have been compiled on an imac maschine - the rest has been compiled on Michael's Game Cube. Compiling and regression testing takes around 30 years (wow, this kind of stuff is damn slow - it is not a good database server *g*). Game Cubes are now fully supported by us ;). Cheers, Michael and Hans = starting postmaster== running on port 65432 with pid 24018 == creating database regression == CREATE DATABASE ALTER DATABASE == dropping regression test user accounts == == installing PL/pgSQL== == running regression test queries== parallel group (13 tests): char int2 name boolean varchar text int8 oid int4 float4 float8 bit numeric boolean ... ok char ... ok name ... ok varchar ... ok text ... ok int2 ... ok int4 ... ok int8 ... ok oid ... ok float4 ... ok float8 ... ok bit ... ok numeric ... ok test strings ... ok test numerology ... ok parallel group (20 tests): point lseg box date circle time path reltime abstime type_sanity timestamptz interval tinterval timestamp timetz comments polygon inet opr_sanity oidjoins point... ok lseg ... ok box ... ok path ... ok polygon ... ok circle ... ok date ... ok time ... ok timetz ... ok timestamp... ok timestamptz ... ok interval ... ok abstime ... ok reltime ... ok tinterval... ok inet ... ok comments ... ok oidjoins ... ok type_sanity ... ok opr_sanity ... ok test geometry ... ok test horology ... ok test insert ... ok test create_function_1... ok test create_type ... ok test create_table ... ok test create_function_2... ok test copy ... ok parallel group (7 tests): create_operator create_aggregate vacuum inherit triggers constraints create_misc constraints ... ok triggers ... ok create_misc ... ok create_aggregate ... ok create_operator ... ok inherit ... ok vacuum ... ok parallel group (2 tests): create_view create_index create_index ... ok create_view ... ok test sanity_check ... ok test errors ... ok test select ... ok parallel group (17 tests): select_distinct select_into select_distinct_on select_having case update union random aggregates transactions select_implicit arrays subselect hash_index join portals btree_index select_into ... ok select_distinct ... ok select_distinct_on ... ok select_implicit ... ok select_having... ok subselect... ok union... ok case ... ok join ... ok aggregates ... ok transactions ... ok random ... ok portals ... ok arrays ... ok btree_index ... ok hash_index ... ok update ... ok test privileges ... ok test misc ... ok parallel group (5 tests): portals_p2 select_views cluster rules foreign_key select_views ... ok portals_p2 ... ok rules... ok foreign_key ... ok cluster ... ok parallel group (14 tests): sequence limit polymorphism without_oid truncate copy2 prepare temp rangefuncs conversion domain stats plpgsql alter_table limit... ok plpgsql ... ok copy2... ok
Re: [HACKERS] [PERFORM] WAL Optimisation - configuration and usage
Neil Conway Simon Riggs wrote: Josh Berkus wrote Simon Riggs wrote Please set WAL_DEBUG to 1 so we can see a bit more info: thanks. I'm pretty sure that WAL_DEBUG requires a compile-time option. I'm surprised, but you are right, the manual does SAY this requires a compile time option; it is unfortunately not correct. Actually, the manual is correct: in 7.4 and earlier releases, enabling wal_debug can be done without also setting a compile-time #ifdef. As of current CVS HEAD, the WAL_DEBUG #ifdef must be defined before this variable is available. Touche! I stand corrected, thank you both. My suggestion does work for Rob, then. [This also implies I have a screwed version on my machine, so thank you also for flushing that lurking issue out for me. I'd had a suspicion for a few weeks. Lucky I'm still just prototyping.] On the other hand, I was just about to change the wal_debug behaviour to allow better debugging of PITR features as they're added. I think it is very important to be able to put the system fairly easily into debug mode; a recompile is easy enough, but it would be even better to avoid this completely. This would mean reversing the change you describe: here's the design: The behaviour I wish to add is: Keep wal_debug as a value between 0 and 16. If =0 then no debug output (default). Use following bitmasks against the value Mask 1 = XLOG Checkpoints get logged Mask 2 = Archive API calls get logged Mask 4 = Transaction - commits get logged Mask 8 = Flush INSERTs get logged That way it should be fairly straightforward to control the amount and type of information available to administrators. The existing design produces too much info to be easily usable, mostly requiring a perl program to filter out the info overload and do record counts. This suggested design allows you to control the volume of messages, since the bitmasks are arranged in volume/frequency order and brings the wal_debug option back into something useful for problem diagnosis on live systems, not just hacking the code. Anybody object to these mods, or have better/different ideas? Getting the diagnostics right is fairly important, IMHO, to making PITR become real. Best regards, Simon Riggs ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] [PERFORM] WAL Optimisation - configuration and usage
Simon Riggs [EMAIL PROTECTED] writes: The behaviour I wish to add is: Keep wal_debug as a value between 0 and 16. If =0 then no debug output (default). Use following bitmasks against the value Mask 1 = XLOG Checkpoints get logged Mask 2 = Archive API calls get logged Mask 4 = Transaction - commits get logged Mask 8 = Flush INSERTs get logged I see no value in reverting Neil's change. The above looks way too much like old-line assembler-programmer thinking to me, anyway. Why not invent a separate, appropriately named boolean variable for each thing you want to control? Even C programmers manage to avoid doing the sort of mental arithmetic that the above would force onto DBAs. As for whether it should be #ifdef'd or not, I'd have no objection to turning WAL_DEBUG on by default in pg_config_manual.h for the duration of PITR development. One should not however confuse short-term debugging needs with features that the average user is going to need indefinitely. (It was not too long ago that there was still debugging code for btree index building in there, for crissakes.) regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [PERFORM] WAL Optimisation - configuration and usage
Simon Riggs wrote: On the other hand, I was just about to change the wal_debug behaviour to allow better debugging of PITR features as they're added. That's a development activity. Enabling the WAL_DEBUG #ifdef by default during the 7.5 development cycle would be uncontroversial, I think. I think it is very important to be able to put the system fairly easily into debug mode It is? Why would this be useful for non-development activities? (It may well be the case that we ought to report more or better information about the status of the WAL subsystem; but WAL_DEBUG is surely not the right mechanism for emitting information intended for an administrator.) -Neil ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] Slony-I makes progress
After some substantial progress on the Slony-I engine development, I'd like to give a little status report and invite everyone who wants to participate in this project to join the mailing list and the development team. The project homepage is here: http://gborg.postgresql.org/project/slony1/projdisplay.php The current development status: Slony-I configures on RedHat Linux and FreeBSD 4.9 against a PostgreSQL 7.4. The engine is capable of hot install, hot subscribe with catch up and cascaded slaves. The config utilities are either not existent yet or ugly, incomplete shell scripts. The replication log trigger is the pototypes version in PL/Tcl. What this odd collection is capable of doing is creating a master, slave and cascaded slave replication system with a pgbench database as master. The kick is, that the master database is originally just a plain, stock PG 7.4.1 with zero changes, and the pgbench application is running non stop through the whole process. Next steps: The log trigger function must be reimplemented in C. I will do this during the next couple of days. Implement the functionality to change the data provider of a slave. With that a slave can be added as a cascaded slave, copy the data, catch up and then switch over to replicate against the master, or an existing slave can become a cascaded one to reduce the load on the master. Implement the failover capability to replace a failed master with a slave. Add backup and replication logging (sort of PITR based on replication information). Add sequence replication. Add script execution support for schema changes. Both, the provider change and the failover need a much more complex configuration than the current shell scripts can setup. The problem is that the configuration and administration tools are not designed yet. So here is a huge field for others to step up and take a key role in this project. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
