[HACKERS] xlog flush request is not satisfied error
Hello I've did pg_dumpall of a number of databases on 7.4.2 install of PostgreSQL then imported that into another 7.4.2 install. This is on Debian Sid, every 5 hours, there's a script /usr/lib/postgresql/bin/do.maintenance which runs on the database, VACUUMing it and such. The next time that script ran, I got the following errors: ERROR: xlog flush request 0/E25BA1C4 is not satisfied --- flushed only to 0/841D994 CONTEXT: writing block 1 of relation 17143/370747 \connect: FATAL: xlog flush request 0/E25BC218 is not satisfied --- flushed only to 0/841D994 CONTEXT: writing block 1 of relation 17143/370749 At that point, it was still running, but due to a misconfiguration, wasn't logging anything. So, I thought I should restart it with logging. It failed to start: 2004-04-19 08:41:15 [3841] LOG: database system shutdown was interrupted at 2004-04-19 08:41:13 EEST 2004-04-19 08:41:15 [3841] LOG: checkpoint record is at 0/789AE1C 2004-04-19 08:41:15 [3841] LOG: redo record is at 0/789AE1C; undo record is at 0/0; shutdown FALSE 2004-04-19 08:41:15 [3841] LOG: next transaction ID: 3711; next OID: 610488 2004-04-19 08:41:15 [3841] LOG: database system was not properly shut down; automatic recovery in progress 2004-04-19 08:41:15 [3841] LOG: redo starts at 0/789AE5C 2004-04-19 08:41:19 [3841] PANIC: heap_delete_redo: no block 2004-04-19 08:41:19 [3835] LOG: startup process (PID 3841) was terminated by signal 6 2004-04-19 08:41:19 [3835] LOG: aborting startup due to startup process failure Subsequent attempts to start it generate: 2004-04-19 08:48:18 [3921] LOG: database system was interrupted while in recovery at 2004-04-19 08:41:15 EEST HINT: This probably means that some data is corrupted and you will have to use the last backup for recovery. 2004-04-19 08:48:18 [3921] LOG: checkpoint record is at 0/789AE1C ... While the data is irrelevant (I can get that from the dump), this could be a bug in PostgreSQL, or whatnot, so I'm asking here if anyone has any ideas what I can do to that. -- George Cristian Birzan gcbirzan (at) wolfheart (dot) ro I respect faith, but doubt is what gives you an education. -- Wilson Mizner signature.asc Description: Digital signature
Re: [HACKERS] ON COMMIT DROP
Christopher Kings-Lynne [EMAIL PROTECTED] writes: Is there any reason why the 'ON COMMIT' behaviour feature is not available if you use CREATE TABLE AS ...? Where exactly would you propose to stick it in the syntax? Can you do it without introducing more fully-reserved words than we have already? Is there any spec or other-product precedent for it? (Offhand I can't even find CREATE TABLE AS in SQL99...) regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] ON COMMIT DROP
On Mon, 2004-04-19 at 02:16, Tom Lane wrote: (Offhand I can't even find CREATE TABLE AS in SQL99...) This is semi-OT, but CREATE TABLE AS is (new) in SQL2003. At few glance, the spec's notion of the command is about the same as ours, except for a few minor syntactic differences (e.g. the [ WITH [ NO ] DATA ] clause the spec allows for; I'm planning to implement that soon). -Neil ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] ON COMMIT DROP
Where exactly would you propose to stick it in the syntax? Good question, I don't know. Can you do it without introducing more fully-reserved words than we have already? No idea. Is there any spec or other-product precedent for it? (Offhand I can't even find CREATE TABLE AS in SQL99...) Weeell. I was just minorly annoyed at having to create table with on commit behaviour, then insert into select from instead of being able to do it in one step... Chris ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] fsync thread on linux-kernel
While it's a couple of weeks old, in case it hasn't been seen by those who need it, the folllowing thread on linux kernel discusses the fsync-and-IDE-writecache interactions. There is also a comment on MacOS X - I have no idea if that was already known, but I thought it might be worth pointing out. http://www.uwsg.iu.edu/hypermail/linux/kernel/0403.2/0527.html //Magnus ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] executing prepared select, missing RowDescription info
- Begin Forwarded Message - From To:[EMAIL PROTECTED] Mon Apr 19 11:08:57 2004 Date: Mon, 19 Apr 2004 11:08:57 METDST To: [EMAIL PROTECTED] (Tom Lane) Subject: Re: [HACKERS] executing prepared select, missing RowDescription info In-Reply-To: [EMAIL PROTECTED]; from Tom Lane at Apr 18, 104 7:11 pm Content-Length: 899 Status: RO Kris Jurka [EMAIL PROTECTED] writes: When executing a prepared select statement, the returned RowDescription protocol message does not have any information for the table oid or column position. Running the equivalent select without prepare provides this information, so I don't see why the act of preparing and executing the statement removes this valuable data. Any insight on why it isn't there or how to fix it? Fixing this would be a tad messy, because the information is not propagated up through a utility-statement Portal. I guess I would ask why you're using EXECUTE at all; it's considerably less efficient than invoking the prepared statement via the protocol-level operation for doing so (Bind, then Execute). regards, tom lane And how would I do this more efficient Bind, then Execute using libpq? TIA Regards, Christoph - End Forwarded Message - ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Tuple sampling
The proposed new sampling method (http://archives.postgresql.org/pgsql-hackers/2004-04/msg00036.php and http://archives.postgresql.org/pgsql-patches/2004-04/msg00045.php) basically incorporates two independant changes: (1) Two-stage sampling: Stage one collects a random sample of pages, stage two collects a random sample of tuples out of these pages. (2) Improved row count estimation: Once a page is read in, all tuples on this page are looked at to see whether they are active (visible, live) or dead. The estimated row count is calculated from the number of active tuples seen, the number of visited pages, and the total number of pages. The preview patch implements three sampling methods: Method 0 is the original sampling method; method 1 uses both changes; and method 2 is sort of change (1) without change (2). Theoretically we could apply only the second change, if we come to the conclusion that we do not want two-stage sampling. But I don't have test results for this. I'd like to present my *interpretation* of the results I got from testing various sampling methods. Whoever is interested in looking at the raw test results can get them from http://www.pivot.at/pg/SamplePerf2.sxc or (for those still without OOo) http://www.pivot.at/pg/SamplePerf2.html. This set of tests measures only the sampling times, actually analysing the sample and storing the results in pg_statistic has been disabled. To eliminate as many sources of noise as possible, following steps have been taken prior to each test run: . SELECT count(*) FROM testtable; to make sure that all visibility bits in the tuple headers are set correctly, . tar cf /dev/null something to fill the OS cache with unrelated stuff, . SELECT count(*) FROM othertable; to fill the shared buffers with unrelated data, . CHECKPOINT; to avoid a checkpoint happening during the test run. The tests have been performed with two types of tables. The tables named x, x2, x3, x3d have initially been copied from tenk1 in the regression database. These tables have between 20 and 30 tuples per page. Tables of the other type -- named y, y3, y3d -- have much smaller tuples, ca. 150 per page. Method 1 vs. method 2: With large tuples method 2 is sometimes faster, sometimes slower than method 1, never enough to worry about. This has been tested for statistics targets 10 and 100. Small tuples, statistics target 10 to 100: No reproduceable difference up to 490 pages (8 tuples). Starting at 1900 pages (32 tuples) method 2 is consistently faster. The difference can easily be explained by hundreds of thousands or even millions of heap_fetch() calls. OTOH with method 2 we get row count estimation errors of up to 20% compared to less than 1% for method 1. So I conclude that change (2) is well worth the CPU cycles it costs. Method 0 produced estimation errors of up to 60%. What about two-stage sampling? Comparing the speeds of method 0 and method 2 we get the following pattern which is basically the same for all statistics targets I tried. Sample size 3000 (3): For tables smaller than 1000 (5000) pages all sampling methods access all or almost all pages and there is hardly any runtime difference. For table sizes between 1000 (5000) and 3000 (3) the new method reads the whole table while the old method starts to skip pages. This doesn't result in a significant runtime difference, however. If the table size grows beyond 3000 (3) pages, the number of page reads continues to grow only for the old method, but up to ca. 3 (30) pages the new method is not much faster (if at all). For tables larger than 3 (30) pages two-stage sampling is reliably faster. Other pros and cons of two-stage sampling Pro and con: Cache pollution. Two-stage sampling reads the same number of pages as one-stage sampling for small tables, slightly more for medium sized tables (worst case is ca. 15%), and a little less to far less for large to very large tables (3000 vs. 2 for 43 pages with statistics target 10, 3 vs. 98000 for the same table with statistsitcs target 100). Con: In Tom's words There's still a risk of not being able to collect N rows out of N blocks, if you are unfortunate enough to select a lot of wholly-empty pages. But that seems like a low-probability scenario; besides such a table would be so desperately in need of VACUUM FULL that the possible low quality of the stats hardly matters. Con: The sample generated is not perfectly random (cf. discussion starting at http://archives.postgresql.org/pgsql-performance/2004-04/msg00196.php). If somebody has an idea how we could steer against that effect of collecting tuples from too few different pages, I'd be happy to implement it. If nothing pops up, I think that the current consensus is that we don't care. Anything else? Servus Manfred ---(end of broadcast)--- TIP 4: Don't 'kill -9' the
[HACKERS] GUC variable set, TODO
Hi, I might have some time left to spend on the item Allow external interfaces to extend the GUC variable set filed under Administration on the TODO list. I'm thinking of an approach with the steps: 1) Increasing my own understanding of the GUC code 2) Suggest a solution on this newsgroup 3) Implement and submit a patch Any advice is greatly appreciated. Especially the ones that might speed up step #1. Comments, suggestions? Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] xlog flush request is not satisfied error
George Cristian Birzan [EMAIL PROTECTED] writes: ERROR: xlog flush request 0/E25BA1C4 is not satisfied --- flushed only to = 0/841D994 CONTEXT: writing block 1 of relation 17143/370747 While the data is irrelevant (I can get that from the dump), this could be a bug in PostgreSQL, or whatnot, so I'm asking here if anyone has any ideas what I can do to that. Run some hardware diagnostics. This looks like your disk drive dropped some bits ... regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] GUC variable set, TODO
Dear Thomas, I might have some time left to spend on the item Allow external interfaces to extend the GUC variable set filed under Administration on the TODO list. I'm thinking of an approach with the steps: 1) Increasing my own understanding of the GUC code Well, I did that yesterday morning;-) So I can share my little experience. . All guc variables are described in static structures initialized at loadtime. if it is to be extendable, it means that you have break this, and build or add some dynamic structure(s), that will have to be initialized somehow to be filled with current existing options. . There are different version of these static arrays depending on the type of the guc variable (boolean, string, integer...). The actual option is stored in a static variable that can be set or reset with some handlers (to check for appropriate submitted new value). . There is also a dynamic sorted array that allows to find all variables descriptions independently of their type, with a binary search. . You must know that static struct variables are by default initialized to 0 in C, so if something is not initialized, its value is really 0. The current guc code RELIES on this, for instance for flags initialization. Some thoughts: Maybe some new flags value should be added to distinguish possibly native guc variables and others added by extensions? Also, maybe you should keep track of the extension origin of a guc variable? 2) Suggest a solution on this newsgroup 3) Implement and submit a patch Any advice is greatly appreciated. Especially the ones that might speed up step #1. Comments, suggestions? Good luck. -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] xlog flush request is not satisfied error
On Mon, Apr 19, 2004 at 09:53:26AM -0400, Tom Lane wrote: Run some hardware diagnostics. This looks like your disk drive dropped some bits ... That would be something, it would be the second disk in two weeks. It's a brand new drive and I haven't had any other problems. (Heck, even the other PostgreSQL installation, the one I got the dump from, is running on that drive). Though, yes, it would be an indication of a problem elsewhere, if that's the case. -- George Cristian Birzan gcbirzan (at) wolfheart (dot) ro I'm tired of all this nonsense about beauty being only skin-deep. That's deep enough. What do you want, an adorable pancreas? -- Jean Kerr signature.asc Description: Digital signature
Re: [HACKERS] signal 11 on AIX: 7.4.2
Andrew Sullivan [EMAIL PROTECTED] writes: On Thu, Apr 15, 2004 at 07:52:59PM -0400, Tom Lane wrote: I can see from your trace that you are using the getaddrinfo code from libc, but where is configure finding a header that declares struct addrinfo? Hrm, I can't seem to tell. I see this in config.log, but it isn't telling me where it found it. Am I looking in the wrong place? What you'd need to do is determine which system headers are being #include'd by that config test, and then look through them to find struct addrinfo. A shortcut is just to grep through /usr/include and its subdirectories for addrinfo. If you only find one definition, then you don't really need to worry too much. But if there's more than one you need to determine which is getting used. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] GUC variable set, TODO
Fabien COELHO [EMAIL PROTECTED] writes: . All guc variables are described in static structures initialized at loadtime. if it is to be extendable, it means that you have break this, No, you don't. . There is also a dynamic sorted array that allows to find all variables descriptions independently of their type, with a binary search. The idea was to provide an API that would allow insertion of new entries into the dynamic array. This does not mean you have to reinvent where the static entries come from. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] GUC variable set, TODO
Dear Tom, . All guc variables are described in static structures initialized at loadtime. if it is to be extendable, it means that you have break this, No, you don't. Well, sorry, I'm used to show my lack of imagination on the list;-) . There is also a dynamic sorted array that allows to find all variables descriptions independently of their type, with a binary search. The idea was to provide an API that would allow insertion of new entries into the dynamic array. This does not mean you have to reinvent where the static entries come from. Ok. I understand that you suggest to Thomas that he should only touch the search array to insert new pointers to guc description structures, which may come from outside or be newly allocated, without touching the existing static versions for internal guc variables. -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] signal 11 on AIX: 7.4.2
On Mon, Apr 19, 2004 at 11:18:07AM -0400, Tom Lane wrote: A shortcut is just to grep through /usr/include and its subdirectories for addrinfo. If you only find one definition, then you don't really need to worry too much. But if there's more than one you need to determine which is getting used. Maybe an easier way is to examine the output of cpp src/include/c.h. -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) En las profundidades de nuestro inconsciente hay una obsesiva necesidad de un universo lógico y coherente. Pero el universo real se halla siempre un paso más allá de la lógica (Irulan) ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] GUC variable set, TODO
Fabien COELHO [EMAIL PROTECTED] writes: Ok. I understand that you suggest to Thomas that he should only touch the search array to insert new pointers to guc description structures, which may come from outside or be newly allocated, without touching the existing static versions for internal guc variables. Right. IIRC there are some comments about this possibility already in the source code. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] Why are these ARC variables per-backend?
I've got a problem with these variables in freelist.c: static int strategy_cdb_found; static int strategy_cdb_replace; static int strategy_get_from; Why are these per-backend? Shouldn't they be in shared state? regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] 'begin transaction' new syntax bug?
It seems to me that new 'begin isolation level ...' syntax does not work with current cvs head: # # old syntax: # psql SELECT VERSION(); version --- PostgreSQL 7.5devel on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.3.3 (Debian 20040401) psql BEGIN; BEGIN psql SET TRANSACTION ISOLATION LEVEL SERIALIZABLE; SET psql SHOW TRANSACTION ISOLATION LEVEL; transaction_isolation --- serializable # # new syntax? # psql BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE; BEGIN psql SHOW TRANSACTION ISOLATION LEVEL; transaction_isolation --- read committed I would have expected 'serializable' ? If the isolation level or read/write mode is specified, the new transaction has those characteristics, as if SET TRANSACTION was executed. -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] signal 11 on AIX: 7.4.2
Tom Lane wrote: Andrew Sullivan [EMAIL PROTECTED] writes: On Thu, Apr 15, 2004 at 07:52:59PM -0400, Tom Lane wrote: I can see from your trace that you are using the getaddrinfo code from libc, but where is configure finding a header that declares struct addrinfo? Hrm, I can't seem to tell. I see this in config.log, but it isn't telling me where it found it. Am I looking in the wrong place? What you'd need to do is determine which system headers are being #include'd by that config test, and then look through them to find struct addrinfo. judging by gdb's structure printing, the crashed postgres instance used the non-43 compatible 64-bit version of the strucure. What I don't really get is that the whole excercise seems to have scribbled over the stack. The hints pointer originating from the on-stack structure in parse_hba is somehow pointing into the blue. Jan A shortcut is just to grep through /usr/include and its subdirectories for addrinfo. If you only find one definition, then you don't really need to worry too much. But if there's more than one you need to determine which is getting used. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Why are these ARC variables per-backend?
Tom Lane wrote: I've got a problem with these variables in freelist.c: static int strategy_cdb_found; static int strategy_cdb_replace; These two most definitely are per backend because they hold status information about the blocks this backend specifically is mucking with. If it all would be in one function, they would be local variables. static int strategy_get_from; This one is questionable. The point of it is to steer the T1/T2 list sizes towards the goal of the strategy (T1_target). It doesn't matter much if this is off by one every now and then, it will be corrected at the next buffer eviction ... or the one after that. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] a couple of ideas
As I was playing around with prelimiary stuff for regression testing the
CSV patch, I had a couple of ideas I thought I might float.
Context: many of the tests will write a table out and read it back into
a duplicate table and then perform a symmetric difference on the 2
tables. Making the query for this is slightly error prone, so I made two
functions, one to generate the query in text, and the other perform the
query and return a set of records.
Idea 1: an eval facility for psql. e.g.:
\eval symdiffgen('foo','bar')
or
\eval select symdiffgen('foo','bar')
This would be a use of the first query. In effect we'd be providing for
a sort of macro facility.
Idea 2: allow a LIKE clause in the column specs where a function
returning a set of records is called. e.g., instead of
select * from symdiff('foo','bar') as diff(_table_name_ text, d date,
x text);
allow
select * from symdiff('foo','bar') as diff(_table_name_ text, like foo);
There may be good reasons not to do either of these, so I am just
floating them for discussion.
I might also have missed something obvious that would let me do things
easily some other way.
cheers
andrew
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] GUC variable set, TODO
Tom Lane wrote: Fabien COELHO [EMAIL PROTECTED] writes: Ok. I understand that you suggest to Thomas that he should only touch the search array to insert new pointers to guc description structures, which may come from outside or be newly allocated, without touching the existing static versions for internal guc variables. Right. IIRC there are some comments about this possibility already in the source code. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend Thanks Fabien for explaing how the static stuff is built and complemented with the dynamic array. My intention is to pursue a suggestion that Tom and Joe Conway discuss in the thread that eventually ended upp in the TODO item. Here's a link to the discussion (I should of course have included this in my original post): http://archives.postgresql.org/pgsql-hackers/2004-02/msg00229.php Kind Regards, Thomas Hallgren ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Why is libpgtcl still in CVS?
Bruce Momjian wrote: Why is libpgtcl still in /src/interfaces? Isn't it now on gborg? Are there two of them there now? I am still waiting for an answer on this. Does no one know? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] Postgresql snapshots broken?
The snapshots in ftp.postgresql.org/pub/dev/ (and mirrors) appears not to have updated since Apr 5th. These are supposed to be generated nightly, right? If so, they seem broken. //Magnus ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Why are these ARC variables per-backend?
Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: I've got a problem with these variables in freelist.c: static int strategy_cdb_found; static int strategy_cdb_replace; These two most definitely are per backend because they hold status information about the blocks this backend specifically is mucking with. If it all would be in one function, they would be local variables. Would you object if I made 'em actual local variables? This would clutter the strategy API slightly since the vars would have to be passed out of some routines and into others, but I think it would be logically cleaner. (In the back of my mind is the idea to support two instances of the ARC datastructure, one for global and one for local buffers, so minimizing the number of static variables is a prerequisite.) static int strategy_get_from; This one is questionable. The point of it is to steer the T1/T2 list sizes towards the goal of the strategy (T1_target). It doesn't matter much if this is off by one every now and then, it will be corrected at the next buffer eviction ... or the one after that. Why do we have it at all? Seems like it would be at least as good to make the T1-or-T2 decision in StrategyGetBuffer, rather than earlier. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] signal 11 on AIX: 7.4.2
On Mon, Apr 19, 2004 at 11:18:07AM -0400, Tom Lane wrote:
What you'd need to do is determine which system headers are being
#include'd by that config test, and then look through them to find
struct addrinfo.
Well, I have this in /usr/include/netdb.h:
struct addrinfo {
int ai_flags; /* AI_PASSIVE, AI_CANONNAME,
AI_NUMERICH
OST */
int ai_family; /* PF_xxx */
int ai_socktype; /* SOCK_xxx */
int ai_protocol; /* 0 or IPPROTO_xxx */
size_t ai_addrlen;/* length of ai_addr */
char*ai_canonname; /* canonical name for
hostname */
struct sockaddr *ai_addr; /* binary address */
struct addrinfo *ai_next; /* next structure in list */
};
Using the cpp trick that Alvaro Herrera suggested, I see that file
mentioned in the output, and this a little way along:
struct addrinfo {
int ai_flags;
int ai_family;
int ai_socktype;
int ai_protocol;
size_t ai_addrlen;
char*ai_canonname;
struct sockaddr *ai_addr;
struct addrinfo *ai_next;
};
So it looks like that must be the one. Dunno if this helps.
A
--
Andrew Sullivan | [EMAIL PROTECTED]
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Why are these ARC variables per-backend?
Tom Lane wrote: Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: I've got a problem with these variables in freelist.c: static int strategy_cdb_found; static int strategy_cdb_replace; These two most definitely are per backend because they hold status information about the blocks this backend specifically is mucking with. If it all would be in one function, they would be local variables. Would you object if I made 'em actual local variables? This would clutter the strategy API slightly since the vars would have to be passed out of some routines and into others, but I think it would be logically cleaner. (In the back of my mind is the idea to support two instances of the ARC datastructure, one for global and one for local buffers, so minimizing the number of static variables is a prerequisite.) Good idea. static int strategy_get_from; This one is questionable. The point of it is to steer the T1/T2 list sizes towards the goal of the strategy (T1_target). It doesn't matter much if this is off by one every now and then, it will be corrected at the next buffer eviction ... or the one after that. Why do we have it at all? Seems like it would be at least as good to make the T1-or-T2 decision in StrategyGetBuffer, rather than earlier. If it'd be possible. The problem is that it can get called multiple times during one logical buffer lookup/fault cycle. Because of the possible IO, which is done in the caller of the strategy, this would screw up the decision process about the queue sizes. The original ARC logic was built on a single process/thread model, where you do the IO just in the middle of the decision process. With our buffer manager who's now calling the strategy logic this needs a few status switches. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Why are these ARC variables per-backend?
Jan Wieck [EMAIL PROTECTED] writes: Tom Lane wrote: Why do we have it at all? Seems like it would be at least as good to make the T1-or-T2 decision in StrategyGetBuffer, rather than earlier. If it'd be possible. The problem is that it can get called multiple times during one logical buffer lookup/fault cycle. Because of the possible IO, which is done in the caller of the strategy, this would screw up the decision process about the queue sizes. How so? It seems like postponing the decision about which queue to take from couldn't make the behavior worse; if anything it would help to track T1target more closely. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [INTERFACES] [HACKERS] Why is libpgtcl still in CVS?
for the second part, yes, there are 2 on gborg: pgtcl and pgtclng --- Bruce Momjian [EMAIL PROTECTED] wrote: Bruce Momjian wrote: Why is libpgtcl still in /src/interfaces? Isn't it now on gborg? Are there two of them there now? I am still waiting for an answer on this. Does no one know? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match __ Do you Yahoo!? Yahoo! Photos: High-quality 4x6 digital prints for 25¢ http://photos.yahoo.com/ph/print_splash ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Remove MySQL Tools from Source?
Shachar, I have a bunch of perl scripts, as well as one user-defined type, for porting from SQL Server. Where should I place these? Give us a few days. We'll be announcing the opening of the new projects site soon (like, wednesday). Regrettably, the perfect feature for this sort of thing, GForge's Code Snippets, has a bug that will prevent us from using it immediately. But I'll see what I can do. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Remove MySQL Tools from Source?
Andrew Dunstan wrote: Jan Wieck wrote: Christopher Kings-Lynne wrote: ... on projects.postgresql.org, or similar.They really aren't doing any good in /contrib. I've already set up a category conversion tools on pgFoundry, and my idea was one project per target system. I reckon that by far the best way to do a mysql2pgsql converter is to just modify mysqldump C source code to output in postgresql format! ... and contribute it to MySQL :-) But you would have to assign the copyright to them If someone is going to make money from my code, I prefer it to be me, or at least that everyone has a chance to do so rather than just one company. You still don't understand their business model. You publish the code under GPL for free, and offer it under something else for sale for everyone who's actually stupid enough to use it. Jan cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] GiST -- making my index faster makes is slower
I tracked the problem down to the penalty function used to build the tree. Basically, it compares the area of the bounding boxes. There wasnt enough precision in the area calculations - instead of giving 0.0 it would give numbers in the[+-]10^-6 range. This really screwed up how the tree was being built. I cast the float to doubles when calculating area, and it all works well now. dave ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [INTERFACES] [HACKERS] Why is libpgtcl still in CVS?
Brett Schwarz wrote: for the second part, yes, there are 2 on gborg: pgtcl and pgtclng My question is whether they have all the CVS history and fixes in libpgtcl so we can remove it from CVS. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] locale issues on win32 (fix?)
Hello! Here is a temp fix for the locale issues on win32. It passes regression tests, but is *NOT* ready to be applied (if nothing else, it at least needs more error checking). The issue is that locale settings were not inherited by the postgres backends when they were execed... Instead, the locale was reset to whatever the system default was. I have a few questions before I continue fixing up the patch, though: * Is it enough to explicitly store/save LC_COLLATE and LC_CTYPE, or does more of the locale stuff need to be stored? I think the rest is handled by GUC and should thus be reloaded by the GUC routines already, but I'm not sure there. * Does pgstat, startup, shutdown, checkpointer and bgwriter need to have the locale set as well, or can they live with defaults? (my fix only fixes standard backends, since they have different exec paths) * Locale can't be changed on-the-fly, if I'm correct. So just setting it upon exec should be enough. Or am I wrong there? //Magnus locale_win32.patch Description: locale_win32.patch ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] 'begin transaction' new syntax bug?
Please ignore the 'bjm' debug code that was part of the patch. It has been removed from CVS. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] 'begin transaction' new syntax bug?
Fabien COELHO wrote:
#
# new syntax?
#
psql BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
BEGIN
psql SHOW TRANSACTION ISOLATION LEVEL;
transaction_isolation
---
read committed
OK, fixed:
test= BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
BEGIN
test= SHOW TRANSACTION ISOLATION LEVEL;
transaction_isolation
---
serializable
(1 row)
I added the ability to use such clauses to BEGIN in January for 7.5, but
I never tested it using SHOW. I thought the START TRANSACTION and BEGIN
code would handle the clause code the same, but BEGIN was ignoring it.
Patch attached and applied.
Thanks for the report.
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: src/backend/tcop/utility.c
===
RCS file: /cvsroot/pgsql-server/src/backend/tcop/utility.c,v
retrieving revision 1.211
diff -c -c -r1.211 utility.c
*** src/backend/tcop/utility.c 23 Mar 2004 19:35:17 - 1.211
--- src/backend/tcop/utility.c 19 Apr 2004 21:16:57 -
***
*** 302,316
switch (stmt-kind)
{
case TRANS_STMT_BEGIN:
- BeginTransactionBlock();
- break;
-
- /*
-* START TRANSACTION, as defined by
SQL99:
-* Identical to BEGIN, except that it
takes a few
-* additional options.
-*/
case TRANS_STMT_START:
{
BeginTransactionBlock();
--- 302,313
switch (stmt-kind)
{
+ /*
+* START TRANSACTION, as defined by SQL99:
+* Identical to BEGIN, except that it takes a
few
+* additional options. Same code for both.
+*/
case TRANS_STMT_BEGIN:
case TRANS_STMT_START:
{
BeginTransactionBlock();
Index: src/backend/utils/misc/guc.c
===
RCS file: /cvsroot/pgsql-server/src/backend/utils/misc/guc.c,v
retrieving revision 1.199
diff -c -c -r1.199 guc.c
*** src/backend/utils/misc/guc.c7 Apr 2004 18:52:26 - 1.199
--- src/backend/utils/misc/guc.c19 Apr 2004 21:17:01 -
***
*** 3394,3399
--- 3394,3401
SetPGVariable(const char *name, List *args, bool is_local)
{
char *argstring = flatten_set_variable_args(name, args);
+ printf(bjm: %s %s\n, name, argstring);
+ fflush(stdout);
/* Note SET DEFAULT (argstring == NULL) is equivalent to RESET */
set_config_option(name,
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [INTERFACES] [HACKERS] Why is libpgtcl still in CVS?
Bruce Momjian wrote: Bruce Momjian wrote: Why is libpgtcl still in /src/interfaces? Isn't it now on gborg? Are there two of them there now? I am still waiting for an answer on this. Does no one know? I replied right away - to you and the lists you posted to. Never got it? It's in the archive: http://archives.postgresql.org/pgsql-interfaces/2004-04/msg00020.php My question is whether they have all the CVS history and fixes in libpgtcl so we can remove it from CVS. I believe I have all the fixes, and I know I have none of the histories, in pgtcl-ng. I think both statements are also true about the Gborg pgtcl project. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [INTERFACES] [HACKERS] Why is libpgtcl still in CVS?
L J Bayuk wrote: Bruce Momjian wrote: Bruce Momjian wrote: Why is libpgtcl still in /src/interfaces? Isn't it now on gborg? Are there two of them there now? I am still waiting for an answer on this. Does no one know? I replied right away - to you and the lists you posted to. Never got it? It's in the archive: http://archives.postgresql.org/pgsql-interfaces/2004-04/msg00020.php Yes, I saw that. The issue is that the CVS history needs to be moved over into one of the gborg TCL project histories. Marc? My question is whether they have all the CVS history and fixes in libpgtcl so we can remove it from CVS. I believe I have all the fixes, and I know I have none of the histories, in pgtcl-ng. I think both statements are also true about the Gborg pgtcl project. Yea, we need to do that before removing from CVS. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] pg_encoding not needed anymore
With initdb written now in C, we don't need a pg_encoding binary anymore. I have removed it from CVS. There were no mentions of it in the docs. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [INTERFACES] [HACKERS] Why is libpgtcl still in CVS?
Bruce Momjian wrote: L J Bayuk wrote: Bruce Momjian wrote: Bruce Momjian wrote: Why is libpgtcl still in /src/interfaces? Isn't it now on gborg? Are there two of them there now? I am still waiting for an answer on this. Does no one know? I replied right away - to you and the lists you posted to. Never got it? It's in the archive: http://archives.postgresql.org/pgsql-interfaces/2004-04/msg00020.php Yes, I saw that. The issue is that the CVS history needs to be moved over into one of the gborg TCL project histories. Marc? My question is whether they have all the CVS history and fixes in libpgtcl so we can remove it from CVS. I believe I have all the fixes, and I know I have none of the histories, in pgtcl-ng. I think both statements are also true about the Gborg pgtcl project. Yea, we need to do that before removing from CVS. OK, Marc has removed libpgtcl. I just removed the docs for them. Can someone comment on pgtclsh? The README says: pgtclsh is an example of a tclsh extended with the new Tcl commands provided by the libpgtcl library. By using pgtclsh, one can write front-end applications to PostgreSQL in Tcl without having to deal with any libpq programming at all. The pgtclsh is an enhanced version of tclsh. Similarly, pgtksh is a wish replacement with PostgreSQL bindings. For details of the libpgtcl interface, please see the Programmer's Guide. Do we need to move that over to gborg? If so, you can pull it out of 7.4 CVS. Anyone? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] 'begin transaction' new syntax bug?
Christopher Kings-Lynne wrote: char *argstring = flatten_set_variable_args(name, args); + printf(bjm: %s %s\n, name, argstring); + fflush(stdout); Did you really mean to include that? :) Nope, posted a followup that it was removed in a later commit. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] 'begin transaction' new syntax bug?
char *argstring = flatten_set_variable_args(name, args); + printf(bjm: %s %s\n, name, argstring); + fflush(stdout); Did you really mean to include that? :) Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] locale issues on win32 (fix?)
Magnus Hagander [EMAIL PROTECTED] writes: * Is it enough to explicitly store/save LC_COLLATE and LC_CTYPE, or does more of the locale stuff need to be stored? The other LC_xxx settings will get fixed by GUC *only* if there are explicit settings in postgresql.conf ... I don't think you can assume that. * Does pgstat, startup, shutdown, checkpointer and bgwriter need to have the locale set as well, or can they live with defaults? You need to fix at least LC_MESSAGES, surely, if you want errors from these processes suitably localized. Although in theory none of these guys do any index operations and so should not need LC_COLLATE set, it scares me to think of what might go wrong; probably best to fix 'em all. Note that I am about to whack around the checkpointer and bgwriter, so you might want to wait a day or two before you try to do this. (my fix only fixes standard backends, since they have different exec paths) If those guys do not go through that variable-saving-and-restoring routine, they are probably broken ipso facto. You sure they don't? * Locale can't be changed on-the-fly, if I'm correct. So just setting it upon exec should be enough. Or am I wrong there? Sounds sufficient to me. All you are doing is implementing a substitute for the Unix copy-on-fork behavior. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] Position available at the Telegraph project
Dear Hackers I apologize in advance if this posting is construed as spam. As many of you are aware, the database research group at UC Berkeley is building TelegraphCQ, a system for processing continuous queries over data streams. TelegraphCQ was built with the PostgreSQL source base. We are looking for a full-time programmer to work with our group. We believe that PostgreSQL hackers will be great candidates for such a position. Appended to this email is a full description of the job requirements. I encourage any hackers interested to apply for the position. To apply for the position please do the following: - Point your web browser to http://jobs.berkeley.edu - Click on External applicants - Click on View job posting / apply for jobs - Check the Information Technology box, Choose Regular from Regular/Temporary, fill in telegraph in the Keyword box and hit the search button. - Click on Programmer/Analyst IV-Ucb, job #001090 Please note: YOU HAVE TO APPLY FOR THE JOB THROUGH THE WEB SITE .. Job Description Posting Title: Programmer/Analyst IV-Ucb Requisition: 001090 Department: Electronics Research Lab Location: Main Campus-Berkeley First Review Date: 03/29/2004 This requisition will remain open until filled. Job Description: The focus of the Telegraph research group is on developing a general-purpose software infastructure for the dataflow applications. Dataflow applications are prevalent in networked environments where information from multiple sources must be flowed through various networked machines and software services for integration and analysis. The unpredictable nature of decoupled, networked machines and services requires novel, very adaptive software infrastructures: the design and study of such infastructure is the goal of the Telegraph project. The position is in support of the Telegraph project, with a focus on integration with two other software projects currently underway in the UC Berkeley database group: TinyDB, a query processing system that runs on ad hoc networks of wireless sensor motes, and YFilter, a distributed XML filtering and brokering system. This position will represent the chief programmer for the Telegraph project, with a focus on overall system architecture, documentation, open-source software delivery, and demonstration development and deployment. This work will be done in collaboration with the students and faculty developing the system. The success of the work will be best measured by the ability of our group to deploy interesting functionality with Telegraph, and for others outside of our group to successfully exploit our open source software in the development of applications in a wide range of scenarios. We want to make Telegraph the system of choice for research groups that are experimenting with data streaming and advanced dataflow applications. This project involves a large amount of existing software developed in the three projects (Telegraph, TinyDB, and YFilter), probably totaling several hundred thousand lines of code. The person holding the position will need to understand database management internals, XML processing and must be able to interface such systems with external devices such as sensor networks. Responsibilities: Design, implement, test, deliver, release, and maintain the core Telegraph system as a viable open source dataflow infastructure. This position will make all client-side design decisions: i.e. they will decide on user interface design, client-server APIs, and client and application design (there are a number of client applications as yet undeveloped which will be the full responsibility of the programmer.) The position will collaborate with students and faculty on server-side design decisions. The position will also be fully responsible for all public code release and installer issues, for documentation, and for testing and other quality assurance of both server and client code. Design, implement, test, deliver and maintain a number of demonstration applications over Telegraph- including conception, execution, and deployment at venues such as international research conferences, departmental meetings, CITRIS functions, etc. Coordinate with students and faculty to integrate the core Telegraph code with ongoing research extensions thereof, including participation in research meetings of the Telegraph group. Outreach activities to promote the adoption of Telegraph for future application, including assistance to selected users as well as general tutorials and presentations. Requirements Qualifications: The position requires a broad-based skill and experience level with the ability to quickly grasp the intricacies of complex software systems. Position
Re: [INTERFACES] [HACKERS] Why is libpgtcl still in CVS?
Bruce Momjian wrote: ... OK, Marc has removed libpgtcl. I just removed the docs for them. Can someone comment on pgtclsh? The README says: ... Do we need to move that over to gborg? If so, you can pull it out of 7.4 CVS. Anyone? Already discussed: http://archives.postgresql.org/pgsql-interfaces/2004-02/msg00076.php and http://archives.postgresql.org/pgsql-interfaces/2004-02/msg00078.php ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [INTERFACES] [HACKERS] Why is libpgtcl still in CVS?
L J Bayuk wrote: Bruce Momjian wrote: ... OK, Marc has removed libpgtcl. I just removed the docs for them. Can someone comment on pgtclsh? The README says: ... Do we need to move that over to gborg? If so, you can pull it out of 7.4 CVS. Anyone? Already discussed: http://archives.postgresql.org/pgsql-interfaces/2004-02/msg00076.php and http://archives.postgresql.org/pgsql-interfaces/2004-02/msg00078.php Great. Thanks you. I heard it wasn't needed, but couldn't remember if that was current information or something old. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [BUGS] Bug in pg_autovacuum ?
Not sure if anyone reported to you but we fixed these in current CVS and will have the fix in 7.4.3. --- Cott Lang wrote: On Wed, 2004-02-11 at 16:25, Bruce Momjian wrote: Sure, shoot them over to hackers or patches. The pg_autovacuum author is looking into this. Here they are. They've worked well for me, but someone wiser in the ways of C should certainly look them over. :) [ Attachment, skipping... ] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Postgresql snapshots broken?
should be fixed now ... let me know if there are any other problems ... On Mon, 19 Apr 2004, Magnus Hagander wrote: The snapshots in ftp.postgresql.org/pub/dev/ (and mirrors) appears not to have updated since Apr 5th. These are supposed to be generated nightly, right? If so, they seem broken. //Magnus Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
