Re: [HACKERS] postgreSQL-8.0.1 compilation with icc-8.1 on Itanium-2 gives error: asm statements not supported
Tom Lane wrote: #if defined(__GNUC__) || defined(__ICC) Can anyone say a reason why the above #if is not wrong ... ie, are there any platforms where icc does handle gcc asm syntax, and if so exactly which ones are they? I believe I added that a few releases ago. The platform is IA32. Evidently, the GCC compatibility on IA64 is not quite as far yet. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Novice Slony User (Was [HACKERS] hi all)
First things first: try posting to the Slony mailing list: [EMAIL PROTECTED] -tfo -- Thomas F. O'Connell Co-Founder, Information Architect Sitening, LLC http://www.sitening.com/ 110 30th Avenue North, Suite 6 Nashville, TN 37203-6320 615-260-0005 On Mar 2, 2005, at 7:23 PM, Qu Tianlian wrote: Hi all: I have a question. How to add table in slony. I try to add table in already being database that using slony . but it's not realize table's replication. I used postgresql version 7.4.2 and slony version 1.0 Can you help me . Thanks Yours, Qu TianLian ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] bitmap AM design
Ühel kenal päeval (teisipäev, 1. märts 2005, 14:54-0500), kirjutas [EMAIL PROTECTED]: Now, it occurs to me that if my document reference table can refer to something other than an indexed primary key, I can save a lot of index processing time in PostgreSQL if I can have a safe analogy to CTID. I guess you could work on making hash indexes better (for concurrent access). 'a safe analogy to CTID' looks remarkably like hash index -- Hannu Krosing [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] cluster table by two-column index ?
I'm wondering, is there any sense to cluster table using two-column index ? Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] consequent btree index scans optimizations ?
Hi there, I'm trying to understand if I have something to optimize in my query which is basically looks like a bunch of many intervals: (ipix = 341288409261670400 AND ipix 341358778005848064) OR (ipix = 341710621726736384 AND ipix 341728213912780800) OR (ipix = 341728213912780800 AND ipix 341745806098825216) OR (ipix = 340531945261760512 AND ipix 340549537447804928) OR (ipix = 340567129633849344 AND ipix 340584721819893760) ... Table is rather big ( 500 mln rows) and clustered by btree index on ipix. Generally I'm quite satisfied with performance, but I'm wondering if optimizer take order of intervals into account ? Looking into pg_stat_user_tables I see there were 168 index scans which is exactly the number of intervals and 98530 tuples fetched. Since table is clustered hit ratio is very good and execution time is 0.5s. Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] 8.0.X and the ARC patent
Dave Cramer [EMAIL PROTECTED] writes: I was just looking at the config parameters, and you have the shared buffers set to 60k, and the effective cache set to 1k I was actually going to suggest that the performance degradation might be because of an excessively high shared_buffers setting. That was before I saw this comment. The only reason I could imagine the performance degradation would be because more and more CPU time is being spent traversing the 2Q LRU buffer lists. I would try it with a shared buffer setting of 10k to see if it levels out sooner at a higher TPM. I would also suggest setting checkpoint_timeout to something more realistic. All your 60m tests that show a single checkpoint in the middle are being deceptive since half the data in the test hasn't even been checkpointed. You should have enough checkpoints in your test that they're represented in the results realistically. If you want 60m to be a reasonably representative sample then I would suggest a checkpoint_timeout of 300-600 (ie, checkpoints every 5-10m) so you get 10-20 checkpoints in the result. And so that a maximum of 5-10% of the data isn't being checkpointed in the test. That would also make those huge performance dropouts a little less dramatic. And it might give us a chance to see how effective the bgwriter is at smoothing them out. Personally, as a user, I think it's more important to look at the maximum transaction latency than the average throughput. -- greg ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] cluster table by two-column index ?
I'm wondering, is there any sense to cluster table using two-column index ? We've had this discussion a few weeks ago. Look at the archives for my post One Big Trend The problem is that while the statistics can resonably deal with the primary column it completely misses the trends produced in the secondary column. This situation can be seen quite clearly using the US Census TIGER database. I imagine the primary and secondary columns both have a discrete index and the combined index is for the cluser or more complex queries. If you execute a query based on the secondary column's index that should return about 100 rows. The smaller trends in the column produced by the cluster are not detected. So, rather then seeing that its probably a few index seeks and a few table seeks because the data is fairly well grouped, it opts, instead, to do a table scan because it doesn't see any correlation. Increasing the number of samples in ANALIZE helps a bit, but the solution is better statistics or maybe hints that can be embedded into the query. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] bitmap AM design
Ühel kenal päeval (teisipäev, 1. märts 2005, 14:54-0500), kirjutas [EMAIL PROTECTED]: Now, it occurs to me that if my document reference table can refer to something other than an indexed primary key, I can save a lot of index processing time in PostgreSQL if I can have a safe analogy to CTID. I guess you could work on making hash indexes better (for concurrent access). 'a safe analogy to CTID' looks remarkably like hash index Yes, I agree, but I don't particularly like linear hash models without the ability to adjust the initial table size estimates. Also, hash tables without access to the hash function typically have a lot of collision, specifically, I am dubious of generic hash functions having an optimally dispersed behavior. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] plperl function has side-effects
Hi,
I have a 7.4.3 installation where a small plperl function seems to have
side-effects. In the example below I run an ordinary SELECT first, nothing
special with the table. Thereafter I call the plperl function and then I
rerun the SELECT query. This time it doesn't return the expected result. The
problem seems to show up only after postgres has run for quite a while.
Futhermore this effect is limited to the current session. I'll upgrade the
machine to 7.4.7 during the weekend, however I haven't seen anything in the
release notes that seems to matter, does anybody know this effect or is this
issue even already solved?
(Crypt::PasswdMD5 1.3, perl 5.8.4)
db= select login from subaccounts where login='web1p1';
login
web1p1
(1 row)
db= select md5password('bla1', 'Ao2ZaGKp');
md5password
$1$Ao2ZaGKp$XBDNeuZM3RSrqq9gruKXH1
(1 row)
db= select login from subaccounts where login='web1p1';
login
---
(0 rows)
The definition of md5password:
CREATE FUNCTION md5password(varchar(20), varchar(50)) RETURNS varchar(50)
SECURITY DEFINER AS '
use Crypt::PasswdMD5;
my $password = $_[0];
my $salt = $_[1];
my $crypted = unix_md5_crypt($password, $salt);
return $crypted;
' LANGUAGE 'plperlu';
Thanks,
Joachim
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] 8.0.X and the ARC patent
Josh Berkus josh@agliodbs.com writes: I should be able to run more OLTP benchmarks, and a DSS benchmark, within the next week. Please wait until I complete those before considering an 8.0.2 release with the new code. Sure, there's no hurry to push out 8.0.2 (and we need to have some beta testing done on it anyway). I have committed the proposed patch into the REL8_0_STABLE branch, so you can just pull the branch tip from CVS to do testing. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] logging as inserts
Josh, I am looking at having one of our guys write up the code to allow logging as insert statements. I have a couple of questions. What would we like the postgresql.conf option to be? I was thinking log_statements_as_inserts = (t/f) Nope. log_destination = 'inserts' #not a new GUC! insert_columns = '%u,%d,%r,%p, ... %$' #this new GUC would define a list of comma-seperated columns as escape codes defined via the same code set as log_line_prefix. The only change would be the addition of %$, which would symbolize the statement being logged. I'd also assert that this option should log the inserts to a stderr and thus take advantage of all of the redirection, rotation, etc that we now support for stderr logging. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] logging as inserts
Josh Berkus josh@agliodbs.com writes: What would we like the postgresql.conf option to be? I was thinking log_statements_as_inserts = (t/f) Nope. log_destination = 'inserts' #not a new GUC! That seems a bit bizarre to me. The facility isn't a new log destination; what it is is a different way of formatting what's sent to the log. insert_columns = '%u,%d,%r,%p, ... %$' #this new GUC would define a list of comma-seperated columns as escape codes defined via the same code set as log_line_prefix. The only change would be the addition of %$, which would symbolize the statement being logged. I think what you'd probably really want to write is something like log_message_format = 'INSERT INTO mytab VALUES(%this, %that, ...);' the point being that otherwise we'll need yet another GUC var to determine the table name used in the INSERT. With a suitable set of escape codes we could probably arrange for the existing behavior to correspond to a particular value of log_message_format, and then there isn't anything weird going on here; you are just changing away from a default format. I wonder whether this could be defined in a way that lets it replace log_line_prefix ... otherwise we have to think about the interaction of the two facilities. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] unexpected and reproducable crash in pl/pgsql function
Ok, I have a fairly nasty situation. I am having a production server that is crashing upon execution of a pl/pgsql function...on code that has been working flawlessly for weeks. My production server is running 8.0 on win32 and I was able to 'sort-of' reproduce the behavior on my development machine here at the office. What happens: On the production machine, upon execution of the function (with a very specific parameter value, all others work ok), all server backends freeze and completely stop working. Any attempt to connect to the server hangs psql in limbo. In addition, the service fails to shut down, and the only way to get working again is to kill postmaster.exe and all instances of postgres.exe. However after that everything runs o.k. until I try to run the function again. There is nothing useful in the log. Following this, I did a dump of the production database and loaded it into my office machine. Here, I try and execute the function and I get: esp=# select generate_oe_bom(18208); server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. The connection to the server was lost. Attempting reset: Succeeded. And server seems to recover from this. Looking at the event log, I see: NOTICE: hello LOG: server process (PID 5720) exited with unexpected status 128 LOG: terminating any other active server processes WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. LOG: all server processes terminated; reinitializing LOG: database system was interrupted at 2005-03-03 14:16:15 Eastern Standard Time LOG: checkpoint record is at 6/D7546E28 LOG: redo record is at 6/D7546E28; undo record is at 0/0; shutdown TRUE LOG: next transaction ID: 11208897; next OID: 62532404 LOG: database system was not properly shut down; automatic recovery in progress LOG: redo starts at 6/D7546E68 LOG: unexpected pageaddr 6/CF5BA000 in log file 6, segment 215, offset 6004736 LOG: redo done at 6/D75B7F90 LOG: database system is ready This will repeat if the function is run again. Only the exact parameter (order# 18208) will cause the crash. Another order, 18150, runs through ok. I would expect data corrumption to be the cause of the problem except I was able to reproduce the problem on a different server following a dump/restore. Unfortunately, this is sensitive data. Attached is the pl/pgsql code. There is a raise notice 'hello'. This gets raised exactly once before the crash. Merlin oebom.sql Description: oebom.sql ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] logging as inserts
Tom, That seems a bit bizarre to me. The facility isn't a new log destination; what it is is a different way of formatting what's sent to the log. It's not, but it functions like one. And ultimately, the destination *is* someplace different; likely the DBA will be piping the log output to another database somewhere. log_message_format = 'INSERT INTO mytab VALUES(%this, %that, ...);' Yeah, good idea. I wonder whether this could be defined in a way that lets it replace log_line_prefix ... otherwise we have to think about the interaction of the two facilities. Well, that's why I like the idea of using log_destination. It makes it clear that log_line_prefix doesn't work if log_destination 'stderr'. -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] unexpected and reproducable crash in pl/pgsql function
I wrote: Ok, I have a fairly nasty situation. I am having a production server that is crashing upon execution of a pl/pgsql function...on code that has been working flawlessly for weeks. My production server is running 8.0 on win32 and I was able to 'sort-of' reproduce the behavior on my development machine here at the office. Ok, problem was due to recursive pl/pgsql function and a recursion loop in the data. I traced this problem to the data: somebody disabled the recursion check constraint. I've never had this actually happen before. It totally nuked the server. Merlin ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] logging as inserts
Josh Berkus josh@agliodbs.com writes: I wonder whether this could be defined in a way that lets it replace log_line_prefix ... otherwise we have to think about the interaction of the two facilities. Well, that's why I like the idea of using log_destination. It makes it clear that log_line_prefix doesn't work if log_destination 'stderr'. But log_line_prefix works fine for all destinations, which is exactly why this new facility isn't a destination. You're just confusing matters by wanting to treat it as one. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] unexpected and reproducable crash in pl/pgsql function
Merlin Moncure [EMAIL PROTECTED] writes: Ok, problem was due to recursive pl/pgsql function and a recursion loop in the data. I traced this problem to the data: somebody disabled the recursion check constraint. I've never had this actually happen before. It totally nuked the server. I thought we'd fixed things so that the stack depth on Windows is actually greater than max_stack_depth? None of this weirdness could happen if the stack depth check were kicking in properly. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] unexpected and reproducable crash in pl/pgsql function
Merlin Moncure [EMAIL PROTECTED] writes: Ok, problem was due to recursive pl/pgsql function and a recursion loop in the data. I traced this problem to the data: somebody disabled the recursion check constraint. I've never had this actually happen before. It totally nuked the server. I thought we'd fixed things so that the stack depth on Windows is actually greater than max_stack_depth? None of this weirdness could happen if the stack depth check were kicking in properly. I thought so too. I'll play with it a bit and see what I come up with. Merlin ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Solving hash table overrun problems
We saw a case recently where a hash join was using much more memory than it was supposed to, causing failure when the server ran out of memory. The hash join code is supposed to spill tuples to disk when the hashtable exceeds work_mem, but this failed to save us because the algorithm is not adaptive. What it really does is to divide the hash key space into N batches where N is chosen at query startup based on the planner's estimate of the number of rows to be processed. If that estimate is way too small then an individual batch can be way too large, but the code can't recover by adjusting N after the fact. A somewhat related problem is that the HashAgg code doesn't have a way to spill hashtable entries to disk at all, so it too can blow out memory if the planner's estimate of the number of entries is way off. We've seen reports of that happening, too. Here's what I'm thinking of doing to fix it: * Determine the number of in-memory hash buckets, K, at plan startup. This is dependent on work_mem more than it is on the planner's estimates, so there's no need for it to be adaptive. A tuple with hash value H will go into bucket (H mod K) when it is processed. * Estimate the number of batches N using the planner's estimate. We will always choose N a power of 2. A tuple's batch number is ((H div K) mod N). * Begin loading the hash table from the inner input. Tuples of batch zero go into the hash table, tuples of higher batch numbers go into holding files, one per batch. * If the hash table size exceeds work_mem, double N (creating a bunch of new empty holding files). Scan through the hash table for tuples whose batch number is no longer zero according to the new calculation, and dump them out to the appropriate one of the new holding files. This should get rid of about half of the hash table entries if the hash values are well dispersed. Essentially, we are looking at one more bit of the hash value than we were using before. * Lather, rinse, repeat until inner join input is completely read. * Now begin scanning the outer join input. Tuples of batch number zero (according to the current calculation) can be matched to the current hashtable contents. Tuples of higher batch numbers are dropped into holding files for the outer input, one per batch. * After exhausting the outer input, we still have to match up tuples of corresponding batches. To do this, we clear the in-memory hash table and load it from tuples in the first unprocessed inner batch file. If we had to increase N on-the-fly then it is possible that some of these tuples no longer belong to batch 1, but to some higher batch number --- write such tuples to the proper batch file instead of putting them into the hash table. * If some batches are more heavily populated than others, it is possible that we exceed work_mem here. No problem: we can play the same game of increasing N even at this stage. This works because increasing N can only cause tuples to be reassigned to later batches, never to earlier ones. (Of course, each on-the-fly increase in N means extra writes and reads of tuples that were initially put in the wrong batch, so it's still best to get as good an estimate as we can to start with.) * While reading from an outer batch file, we have to check whether each tuple is still considered to belong to the current batch, and dump it out to the proper later batch file if not. We can use basically the same ideas to fix HashAgg. Here, the aggregate state values play the part of the inner join tuples, and the incoming data to be aggregated plays the part of the outer join tuples. When the hash table gets too big, we double the number of batches and dump currently accumulated aggregate states into a per-batch holding file. Incoming data that hashes into the current batch can be accumulated into its aggregate value and discarded. Incoming data that hashes into a later batch is put into a to-do file. After we scan all the input, we emit the current aggregate states, load up the hash table from the next batch holding file, and then scan the current to-do file for inputs of the new batch. Note that we'll use only one to-do file in each pass, not one per batch. This implies more I/O but it is the only way to preserve the guarantee that incoming values are accumulated into their aggregate in order of arrival. The standard aggregates don't care about that, but user-defined aggregate functions often do. Comments? regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] logging as inserts
Tom, But log_line_prefix works fine for all destinations, which is exactly why this new facility isn't a destination. You're just confusing matters by wanting to treat it as one. Hmmm ... hey, if we just allowed extra text in log_line_prefix, and allowed %$ to enclose the statement with literal quoting, you could do this all through log_line_prefix, as: log_line_prefix='INSERT INTO pg_log VALUES ( %u, %p, ''%t'', ''%$'')' -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] logging as inserts
Josh Berkus wrote: Tom, But log_line_prefix works fine for all destinations, which is exactly why this new facility isn't a destination. You're just confusing matters by wanting to treat it as one. Hmmm ... hey, if we just allowed extra text in log_line_prefix, and allowed %$ to enclose the statement with literal quoting, you could do this all through log_line_prefix, as: log_line_prefix='INSERT INTO pg_log VALUES ( %u, %p, ''%t'', ''%$'')' You can have extra text today, apart from the fact that there's no escape for the statement. try it and see. log_line_prefix is (not coincidentally) very similar to a printf-type format string. Indeed, use of admin-supplied fixed text was always intended - see the discussions that led up to it. Incidentally, %$ doesn't tstrike me as ideal - to conform to the way the rest are done we should use an alphabetic character, not $. cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] logging as inserts
Andrew, Incidentally, the fly in this particular pot of ointment is that we potentially log a lot more than just statements. Oh, yeah, but just about anything can be put in the statement field; errors, disconnects, etc. Hmmm ... though we don't currently apply log line prefix to those, do we? Incidentally, %$ doesn't tstrike me as ideal - to conform to the way the rest are done we should use an alphabetic character, not $. Sorry, I'm being perlish ;-) -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Where to see the patch queue (was Re: [PATCHES] Patch
Thomas F.O'Connell wrote: I have a feeling Bruce was referring to item 1.4: http://developer.postgresql.org/readtext.php?src/FAQ/ FAQ_DEV.html+Developers-FAQ#1.4 It has never been standard practice to ask for comments before the development of small features, such as this one. The recently duplicated work on allowing multiple -t and -n options in pg_dump is another example (although that is complex enough an RFC might be worth doing). Anyway, I think this is missing the point. Checking has this work already been done and is sitting in some patch queue somewhere? before doing anything presupposes the existence of a sizeable queue of unapplied patches that is hard to find. I don't think either of those should be true. I think Matthias' comment is well-founded: the committers, myself included, deserve some blame for not making more rapid progress on the queue of unapplied patches for 8.1. In the meanwhile, the queue should be easier for folks to find (why is the pgpatches queue the only one linked from postgresql.org, but it is almost empty?) -Neil ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] : Novice Slony User (Was [HACKERS] hi all)
Thanks I try it. | -- | : [EMAIL PROTECTED] | [mailto:[EMAIL PROTECTED] Thomas F.O'Connell | : 200533 20:38 | : Qu Tianlian | : PostgreSQL-development; Slony Mailing List | : Novice Slony User (Was [HACKERS] hi all) | | First things first: try posting to the Slony mailing list: | | [EMAIL PROTECTED] | | -tfo | | -- | Thomas F. O'Connell | Co-Founder, Information Architect | Sitening, LLC | http://www.sitening.com/ | 110 30th Avenue North, Suite 6 | Nashville, TN 37203-6320 | 615-260-0005 | | On Mar 2, 2005, at 7:23 PM, Qu Tianlian wrote: | | Hi all: | I have a question. | How to add table in slony. | I try to add table in already being database that using slony . but | it's not | realize table's replication. | I used postgresql version 7.4.2 and slony version 1.0 | | Can you help me . Thanks | | | Yours, | Qu TianLian | | | ---(end of broadcast)--- | TIP 8: explain analyze is your friend ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] refactoring fork() and EXEC_BACKEND
While going through the usual motions needed to fork a child process of
the postmaster, it occurred to me that there's a fair bit of duplicated
code involved. There are also #ifdef for various situations (BeOS,
LINUX_PROFILE, and EXEC_BACKEND), which makes the code yet more ugly. I
think we could make this a lot cleaner.
I'd like to define an API like so:
pid_t fork_process(int proc_type);
pid_t fork_backend(Port *port);
If the process needs to add a lot of private information to the argv in
the case of EXEC_BACKEND, they could invoke a third variant:
#ifdef EXEC_BACKEND
pid_t forkexec_process(int proc_type, int argc, char **argv);
#endif
(Or possibly using varargs, if that is cleaner for most call-sites).
Hopefully most call sites could just use fork_process().
These functions would then take care of all the necessary
platform-specific judo:
- flush stdout, stderr
- invoke BeOS hooks as necessary
- save and restore profiling timer, if necessary
- if EXEC_BACKEND, use proc_type to lay out the argv for the new process
and then invoke internal_forkexec()
- otherwise, just invoke fork()
- return result to client
So, most call sites would be quite nice:
pid_t result = fork_process(PROC_TYPE_FOO);
if (result == -1) { /* fork failed, in parent */ }
else if (result == 0) { /* in child */ }
else { /* in parent, `result' is pid of child */ }
I'd also like to move the implementation of fork_process() and friends,
as well as internal_forkexec(), into a separate file -- I'd rather not
clutter up postmaster.c with it.
Comments?
-Neil
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [HACKERS] postgreSQL-8.0.1 compilation with icc-8.1 on Itanium-2 gives error: asm statements not supported
Tom, Peter,
I have been able to compile and sucessfully run pgSQL after replacing
the asm statement in postgresql-8.0.1/src/include/storage/s_lock.h
with an equivalent intrinsic for the Itanium platform-
--BEGIN OLD
s_lock.h--
#if defined(__ia64__) || defined(__ia64) /* __ia64 used by ICC compiler? */
#define HAS_TEST_AND_SET
typedef unsigned int slock_t;
#define TAS(lock) tas(lock)
static __inline__ int
tas(volatile slock_t *lock)
{
long intret;
__asm__ __volatile__(
xchg4 %0=%1,%2\n
: =r(ret), +m(*lock)
: r(1)
: memory);
return (int) ret;
}
#endif /* __ia64__ || __ia64 */
---END OLD
s_lock.h--
--BEGIN NEW
s_lock.h--
#if defined(__ia64__) || defined(__ia64) /* __ia64 used by ICC compiler? */
#define HAS_TEST_AND_SET
typedef unsigned int slock_t;
#define TAS(lock) tas(lock)
static __inline__ int
tas(volatile slock_t *lock)
{
int ret;
ret = _InterlockedExchange(lock,1);
return ret;
}
#endif /* __ia64__ || __ia64 */
--END NEW
s_lock.h--
The binary appears to be stable and the tpc-H benchmark executed
successfully against it as well. I also ran the regression test but
the following tests failed, the reasons for which I haven't
investigated yet
(http://www.cse.psu.edu/~kalsi/files/regression.diffs)-
test create_function_1... FAILED
test create_type ... FAILED
test create_table ... FAILED
test create_function_2... FAILED
test triggers ... FAILED
test create_operator ... FAILED
test create_view ... FAILED
test transactions ... FAILED
test misc ... FAILED
test select_views ... FAILED
test rules... FAILED
test plpgsql ... failed (ignored)
test copy2... FAILED
test rangefuncs ... FAILED
test conversion ... FAILED
test stats... FAILED
The _InterlockedExchange() function is defined in ia64intrin.h header file
int _InterlockedExchange(volatile int *Target, long value)
Do an exchange operation atomically. Maps to the xchg4 instruction.
More information is available at
http://www.intel.com/software/products/compilers/clin/docs/ug_cpp/lin1072.htm
Also, some other points to note, _ICC wasn't defined on my
installation when I was using icc by setting env var CC=icc. So, when
I tried to put a #if defined for using asm() for gcc and
_InterlockedExchange(), it didn't work. So, after this change gcc
compilation fails.
As of now, I am trying to test the binary further to see if it is
stable. Would you be knowing some good way to test this change?
I am not aware of the procedure of building patches but if this
resolves this issue and you would like me to make some sort of a
patch, then please let me know.
Thanks,
-Vikram
On Thu, 3 Mar 2005 09:55:18 +0100, Peter Eisentraut [EMAIL PROTECTED] wrote:
Tom Lane wrote:
#if defined(__GNUC__) || defined(__ICC)
Can anyone say a reason why the above #if is not wrong ... ie,
are there any platforms where icc does handle gcc asm syntax,
and if so exactly which ones are they?
I believe I added that a few releases ago. The platform is IA32.
Evidently, the GCC compatibility on IA64 is not quite as far yet.
--
Peter Eisentraut
http://developer.postgresql.org/~petere/
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] postgreSQL-8.0.1 configure --enable-thread-safety with icc-8.1 on RHEL-AS3 Itanium-2 gives error
Hi, I am trying to build postgresql-8.0.1 with icc-8.1.028 on a Linux RHEL AS3 SMP Itanium2 machine and I get an error as follows when I run configure --enable-thread-safety as follows- shellexport CC=icc shellexport CFLAGS=-static -fPIC shellexport LDFLAGS=-L/opt/intel_cc_80/lib shellexport CPPFLAGS=-I/opt/intel_cc_80/include shellconfigure --prefix=$MY_HOME/dbms/pgsql --enable-thread-safety --disable-shared --with-low-memory --with-pgport=5410 .. .. .. configure:18836: icc -o conftest -static -fPIC -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wold-style-definition -Wendif-labels -fno-strict-aliasing -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -DIN_CONFIGURE -D_GNU_SOURCE -L/opt/intel_cc_80/lib conftest.c -lz -lreadline -ltermcap -lcrypt -lresolv -lnsl -ldl -lm -lbsd 5 ./src/tools/thread/thread_test.c(75): remark #1418: external definition with no prior declaration char *temp_filename_1; ^ ./src/tools/thread/thread_test.c(76): remark #1418: external definition with no prior declaration char *temp_filename_2; ^ ./src/tools/thread/thread_test.c(78): remark #1418: external definition with no prior declaration pthread_mutex_t init_mutex = PTHREAD_MUTEX_INITIALIZER; ^ ./src/tools/thread/thread_test.c(80): remark #1418: external definition with no prior declaration volatile int thread1_done = 0; ^ ./src/tools/thread/thread_test.c(81): remark #1418: external definition with no prior declaration volatile int thread2_done = 0; ^ ./src/tools/thread/thread_test.c(83): remark #1418: external definition with no prior declaration volatile int errno1_set = 0; ^ ./src/tools/thread/thread_test.c(84): remark #1418: external definition with no prior declaration volatile int errno2_set = 0; ^ ./src/tools/thread/thread_test.c(105): remark #1418: external definition with no prior declaration bool platform_is_threadsafe = true; ^ /tmp/iccQ3B36U.o(.text+0x1d2): In function `main': : undefined reference to `pthread_mutex_lock' /tmp/iccQ3B36U.o(.text+0x202): In function `main': : undefined reference to `pthread_create' /tmp/iccQ3B36U.o(.text+0x232): In function `main': : undefined reference to `pthread_create' /tmp/iccQ3B36U.o(.text+0x2e2): In function `main': : undefined reference to `pthread_mutex_unlock' /tmp/iccQ3B36U.o(.text+0x302): In function `main': : undefined reference to `pthread_join' /tmp/iccQ3B36U.o(.text+0x322): In function `main': : undefined reference to `pthread_join' /tmp/iccQ3B36U.o(.text+0x602): In function `func_call_1': : undefined reference to `pthread_mutex_lock' /tmp/iccQ3B36U.o(.text+0x612): In function `func_call_1': : undefined reference to `pthread_mutex_unlock' /tmp/iccQ3B36U.o(.text+0x872): In function `func_call_2': : undefined reference to `pthread_mutex_lock' /tmp/iccQ3B36U.o(.text+0x882): In function `func_call_2': : undefined reference to `pthread_mutex_unlock' configure:18839: $? = 1 configure: program exited with status 1 configure: failed program was: #line 18830 configure #include confdefs.h #include ./src/tools/thread/thread_test.c configure:18853: result: no configure:18863: error: *** Thread test program failed. Your platform is not thread-safe. *** Check the file 'config.log'for the exact reason. *** *** You can use the configure option --enable-thread-safety-force *** to force threads to be enabled. However, you must then run *** the program in src/tools/thread and add locking function calls *** to your applications to guarantee thread safety. The complete log is online at http://www.cse.psu.edu/~kalsi/files2/config.log The same works when I use gcc(3.2.3) and configure also works with icc-8.1 if I dont use --enable-thread-safety! Can anybody see if I am doing it wrong? Any suggestions for resolving this error? Thanks, -Vikram ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Solving hash table overrun problems
We saw a case recently where a hash join was using much more memory than it was supposed to, causing failure when the server ran out of memory. Yes. I had the same problem a few month ago, http://archives.postgresql.org/pgsql-general/2004-09/msg00410.php It turned out that the cost estimates were so way off no matter what tunables were modified, so I never was ever able to execute the query fully. I analyzed the code and devised a solution that was similar what you proposed, though I didn't consider HashAggregates at the time. Unfortunately, I lost all work in a hard drive failure and was never able to get back to working on it, so I can't really refer to my old notes. For what it's worth, your solution looks very reasonable to me. This also brings up a line of thought I had a while ago on a related topic. Something like a HashDistinct might be useful, if it had no startup cost. It would basically be a plan node in the executor that would dynamically build a hashtable so that it can pull rows from its child node (discarding if they appear in the hashtable) until it can pass on a novel row. I have some reservations about it, though. At best, in queries with minimal startup cost from the get-go, it would seem to be a tradeoff favoring latency over throughput (assuming the HashDistinct would be a slower operation overall than separate aggregation and distinct operations). Then we have the issue of really big hash tables... I was hoping to get some time in the upcoming months to hash out these issues to see if it's worth it, and if it would be generally useful at all. -Aaron ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
