[HACKERS] Invitation to connect on LinkedIn
LinkedIn I'd like to add you to my professional network on LinkedIn. - somasekhar somasekhar Ramadurgam Student at Indian Institute of Technology, Bombay Mumbai Area, India Confirm that you know somasekhar Ramadurgam https://www.linkedin.com/e/isd/1231429279/vRUTcgAL/ -- (c) 2010, LinkedIn Corporation
Re: [HACKERS] solaris sparc 64bit binary release
Bjorn Munch wrote: I will have a look into it. as of right now, I suspect what I need are the following files from a recent Solaris 64bit build... include/server/pg_config.h lib/64/pgxs/src/Makefile.global there are a few more files that could be involved, but AFAIK, they are invariant or don't matter to my plugin builds -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
On Fri, 2010-04-16 at 11:10 -0400, Tom Lane wrote: > Simon Riggs writes: > > On Fri, 2010-04-16 at 10:39 -0400, Tom Lane wrote: > >> I think you're outsmarting yourself there. A binary search will in fact > >> *not work* with circular xid comparison (this is exactly why there's no > >> btree opclass for XID). > > > I don't understand the exact, please explain more. > > I'm not using bsearch() just a quick chop based upon xid comparison, > > which looks to me like it will work. > > Implementing it yourself doesn't get you out of the fact that it won't > work. Consider > > 1 2 3 ... 30 ... 39 > > and suppose that 30 is the array's middle element. If you > search for 100, your first probe will conclude that it is to the right > of 30, which is the wrong direction. Binary search, or indeed > the entire concept that the array is "sorted" in the first place, > depends on a transitive comparison operator. TransactionIdFollows does > not satisfy the transitive law. AFAICS the example you give isn't correct. We would lay out the values like this W-3 W-2 W-1 W 3 4 5 where W is the wrap value and in this example we have 7 values in the current array, with tail at W-3 and head at 5. Note the gap between W and 3 where we would skip the values 1 and 2 because they are special. Each element's xid is TransactionIdAdvanced(previous element). So when we search for value 3 we would start from W, then decide it is to the right, which is correct and continue from there. The values are laid out in TransactionIdFollows order, not in numeric order, hence we need to use TransactionIdFollows to decide which way to branch. As long as it works I'm not worried if the array is not technically "sorted". -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invitation to connect on LinkedIn
somasekhar Ramadurgam writes: > I'd like to add you to my professional network on LinkedIn. One reason I will never, ever join LinkedIn is that they encourage their members to spam the world like this. If I were running this mailing list, messages like this one would be sufficient grounds for getting banned. There. I feel better now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invitation to connect on LinkedIn
On Sat, Apr 17, 2010 at 11:01 AM, Tom Lane wrote: > somasekhar Ramadurgam writes: >> I'd like to add you to my professional network on LinkedIn. > > > One reason I will never, ever join LinkedIn is that they encourage > their members to spam the world like this. If I were running this > mailing list, messages like this one would be sufficient grounds for > getting banned. > > > There. I feel better now. How about we blacklist that subject line? I joined linkedin and I'm pretty sure I didn't spam any mailing lists when I did so, but we seem to keep having this problem... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
Simon Riggs writes: > AFAICS the example you give isn't correct. > We would lay out the values like this > W-3 W-2 W-1 W 3 4 5 > where W is the wrap value Stop right there, you're already failing to think clearly. There is no unique "wrap value", all values act the same in circular XID space. The fundamental problem here is that if the set of XIDs involved spans a sufficiently long range, your array will have a[0] < a[1] < ... < a[n] but it will fail to be true that a[0] < a[n]. If that's also true for, say, a[0] vs the midpoint element, then a binary search for a[0] will fail because it will make the wrong decision while probing the midpoint. > The values are laid out in TransactionIdFollows order, They *cannot* be "laid out in TransactionIdFollows order". It's not possible, because that relationship isn't a total ordering. Now it might be the case that this is OK for HS purposes because the set of XIDs that are relevant at any one instant should never span more than half of the XID space. But I'd just as soon not use that assumption here if it's unnecessary. It'd be cheaper anyway to sort and search the array using plain <, so why are you so eager to use TransactionIdFollows? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
On Sat, Apr 17, 2010 at 11:13 AM, Tom Lane wrote: > Simon Riggs writes: >> AFAICS the example you give isn't correct. > >> We would lay out the values like this > >> W-3 W-2 W-1 W 3 4 5 > >> where W is the wrap value > > Stop right there, you're already failing to think clearly. There is no > unique "wrap value", all values act the same in circular XID space. Or to put it a different way, what happens when the "wrap value" changes? An array that was in order under one wrap value can cease to be in order under another. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invitation to connect on LinkedIn
On Sat, Apr 17, 2010 at 4:11 PM, Robert Haas wrote: > On Sat, Apr 17, 2010 at 11:01 AM, Tom Lane wrote: >> somasekhar Ramadurgam writes: >>> I'd like to add you to my professional network on LinkedIn. >> >> >> One reason I will never, ever join LinkedIn is that they encourage >> their members to spam the world like this. If I were running this >> mailing list, messages like this one would be sufficient grounds for >> getting banned. >> >> >> There. I feel better now. > > How about we blacklist that subject line? I joined linkedin and I'm > pretty sure I didn't spam any mailing lists when I did so, but we seem > to keep having this problem... They have an option for importing your address book which I suspect is the cause of this. It does take a few clicks to send anything though, if memory serves. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invitation to connect on LinkedIn
Dave Page wrote: On Sat, Apr 17, 2010 at 4:11 PM, Robert Haas wrote: On Sat, Apr 17, 2010 at 11:01 AM, Tom Lane wrote: somasekhar Ramadurgam writes: I'd like to add you to my professional network on LinkedIn. One reason I will never, ever join LinkedIn is that they encourage their members to spam the world like this. If I were running this mailing list, messages like this one would be sufficient grounds for getting banned. There. I feel better now. How about we blacklist that subject line? I joined linkedin and I'm pretty sure I didn't spam any mailing lists when I did so, but we seem to keep having this problem... They have an option for importing your address book which I suspect is the cause of this. It does take a few clicks to send anything though, if memory serves. Yes, just importing contacts does not in itself cause an invitation to be sent. Sadly though, there is no shortage of people who will mindlessly click a buttons. I've actually found LinkedIn to be tolerably unobtrusive. And they actually advise people fairly prominently to be careful about selecting contacts, e.g.: People you invite to connect will become your first degree connections. Thoughtfully select those people you know and trust because these are the people you will seek advice from and request Recommendations about your/other's quality of work. Because of this, the quality of your connections is always more important than the quantity of connections. I agree with Robert's suggestion of filtering out this subject. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
On Sat, 2010-04-17 at 11:13 -0400, Tom Lane wrote: > Simon Riggs writes: > > AFAICS the example you give isn't correct. > > > We would lay out the values like this > > > W-3 W-2 W-1 W 3 4 5 > > > where W is the wrap value > > Stop right there, you're already failing to think clearly. There is no > unique "wrap value", all values act the same in circular XID space. > > The fundamental problem here is that if the set of XIDs involved spans a > sufficiently long range, your array will have a[0] < a[1] < ... < a[n] > but it will fail to be true that a[0] < a[n]. If that's also true for, > say, a[0] vs the midpoint element, then a binary search for a[0] will > fail because it will make the wrong decision while probing the midpoint. I understand that the xids are circular, but there is only one point where the next xid has a lower integer value but is still "later" from the perspective of TransactionIdFollows. Apart from that single discontinuity all other values are monotonic. From your insistence I presume I must be missing something, but I just don't see it. Perhaps we are just misunderstanding each other about the "sufficiently long range". > > The values are laid out in TransactionIdFollows order, > > They *cannot* be "laid out in TransactionIdFollows order". It's not > possible, because that relationship isn't a total ordering. > > Now it might be the case that this is OK for HS purposes because the set > of XIDs that are relevant at any one instant should never span more than > half of the XID space. Agree that is true. > But I'd just as soon not use that assumption > here if it's unnecessary. I understand what your saying. I think it is necessary here. > It'd be cheaper anyway to sort and search the > array using plain <, so why are you so eager to use > TransactionIdFollows? The array grows to the right and is laid out one xid per element, resulting in a sequence of values that are transactionid-monotonic. As the values increase there will eventually reach the discontinuity where they cease being normally monotonic. Doing it this way means that we can add rows past the head of the array and then move the head atomically, so that we can make adding xids lock-free. If we try to actually sort the values then the algorithm is both more complex and requires locking. It would be easier to just remember where the discontinuity is and act accordingly. So I'm not eager to use either way, but I only currently see one way that would work. If there's a different way, that gives the same or better algorithmic characteristics, I will be happy to code it. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
Simon Riggs writes: > On Sat, 2010-04-17 at 11:13 -0400, Tom Lane wrote: >> It'd be cheaper anyway to sort and search the >> array using plain <, so why are you so eager to use >> TransactionIdFollows? > The array grows to the right and is laid out one xid per element, > resulting in a sequence of values that are transactionid-monotonic. How do you know that just adding items at the right will produce a sorted array? It seems quite unlikely to me that transactions can be guaranteed to arrive at this code in XID order. I think you need to do an explicitly sorted insertion. > ... Doing it this way means that we can > add rows past the head of the array and then move the head atomically, > so that we can make adding xids lock-free. ... and even without that issue, this seems like utter fantasy. How are you going to do that "atomically"? Have you considered what will happen on weak-memory-ordering machines like PPC, in particular? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
On Sat, 2010-04-17 at 15:45 -0400, Tom Lane wrote: > Simon Riggs writes: > > On Sat, 2010-04-17 at 11:13 -0400, Tom Lane wrote: > >> It'd be cheaper anyway to sort and search the > >> array using plain <, so why are you so eager to use > >> TransactionIdFollows? > > > The array grows to the right and is laid out one xid per element, > > resulting in a sequence of values that are transactionid-monotonic. > > How do you know that just adding items at the right will produce a > sorted array? It seems quite unlikely to me that transactions can be > guaranteed to arrive at this code in XID order. I think you need to do > an explicitly sorted insertion. Xids don't arrive in sequence, but "known assigned xids" are added in sequence because we infer the existence of the intermediate xids and assuming they are running for the snapshot. > > ... Doing it this way means that we can > > add rows past the head of the array and then move the head atomically, > > so that we can make adding xids lock-free. > > ... and even without that issue, this seems like utter fantasy. How > are you going to do that "atomically"? Have you considered what will > happen on weak-memory-ordering machines like PPC, in particular? We search the array between tail and head. If the head moves by integer overwrite just as already happens for xid assignment, then we would use the new head for the search. The code is careful to fetch only once. I would freely admit I know absolutely nothing about details of weak-memory-ordering machines and have not considered them at all. How would what I have proposed fail to work, yet what we already rely on work correctly? Do the circumstances differ? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
Simon Riggs writes: > On Sat, 2010-04-17 at 15:45 -0400, Tom Lane wrote: >> How do you know that just adding items at the right will produce a >> sorted array? > Xids don't arrive in sequence, but "known assigned xids" are added in > sequence because we infer the existence of the intermediate xids and > assuming they are running for the snapshot. Hm. Okay, maybe that will work. >> ... and even without that issue, this seems like utter fantasy. How >> are you going to do that "atomically"? Have you considered what will >> happen on weak-memory-ordering machines like PPC, in particular? > We search the array between tail and head. If the head moves by integer > overwrite just as already happens for xid assignment, then we would use > the new head for the search. The code is careful to fetch only once. ... but this will not. You need to use a lock, because there is otherwise no guarantee that other processors see the write into the array element before they see the change in the head pointer. > I would freely admit I know absolutely nothing about details of > weak-memory-ordering machines and have not considered them at all. How > would what I have proposed fail to work, yet what we already rely on > work correctly? Do the circumstances differ? Yes. We have memory ordering instructions inserted in the lock acquisition/release code. Trying to access and modify a shared-memory data structure without any locking will not work. There are some places where we suppose that a *single* write into shared memory can safely be done without a lock, if we're not too concerned about how soon other transactions will see the effects. But what you are proposing here requires more than one related write. I've been burnt by this myself: http://archives.postgresql.org/pgsql-committers/2008-06/msg00228.php regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] master in standby mode croaks
On Wed, Apr 14, 2010 at 7:52 AM, Simon Riggs wrote: > On Wed, 2010-04-14 at 07:07 -0400, Robert Haas wrote: >> On Wed, Apr 14, 2010 at 4:21 AM, Simon Riggs wrote: >> > On Sat, 2010-04-10 at 09:02 -0400, Robert Haas wrote: >> > >> >> So this can fail in either of two ways >> > >> > If I understand this correctly, it is unconvincing as a failure mode >> > since it doesn't follow any of the documented procedures for creating a >> > standby. There are many ways to screw up that ignore the manual, which >> > is why the manual exists. >> > >> > If you can show a full test case, with failure, then I'll follow it >> > through. >> >> Huh? If I had done everything correctly, of course I wouldn't have >> gotten an error message at all. Surely the point is that if I do >> something wrong, I should get an error message that describes what I >> actually did wrong rather than an error message telling me that I did >> something wrong which I clearly did not do. > > I will change the error message. I gave a good deal of thought to trying to figure out a cleaner solution to this problem than just changing the error message and failed. So let's change the error message. Of course I'm not quite sure what we should change it TO, given that the situation is the result of an interaction between three different GUCs and we have no way to distinguish which one(s) are the problem. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] master in standby mode croaks
On Sat, 2010-04-17 at 17:44 -0400, Robert Haas wrote: > > I will change the error message. > > I gave a good deal of thought to trying to figure out a cleaner > solution to this problem than just changing the error message and > failed. So let's change the error message. Of course I'm not quite > sure what we should change it TO, given that the situation is the > result of an interaction between three different GUCs and we have no > way to distinguish which one(s) are the problem. "You need all three" covers it. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
On Sat, 2010-04-17 at 16:48 -0400, Tom Lane wrote: > > We search the array between tail and head. If the head moves by integer > > overwrite just as already happens for xid assignment, then we would use > > the new head for the search. The code is careful to fetch only once. > > ... but this will not. You need to use a lock, because there is > otherwise no guarantee that other processors see the write into the > array element before they see the change in the head pointer. > > > I would freely admit I know absolutely nothing about details of > > weak-memory-ordering machines and have not considered them at all. How > > would what I have proposed fail to work, yet what we already rely on > > work correctly? Do the circumstances differ? > > Yes. We have memory ordering instructions inserted in the lock > acquisition/release code. Trying to access and modify a shared-memory > data structure without any locking will not work. > > There are some places where we suppose that a *single* write into shared > memory can safely be done without a lock, if we're not too concerned > about how soon other transactions will see the effects. But what you > are proposing here requires more than one related write. > > I've been burnt by this myself: > http://archives.postgresql.org/pgsql-committers/2008-06/msg00228.php W O W - thank you for sharing. What I'm not clear on is why you've used a spinlock everywhere when only weak-memory thang CPUs are a problem. Why not have a weak-memory-protect macro that does does nada when the hardware already protects us? (i.e. a spinlock only for the hardware that needs it). -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
Simon Riggs writes: > What I'm not clear on is why you've used a spinlock everywhere when only > weak-memory thang CPUs are a problem. Why not have a weak-memory-protect > macro that does does nada when the hardware already protects us? (i.e. a > spinlock only for the hardware that needs it). Well, we could certainly consider that, if we had enough places where there was a demonstrable benefit from it. I couldn't measure any real slowdown from adding a spinlock in that sinval code, so I didn't propose doing so at the time --- and I'm pretty dubious that this code is sufficiently performance-critical to justify the work, either. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] master in standby mode croaks
On Sat, Apr 17, 2010 at 6:41 PM, Simon Riggs wrote: > On Sat, 2010-04-17 at 17:44 -0400, Robert Haas wrote: > >> > I will change the error message. >> >> I gave a good deal of thought to trying to figure out a cleaner >> solution to this problem than just changing the error message and >> failed. So let's change the error message. Of course I'm not quite >> sure what we should change it TO, given that the situation is the >> result of an interaction between three different GUCs and we have no >> way to distinguish which one(s) are the problem. > > "You need all three" covers it. Actually you need standby_connections and either archive_mode=on or max_wal_senders>0, I think. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] enable_material patch
Here's a patch to add enable_material, per previous discussion. I still think we should add enable_joinremoval also, but there wasn't a clear consensus for that. I'd appreciate it if someone could check this over for sanity - like, did I get all the places where materialize nodes can be created? and, did i prevent them from being inserted anywhere that they are necessary for correctness? Thanks, ...Robert enable-material.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] patch: Distinguish between unique indexes and unique constraints
Addressing TODO item "Distinguish between unique indexes and unique constraints in \d+" for psql, and picking up from thread: http://archives.postgresql.org/message-id/8780.1271187...@sss.pgh.pa.us Attached is a simple patch which clarifies unique constraints with "UNIQUE CONSTRAINT" in psql's \d+ description of a table. The appearance of unique indexes is left as-is. == Old \d+ display == Indexes: "name_uniq_constr" UNIQUE, btree (name) == New \d+ display == Indexes: "name_uniq_constr" UNIQUE CONSTRAINT, btree (name) Josh psql_constraints.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] fix segfault with DO and plperl/plperlu
If you do:
# DO $do$ 1; $do$ LANGUAGE plperlu;
# DO $do$ 1; $do$ LANGUAGE plperl;
You get a segfault as we try to SvREFCNT_dec(...); for the wrong
interpreter. To fix push down the restore_context() so that we do the
above on the correct perl interpreter.
--
*** a/src/pl/plperl/plperl.c
--- b/src/pl/plperl/plperl.c
***
*** 1154,1170 plperl_inline_handler(PG_FUNCTION_ARGS)
PG_CATCH();
{
current_call_data = save_call_data;
- restore_context(oldcontext);
if (desc.reference)
SvREFCNT_dec(desc.reference);
PG_RE_THROW();
}
PG_END_TRY();
current_call_data = save_call_data;
- restore_context(oldcontext);
if (desc.reference)
SvREFCNT_dec(desc.reference);
error_context_stack = pl_error_context.previous;
--- 1154,1170
PG_CATCH();
{
current_call_data = save_call_data;
if (desc.reference)
SvREFCNT_dec(desc.reference);
+ restore_context(oldcontext);
PG_RE_THROW();
}
PG_END_TRY();
current_call_data = save_call_data;
if (desc.reference)
SvREFCNT_dec(desc.reference);
+ restore_context(oldcontext);
error_context_stack = pl_error_context.previous;
*** a/src/pl/plperl/plperl.c
--- b/src/pl/plperl/plperl.c
***
*** 1154,1170 plperl_inline_handler(PG_FUNCTION_ARGS)
PG_CATCH();
{
current_call_data = save_call_data;
- restore_context(oldcontext);
if (desc.reference)
SvREFCNT_dec(desc.reference);
PG_RE_THROW();
}
PG_END_TRY();
current_call_data = save_call_data;
- restore_context(oldcontext);
if (desc.reference)
SvREFCNT_dec(desc.reference);
error_context_stack = pl_error_context.previous;
--- 1154,1170
PG_CATCH();
{
current_call_data = save_call_data;
if (desc.reference)
SvREFCNT_dec(desc.reference);
+ restore_context(oldcontext);
PG_RE_THROW();
}
PG_END_TRY();
current_call_data = save_call_data;
if (desc.reference)
SvREFCNT_dec(desc.reference);
+ restore_context(oldcontext);
error_context_stack = pl_error_context.previous;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
